Fixed README, gitignore and analysis description in radsex

.gitignore

@@ -92,6 +92,7 @@ ENV/
 *.test
 test.*
 test/
+tests/
 resources.txt
 *denovo_map.sh
 user.json
@@ -101,3 +102,5 @@ output/*
 results*/*
 data/*
 html/*
+bin/
+build/

README.md

-## RADSeq_analysis
+## RadSex
 
 ### Overview
 
-This software is part of the RADSex pipeline, a method to detect sex-linked sequences in RADSeq data. As such, it is only designed to work with the output of Stacks using specific parameter values. This pipeline was developed for the PhyloSex project, which investigates sex determining factors in a wide range of fish species.
+The RADSex pipeline is used to analyze RADSeq data with focus on sex. This pipeline was developed for the PhyloSex project, which investigates sex determining factors in a wide range of fish species.
 
 ### Requirements
 
-- Python 3.5 or higher
+- A C++11 compliant compiler (GCC >= 4.8.1, Clang >= 3.3)
+- The zlib library (should be installed on linux by default)
 - *Optional (for visualization)* : R 3.3 or higher with the following packages:
     + readr
     + ggplot2
@@ -18,13 +19,12 @@ This software is part of the RADSex pipeline, a method to detect sex-linked sequ
     + svglite
     + scales
 
-- *Optional* : package `progress` to display progress bars in lengthy steps
-
 ### Installation
 
-- Clone: `git clone git@github.com:INRA-LPGP/radseq_analysis.git`
-- Download the archive and unzip it
-- *Optional* : install recommended python packages with `pip3 install -r requirements.txt`
+- Clone: `git clone git@github.com:INRA-LPGP/RadSex.git`
+- Alternative: Download the archive and unzip it
+- Go to the RadSex directory (`cd RadSex`)
+- Run `make`
 - *Optional* : install R packages for visualization with `Rscript install_packages.R`
 
 ### Usage
@@ -32,121 +32,75 @@ This software is part of the RADSex pipeline, a method to detect sex-linked sequ
 
 #### General
 
-`python3 radseq_analysis.py <command> [options]`
+`radsex <command> [options]`
 
 **Available commands** :
 
 Command            | Description
-------------- | ------------
-`heatmap`     | Generates a matrix of haplotypes sex distribution
-`haplotypes`  | Extract haplotypes present in a given number of males and females
-`frequencies` | Calculate haplotypes frequencies distribution in the population
-`rescue`      | Find all alleles in sex-linked loci from the Stacks catalog
-`visualize`   | Visualize analyses results using R
-
-<br/>
+------------------ | ------------
+`process_reads`    | Compute a matrix of coverage from a set of demultiplexed reads files
+`sex_distribution` | Calculate a distribution of sequences between sexes
+`subset` | Extract a subset of the coverage matrix
 
-#### Heatmap
-
-`python3 radseq_analysis.py heatmap -i input_folder -m popmap [-o output_file]`
-
-*Generates a matrix of dimension (Number of males) x (Number of females). The value at coordinates **(i, j)** corresponds to the number of haplotypes found in precisely **i** males and **j** females.*
-
-**Options** :
-
-Option | Full name | Description
---- | --- | ---
-`-i` | `--input-folder` | Path to a folder containing the output of denovo_map |
-`-m` | `--popmap` | Path to a population map file |
-`-o` | `--output-file` | Path to the output file (default: *haplotypes_matrix.tsv*)
 
 <br/>
 
-#### Haplotypes
-
-`python3 radseq_analysis.py haplotypes -i input_folder -m popmap -p positions_list [-o output_file]`
+#### Process reads
 
-*Extracts all the haplotypes found in a given number of males and females (a position in the haplotypes matrix). The output is a tabulated file with the following fields :*
+`radsex process_reads -d input_dir_path -o output_file_path [ -t n_threads -c min_cov ]`
 
-- *Locus* : catalog ID of the haplotype
-- *Males* : number of males in which the haplotype was found
-- *Females* : number of females in which the haplotype was found
-- *Sequence* : haplotype sequence
-- *Male_outliers* : ID of the males in which the haplotype was not found
-- *Female_outliers* : ID of the males in which the haplotype was not found
+*Generates a matrix of coverage for all individuals and all sequences. The output is a tabulated file, where each line contains the ID, sequence and coverage for each individual of a marker.*
 
 **Options** :
 
 Option | Full name | Description
 --- | --- | ---
-`-i` | `--input-folder` | Path to a folder containing the output of denovo_map |
-`-m` | `--popmap` | Path to a population map file |
-`-p` | `--positions` | Path to a file containing the list of positions to extract |
-`-o` | `--output-file` | Path to the output file (default: *extracted_haplotypes.tsv*)
+`-d` | `input_dir_path` | Path to a folder containing demultiplexed reads |
+`-o` | `output_file_path` | Path to the output file |
+`-t` | `n_threads` | Number of threads to use (default: 1) |
+`-c` | `min_cov` | Minimum coverage to consider a marker in an individual (default: 1) |
 
 <br/>
 
-#### Frequencies
+#### Sex distribution
 
-`python3 radseq_analysis.py frequencies -i input_folder [-o output_file]`
+`radsex sex_distribution -f input_file_path -o output_file_path -p popmap_file_path [ -c min_cov ]`
 
-*Computes the distribution of haplotypes frequencies in the population. The output is a tabulated file with the following fields*
-
-- *Frequency* : number of individuals in which a haplotype is found
-- *Count* : number of haplotypes with the associated frequency in the population
+*Generates a matrix of dimensions (Number of males) x (Number of females). The value at coordinates **(i, j)** corresponds to the number of haplotypes found in precisely **i** males and **j** females.*
 
 **Options** :
 
 Option | Full name | Description
 --- | --- | ---
-`-i` | `--input-folder` | Path to a folder containing the output of denovo_map |
-`-o` | `--output-file` | Path to the output file (default: *haplotypes_frequencies.tsv*)
+`-f` | `input_file_path` | Path to an input file (result of process_reads) |
+`-o` | `output_file_path` | Path to the output file |
+`-p` | `popmap_file_path` | Path to a popmap file (indicating the sex of each individual) |
+`-c` | `min_cov` | Minimum coverage to consider a marker in an individual (default: 1) |
 
 <br/>
 
-#### Rescue
-
-`python3 radseq_analysis.py rescue -i input_folder -s sequences_file [-c coverage_file -o output_file]`
+#### Subset
 
-*Find all alleles in sex-linked loci from the Stacks catalog by blasting sex-linked sequences and filtering by similarity. If a coverage file is provided, loci coverage will be corrected based on global coverage differences between individuals. The output is a tabulated file with the following fields :*
+`radsex subset -f input_file_path -o output_file_path -p popmap_file_path [ -c min_cov --min-males min_males --min-females min_females --max-males max_males --max-females max_females ]`
 
-- Stack_ID: catalog ID of the sex-linked haplotype
-- Haplotype_ID: catalog ID of the rescued haplotype (other allele)
-- Sequence: rescued haplotype sequence
-- Matches: number of matching bases between the sex-linked haplotype and the rescued haplotype
-- Mismatches: number of non-matching bases between the sex-linked haplotype and the rescued haplotype
-- Gaps: number of gaps between the sex-linked haplotype and the rescued haplotype
+*Filters the coverage matrix to export markers matching the values of min_males, min_females, max_males, and max_females (i.e. markers found in M males with min_males <= M <= max_males and F females with min_females <= F <= max_females)*
 
 **Options** :
 
 Option | Full name | Description
 --- | --- | ---
-`-i` | `--input-folder` | Path to a folder containing the output of denovo_map |
-`-s` | `--sequences` | Path to a sequences file (result of *haplotypes*)|
-`-c` | `--coverage-file` | Path to a coverage file (result of *coverage*) |
-`-o` | `--output-file` | Path to the output file (default: *extracted_alleles.tsv*)
+`-f` | `input_file_path` | Path to an input file (result of process_reads) |
+`-o` | `output_file_path` | Path to the output file |
+`-p` | `popmap_file_path` | Path to a popmap file (indicating the sex of each individual) |
+`-c` | `min_cov` | Minimum coverage to consider a marker in an individual (default: 1) |
+`--min-males` | `min_males` | Minimum number of males with a marker |
+`--min-females` | `min_females` | Minimum number of females with a marker |
+`--max-males` | `max_males` | Maximum number of males with a marker |
+`--max-females` | `max_females` | Maximum number of females with a marker |
 
 <br/>
 
-#### Visualize
-
-`python3 radseq_analysis.py visualize -i input_file -o output_file -m popmap`
-
-*Generate plots to visualize output from heatmap, rescue, or frequencies commands. The input file type is automatically detected. The following plots are generated :*
-
-- **heatmap** : a heatmap representation of the loci matrix, with number of males as abscissis and number of females as ordinates. The color of a tile at position **(i, j)** shows the number of haplotypes shared by exactly **i** males and **j** females.
-- **rescue** : two heatmaps reprensentations of the results of clustering for both alleles and individuals; in the first heatmap, the values are presence/absence of loci. In the second heatmap, the values are individual coverage for each locus.
-- **frequencies** : a barplot showing the distribution of haplotypes frequencies in the population.
-
-**Options** :
-
-Option | Full name | Description
---- | --- | ---
-`-i` | `--input-file` | Path to a file generated by this pipeline |
-`-m` | `--popmap` | Path to a population map file |
-`-o` | `--output-file` | Path to the output file |
-
-**Examples** :
+**Example output** :
 
 - heatmap :
 ![Heatmap](./examples/plots/heatmap.png)
@@ -160,7 +114,7 @@ Option | Full name | Description
 
 MIT License
 
-Copyright (c) 2017 Romain Feron
+Copyright (c) 2017-2018 Romain Feron and INRA LPGP
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

benchmarks.txt

-#### Buffer implementation (can't handle FASTQ correctly)
-
-./bin/radsex process_reads -d ./test/fastq/ -o ./test.tsv  25,72s user 1,32s system 92% cpu 29,297 total
-2628535 ./test.tsv
-
-
-./bin/radsex process_reads -d ./test/fasta/ -o ./test.tsv  23,75s user 2,10s system 77% cpu 33,437 total
-2628991 ./test.tsv
-
-
-./bin/radsex process_reads -d ./test/fastqgz/ -o ./test.tsv  44,65s user 1,52s system 84% cpu 54,520 total
-2628535 ./test.tsv
-
-
-./bin/radsex process_reads -d ./test/fastagz/ -o ./test.tsv  33,05s user 1,04s system 83% cpu 40,755 total
-2628991 ./test.tsv
-
-
-
-#### Using KSEQ
-
-./bin/radsex process_reads -d ./test/fastq/ -o ./test.tsv  39,05s user 2,14s system 71% cpu 57,846 total
-2631957 ./test.tsv
-
-
-./bin/radsex process_reads -d ./test/fasta/ -o ./test.tsv  27,41s user 1,27s system 87% cpu 32,833 total
-2631957 ./test.tsv
-
-
-./bin/radsex process_reads -d ./test/fastqgz/ -o ./test.tsv  49,03s user 1,07s system 95% cpu 52,449 total
-
-./bin/radsex process_reads -d ./test/fastagz/ -o ./test.tsv  36,66s user 0,86s system 94% cpu 39,526 total

src/radsex.h

@@ -29,7 +29,7 @@ class RadSex {
                                                                               std::vector<std::string> {"input_dir_path", "output_file_path", "n_threads", "min_cov"},
                                                                               process_reads)},
                                                    {"subset", Analysis("subset",
-                                                                       "Extracts a subset of a coverage matrix, from a sex distribution file",
+                                                                       "Extracts a subset of the coverage matrix",
                                                                        std::vector<std::string> {"input_file_path", "output_file_path", "min_cov", "popmap_file_path",
                                                                                                  "min_males", "min_females", "max_males", "max_females"},
                                                                        subset)}