README.md 22.2 KB
Newer Older
1
2
# RADSex

3
4
5
The RADSex pipeline is **currently under development** and has not been officially released yet. 
Missing features are been implemented, and some bugs are to be expected in this current development version. 
Please contact me by email or on Github, or open an issue if you encounter bugs or would like to discuss a feature !
6

Romain Feron's avatar
Romain Feron committed
7
## Overview
8

9
10
11
12
RADSex is a software package for the analysis of sex-determination using RAD-Sequencing data. 
The `process` function generates a data structure summarizing a set of demultiplexed RAD reads, 
and other functions use this data structure to infer information about the type of sex-determination system, identify sex-biased sequences, and map the RAD sequences to a reference genome. 
The results of RADSex are meant to be visualized with the `radsex-vis` R package, available here: https://github.com/INRA-LPGP/radsex-vis.
13

14
This pipeline was developed for the PhyloSex project, which investigates sex determining factors in a wide range of fish species.
Romain Feron's avatar
Romain Feron committed
15
16

## Requirements
17

18
- A C++11 compliant compiler (GCC >= 4.8.1, Clang >= 3.3)
Romain Feron's avatar
Romain Feron committed
19
20
21
- The zlib library (which should be installed on linux by default)

## Installation
22

23
- Clone: `git clone https://github.com/RomainFeron/RadSex.git`
24
- *Alternative: download the archive and unzip it*
25
26
- Go to the RadSex directory (`cd RadSex`)
- Run `make`
Romain Feron's avatar
Romain Feron committed
27
- The compiled `radsex` binary is located in `RadSex/bin/`
28

29
30
31
32
33
## Quick start

#### Before starting

Before running the pipeline, you should prepare the following elements:
34
35
36
37
38
39
40
41
42
43
- A **set of demultiplexed reads**. The current version of RADSex does not implement demultiplexing;
  raw sequencing reads can be demultiplexed using [Stacks](http://catchenlab.life.illinois.edu/stacks/comp/process_radtags.php)
  or [pyRAD](http://nbviewer.jupyter.org/gist/dereneaton/af9548ea0e94bff99aa0/pyRAD_v.3.0.ipynb#The-seven-steps-described).
- A **population map** (popmap): a tabulated file with individual ID as the first column and sex as the second column.
  It is important that the individual IDs in the popmap are the same as the names of the demultiplexed reads files (see the [popmap section](#population-map) for details).
- If you want to map the sequences to a reference genome: a **reference genome** in fasta format.
  Note that when visualizing `mapping` results with `radsex-vis`, linkage groups / chromosomes are automatically inferred from scaffold names in the reference sequence
  if their name starts with *LG*, *chr*, or *chromosome* (case unsensitive).
  If chromosomes are named differently in the reference genome, you should prepare a tabulated file with
  reference scaffold ID in the first column and corresponding chromosome name in the second column (see the [chromosomes names section](#chromosomes-names) for details)
44
45
46
47
48

#### Computing the coverage table

The first step of RADSex is to create a table of coverage for the dataset using the `process` command:

49
`radsex process --input-dir ./samples --output-file coverage_table.tsv --threads 16 --min-coverage 1`
50

51
In this example, demultiplexed reads are stored in `./samples` and the coverage table generated by `process` will be stored in `coverage_table.tsv`. The parameter `--threads` specifies the number of threads to use.
52

53
54
55
The parameter `--min-coverage` specifies the minimum coverage value to consider a sequence present in an individual: 
sequences which are not present with coverage higher than this value in at least one individual will not be retained in the coverage table. 
It is advised to keep the minimum coverage to 1 for this step, as it can be adjusted for each analysis later.
56
57
58
59
60

#### Computing the distribution of sequences between sexes

After generating the coverage table, the `distrib` command is used to compute the distribution of sequences between sexes:

61
`radsex distrib --input-file coverage_table.tsv --output-file distribution.tsv --popmap-file popmap.tsv --min-coverage 5`
62

63
64
65
In this example, the input file `--input-file` is the coverage table generated in the [previous step](#computing-the-coverage-table), and the distribution of sequences between sexes will be stored in `distribution.tsv`. 
The sex of each individual in the population is given by `popmap.tsv` (see the [popmap section](#population-map) for details). 
The minimum coverage to consider a sequence present in an individual is set to 5, meaning that sequences present with coverage (depth) lower than 5 in one individual will not be counted in this individual.
66
67
68
69
70
71
72
73
74
75
76
77
78

The resulting file `distribution.tsv` is a table with four columns:
- **Males** : number of males in which a sequence was present.
- **Females** : number of females in which a sequence was present.
- **Sequences** : number of sequences present in the corresponding number of males and females.
- **P** : p-value of a chi-squared test for association with sex.

This distribution can be visualized with the `plot_sex_distribution()` function of `radsex-vis`, which generates a [distribution heatmap](./examples/figures/sex_distribution.png).

#### Extracting sequences significantly associated with sex

Sequences significantly associated with sex can be obtained with the `signif` command:

79
`radsex signif --input-file coverage_table.tsv --output-file sequences.tsv --popmap-file popmap.tsv --min-coverage 5 [ --output-format fasta ]`
80

81
82
83
84
In this example, the input file `--input-file` is the coverage table generated in the [first step](#computing-the-coverage-table), and the sequences significantly associated with sex will be stored in `sequences.tsv`. 
The sex of each individual in the population is given by `popmap.tsv` (see the [popmap section](#population-map) for details), 
and the minimum coverage to consider a sequence present in an individual is set to 5 (see the [previous section](#computing-the-distribution-of-sequences-between-sexes)). 
By default, the `signif` function exports a small coverage table; sequences can be exported to fasta using the `--output-format` parameter.
85
86
87
88
89
90
91

The coverage table generated by `signif` can be visualized with the `plot_coverage()` function of `radsex-vis`, which generates a [coverage heatmap](./examples/figures/coverage.png)

#### Mapping sequences to a reference genome

Sequences can be mapped to a reference genome using the `map` command:

92
`radsex map --input-file coverage_table.tsv --output-file mapping.tsv --popmap-file popmap.tsv --genome-file genome.fasta --min-quality 20 --min-frequency 0.1 --min-coverage 5`
93

94
95
96
97
98
In this example, the input file `--input-file` is the coverage table generated in the [first step](#computing-the-coverage-table), the mapping results will be stored in `sequences.tsv`, 
and the path to the reference genome file is given by `--genome-file`. The sex of each individual in the population is given by `popmap.tsv` (see the [popmap section](#population-map) for details), 
and the minimum coverage to consider a sequence present in an individual is set to 5 (see the [previous section](#computing-the-distribution-of-sequences-between-sexes)). 
The parameter `--min-quality` specifies the minimum mapping quality (as defined in [BWA](http://bio-bwa.sourceforge.net/bwa.shtml)) to consider a sequence mapped (`--min-quality`), here set to 20. 
The parameter `--min-frequency` specifies the minimum frequency of a sequence in at least one sex; it is set to 0.1 here, meaning that only sequences present in at least 10% of individuals of one sex are retained for mapping.
99
100
101
102
103
104
105
106

The resulting file `mapping.tsv` is a table with five columns:
- **Sequence :** ID of the mapped sequence
- **Contig :** ID of the contig where the sequence mapped
- **Position :** position of the mapped sequence on the contig
- **SexBias :** sex-bias of the mapped sequence, defined as (Males / Total males ) - (Females / Total females)
- **P :** p-value of a chi-squared test for association with sex

107
108
The mapping results generated by `map` can be visualized with the `plot_genome()` function of `radsex-vis`, which generates a [circular plot](./examples/figures/genome.png). 
Mapping results for a specific scaffold can be visualized with the `plot_scaffold()` function to generate a [linear plot](./examples/figures/scaffold.png).
109

Romain Feron's avatar
Romain Feron committed
110
## Usage
111

Romain Feron's avatar
Romain Feron committed
112
### General
113

114
`radsex <command> [options]`
115
116
117

**Available commands** :

118
119
Command            | Description
------------------ | ------------
120
`process`    | Compute a table of coverage from a set of demultiplexed reads
Romain Feron's avatar
Romain Feron committed
121
`distrib` | Compute the distribution of sequences between sexes
122
`subset` | Extract a subset of the coverage table
Romain Feron's avatar
Romain Feron committed
123
`signif` | Extract sequences significantly associated with sex
124
`loci` | Recreate polymorphic loci from a subset of coverage table
Romain Feron's avatar
Romain Feron committed
125
126
`mapping` | Map a subset of sequences (coverage table or fasta) to a reference genome and output sex-association metrics for each mapped sequence
`freq` | Compute sequence frequencies for the population
127

Romain Feron's avatar
Romain Feron committed
128
### process
129

130
`radsex process --input-dir input_dir_path --output-file output_file_path [ --threads n_threads --min-coverage min_cov ]`
131

132
*Generates a table of coverage for all individuals and all sequences. The output is a tabulated file, where each line contains the ID, sequence and coverage for each individual of a sequence.*
133
134
135
136
137

**Options** :

Option | Full name | Description
--- | --- | ---
138
139
140
141
`--input-dir` | `input_dir_path` | Path to a folder containing demultiplexed reads |
`--output-file``output_file_path` | Path to the output file |
`--threads``n_threads` | Number of threads to use (default: 1) |
`--min-coverage``min_cov` | Minimum coverage to consider a sequence in an individual (default: 1) |
142

Romain Feron's avatar
Romain Feron committed
143
### distrib
144

145
`radsex distrib --input-file input_file_path --output-file output_file_path --popmap-file popmap_file_path [ --min-coverage min_cov --output-matrix ]`
146

Romain Feron's avatar
Romain Feron committed
147
*Generates a table which contains the number of sequences present with coverage higher than min_cov and the probability of association with sex for every combination of number of males and number of females.*
148
149
150
151

**Options** :

Option | Full name | Description
152
------ | --------- | -------------
153
154
155
156
`--input-file` | `input_file_path` | Path to an coverage table obtained with `process` |
`--output-file``output_file_path` | Path to the output file |
`--popmap-file``popmap_file_path` | Path to a popmap file indicating the sex of each individual |
`--min-coverage``min_cov` | Minimum coverage to consider a sequence present in an individual (default: 1) |
157
`--output-matrix``output_matrix` | If true, outputs the resutls as a matrix with males in columns and females in rows instead of a table (default: false) |
158

159
### subset
160

161
`radsex subset --input-file input_file_path --output-file output_file_path --popmap-file popmap_file_path [ --output-format output_format --min-coverage min_cov --min-males min_males --min-females min_females --max-males max_males --max-females max_females --min-individuals min_individuals --max-individuals max_individuals]`
162

163
*Filters the coverage table to only export sequences present in any combination of M males and F females, with min_males ≤ M ≤ max_males, min_females ≤ F ≤ max_females, and min_individuals ≤ M + F ≤ max_individuals*
164
165
166
167
168

**Options** :

Option | Full name | Description
--- | --- | ---
169
170
171
172
173
`--input-file` | `input_file_path` | Path to an coverage table obtained with `process` |
`--output-file``output_file_path` | Path to the output file |
`--popmap-file``popmap_file_path` | Path to a popmap file indicating the sex of each individual |
`--output-format``output-format` | Output format, either "table" or "fasta" |
`--min-coverage``min_cov` | Minimum coverage to consider a sequence present in an individual (default: 1) |
174
175
176
177
178
179
180
181
182
`--min-males``min_males` | Minimum number of males with a retained sequence |
`--min-females``min_females` | Minimum number of females with a retained sequence |
`--max-males``max_males` | Maximum number of males with a retained sequence |
`--max-females``max_females` | Maximum number of females with a retained sequence |
`--max-individuals``max_individuals` | Maximum number of individuals with a retained sequence |
`--max-individuals``max_individuals` | Maximum number of individuals with a retained sequence |

### signif

183
`radsex signif --input-file input_file_path --output-file output_file_path --popmap-file popmap_file_path [ --output-format output_format --min-coverage min_cov ]`
184
185
186
187
188
189
190

*Filters the coverage table to only export sequences significantly associated with sex, defined as sequences for which p < 0.05 (after Bonferroni correction), with p being the p-value of a chi-squared test on the numbers of males and females.*

**Options** :

Option | Full name | Description
--- | --- | ---
191
192
193
194
195
`--input-file` | `input_file_path` | Path to an coverage table obtained with `process` |
`--output-file``output_file_path` | Path to the output file |
`--popmap-file``popmap_file_path` | Path to a popmap file indicating the sex of each individual |
`--output-format``output-format` | Output format, either "table" or "fasta" |
`--min-coverage``min_cov` | Minimum coverage to consider a sequence present in an individual (default: 1) |
196
197
198

### map

199
`radsex map --input-file input_file_path --output-file output_file_path --popmap-file popmap_file_path --genome-file genome_file_path [ --min-coverage min_cov --min-quality min_quality --min-frequency min_frequency ]`
200
201
202
203
204
205
206

*Maps the sequences from the coverage table to a reference genome and outputs mapping position, sex bias, and p-value of association with sex for each mapped sequence.*

**Options** :

Option | Full name | Description
--- | --- | ---
207
208
209
210
211
212
`--input-file` | `input_file_path` | Path to an coverage table obtained with `process` |
`--output-file``output_file_path` | Path to the output file |
`--popmap-file``popmap_file_path` | Path to a popmap file indicating the sex of each individual |
`--genome-file``genome_file_path` | Path to a reference genome file in fasta format |
`--min-coverage``min_cov` | Minimum coverage to consider a sequence present in an individual (default: 1) |
`--min-quality``min_quality` | Minimum mapping quality, as defined in BWA, to consider a sequence properly mapped (default: 20) |
213
214
215
216
`--min-frequency``min_frequency` | Minimum frequency in at least one sex for a sequence to be retained (default: 0.25) |

### freq

217
`radsex freq --input-file input_file_path --output-file output_file_path [ --min-coverage min_cov ]`
218
219
220
221
222
223
224

*Computes the sequences frequencies for the entire population*

**Options** :

Option | Full name | Description
--- | --- | ---
225
226
227
`--input-file` | `input_file_path` | Path to an coverage table obtained with `process` |
`--output-file``output_file_path` | Path to the output file |
`--min-coverage``min_cov` | Minimum coverage to consider a sequence present in an individual (default: 1) |
228

Romain Feron's avatar
Romain Feron committed
229
230
231
232
## FILE FORMATS

### Population map

233
234
A population map file is a tabulated file without header, with individual ID in the first column and sex in the second column. 
Sex is encoded as 'M' for males, 'F' for females, and 'N' for undetermined. An example of population map is given below:
Romain Feron's avatar
Romain Feron committed
235
236
237
238
239
240
241
242
243

```
individual_1    M
individual_2    M
individual_3    F
individual_4    N
individual_5    F
```

244
245
246
Individual IDs can be anything, but it is important that they correspond to the name of the demultiplexed files. 
For instance, the reads file for *individual_1* should be named `individual_1.fastq.gz` (in any format supported by your demultiplexer). 
If you are using Stacks with a barcodes file for demultiplexing, just make sure that individual IDs in the barcodes file and in the population map are the same.
Romain Feron's avatar
Romain Feron committed
247
248
249

### Chromosomes names

250
251
252
253
254
Genome-wide results from the `map` command are visualized using the `plot_genome()` function of `radsex-vis`. 
This function can automatically detect chromosomes in the reference file if their name starts with 'LG' or 'chr' (case unsensitive). 
If this is not the case, you should provide a chromosomes names file to `plot_genome()`. 
This file should be a tabulated file without header, with scaffold ID in the reference in the first column and corresponding chromosome name in the second column. 
An example of chromosomes names file is given below for the [Northern Pike genome](https://www.ncbi.nlm.nih.gov/genome/?term=esox%20lucius) :
Romain Feron's avatar
Romain Feron committed
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289

```
NC_025968.3     LG01
NC_025969.3     LG02
NC_025970.3     LG03
NC_025971.3     LG04
NC_025972.3     LG05
NC_025973.3     LG06
NC_025974.3     LG07
NC_025975.3     LG08
NC_025976.3     LG09
NC_025977.3     LG10
NC_025978.3     LG11
NC_025979.3     LG12
NC_025980.3     LG13
NC_025981.3     LG14
NC_025982.3     LG15
NC_025983.3     LG16
NC_025984.3     LG17
NC_025985.3     LG18
NC_025986.3     LG19
NC_025987.3     LG20
NC_025988.3     LG21
NC_025989.3     LG22
NC_025990.3     LG23
NC_025991.3     LG24
NC_025992.3     LG25
```

The chromosomes names can be anything starting with 'LG' or 'chr' (LG1, LG_01, chr1, chromosome_01 ...).

### RADSex output files

#### Coverage table

290
291
292
Coverage tables tabulated files with header generated by the `process` command for the entire dataset, and by the `subset` and `signif` commands for a subset of sequences. 
The first column contains the sequence ID, and the second column contains the sequence itself. Each other column contains the coverage of the corresponding sequence in a given individual. 
An example of coverage table is given below (the sequence was shortened for visual reasons):
Romain Feron's avatar
Romain Feron committed
293
294
295
296
297
298
299
300
301

```
ID    Sequence    individual_1    individual_2    individual_3    individual_4    individual_5
0    TGCA..TATT         0              15              24              17              21
1    TGCA..GACC        20              18               3              26               4
2    TGCA..ATCG         2               1               5              16               0
3    TGCA..CCGA        14              29              23               2              19
```

302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
#### FASTA file

FASTA files can be generated by the `subset` and `signif` commands for a subset of sequences.

In the `subset` analysis, FASTA headers are generated as follows:

```
<ID>_<number of males>M_<number of females>F_cov:<minimum coverage>
```

In the `signif` analysis, another field containing the p-value of association with sex is added:

```
<ID>_<number of males>M_<number of females>F_cov:<minimum coverage>_p:<p-value>
```
Romain Feron's avatar
Romain Feron committed
317
318
319
320
321

#### Distribution of sequences between sexes

##### Table format

322
323
324
325
A table of distribution of sequences between sexes is a tabulated file with header generated by the `distrib` command. 
The first and second columns indicate the number of males and females in which a sequence is present, the third column contains the number of sequences found in the corresponding number of males and females, 
the fourth column contains the p-value of a chi-squared test for association with sex, and the fifth column indicates whether this p-value is significant after Bonferroni correction. 
An example of sex distribution table is given below for 3 males and 3 females:
Romain Feron's avatar
Romain Feron committed
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349

```
Males    Females    Sequences      P      Signif
  0         1           7          1      False
  0         2           3       0.39      False
  0         3           1       0.10      False
  1         0           6          1      False
  1         1           5          1      False
  1         2           1          1      False
  1         3           2       0.39      False
  2         0           3       0.39      False
  2         1           8          1      False
  2         2           4          1      False
  2         3           2          1      False
  3         0           4       0.10      False
  3         1           7       0.39      False
  3         2           6          1      False
  3         3           9          1      False
```

In this example, there are 68 sequences in total, therefore sequences are significantly associated with sex if the p-value of a chi-squared test on the number of males and females is lower than 0.05 / 68 = 0.00074.

##### Matrix format

350
351
The distribution of sequences between sexes can also be output as a matrix, which is a tabulated file without header, with number of females as rows and number of males as rows. 
The sex distribution matrix for the example described above is given below:
Romain Feron's avatar
Romain Feron committed
352
353
354
355
356
357
358
359
360
361

```
0    6    3    4
7    5    8    7
3    1    4    6
1    2    2    9
```

#### Mapping results

362
363
364
365
366
Results from the `map` command are output as a tabulated file with header. 
The first column contains the sequence ID, the second column contains the contig to which the sequence mapped in the reference genome, and the third columns contains the position where the sequence mapped on the contig. 
The fourth column contains a sex-bias value, defined as `(number of males with the sequence) / (total number of males) - (number of females with the sequence) / (total number of females)`. 
The fifth column contains the p-value of a chi-squared test for association with sex, and the sixth column indicates whether this p-value is significant after Bonferroni correction. 
An example of mapping results is given below:
Romain Feron's avatar
Romain Feron committed
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387

```
Sequence      Contig      Position      SexBias        P     Signif
0             LG09        10052920            0        1     False
1             LG45         4008419            0        1     False
2             LG06        20521435            0        1     False
3             LG24         7643946         0.13     0.44     False
4             LG06        16975491            0        1     False
5             LG27        16580048            0        1     False
6             LG49         7206356         0.03        1     False
7             LG30         5571989            0        1     False
8             LG05        20094761            0        1     False
9             LG14        20088495            0        1     False
10            LG34        11566459        -0.04        1     False
11            LG21        17338149            0        1     False
12            LG05        14652417         0.13     0.55     False
13            LG25        23851527         0.75    0.001     True

```

## LICENSE
388

389
390
Copyright (C) 2018 Romain Feron and INRA LPGP

391
392
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, 
either version 3 of the License, or (at your option) any later version.
393

394
395
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. 
See the GNU General Public License for more details.
396
397

You should have received a copy of the GNU General Public License along with this program. If not, see https://www.gnu.org/licenses/