README.md 5.97 KB
Newer Older
peguerin's avatar
peguerin committed
1
2
Scripts to assign environmental descriptors to samples 
======================================================
peguerin's avatar
peguerin committed
3
4
5
6
7
8
2019

# Table of contents

1. [Introduction](#1-introduction)
2. [Installation](#2-installation)
peguerin's avatar
peguerin committed
9
	1. [Prerequisites](#21-prerequisite)
peguerin's avatar
peguerin committed
10
11
12
13
14
15
16
	2. [Data files](#22-data_files)
3. [Reporting bugs](#3-reporting-bugs)
4. [Running the pipeline](#4-running-the-pipeline)
5. [Results](#5-results)

# 1. Introduction

peguerin's avatar
peguerin committed
17
R scripts to assign environmental data to samples according to their coordinates.
peguerin's avatar
peguerin committed
18
19
20

# 2. Installation

peguerin's avatar
peguerin committed
21
22
23
24
25
26
27
28
29
30
31
open a shell
make a folder, name it yourself, I named it workdir
```
mkdir workdir
cd workdir
```
clone the project and switch to the main folder, it's your working directory
```
git clone https://gitlab.mbb.univ-montp2.fr/reservebenefit/environmental_assign_data.git
cd environmental_assign_data
```
peguerin's avatar
peguerin committed
32

peguerin's avatar
peguerin committed
33
## 2.1 Prerequisites
peguerin's avatar
peguerin committed
34
35
36
You must install the following softwares and packages :

- [R Version 3.2.3](https://cran.r-project.org/) to run the scripts
peguerin's avatar
peguerin committed
37
38
    - `R-package` raster
    - `R-package` rgdal
peguerin's avatar
peguerin committed
39
    - `R-package` ncdf4
peguerin's avatar
peguerin committed
40
41
42
43

- [FILEZILLA](https://filezilla-project.org/download.php) to download environmental data from our private SFTP server

## 2.2 Data files
peguerin's avatar
peguerin committed
44
- The included data files are :
peguerin's avatar
peguerin committed
45

peguerin's avatar
peguerin committed
46
	* [01-sample_coords/all_coords.csv](01-sample_coords/all_coords.csv) : label and WGS84 coordinates of all the samples from RESERVEBENEFIT
peguerin's avatar
peguerin committed
47

peguerin's avatar
peguerin committed
48
- You must download the following environmental descriptor raw data into the folder [01-infos](01-infos) :
peguerin's avatar
peguerin committed
49
50
51
52
53
54
55

You will need [FILEZILLA](https://filezilla-project.org/download.php) !
To access to our private server, please follow these instructions

1) Launch [FILEZILLA](https://filezilla-project.org/download.php)
2) click on file > import...
3) select [22-server-infos/SFTPCEFE_PEG.xml](22-server-infos/SFTPCEFE_PEG.xml)
peguerin's avatar
peguerin committed
56
4) click on the site manager button (the button under "File")
peguerin's avatar
peguerin committed
57

peguerin's avatar
peguerin committed
58
![site manager button](22-server-infos/image1.png)
peguerin's avatar
peguerin committed
59

peguerin's avatar
peguerin committed
60
5) select site "CEFE-STFP" and then click on "Connect"
peguerin's avatar
peguerin committed
61

peguerin's avatar
peguerin committed
62
63
64
65
![site manager button](22-server-infos/image2.png)

Now you are connected to our server and you can download the required files. They are stored into : /DataEnvironment/resources

peguerin's avatar
peguerin committed
66
Download the content of /DataEnvironment/resources and copy it into the folder [01-infos](01-infos). Your data should be stored this way :
peguerin's avatar
peguerin committed
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85

```
01-infos/
├── chlorophyll
│   ├── benthic
│   │   ├── Present.Benthic.Mean.Depth.Chlorophyll.Max.tif.zip
│   │   ├── Present.Benthic.Mean.Depth.Chlorophyll.Mean.tif.zip
│   │   └── Present.Benthic.Mean.Depth.Chlorophyll.Min.tif.zip
│   └── surface
│       ├── Present.Surface.Chlorophyll.Max.tif.zip
│       ├── Present.Surface.Chlorophyll.Mean.tif.zip
│       └── Present.Surface.Chlorophyll.Min.tif.zip
├── habitat
│   ├── EUSeaMap2016_substrate_levels_med_res200m.tsv
│   └── EUSeaMap2016_substrate_raster_med_res200m.tif
├── salinity_et_temperature
    └── CMEMS-MedSea_Reanalysis_Phys_006_004_0to210m_1987-2015_1m_votemper_vosaline_SampleLoc.nc
```

eboulanger's avatar
eboulanger committed
86
87
88
89
90
91
### 2.2.a data file sources

* Chlorophyll : https://www.bio-oracle.org
* Habitat     : https://www.emodnet-seabedhabitats.eu
* Salinity    : GEOMAR
* Temperature : GEOMAR
peguerin's avatar
peguerin committed
92

peguerin's avatar
peguerin committed
93
94
95
96
97
98
# 3. Reporting bugs

If you're sure you've found a bug — e.g. if one of my programs crashes
with an obscur error message, or if the resulting file is missing part
of the original data, then by all means submit a bug report.

peguerin's avatar
peguerin committed
99
I use [GitLab's issue system](https://gitlab.mbb.univ-montp2.fr/reservebenefit/environmental_assign_data/issues)
peguerin's avatar
peguerin committed
100
101
102
103
104
as my bug database. You can submit your bug reports there. Please be as
verbose as possible — e.g. include the command line, etc

# 4. Running the pipeline

peguerin's avatar
peguerin committed
105
:point_right: You must have download required data files into [01-infos](01-infos) (see [Data files](#22-data_files) section)
peguerin's avatar
peguerin committed
106

peguerin's avatar
peguerin committed
107
108
109
### 4.1. Assign `habitat` (substrat) values to samples

To attribute `habitat` information (this will generate [03-results/all_habitat.tsv](03-results/all_habitat.tsv))
peguerin's avatar
peguerin committed
110
111
112
113
```
Rscript 00-scripts/habitat.R
```

peguerin's avatar
peguerin committed
114
### 4.2. Assign `chlorophyll A abundance` values to samples
peguerin's avatar
peguerin committed
115
116
117
118
119
120
To attribute "chlorophyll A" information (this will generate [03-results/all_habitat.tsv](03-results/all_chlorophyll.tsv))
```
Rscript 00-scripts/chlorophyll.R

```

peguerin's avatar
peguerin committed
121
### 4.3. Assign `mounthly temperature & salinity` values to samples for each sea depth
peguerin's avatar
peguerin committed
122

peguerin's avatar
peguerin committed
123
To attribute `mounthly temperature & salinity` by sea depth information (this will generate [03-results/all_depth_repro_temperature_salinity.tsv](03-results/all_depth_repro_temperature_salinity.tsv))
peguerin's avatar
peguerin committed
124
```
peguerin's avatar
peguerin committed
125
Rscript 00-scripts/salt_temp.R
peguerin's avatar
peguerin committed
126
127
```

peguerin's avatar
peguerin committed
128
129
130
131
132
133
134

### 4.4. Correct environmental data values

* Correct `mounthly temperature & salinity` values

:warning: `mounthly temperature & salinity` values for these following samples were missing data and then have been **_manually_** fullfiled:  
(ce serait bien de savoir _COMMENT ?_)
peguerin's avatar
peguerin committed
135
```
peguerin's avatar
peguerin committed
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
mul_bal_115
mul_bal_116
mul_bal_117
mul_bal_118
mul_bal_119
mul_bal_120
mul_bal_121
mul_bal_122
mul_bal_123
mul_bal_124
mul_bal_125
mul_bal_126
mul_bal_127
mul_bal_128
mul_bal_129
mul_bal_130
mul_bal_131
```
(see file [03-results/all_depth_repro_CORRECTED_temperature_salinity.csv](03-results/03-results/all_depth_repro_CORRECTED_temperature_salinity.csv))





### 4.5. All environmental descriptors into one single table
peguerin's avatar
peguerin committed
161

peguerin's avatar
peguerin committed
162
To merge all data into one single table (this will generate [03-results/final_chloro_temperature_salinity.tsv](03-results/final_chloro_temperature_salinity.tsv)
peguerin's avatar
peguerin committed
163
164
165
166
167
```
Rscript 00-scripts/merge_table.R
```


peguerin's avatar
peguerin committed
168
# 5. Results
peguerin's avatar
peguerin committed
169
170

* [03-results/all_habitat.tsv](03-results/all_habitat.tsv) : table of samples with WGS84 coordinates and predicted habitat
peguerin's avatar
peguerin committed
171
* [03-results/all_CHECKED_chlorophyll.tsv](03-results/all_CHECKED_chlorophyll.tsv) : table of samples with WGS84 coordinates and chlorophyll A - benthic and surface - mean,max,min values
eboulanger's avatar
eboulanger committed
172
* [03-results/final_chloro_temperature_salinity.tsv](03-results/final_chloro_temperature_salinity.tsv) :  table of samples with WGS84 coordinates and all the environmental data (predicted habitat, chlorophyll A, temperature, salinity by mounth and by depth values)