Following GCC2016 lightning talk, and in the context of the french National Museum of Natural History MNHN project "65 Millions d'observateurs" dedicated to enhance and expand participation to citizen sciences projects studying biodiversity, a proof of concept of analysis web platform in macroecology will be made. We propose for this to use the Galaxy web platform. Here is the origin of a Galaxy-E, for Ecology ?
This repository will gather ideas and development of Galaxy-E tools
Galaxy i18n is almost here, translations still welcomed!
- Using R for occurences data through the spocc package: https://github.com/ropensci/spocc
- Using mapr to display species occurences https://github.com/ropensci/mapr
- Clean up occurences with scrubr https://github.com/ropensci/scrubr
- Global Biodiversity Information Facility (GBIF) : https://www.gbif.org/developer/summary
- IDigBio
- PlutoF
- BARCODE OF LIFE DATA SYSTEM (BOLD), Advancing biodiversity science through DNA-based species identification: http://boldsystems.org/index.php/resources/api?type=webservices. Example of API request can be:
http://boldsystems.org/index.php/API_Public/specimen?taxon=Aves&geo=Costa%20Rica&format=tsvhttp://boldsystems.org/index.php/API_Public/specimen?taxon=Dicentrarchus%20labrax&geo=France&format=tsvhttp://boldsystems.org/index.php/API_Public/specimen?taxon=taxon=Aves|Reptilia&geo=France&format=tsv
- IPSL https://cse.ipsl.fr/donnees/114-prodiguer
- WorldClim - Global Climate Data http://www.worldclim.org/
- Copernicus Climate Change Service Providing climate data http://climate.copernicus.eu/
- Copernicus Marine Environment Monitoring Service Providing products & services for all marine applications http://marine.copernicus.eu/
- Copernicus Atmosphere air quality & atmospheric composition https://atmosphere.copernicus.eu/catalogue#/
- Copernicus Global Land Service Providing bio-geophysical products of global land surface http://land.copernicus.eu/global/products/NDVI (included Corine Land Cover products)
- CESBIO Carte d'occupation des sols. http://www.cesbio.ups-tlse.fr/multitemp/?p=10104
- GEONETCast global network of satellite-based data dissemination systems providing environmental data http://www.eumetsat.int/website/home/Data/DataDelivery/EUMETCast/GEONETCast/index.html
- Donéens agricoles via Agreste: http://agreste.agriculture.gouv.fr/
- Bio-ORACLE Marine data layers for ecological modelling: http://www.bio-oracle.org/ (R invocation through
sdmpredictors::list_layers("Bio-ORACLE", version=2)) - MARSPEC monthly layers for temperature and salinity (R invocation through
paleo: sdmpredictors::list_layers("MARSPEC")) and paleo layers for these (R invocation throughpaleo: sdmpredictors::list_layers_paleo("MARSPEC")): http://onlinelibrary.wiley.com/doi/10.1890/12-1358.1/abstract - Use of sdmpredictor R package:
# exploring the marine (you can also choose terrestrial) datasets
datasets <- list_datasets(terrestrial = FALSE, marine = TRUE)
- downloading and working with the Marine SPEcies and Environmental Data (MarineSPEED.org) benchmark dataset: https://github.com/lifewatch/marinespeed
- Hub’eau Water related data (fishes,…) http://www.hubeau.fr/
- CRBPO data https://crbpodata.mnhn.fr/
- Movebank data https://www.movebank.org/
- API-Agro import agrifood related data http://www.api-agro.fr/
- Free GIS geographic datasets: http://freegisdata.rtwilson.com/
- Global Environmental Layers: http://worldgrids.org/doku.php
- World Conservation Monitoring Centre: https://www.unep-wcmc.org/
- World Database on Protected Areas (WDPA) is the most comprehensive global database on terrestrial and marine protected areas.: https://protectedplanet.net/
- Chelsea Climate (Climatologies at high resolution for the earth’s land surface areas ): http://chelsa-climate.org/
- E-OBS gridded dataset : http://www.ecad.eu/download/ensembles/download.php
- Global Lithological Map Database v1.0 https://doi.pangaea.de/10.1594/PANGAEA.788537
-
Work with messy data from db
-
OpenRefine Implemented as GIE [docker repo](DockerHub repo : https://hub.docker.com/r/valentinchdock/openrefine-galaxy-ie/)
-
FAIRifier based on OpenRefine and associated to FAIR-metadata-editor. To test through GIE and linked to EML!
-
Other solutions:
- DataCleaner
- Karma A Data Integration Tool
-
Tadarida tools suite: A Toolbox for Animal Detection on Acoustic Recordings Integrating Discriminant Analysis
-
Ubat slicer: Tadarida pre-processing
-
STOC (Temporal monitoring of common birds)
-
Simple punctual sampling
- Regional scale: scriptSTOCeps.R
-
Capture
-
STERF (Temporal Follow-up of France Rhopalocera)
- TRIM is now also available as R package (rtrim, via install.packages("rtrim") from CRAN). This will make it much more easy for many of you to calculate trends. But remember to have the input file in good order (so with missing values and zeroes). Of course you can also use the result of Reto's regional_gam (https://github.com/RetoSchmucki/regionalGAM/blob/master/README.md) as input.
- Manuals and helpfiles are available via https://github.com/markvanderloo/rtrim
- CBS also made their Multi Species Indicator tool available: https://www.cbs.nl/nl-nl/maatschappij/natuur-en-milieu/indexen-en-trends--trim--/msi-tool. With this tool you can build your own indicators from the results of rtrim.
- regionalGAM
- rtrim
- MSI-tool
-
GIS data handling
- Geospatial Abstracation Data Librairy
- Impute missing value: https://github.com/RetoSchmucki/CESCO_R-scripts/blob/master/replace%20missing%20values%20in%20raster.r
- Sites extraction
- Conversion
- Buffering
- Calculate mean by buffer
-
Visualize GIS data
- Through "classical" GIS specialists oriented solutions:
- Geoserver
- QGIS server ou QGIS desktop. A particular interesting QGIS based tool : LizMap et LizMap Docker
- GeoCMS GeoCMS is a complete open source solution for consuming and visualizing geospatial data
OpenEV a software library and application for viewing and analysing raster and vector geospatial data (last release 2007!)
- To manage data:
- PostGIS with a Docker version usable through the Galaxy pg datatype and relatde tools implementation /Leaflet through Interactive Environment ?
- H2GIS light and standalone GIS database
- Through GIS non-specialists oriented solutions:
- Magrit for thematic GIS (in french and english)
- Through "classical" GIS specialists oriented solutions:
-
GIS data analysis
-
Taxa automated recognition through TensorFlow
-
Dashboards for a community intensively oriented toward R
- R-Shiny Interactive Environment
- GIS shiny GIE through leaflet based shiny apps to display data by french regions and related plots
- Statistics shiny GIE through radiant
- Dashboard / restitution shiny GIE through flexdashboard+shiny or shiny dashboard
- Macroecology through Wallace
- Marxan, a shiny app for systematic conservation planning
- Shiny and reproducibility through
interactive documentconcept- rmarkdown, a new way to build Shiny apps through interactive documents
Interactive documents will not replace standard Shiny apps since they cannot provide the design options that come with a ui.R or index.html file. However, interactive documents do create some easy wins:
The R Markdown workflow makes it easy to build light-weight apps. You do not need to worry about laying out your app or building an HTML user interface for the app.
You can use R Markdown to create interactive slideshows, something that is difficult to do with Shiny alone. To create a slideshow, change output: html_document to output: ioslides_presentation in the YAML front matter of your .Rmd file. R Markdown will divide your document into slides when you click “Run Document.” A new slide will begin whenever a header or horizontal rule (***) appears.
Interactive documents enhance the existing R Markdown workflow. R Markdown makes it easy to write literate programs and reproducible reports. You can make these reports even more effective by adding Shiny to the mix.
- R-Shiny Interactive Environment
To learn more about R Markdown and interactive documents, please visit rmarkdown.rstudio.com. ```
- Alan Amossé (MNHN CESCO & Concarneau marine biology station)
- Björn Grüning (Freiburg University)
- Boyan Angelov
- Clara Urfer (MNHN CESCO, Concarneau marine biology station) & Brest University
- Elisa Michon (MNHN CESCO, Concarneau marine biology station) & Montpellier University
- Eloïse Trigodet (MNHN CESCO, Concarneau marine biology station) & Brest IUEM University
- Mathias Rouan (LETG)
- Nicolas Dubos (MNHN CESCO)
- Thimothée Virgoulay (MNHN CESCO, Concarneau marine biology station & Montpellier University)
- Valentin Chambon (MNHN CESCO & Concarneau marine biology station)
- Yvan Le Bras (MNHN CESCO & Concarneau marine biology station)
- Yves Bas (UMR CEFE, MNHN CESCO)