Skip to content

Latest commit

 

History

History
150 lines (125 loc) · 6.13 KB

outputs.md

File metadata and controls

150 lines (125 loc) · 6.13 KB
Raw

Pipeline output

Introduction

This document describes the output produced by the pipeline.

Pipeline summary

The pipeline is built using Nextflow and processes data using the following steps:

1. Input preparation - Checks, validates and prepares input files

  • gunzip - If required, uncompresses input genome FASTA file
  • gawk - Cleans genome FASTA headers
  • cat - Merges FASTQ files of re-sequenced samples based on samplesheet ID values

2. Read QC and trimming - Quality control and trimming of raw reads

  • FastQC (raw) - Raw read QC
  • TrimGalore! - Adapter and quality trimming
  • FastQC (trimmed) - Trimmed read QC
  • TrimGalore! (hard-trimming) - Hard trim reads to a fixed length
  • FastQC (hardtrimmed) - Hard trimmed read QC

3. Read alignment - Alignment of reads to the genome

  • STAR/genomeGenerate - Indexes genome for alignment
  • STAR/align - Aligns reads to the genome
  • picard/CollectAlignmentSummaryMetrics - Collects alignment summary metrics

4. Genome QC

  • BUSCO (genome) - BUSCO completeness assessment of the genome assembly

5. Repeat masking - Repeat masking of the genome assembly

  • RepeatModeler - De novo repeat identification
  • RepeatMasker - Repeat masking of the genome assembly

6. Genome annotation - Genome annotation and QC

  • BRAKER - Genome annotation using BRAKER, using the aligned reads as input
  • BUSCO (annotation) - BUSCO completeness assessment of the genome annotation
  • AGAT (spstatistics) - Calculates statistics for the genome annotation

7. Output reporting - Output processing and reporting

  • MultiQC - Aggregate report describing results and QC from the whole pipeline
  • Pipeline information - Report metrics generated during the workflow execution, including software versions

Output files

The directories listed below will be created in the directory specified with --outdir <OUTDIR> after the pipeline has finished. All paths are relative to the top-level results directory, with <ASSEMBLY_NAME> being the name of given with --assembly_name <ASSEMBLY_NAME>.

<OUTPUTDIR>
├── agat-spstatistics/
│   └── <ASSEMBLY_NAME>.stats.txt
├── braker/
│   └── <ASSEMBLY_NAME>/
│       ├── braker.aa
│       ├── braker.codingseq
│       ├── braker.gff3
│       ├── braker.gtf
│       ├── braker.log
│       ├── hintsfile.gff
│       └── what-to-cite.txt
├── busco/
│   ├── annotation/
│   │   └── <ASSEMBLY_NAME>/
│   └── genome/
│       └── <ASSEMBLY_NAME>>/
├── fastqc/
│   ├── hardtrim/
│   │   ├── *_fastqc.html
│   │   └── *_fastqc.zip
│   ├── raw/
│   │   ├── *_fastqc.html
│   │   └── *_fastqc.zip
│   └── trim/
│   │   ├── *_fastqc.html
│   │   └── *_fastqc.zip
├── gawk-fastaheader/
│    └── <ASSEMBLY_NAME>.clean.fna
├── gunzip/
│    └── <ASSEMBLY_NAME>.fna
├── multiqc/
│    ├── multiqc_data/
│    ├── multiqc_plots/
│    └── multiqc_report.html
├── picard/
│    └── alignment-metrics/
│        └── <ASSEMBLY_NAME>.txt
├── pipeline_info/
│    ├── execution_report-*.html
│    ├── execution_timeline-*.html
│    ├── execution_trace-*.txt
│    ├── pipeline_dag-*.html
│    └── software_versions.yml
├── repeatmasker/
│   └── <ASSEMBLY_NAME>>
│       ├── <ASSEMBLY_NAME>>.clean.fna.cat.gz
│       ├── <ASSEMBLY_NAME>>.clean.fna.masked
│       ├── <ASSEMBLY_NAME>>.clean.fna.out
│       ├── <ASSEMBLY_NAME>>.clean.fna.out.gff
│       └── <ASSEMBLY_NAME>>.clean.fna.tbl
├── repeatmodeler/
│    ├── <ASSEMBLY_NAME>-families.fa
│    ├── <ASSEMBLY_NAME>-families.stk
│    ├── <ASSEMBLY_NAME>-rmod.log
│    └── RM_XXX.DATETIME/
├── star/
│   ├── alignment/
│   │   ├── <ASSEMBLY_NAME>.Aligned.sortedByCoord.out.bam
│   │   ├── <ASSEMBLY_NAME>.Log.final.out
│   │   ├── <ASSEMBLY_NAME>.Log.out
│   │   ├── <ASSEMBLY_NAME>.Log.progress.out
│   │   └── <ASSEMBLY_NAME>.SJ.out.tab
│   └── index/
│       └── star/
│           ├── chrLength.txt
│           ├── chrNameLength.txt
│           ├── chrName.txt
│           ├── chrStart.txt
│           ├── Genome
│           ├── genomeParameters.txt
│           ├── SA
│           └── SAindex
└── trimgalore
    ├── hardtrim
    │   └── *.fq.gz
    └── trim
        ├── *.fastq.gz_trimming_report.txt
        └── *.fq.gz

MultiQC

MultiQC is a visualization tool that generates a single HTML report summarising all samples in your project. Most of the pipeline QC results are visualised in the report and further statistics are available in the report data directory.

Results generated by MultiQC collate pipeline QC from supported tools e.g. FastQC. The pipeline has special steps which also allow the software versions to be reported in the MultiQC output for future traceability. For more information about how to use MultiQC reports, see http://multiqc.info.

Pipeline information

Output files
  • pipeline_info/
    • Reports generated by Nextflow: execution_report.html, execution_timeline.html, execution_trace.txt and pipeline_dag.dot/pipeline_dag.svg.
    • Reports generated by the pipeline: pipeline_report.html, pipeline_report.txt and software_versions.yml. The pipeline_report* files will only be present if the --email / --email_on_fail parameters are used when running the pipeline.

Nextflow provides excellent functionality for generating various reports relevant to the running and execution of the pipeline. This will allow you to troubleshoot errors with the running of the pipeline, and also provide you with other information such as launch commands, run times and resource usage.