NF_MAAffymetrix: update pipeline version from GL-DPPD-7114 to GL-DPPD-7114-A

Author: cyouh95

Microarray/Affymetrix/Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114-A.md

@@ -1,14 +1,14 @@
 # GeneLab bioinformatics processing pipeline for Affymetrix microarray data <!-- omit in toc -->
 
-> **This page holds an overview and instructions for how GeneLab processes Affymetrix microarray datasets. Exact processing commands and GL-DPPD-7114 version used for specific GeneLab datasets (GLDS) are provided with their processed data in the [Open Science Data Repository (OSDR)](https://osdr.nasa.gov/bio/repo).**  
+> **This page holds an overview and instructions for how GeneLab processes Affymetrix microarray datasets. Exact processing commands and GL-DPPD-7114-A version used for specific GeneLab datasets (GLDS) are provided with their processed data in the [Open Science Data Repository (OSDR)](https://osdr.nasa.gov/bio/repo).**  
 > 
 > \* The pipeline detailed below currently supports gene annotations for Arabidopsis Thaliana via Ensembl FTP, all animals available in Biomart, and custom annotations (see [Step 8a](#8a-get-probeset-annotations)).
 
 ---
 
 **Date:** March 31, 2023  
 **Revision:** -  
-**Document Number:** GL-DPPD-7114   
+**Document Number:** GL-DPPD-7114-A   
 
 **Submitted by:**  
 Jonathan Oribello (GeneLab Data Processing Team)

Microarray/Affymetrix/README.md

@@ -1,7 +1,7 @@
 # GeneLab bioinformatics processing pipeline for Affymetrix microarray data
 
 
-> **The document [`GL-DPPD-7114.md`](Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114.md) holds an overview and example commands for how GeneLab processes Affymetrix microarray datasets. See the [Repository Links](#repository-links) descriptions below for more information. Processed data output files and processing code is provided for each GLDS dataset along with the processed data in the [Open Science Data Repository (OSDR)](https://osdr.nasa.gov/bio/repo/).**
+> **The document [`GL-DPPD-7114-A.md`](Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114-A.md) holds an overview and example commands for how GeneLab processes Affymetrix microarray datasets. See the [Repository Links](#repository-links) descriptions below for more information. Processed data output files and processing code is provided for each GLDS dataset along with the processed data in the [Open Science Data Repository (OSDR)](https://osdr.nasa.gov/bio/repo/).**
 
 --- 
 
@@ -20,6 +20,10 @@
 
   - Contains instructions for installing and running the GeneLab NF_MAAffymetrix workflow
 
+* [**Array_Annotations**](Array_Annotations)
+
+  - Contains the custom annotations table used in the GeneLab NF_MAAffymetrix
+
 ---
 **Developed by:**  
 Jonathan Oribello  

Microarray/Affymetrix/Workflow_Documentation/NF_MAAffymetrix/CHANGELOG.md

@@ -5,6 +5,12 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [TBD](#) - YYYY-MM-DD
+
+### Changed
+
+- Better support for custom annotations, see [specification](examples/annotations/README.md) ([#113](https://github.com/nasa/GeneLab_Data_Processing/issues/113))
+
 ## [1.0.5](https://github.com/nasa/GeneLab_Data_Processing/tree/NF_MAAffymetrix_1.0.5/Microarray/Affymetrix/Workflow_Documentation/NF_MAAffymetrix) - 2024-08-30
 
 ### Added

Microarray/Affymetrix/Workflow_Documentation/NF_MAAffymetrix/README.md

@@ -4,7 +4,7 @@
 
 ### Implementation Tools <!-- omit in toc -->
 
-The current GeneLab Affymetrix Microarray consensus processing pipeline (NF_MAAffymetrix), [GL-DPPD-7114](../../Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114.md), is implemented as a [Nextflow](https://nextflow.io/) DSL2 workflow and utilizes [Singularity](https://docs.sylabs.io/guides/3.10/user-guide/introduction.html) to run all tools in containers. This workflow (NF_MAAffymetrix) is run using the command line interface (CLI) of any unix-based system.  While knowledge of creating workflows in Nextflow is not required to run the workflow as is, [the Nextflow documentation](https://nextflow.io/docs/latest/index.html) is a useful resource for users who want to modify and/or extend this workflow.   
+The current GeneLab Affymetrix Microarray consensus processing pipeline (NF_MAAffymetrix), [GL-DPPD-7114-A](../../Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114-A.md), is implemented as a [Nextflow](https://nextflow.io/) DSL2 workflow and utilizes [Singularity](https://docs.sylabs.io/guides/3.10/user-guide/introduction.html) to run all tools in containers. This workflow (NF_MAAffymetrix) is run using the command line interface (CLI) of any unix-based system.  While knowledge of creating workflows in Nextflow is not required to run the workflow as is, [the Nextflow documentation](https://nextflow.io/docs/latest/index.html) is a useful resource for users who want to modify and/or extend this workflow.   
 
 ### Workflow & Subworkflows <!-- omit in toc -->
 
@@ -14,8 +14,8 @@ The current GeneLab Affymetrix Microarray consensus processing pipeline (NF_MAAf
 
 ---
 The NF_MAAffymetrix workflow is composed of three subworkflows as shown in the image above.
-Below is a description of each subworkflow and the additional output files generated that are not already indicated in the [GL-DPPD-7114 pipeline 
-document](../../Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114.md):
+Below is a description of each subworkflow and the additional output files generated that are not already indicated in the [GL-DPPD-7114-A pipeline 
+document](../../Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114-A.md):
 
 1. **Analysis Staging Subworkflow**
 
@@ -26,7 +26,7 @@ document](../../Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114.md):
 2. **Affymetrix Microarray Processing Subworkflow**
 
    - Description:
-     - This subworkflow uses the staged raw data and metadata parameters from the Analysis Staging Subworkflow to generate processed data using the [GL-DPPD-7114 pipeline](../../Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114.md).
+     - This subworkflow uses the staged raw data and metadata parameters from the Analysis Staging Subworkflow to generate processed data using the [GL-DPPD-7114-A pipeline](../../Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114-A.md).
 
 1. **V&V Pipeline Subworkflow**
 
@@ -200,7 +200,7 @@ All R code steps and output are rendered within a Quarto document yielding the f
   
 
 The outputs from the Analysis Staging and V&V Pipeline Subworkflows are described below:
-> Note: The outputs from the Affymetrix Microarray Processing Subworkflow are documented in the [GL-DPPD-7114.md](../../../Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114.md) processing protocol.
+> Note: The outputs from the Affymetrix Microarray Processing Subworkflow are documented in the [GL-DPPD-7114-A.md](../../../Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114-A.md) processing protocol.
 
 **Analysis Staging Subworkflow**
 

Microarray/Affymetrix/Workflow_Documentation/NF_MAAffymetrix/workflow_code/modules/POST_PROCESSING/GENERATE_PROTOCOL/resources/usr/bin/generate_protocol.sh

@@ -80,7 +80,7 @@ else
 fi
 
 # Read the template file
-template="Data were processed as described in GL-DPPD-7114 (https://github.com/nasa/GeneLab_Data_Processing/blob/master/Microarray/Affymetrix/Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114.md) using NF_MAAffymetrix version $1 (https://github.com/nasa/GeneLab_Data_Processing/tree/NF_MAAffymetrix_$1/Microarray/Affymetrix/Workflow_Documentation/NF_MAAffymetrix). In short, a RunSheet containing raw data file location and processing metadata from the study's *ISA.zip file was generated using dp_tools (version ${dp_tools_VERSION}). The raw array data files were loaded into R (version ${R_VERSION}) using oligo (version ${oligo_VERSION}). Raw data quality assurance density plot, pseudo images, MA plots, and boxplots were generated using oligo (version ${oligo_VERSION}). The raw probe level intensity data was background corrected and normalized across arrays via the oligo (version ${oligo_VERSION}) quantile method. Normalized probe level data quality assurance density plot, pseudo images, MA plots, and boxplots were generated using oligo (version ${oligo_VERSION}). Normalized probe level data was summarized to the probeset level using the oligo (version ${oligo_VERSION}) RMA method. ${GENE_MAPPING_STEP} ${DE_STEP} ${ANNOT_STEP}"
+template="Data were processed as described in GL-DPPD-7114-A (https://github.com/nasa/GeneLab_Data_Processing/blob/master/Microarray/Affymetrix/Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114-A.md) using NF_MAAffymetrix version $1 (https://github.com/nasa/GeneLab_Data_Processing/tree/NF_MAAffymetrix_$1/Microarray/Affymetrix/Workflow_Documentation/NF_MAAffymetrix). In short, a RunSheet containing raw data file location and processing metadata from the study's *ISA.zip file was generated using dp_tools (version ${dp_tools_VERSION}). The raw array data files were loaded into R (version ${R_VERSION}) using oligo (version ${oligo_VERSION}). Raw data quality assurance density plot, pseudo images, MA plots, and boxplots were generated using oligo (version ${oligo_VERSION}). The raw probe level intensity data was background corrected and normalized across arrays via the oligo (version ${oligo_VERSION}) quantile method. Normalized probe level data quality assurance density plot, pseudo images, MA plots, and boxplots were generated using oligo (version ${oligo_VERSION}). Normalized probe level data was summarized to the probeset level using the oligo (version ${oligo_VERSION}) RMA method. ${GENE_MAPPING_STEP} ${DE_STEP} ${ANNOT_STEP}"
 
 # Output the filled template
 echo "$template" > PROTOCOL_GLmicroarray.txt

Microarray/Affymetrix/Workflow_Documentation/NF_MAAffymetrix/workflow_code/nextflow.config

@@ -41,7 +41,7 @@ profiles {
 
 manifest {
     homePage = 'https://github.com/nasa/GeneLab_Data_Processing/tree/master/Microarray/Affymetrix'
-    description = 'Affymetrix Microarray Workflow for Document GL-DPPD-7114'
+    description = 'Affymetrix Microarray Workflow for Document GL-DPPD-7114-A'
     mainScript = 'main.nf'
     defaultBranch = 'main'
     nextflowVersion = '>=23.10.1'

Microarray/Affymetrix/Workflow_Documentation/README.md

@@ -1,14 +1,14 @@
 # GeneLab RNAseq Workflow Information
 
 > ** For the processing pipeline for Affymetrix microarray data,
-[`GL-DPPD-7114.md`](../Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114.md),
+[`GL-DPPD-7114-A.md`](../Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114-A.md),
 GeneLab has wrapped each step of the pipeline into a workflow with validation and verification of output files built in after each step. The table below lists (and links to) each NF_MAAffymetrix version and the corresponding workflow subdirectory, the current NF_MAAffymetrix/workflow implementation is indicated. Each workflow subdirectory contains information about the workflow along with instructions for installation and usage.**  
 
 ## NF_MAAffymetrix Version and Corresponding Workflow
 
 |Pipeline Version|Current Workflow Version (for respective pipeline version)|Nextflow Version|
 |:---------------|:---------------------------------------------------------|:---------------|
-|*[GL-DPPD-7114.md](../Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114.md)|[1.0.4](NF_MAAffymetrix)|23.10.1|
+|*[GL-DPPD-7114-A.md](../Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114-A.md)|[1.0.5](NF_MAAffymetrix)|23.10.1|
 
 *Current GeneLab Pipeline/Workflow Implementation