4DN Pipelines Development Guide

Before start developing a pipeline you should get familiar with the following software:

Github: https://help.github.com/en

Docker: https://docs.docker.com/

DockerHub: https://docs.docker.com/docker-hub/

CWL: https://www.commonwl.org/user_guide/

Travis: https://docs.travis-ci.com/user/tutorial/#to-get-started-with-travis-ci

Tibanna: https://tibanna.readthedocs.io/en/latest/

Main Steps:

• Overview
• Setting Up GitHub Repo
  • Dockerization
  • Testing the docker image locally
  • CWL File(s)
  • Travis Testing
  • The .travis.yml file
  • Tests files
  • Configuring Travis in Github
• Setting Up Data Portal
  • Create Workflow Object
  • Test with Tibanna
  • Prepare Tibanna Input
  • Run Tibanna Test
  • Benchmarking
• Common Use Cases
  • Output Files
  • Extra files
  • QC metrics that are attached to existing files
  • QC metrics that are patched by tibanna to a specific field