Code

Sfd5311/cells index#294

Code

Open

SFD5311 wants to merge 10 commits intodevelhubmapconsortium/ingest-pipeline:develfrom sfd5311/cells-indexhubmapconsortium/ingest-pipeline:sfd5311/cells-indexCopy head branch name to clipboard

+243

Commits

Commits on Mar 14, 2021

• Adding submodules for cross-dataset cells indexing pipelines

  SFD5311

  committed
  f301fe3

Commits on Mar 15, 2021

• Adding cells_index dag

  SFD5311

  committed
  fd0a30e

• Switching to hitting production instance of cells

  SFD5311

  committed
  d9369d0

Commits on Mar 16, 2021

• cells_index DAG: remove indirection with DAG generator function

  Show description for ab78a17
  mruffalo

  committed
  ab78a17

• WIP DAG-level parallelization of modality-specific pipelines

  Show description for 366ca35
  mruffalo

  committed
  366ca35

Commits on Mar 17, 2021

• Adding join operators (maybe_run_pipeline_N) to reflect the fact that each pipeline may or may not run, depending on whether or not there's new datasets in that modality

  SFD5311

  committed
  894f460

Commits on Apr 12, 2021

• Parameterizing differences between dags, making task ids globally unique, programatically generating dags

  SFD5311

  committed
  86dfeee

• Switching from requests to airflow http hooks for api calls

  SFD5311

  committed
  108c5b2

Commits on Apr 14, 2021

• Fix problem with cross-dataset-atac referencing reverted commit; set to HEAD

  

  jswelling

  authored and

  Hubmap Hive

  committed
  87b9780

Commits on Apr 20, 2021

• WIP. Updating cells_index.py for compatibility with our Airflow environment.

  

  Hubmap Hive

  committed
  869405a