Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: trigger release process #792

Closed
wants to merge 92 commits into from
Closed

chore: trigger release process #792

wants to merge 92 commits into from

Conversation

github-actions[bot]
Copy link

⚠️ This PR requires a MERGE COMMIT (Don't squash!)

project-defiant and others added 30 commits July 25, 2024 11:26
@project-defiant
feat: expose summary statistics qc and locus breaker steps to hydra c…
ea80fbf 
…li (#716)

* feat: expose summary statistics qc to hydra cli

* feat: expose locus breaker clumping step

---------

Co-authored-by: Szymon Szyszkowski <[email protected]>
@project-defiant
fix: change config params to match new name (#721)
a5c96b9 
Co-authored-by: Szymon Szyszkowski <[email protected]>
@pre-commit-ci
chore: pre-commit autoupdate (#715)
897ae12 
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.5.2 → v0.5.5](astral-sh/ruff-pre-commit@v0.5.2...v0.5.5)
- [github.com/pre-commit/mirrors-mypy: v1.10.1 → v1.11.0](pre-commit/mirrors-mypy@v1.10.1...v1.11.0)
- [github.com/jsh9/pydoclint: 0.5.4 → 0.5.6](jsh9/pydoclint@0.5.4...0.5.6)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@dependabot @project-defiant
build(deps-dev): bump pymdown-extensions from 10.8.1 to 10.9 (#720)
397f1e9 
Bumps [pymdown-extensions](https://github.com/facelessuser/pymdown-extensions) from 10.8.1 to 10.9.
- [Release notes](https://github.com/facelessuser/pymdown-extensions/releases)
- [Commits](facelessuser/pymdown-extensions@10.8.1...10.9)

---
updated-dependencies:
- dependency-name: pymdown-extensions
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Szymon Szyszkowski <[email protected]>
@dependabot
build(deps-dev): bump deptry from 0.17.0 to 0.18.0 (#723)
0b9af70 
Bumps [deptry](https://github.com/fpgmaas/deptry) from 0.17.0 to 0.18.0.
- [Release notes](https://github.com/fpgmaas/deptry/releases)
- [Changelog](https://github.com/fpgmaas/deptry/blob/main/CHANGELOG.md)
- [Commits](fpgmaas/deptry@0.17.0...0.18.0)

---
updated-dependencies:
- dependency-name: deptry
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@Daniel-Considine
feat: notebook for locus breaker and susie finemapping benchmark (#717)
ae734a8 
* feat: notebook for locus breaker and susie finemapping benchmark

* fix: won't pass tests with matplotlib imported

* feat: ukb_ppp_fm benchmark notebook

* fix: check for nulls as well as nans

* fix: removing matplotlib

* fix: update notebooks

* chore: re-run ukb ppp notebook

* chore: rerun notebook
@Daniel-Considine
fix: updating config paths and fine-mapping methods (#725)
5b5b408 
* fix: updating config paths and fine-mapping methods

* Update ot_locus_to_gene_train.yaml
@Daniel-Considine
docs: macos fix for some functions (#729)
e45f295 
* docs: macos fix for some functions

* docs: formatting
@dependabot @d0choa @Daniel-Considine
build(deps-dev): bump ruff from 0.5.1 to 0.6.1 (#732)
f49a5c5 
* build(deps-dev): bump ruff from 0.5.1 to 0.6.1

Bumps [ruff](https://github.com/astral-sh/ruff) from 0.5.1 to 0.6.1.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](astral-sh/ruff@0.5.1...0.6.1)

---
updated-dependencies:
- dependency-name: ruff
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* fix: linting issues associated with ruff 0.6

* chore: fixing imports for notebooks

* chore: removing old notebook

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: David Ochoa <[email protected]>
Co-authored-by: Daniel Considine <[email protected]>
@pre-commit-ci @d0choa
chore: pre-commit autoupdate (#724)
4169b81 
* chore: pre-commit autoupdate

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.5.5 → v0.6.1](astral-sh/ruff-pre-commit@v0.5.5...v0.6.1)
- [github.com/pre-commit/mirrors-mypy: v1.11.0 → v1.11.1](pre-commit/mirrors-mypy@v1.11.0...v1.11.1)

* chore: pre-commit auto fixes [...]

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: David Ochoa <[email protected]>
@dependabot @d0choa
build(deps-dev): bump deptry from 0.18.0 to 0.19.1 (#728)
02b006d 
Bumps [deptry](https://github.com/fpgmaas/deptry) from 0.18.0 to 0.19.1.
- [Release notes](https://github.com/fpgmaas/deptry/releases)
- [Changelog](https://github.com/fpgmaas/deptry/blob/main/CHANGELOG.md)
- [Commits](fpgmaas/deptry@0.18.0...0.19.1)

---
updated-dependencies:
- dependency-name: deptry
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: David Ochoa <[email protected]>
@dependabot
build(deps-dev): bump lxml from 5.2.2 to 5.3.0 (#727)
5984de9 
Bumps [lxml](https://github.com/lxml/lxml) from 5.2.2 to 5.3.0.
- [Release notes](https://github.com/lxml/lxml/releases)
- [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt)
- [Commits](lxml/lxml@lxml-5.2.2...lxml-5.3.0)

---
updated-dependencies:
- dependency-name: lxml
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot @project-defiant @d0choa
build(deps-dev): bump pre-commit from 3.7.1 to 3.8.0 (#719)
4db64a3 
* build(deps-dev): bump pre-commit from 3.7.1 to 3.8.0

Bumps [pre-commit](https://github.com/pre-commit/pre-commit) from 3.7.1 to 3.8.0.
- [Release notes](https://github.com/pre-commit/pre-commit/releases)
- [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md)
- [Commits](pre-commit/pre-commit@v3.7.1...v3.8.0)

---
updated-dependencies:
- dependency-name: pre-commit
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* fix: update lock

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Szymon Szyszkowski <[email protected]>
Co-authored-by: David Ochoa <[email protected]>
Co-authored-by: David Ochoa <[email protected]>
@ireneisdoomed @pre-commit-ci @project-defiant
feat(variant_index): changes for a successful run (#735)
dc8c9dc 
* feat: update variant sources for variant idx and drop rsid mapping

* chore: remove variant_index step from etl dag

* chore: move gnomad annotation from dev to static assets

* chore: change gentropy docker image

* feat(convert_to_vcf): write data partitioned

* feat: add vcf merging step

* chore: assert merge is succesful

* fix: remove non canonical variants from vcfs

* chore: pass vcf types to merging rule

* fix: match vep parser order of columns to variant index schema

* fix: ditch `assign_variant_id` in favour of `hash_long_variant_ids`

* chore: update `test_as_vcf_df_without_variant_id`

* fix: correct test mock data

* chore: pre-commit auto fixes [...]

* chore: merge from dev

* chore: pre-commit auto fixes [...]

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Szymon Szyszkowski <[email protected]>
@project-defiant
fix: update cluster creation command (#739)
348b6f0 
Co-authored-by: Szymon Szyszkowski <[email protected]>
@project-defiant
fix: revert recursiveFileLookup to False (#738)
b6746fb 
Co-authored-by: Szymon Szyszkowski <[email protected]>
@project-defiant @pre-commit-ci @DSuveges
feat: Finngen r11 ingestion (#733)
cf1ec24 
* feat: finngen r11 study index paths

* feat: simplification of finngen susie finemapping ingestion

* feat: efos in finngne study index

* docs: updated docstring

* fix: pass spark_session not gentropy session to step

* fix: read uploaded file  requires file:///

* fix: parallelize csv from url

* fix: typo in docstring

* chore: pre-commit auto fixes [...]

---------

Co-authored-by: Szymon Szyszkowski <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Suveges <[email protected]>
@Daniel-Considine
fix: using h4 instead of log2(h4/h3) (#740)
2612aa7
@addramir
fix: adding carma_tau parameter to susie_finemapper (#743)
93a6e60 
* fix: adding carma_tau parameter to susie_finemapper

* fix: changing default

* fix: defaults
@DSuveges @d0choa
@pre-commit-ci @project-defiant
feat: logic and airflow pipeline for validation (#730)
bb8558c 
* refactor: generalised validation logic

* refactor: generalised validation logic

* fix(airflow): fine-tuning DAG for data validation

* fix(validation): study locus uniqueness fixed

* feat: add invalid/valid dataset generation in validation steps/dag (#734)

* fix: does not belong in this PR

* refactor: to be moved to orchestration repo

* docs: respective docs pages for the steps

* docs: ammend docstrings

* revert: maintain dag for now but it should be removed eventually

* feat: validate study_locus dataset to produce valid or invalid df

* feat: adjust DAG to parametrise dataset validation

* fix: duplicated row

* feat: increase abstraction of Dataset validation of rows

* docs: increase clarity of what the function does

* fix: error message

* revert: unintended change

* test: testing dataset filtering by quality flag

---------

Co-authored-by: DSuveges <[email protected]>

* chore: pre-commit auto fixes [...]

---------

Co-authored-by: David Ochoa <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Szymon Szyszkowski <[email protected]>
@DSuveges @project-defiant
feat(variant index): improved data structure (#710)
1a7b0d7 
* feat(variant index): upating schema and logic

* feat(variant index): vep transcript annotation improved with consequence score + transcript index

* refactor: removed hardcoded schema definition from parser

* fix(test): fixing mock variant index

* fix: schema extraction fixed

* fix: doctest issue resolved

* fix: updating vep command in the DAG

* fix: removing un-used sequence ontology terms

* fix: adding example to sorter method + changing variable names

---------

Co-authored-by: Szymon Szyszkowski <[email protected]>
@project-defiant
fix: remove finngen prefix from credible set (#746)
3ea47a9
@project-defiant @ireneisdoomed
fix(finngen_r11): preserve all studyIds (#747)
12ff35b 
* fix(finngen_r11): preserve all studyIds

Preserve all studyIds, even if EFO mapping is missing, so mapping
between studyIndex and StudyLocus is by studyId column is accurate.

* fix: typo in docstring

Co-authored-by: Irene López Santiago <[email protected]>

---------

Co-authored-by: Szymon Szyszkowski <[email protected]>
Co-authored-by: Irene López Santiago <[email protected]>
@Daniel-Considine @project-defiant
feat: adding finemapping method to studylocusid hash (#744)
15a058e 
* feat: adding finemapping method to studylocusid hash

* chore: updating function in needed files

* test: update doc test

* fix: mistakes

* fix: tidying logic

* chore: adding studyLocusId recalculation to pics finemapping method

---------

Co-authored-by: Szymon Szyszkowski <[email protected]>
@dependabot @project-defiant
build(deps-dev): bump ipython from 8.26.0 to 8.27.0 (#741)
3c1e81a 
Bumps [ipython](https://github.com/ipython/ipython) from 8.26.0 to 8.27.0.
- [Release notes](https://github.com/ipython/ipython/releases)
- [Commits](ipython/ipython@8.26.0...8.27.0)

---
updated-dependencies:
- dependency-name: ipython
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Szymon Szyszkowski <[email protected]>
@dependabot @project-defiant
build(deps-dev): bump deptry from 0.19.1 to 0.20.0 (#742)
151b4ec 
Bumps [deptry](https://github.com/fpgmaas/deptry) from 0.19.1 to 0.20.0.
- [Release notes](https://github.com/fpgmaas/deptry/releases)
- [Changelog](https://github.com/fpgmaas/deptry/blob/main/CHANGELOG.md)
- [Commits](fpgmaas/deptry@0.19.1...0.20.0)

---
updated-dependencies:
- dependency-name: deptry
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Szymon Szyszkowski <[email protected]>
@project-defiant
fix: validation name mapping (#753)
0b216f6 
* fix: use mapping instead of enum values in valid_rows

* fix: typos

* fix: swap valid and invalid paths

---------

Co-authored-by: Szymon Szyszkowski <[email protected]>
@addramir @project-defiant
fix: removing old functions (#752)
010c881 
Co-authored-by: Szymon Szyszkowski <[email protected]>
@tskir @project-defiant
refactor: generalise per-chromosome processing (#754)
feb9cad 
* refactor: move VA preparation logic into a separate module

* refactor: generalise per-chromosome summary stats ingestion

* Update src/gentropy/common/per_chromosome.py

Co-authored-by: Szymon Szyszkowski <[email protected]>

---------

Co-authored-by: Szymon Szyszkowski <[email protected]>
@tskir
refactor: generalise the harmonisation pipeline (#755)
d10cc20 
* refactor: move harmonisation into a separate module

* refactor: make column names configurable

* feat: make INFO and A1FREQ columns optional

* docs: expand comments on tmp_variant_annotation_path and variant types
d0choa and others added 26 commits October 1, 2024 10:02
@d0choa @DSuveges
feat: out sample LD qc reason (#798)
c3b8c2c 
Co-authored-by: Daniel Suveges <[email protected]>
@project-defiant
chore: drop redundant parameter (#802)
a5588ae 
Co-authored-by: Szymon Szyszkowski <[email protected]>
@project-defiant
fix: align the schema of study_index for ukb ppp eur (#803)
d4b5070 
* fix(ukb_ppp_study_index): update column name to match schema

* chore: add note that notebooks are not supported

---------

Co-authored-by: Szymon Szyszkowski <[email protected]>
@project-defiant
feat: force reinstallation of the gentropy on the cluster (#804)
ccb484e 
Co-authored-by: Szymon Szyszkowski <[email protected]>
@DSuveges
feat(validation): adding credible set confidence annotation at valida…
1c396d2 
…tion time (#801)

* feat: adding credible set confidence

* feat: adding confidence assignment to the credible set validation step

* fix: adding confidence field to schema

* fix: schema

* fix: confidence criteria

* fix: finalising labels

* fix: docstring typo

* test: for credible set confidence assignment
@project-defiant
chore: make the lb clumping ingest the partitionned data (#806)
9177dd4 
Co-authored-by: Szymon Szyszkowski <[email protected]>
@addramir @Daniel-Considine
feat: adding filtering to susie finemapper (#796)
174f8f9 
* feat: adding filltering to susie fine_mapper

* fix: correct options

* fix: fix simulations

* fix: correct clumping

* fix: fix options

* fix: fix for catching None

* fix: adding LD and gwas_df as output

* chore: changing defaults

* chore: v1

* fix: spelling in susie_finemapper.py

---------

Co-authored-by: Daniel-Considine <[email protected]>
@addramir
feat: add sumstat QC fields to schema (#809)
25a4820 
* feat: add sumstat QC fields to schema

* fix: fix of type

* fix: fix for struct
@Tobi1kenobi
feat: fix biosample study validation (#810)
70fd593 
* fix(biosample index): update biosample index validation step to only apply to non-gwas studies

* fix(biosample index): revert to dropping

* feat(biosample index): update test study index to test biosample index too

* feat(biosample index): add biosampleId to study index schema
@ireneisdoomed
refactor(vep_parser): store consequence to impact score as a project …
c286c3b 
…config (#811)

* refactor: set variant functional consequence to pathogenicity score as class attribute

* chore: drop `VariantIndex.get_most_severe_gene_consequence`

* chore(VariantIndex): make `CONSEQUENCE_TO_PATHOGENICITY_SCORE` a class attribute

* fix(vep): convert `id_to_score_map` to `label_to_score_map`

* chore: remove comment

* refactor: move `consequence_to_pathogenicity_score` to `VariantIndexConfig`
@ireneisdoomed
feat(l2g): implement variant consequence features from VEP (#805)
8876fc1 
* feat(variant_index): set variant functional consequence to pathogenicity score as class attribute

* feat: add and test vep features

* feat: add and test vep neighbourhood features

* feat(vep_features): use only protein coding genes to look in the vicinity

* chore: update feature list and other bits

* fix: docs ref

* refactor: set variant functional consequence to pathogenicity score as class attribute

* chore: drop `VariantIndex.get_most_severe_gene_consequence`

* chore(VariantIndex): make `CONSEQUENCE_TO_PATHOGENICITY_SCORE` a class attribute

* fix(vep): convert `id_to_score_map` to `label_to_score_map`

* chore: remove comment

* chore: access max consequence score from variant index
@addramir
feat: optimisation of qc step (#813)
fca55be 
* feat: optimisation of qc step

* fix: adding Z2 filter

* fix: v1
@project-defiant
fix(validation): add qualityControls column if missing in StudyLocu…
68c0168 
…s dataset when perfroming validation (#814)

* fix: coalesce qualityControl column in credible set

* fix: resolve missing qualityControls column when validating studyLocus

* chore: whitespace

* chore: use getter to infer qc column name

* chore: drop show

* chore: drop warnings for step_test(s)

---------

Co-authored-by: Szymon Szyszkowski <[email protected]>
@addramir
fix: fix of type error in schema checking (#817)
b325eaa 
* fix: fix of type error in schma checking

* fix: fix fix fix
@addramir
fix: adding studId to FM log (#816)
99b1d3c
@Tobi1kenobi
fix: biosample id duplication (#822)
8b29168 
* fix(biosample_index): remove biosampleId column if present in validation and tests

* fix(biosample index): update tests for biosample index
@vivienho
feat: change LD annotation for PICS fine-mapping to use major ancestry (
27d8206 
#821)

* feat: add functions to get the major population and calculate r2

* refactor: remove functions not needed anymore

* refactor: change r2Overall to r2Major

* test: add tests for both of the added functions

* test: remove tests no longer needed

* revert: change r2Major back to r2Overall
@dependabot @project-defiant
build(deps-dev): bump ipython from 8.27.0 to 8.28.0 (#819)
f3490d3 
Bumps [ipython](https://github.com/ipython/ipython) from 8.27.0 to 8.28.0.
- [Release notes](https://github.com/ipython/ipython/releases)
- [Commits](ipython/ipython@8.27.0...8.28.0)

---
updated-dependencies:
- dependency-name: ipython
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Szymon Szyszkowski <[email protected]>
@dependabot
build(deps-dev): bump pre-commit from 3.8.0 to 4.0.0 (#820)
60f6bfa 
Bumps [pre-commit](https://github.com/pre-commit/pre-commit) from 3.8.0 to 4.0.0.
- [Release notes](https://github.com/pre-commit/pre-commit/releases)
- [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md)
- [Commits](pre-commit/pre-commit@v3.8.0...v4.0.0)

---
updated-dependencies:
- dependency-name: pre-commit
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@ireneisdoomed
feat: decouple feature generation from L2G training step (#823)
b7dce8f 
* fix: join between gold standard and credible set based on studyId and variantId

* fix: minor bugs to generate the feature matrix

* fix(colocalisation): safeguard existing rightStudyType when applying `append_study_metadata`

* feat: add feature generation step (working interactively)

* feat(l2g): remove feature generation from `LocusToGeneStep`

* fix: correct feature names

* feat: filter gwas credible sets in `L2GPrediction.from_credible_set`

* chore: update docs

* chore: pass credible set to  `L2GGoldStandard.build_feature_matrix` in test

* chore: uncomment code
@d0choa
fix: mhc flag incorrect (#825)
9f0111c
@d0choa
revert: no longer rescuing variants not in LD matrix when overlapping…
31e217b 
… with SuSiE (#827)

* revert: no longer rescueing variants not in LD matrix

* test: update test to cover new expectation
@DSuveges
fix: empty inSilicoPredictors object in GnomAD variant index (#807)
58333c0 
* fix: empty inSilicoPredictors object in variant index

* fix: object filtering expression

* fix(hail): limit removed
@d0choa
fix: write mode added to validation steps (#826)
c7c602a
@ireneisdoomed
feat(l2g): merge sQTL and tuQTL colocalisation features (#824)
e3d32ba 
* feat: merge tuQTL colocalisation results into sQTL features

* fix: add colocalisation neighbourhood features in the l2g default features list

* fix: minor bug
@DSuveges
fix: adding single point statistics to pics loci (#832)
fb6111d 
* fix: adding single point statistics to pics loci

* fix: patching field update logic

* refactor: simplifying the pics process slightly

* test: for lead statistics propagation

* refactor: udf as decorator

* fix: removing commented rows

* fix: reverting udf decorator - tests would fail otherwise
@d0choa d0choa closed this Oct 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
auto-pr Chore documentation Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
9 participants
@d0choa @project-defiant @Daniel-Considine @ireneisdoomed @addramir @DSuveges @tskir @vivienho @Tobi1kenobi