4.1.1

Bug fix and maintenance

Important CRAM writing bug fix

A serious bug which can cause corrupted reads in CRAM files was discovered and fixed in this release. This bug was introduced in HTSJDK 3.0.0, and affects Picard versions 2.27.3 through 3.1.1 and GATK versions 4.3 through 4.5.

The bug occurs in cases where there is a read aligned starting at exactly position 1 on a reference contig. This means that the bug doesn't generally impact human autosome and X/Y contigs because they tend to start with a large number of N bases and reads are not aligned at exactly position 1. The exceptions to this would be T2T references and things like mitochondrial calling.

For more information on the conditions that trigger this bug, see this post.

GATK 4.6 includes a tool called CRAMIssue8768Detector that can scan a CRAM file and report whether it is affected, and if so which regions in the file are corrupt. If you suspect that some of your CRAM files may have been affected, please run this tool on them for confirmation!

See also #1708 for more information.

What's Changed

• Fix CRAMReferenceRegion updating. by @cmnbroad in #1708
• Updating tests in SAMRecordUnitTest to remove ambiguity by @lbergelson in #1696
• Update to gradle 8.5 by @lbergelson in #1697
• Updating our dependencies to the the newest version by @lbergelson in #1699
• Upgrade to samtools 1.19.1 for interop testing. by @cmnbroad in #1706

Full Changelog: 4.1.0...4.1.1

4.1.0

What's Changed

• Updated the way that tribble creates SeekableStreams. It will now prefer available http / ftp FileSystemProvider plugins if they are available instead of using the legacy http / ftp support. This is intended to support downstream use of plugins such as org.broadinstitute:http-nio which may provide more robust handling of various URIs.

Compatibility note

• Tribble now requires all fully specified absolute paths to be already encoded. Specifically, this means that ftp / http URIs which used
  to be written with spaces or special characters must now be already percent encoded before handing them to tribble.
  For example, the previous ftp path style:

• ftp://example.org/ftp/my file with spaces.txt should now be entered as

• ftp://example.org/ftp/my%20file%20with%20spaces.txt
  This is more awkward in some cases but it aligns the legacy ftp/http access with the IOPath standard of always requiring a fully encoded URI if a scheme is specified. File paths that don't include a file:// scheme are still handled unencoded so this should have limited impact. See the documentation for HtsPath for more details information.

• Deprecated SeekableStreamFactory.isFilePath() and replaced it with the clearer but more verbose isBeingHandledByLegacyUrlSupport

• Some changes to error handling in HtsPath now reject certain paths which previously would be treated as local file paths.

  • Paths with only a scheme, or a missing scheme are invalid now:
    • file://, ://mypath, ://
  • Paths which contain a scheme but are not parseable as a uri are now rejected instead of being treated as a local file path
    • http://example.org/filename I forgot to encode
    • These are much more likely to be an attempt to access an unencoded URI instead of an attempt to create a local file path
      named file:///<workingdir>/http:/example.org/filename I forgot to encode

Full Changelog: 4.0.2...4.1.0

4.0.2

Small release with a few bug fixes and small features.

What's Changed

• Add IntervalFileFeature for common interface between bed and interval_list by @rickymagner in #1680
• Update snappy-java to fix vulnerability by @bbimber in #1687
• VCF 4.4 optimistic read switch to allow reading VCF 4.4 as if it's 4.3 (use at your own risk) by @cmnbroad in #1683
• Update some methods to accept Path input by @takutosato in #1681
• Fix IOUtil.unrollPaths for http paths with query parameters by @lbergelson in #1688
• Update test dependencies to newest versions by @lbergelson in #1690

New Contributors

• @takutosato made their first contribution in #1681

Full Changelog: 4.0.1...4.0.2

4.0.1

This is a tiny release which bumps some dependency versions and adds two small convenience methods to Genotype.

What's Changed

• Move the EnaRefServiceTest to the externalAPI test group by @lbergelson in #1675
• two methods for Genotype: HasRefAllele HasAltAllele by @lindenb in #1678
• Update two remaining out of date dependencies by @lbergelson in #1677

Full Changelog: 4.0.0...4.0.1

4.0.0

Moving forward

This is the first release to be built exclusively for java 17. Java 17 features are now allowed in our source code and we will no longer support older versions of java. We've also updated dependencies to fix security issues. There are several small bug fixes as well.

JSON dependency:

We've dropped the MJSON library which was no longer being updated and replaced it with a similarly small json library from org.json

What's Changed

• Migrate to Java 17 by @lbergelson in #1649
• Remove low-value progress logging message by @nh13 in #1659
• removed redundant code by @KleinSamuel in #1664
• Update snappy-java and migrate mjson to org.json to address CVEs by @bbimber in #1670
• Remove incorrect zero-length-B-array checks @gileshall and @jmarshall in #1674
• add SINGULAR platform to read group by @omicsorama in #1635

New Contributors

• @KleinSamuel made their first contribution in #1664
• @gileshall made their first contribution in #1674
• @omicsorama made their first contribution in #1635

Full Changelog: 3.0.5...4.0.0

3.0.5

The last gasp of Java 8

This release includes very minor new features as well as a bug fix. It is mostly notable because it is intended to be the last release supporting Java 8. Future releases will target Java 17.

What's Changed

• Update commons-compress to close vulnerabilities by @lbergelson in #1639
• Removing scala test infrastructure by @lbergelson in #1640
• Adding Cigar.fromCigarString() by @lbergelson in #1647
• Updating gradle to 7.6 by @lbergelson in #1650
• Expose the ability to encode a Genotoype into a GT field by @lbergelson in #1648
• Fix typos in README by @Manice18 in #1654
• Fix an NPE in CRAMRecordReadFeatures.restoreReadBases by @lbergelson in #1655

New Contributors

• @Manice18 made their first contribution in #1654

Full Changelog: 3.0.4...3.0.5

3.0.4

3.0.4

Hotfix for #1637

This is a single change e2943b7 which reverts #1593

3.0.3

Serious Known issue

We've introduced a serious bug by making the VCF sort order more stable in many situations when outputting vcfs. This introduced the side effect of causing many valid VCF's to be incorrectly flagged as invalid when reading existing VCFS.

We recommend against updating to this version, prefer 3.0.4 or 3.0.2

What's Changed

• Use allele info in VariantContext comparisons for stable sorts by @clintval in #1593
• Fix HtsCRAMCodec test data provider. by @cmnbroad in #1632
• Allow the BAM index to be up to 5 seconds older than the BAM before e… by @tfenne in #1634

Full Changelog: 3.0.2...3.0.3

3.0.2

What's Changed

• Minor improvements to AbstractLocusIterator by @lbergelson in #1624
• Remove deprecation from Allele.acceptableAlleleBases by @lbergelson in #1625

Full Changelog: 3.0.1...3.0.2

3.0.1

Fix for a long standing vulnerability around temporary directory creation which could expose data to malicious users with access to a shared system. See for more information #1617

Compatibility note:

The previous implementation of IOUtil.createTempDir() could accept a prefix with a complete file path:
IOUtil.createTempDir("path/to/my/tempdir/prefix", ""). The new implementation will now throw in that case. You can use Files.createTemporaryDirectory(path, prefix) for those use cases instead.

4a4024a Fix temporary directory hijacking or temporary directory information disclosure (#1621)
9fd0ecf Disable codecov until we can fix the uploader (#1622)
347c0ac Fix EdgeReadIterator (#1616)
d15a5ba Added ULTIMA and ELEMENT as valid value for RG-PL according to SAM spec. (#1619)