Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Usage recommendations, confusion about mosdepth example command #30

Open
SergeWielhouwer opened this issue Aug 16, 2024 · 3 comments
Open

Comments

@SergeWielhouwer
Copy link

SergeWielhouwer commented Aug 16, 2024

Hi,

I am currently evaluating Spectre for CNV calling on some GIAB HG002 ONT data and I encountered a few issues which I wasn't able to resolve myself.

Using mosdepth I tried to create required regions.bed.gz file for use with Spectre, which worked fine with the following:
mosdepth -t {threads} -x -b 1000 -Q 20 "variants/{wildcards.sample}/spectre/mosdepth/{wildcards.sample}"' "{input.bam}" 2>{log}'
But, I don't understand why the "-c X" parameter is included in your example command. This would lead to only Chromosome X being evaluated on read depth right? Could you explain why this parameter is there?

When I tried to run Spectre itself I only got some CNVs on chrX and not on any autosomes or other contigs (I don't know the known CNVs for HG002 atm, but was surprised that I only got this result). Any idea what's causing this? The logs mention something about expecting trouble as the Minimum CNV length is smaller than 1000000 base pairs.

spectre::2024-08-16 08:56:06,869::INFO::spectre.spectre> Spectre input: CNVCaller --coverage variants/PAO89685_90Gbp/spectre/mosdepth/PAO89685_90Gbp.regions.bed.gz --sample-id PAO89685_90Gbp --output-dir variants/PAO89685_90Gbp/spectre/ --reference /mnt/shared/genomes/GRCh38/ncbi/GCA_000001405.15_GRCh38_no_alt_analysis_set.fna --blacklist /mnt/shared/tools/spectre-0.2.1/data/grch38_blacklist_spectre.bed --metadata /mnt/shared/tools/spectre-0.2.1/data/grch38.mdr
spectre::2024-08-16 08:56:06,869::INFO::spectre.spectre> Spectre version: 0.2.1
spectre::2024-08-16 08:56:06,869::INFO::spectre.spectre> Starting spectre
spectre::2024-08-16 08:56:06,869::INFO::spectre.spectre> Evaluate if a new .mdr file needs to be created
spectre::2024-08-16 08:56:06,992::INFO::spectre.spectre> Using existing metadata file /mnt/shared/tools/spectre-0.2.1/data/grch38.mdr
spectre::2024-08-16 08:56:07,053::INFO::spectre.spectreCNV> Spectre calculating for: variants/PAO89685_90Gbp/spectre/mosdepth/PAO89685_90Gbp.regions.bed.gz
spectre::2024-08-16 08:56:07,054::INFO::spectre.spectreCNV> Data normalization and outlier removal (right tail)
[W::hts_idx_load3] The index file is older than the data file: variants/PAO89685_90Gbp/spectre/mosdepth/PAO89685_90Gbp.regions.bed.gz.csi
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
spectre::2024-08-16 08:56:07,630::INFO::spectre.analysis.analysis> Determined bin size from Mosdepth coverage: 1000
spectre::2024-08-16 08:56:07,692::WARNING::spectre.analysis.analysis> Minimum CNV length is smaller then 1000000 base pairs
spectre::2024-08-16 08:56:07,692::WARNING::spectre.analysis.analysis> Expect trouble!
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
[W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option?
spectre::2024-08-16 08:56:12,241::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr1: 248957
spectre::2024-08-16 08:56:12,342::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr2: 242194
spectre::2024-08-16 08:56:12,437::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr3: 198296
spectre::2024-08-16 08:56:12,515::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr4: 190215
spectre::2024-08-16 08:56:12,592::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr5: 181539
spectre::2024-08-16 08:56:12,664::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr6: 170806
spectre::2024-08-16 08:56:12,733::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr7: 159346
spectre::2024-08-16 08:56:12,798::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr8: 145139
spectre::2024-08-16 08:56:12,858::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr9: 138395
spectre::2024-08-16 08:56:12,916::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr10: 133798
spectre::2024-08-16 08:56:12,969::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr11: 135087
spectre::2024-08-16 08:56:13,024::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr12: 133276
spectre::2024-08-16 08:56:13,077::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr13: 114365
spectre::2024-08-16 08:56:13,125::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr14: 107044
spectre::2024-08-16 08:56:13,169::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr15: 101992
spectre::2024-08-16 08:56:13,212::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr16: 90339
spectre::2024-08-16 08:56:13,251::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr17: 83258
spectre::2024-08-16 08:56:13,286::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr18: 80374
spectre::2024-08-16 08:56:13,318::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr19: 58618
spectre::2024-08-16 08:56:13,343::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr20: 64445
spectre::2024-08-16 08:56:13,369::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr21: 46710
spectre::2024-08-16 08:56:13,390::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr22: 50819
spectre::2024-08-16 08:56:13,412::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chrX: 156041
spectre::2024-08-16 08:56:13,479::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chrY: 57228
spectre::2024-08-16 08:56:13,505::WARNING::spectre.analysis.analysis> Chromosome chrY: no data available!
spectre::2024-08-16 08:56:13,505::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr16_KI270728v1_random: 1873
spectre::2024-08-16 08:56:38,297::INFO::spectre.analysis.analysis> Selected threshold for DEL=1.3083 and DUP=2.626 for a ploidy of 2
spectre::2024-08-16 08:56:38,326::INFO::spectre.spectreCNV> CNV calling - Coverage for sample: PAO89685_90Gbp
spectre::2024-08-16 08:56:38,326::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr1
spectre::2024-08-16 08:56:38,844::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr2
spectre::2024-08-16 08:56:39,419::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr3
spectre::2024-08-16 08:56:39,919::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr4
spectre::2024-08-16 08:56:40,427::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr5
spectre::2024-08-16 08:56:40,937::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr6
spectre::2024-08-16 08:56:41,499::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr7
spectre::2024-08-16 08:56:41,979::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr8
spectre::2024-08-16 08:56:42,428::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr9
spectre::2024-08-16 08:56:42,817::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr10
spectre::2024-08-16 08:56:43,260::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr11
spectre::2024-08-16 08:56:43,739::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr12
spectre::2024-08-16 08:56:44,231::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr13
spectre::2024-08-16 08:56:44,613::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr14
spectre::2024-08-16 08:56:44,974::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr15
spectre::2024-08-16 08:56:45,298::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr16
spectre::2024-08-16 08:56:45,573::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr17
spectre::2024-08-16 08:56:45,867::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr18
spectre::2024-08-16 08:56:46,173::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr19
spectre::2024-08-16 08:56:46,413::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr20
spectre::2024-08-16 08:56:46,659::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr21
spectre::2024-08-16 08:56:46,806::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr22
spectre::2024-08-16 08:56:46,969::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chrX
spectre::2024-08-16 08:56:47,905::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr16_KI270728v1_random
spectre::2024-08-16 08:57:31,868::INFO::spectre.analysis.cnv_metrics> DEL border:1.3082655826558265 DUP border:2.6260162601626016
spectre::2024-08-16 08:57:32,358::INFO::spectre.analysis.analysis> refining cnv calls
spectre::2024-08-16 08:57:44,589::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:57:44,589::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:57:44,589::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:57:44,589::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:57:57,730::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:57:57,730::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:57:57,730::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:57:57,730::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:58:07,912::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:58:07,912::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:58:07,912::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:58:07,912::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:58:17,715::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:58:17,715::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:58:17,715::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:58:17,715::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:58:27,048::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:58:27,049::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:58:27,049::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:58:27,049::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:58:35,837::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:58:35,837::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:58:35,837::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:58:35,837::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:58:43,679::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:58:43,679::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:58:43,679::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:58:43,679::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:58:50,723::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:58:50,723::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:58:50,723::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:58:50,723::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:58:56,104::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:58:56,104::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:58:56,104::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:58:56,104::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:59:02,721::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:59:02,721::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:59:02,721::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:59:02,721::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:59:09,832::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:59:09,832::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:59:09,832::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:59:09,832::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:59:16,911::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:59:16,911::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:59:16,911::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:59:16,911::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:59:22,050::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:59:22,051::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:59:22,051::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:59:22,051::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:59:26,546::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:59:26,546::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:59:26,546::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:59:26,546::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:59:30,397::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:59:30,397::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:59:30,397::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:59:30,397::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:59:33,459::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:59:33,459::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:59:33,459::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:59:33,459::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:59:36,942::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:59:36,942::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:59:36,942::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:59:36,942::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:59:40,770::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:59:40,770::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:59:40,770::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:59:40,770::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:59:43,788::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:59:43,788::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:59:43,788::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:59:43,788::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:59:46,763::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:59:46,763::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:59:46,763::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:59:46,763::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:59:48,312::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:59:48,312::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:59:48,312::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:59:48,312::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:59:49,993::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:59:49,993::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:59:49,993::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:59:49,993::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:59:52,800::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:59:52,800::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:59:52,800::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:59:52,800::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:59:52,845::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1.
spectre::2024-08-16 08:59:52,845::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1
spectre::2024-08-16 08:59:52,845::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3.
spectre::2024-08-16 08:59:52,845::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3
spectre::2024-08-16 08:59:52,845::INFO::spectre.spectreCNV> Calculate CNV metrics
spectre::2024-08-16 09:00:20,053::INFO::spectre.analysis.cnv_metrics> DEL border:1.3082655826558265 DUP border:2.6260162601626016
spectre::2024-08-16 09:00:20,076::INFO::spectre.spectreCNV> Merge CNV and LoH candidates
spectre::2024-08-16 09:00:20,076::DEBUG::spectre.classes.loh_candidate> CNV cand: 9
spectre::2024-08-16 09:00:20,076::INFO::spectre.spectreCNV> Final candidates are written to .spc file
spectre::2024-08-16 09:00:26,716::INFO::spectre.spectreCNV> Results are writen to bed file
spectre::2024-08-16 09:00:26,813::INFO::spectre.spectreCNV> Results are writen to VCF file
spectre::2024-08-16 09:00:26,881::INFO::spectre.spectreCNV> Result plot in progress
spectre::2024-08-16 09:00:27,034::INFO::spectre.plots.plot> CNVs in chromosome: chr1
spectre::2024-08-16 09:00:27,448::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr1.png
spectre::2024-08-16 09:00:27,618::INFO::spectre.plots.plot> CNVs in chromosome: chr2
spectre::2024-08-16 09:00:28,002::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr2.png
spectre::2024-08-16 09:00:28,064::INFO::spectre.plots.plot> CNVs in chromosome: chr3
spectre::2024-08-16 09:00:28,464::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr3.png
spectre::2024-08-16 09:00:28,526::INFO::spectre.plots.plot> CNVs in chromosome: chr4
spectre::2024-08-16 09:00:28,893::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr4.png
spectre::2024-08-16 09:00:28,954::INFO::spectre.plots.plot> CNVs in chromosome: chr5
spectre::2024-08-16 09:00:29,326::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr5.png
spectre::2024-08-16 09:00:29,388::INFO::spectre.plots.plot> CNVs in chromosome: chr6
spectre::2024-08-16 09:00:29,759::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr6.png
spectre::2024-08-16 09:00:29,819::INFO::spectre.plots.plot> CNVs in chromosome: chr7
spectre::2024-08-16 09:00:30,175::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr7.png
spectre::2024-08-16 09:00:30,233::INFO::spectre.plots.plot> CNVs in chromosome: chr8
spectre::2024-08-16 09:00:30,563::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr8.png
spectre::2024-08-16 09:00:30,621::INFO::spectre.plots.plot> CNVs in chromosome: chr9
spectre::2024-08-16 09:00:30,903::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr9.png
spectre::2024-08-16 09:00:31,075::INFO::spectre.plots.plot> CNVs in chromosome: chr10
spectre::2024-08-16 09:00:31,382::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr10.png
spectre::2024-08-16 09:00:31,441::INFO::spectre.plots.plot> CNVs in chromosome: chr11
spectre::2024-08-16 09:00:31,759::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr11.png
spectre::2024-08-16 09:00:31,818::INFO::spectre.plots.plot> CNVs in chromosome: chr12
spectre::2024-08-16 09:00:32,159::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr12.png
spectre::2024-08-16 09:00:32,241::INFO::spectre.plots.plot> CNVs in chromosome: chr13
spectre::2024-08-16 09:00:32,533::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr13.png
spectre::2024-08-16 09:00:32,617::INFO::spectre.plots.plot> CNVs in chromosome: chr14
spectre::2024-08-16 09:00:32,877::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr14.png
spectre::2024-08-16 09:00:32,966::INFO::spectre.plots.plot> CNVs in chromosome: chr15
spectre::2024-08-16 09:00:33,224::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr15.png
spectre::2024-08-16 09:00:33,282::INFO::spectre.plots.plot> CNVs in chromosome: chr16
spectre::2024-08-16 09:00:33,531::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr16.png
spectre::2024-08-16 09:00:33,589::INFO::spectre.plots.plot> CNVs in chromosome: chr17
spectre::2024-08-16 09:00:33,838::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr17.png
spectre::2024-08-16 09:00:33,898::INFO::spectre.plots.plot> CNVs in chromosome: chr18
spectre::2024-08-16 09:00:34,286::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr18.png
spectre::2024-08-16 09:00:34,344::INFO::spectre.plots.plot> CNVs in chromosome: chr19
spectre::2024-08-16 09:00:34,570::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr19.png
spectre::2024-08-16 09:00:34,627::INFO::spectre.plots.plot> CNVs in chromosome: chr20
spectre::2024-08-16 09:00:34,875::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr20.png
/usr/local/lib/python3.10/site-packages/spectre/plots/plot.py:39: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (matplotlib.pyplot.figure) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam figure.max_open_warning). Consider using matplotlib.pyplot.close().
self.figure = plot_engine.figure(figsize=(8, 4))
spectre::2024-08-16 09:00:34,953::INFO::spectre.plots.plot> CNVs in chromosome: chr21
spectre::2024-08-16 09:00:35,140::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr21.png
spectre::2024-08-16 09:00:35,218::INFO::spectre.plots.plot> CNVs in chromosome: chr22
spectre::2024-08-16 09:00:35,426::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr22.png
spectre::2024-08-16 09:00:35,492::INFO::spectre.plots.plot> CNVs in chromosome: chrX
spectre::2024-08-16 09:00:35,767::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chrX.png
spectre::2024-08-16 09:00:35,823::INFO::spectre.plots.plot> CNVs in chromosome: chr16_KI270728v1_random
spectre::2024-08-16 09:00:36,054::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr16_KI270728v1_random.png
spectre::2024-08-16 09:00:36,088::INFO::spectre.spectreCNV> Output dir: /mnt/scratch2/s.wielhouwer/ONT_variant_calling/pipeline/test_run/variants/PAO89685_90Gbp/spectre
spectre::2024-08-16 09:00:36,088::INFO::spectre.spectreCNV> Done with sample PAO89685_90Gbp
spectre::2024-08-16 09:00:36,088::INFO::spectre.spectre> Spectre finished

I am using GCA_000001405.15_GRCh38_no_alt_analysis_set.fna so I assume the metadata and blacklist are still compatible right? The contig names are identical (chr prefix) and it concerns hg38.

When including a snv file for marking LoH regions I get the following error.

spectre::2024-08-16 09:16:58,350::INFO::spectre.spectre> Spectre input: CNVCaller --coverage variants/PAO89685_90Gbp/spectre/mosdepth/PAO89685_90Gbp.regions.bed.gz --sample-id PAO89685_90Gbp --output-dir variants/PAO89685_90Gbp/spectre/ --reference /mnt/shared/genomes/GRCh38/ncbi/GCA_000001405.15_GRCh38_no_alt_analysis_set.fna --blacklist /mnt/shared/tools/spectre-0.2.1/data/grch38_blacklist_spectre.bed --metadata /mnt/shared/tools/spectre-0.2.1/data/grch38.mdr --snv variants/PAO89685_90Gbp/PAO89685_90Gbp.snp.vcf.gz
spectre::2024-08-16 09:16:58,350::INFO::spectre.spectre> Spectre version: 0.2.1
spectre::2024-08-16 09:16:58,350::INFO::spectre.spectre> Starting spectre
spectre::2024-08-16 09:16:58,350::INFO::spectre.spectre> Evaluate if a new .mdr file needs to be created
spectre::2024-08-16 09:16:58,350::INFO::spectre.spectre> Using existing metadata file /mnt/shared/tools/spectre-0.2.1/data/grch38.mdr
spectre::2024-08-16 09:16:58,362::INFO::spectre.spectreCNV> Spectre calculating for: variants/PAO89685_90Gbp/spectre/mosdepth/PAO89685_90Gbp.regions.bed.gz
spectre::2024-08-16 09:16:58,363::INFO::spectre.spectreCNV> Analysing CN neutral state from SNV data
spectre::2024-08-16 09:16:58,460::DEBUG::spectre.analysis.snv_analysis> Starting SNV Copy number state compute ...
spectre::2024-08-16 09:16:58,460::DEBUG::spectre.analysis.snv_analysis> Het sites count per chromosome
[W::vcf_parse] Contig '##cmdline=/opt/bin/run_clair3.sh --bam_fn=alignments/PAO89685_90Gbp.bam --ref_fn=/mnt/shared/genomes/GRCh38/ncbi/GCA_000001405.15_GRCh38_no_alt_analysis_set.fna --platform=ont --threads=64 --model_path=/mnt/shared/tools/Clair3-1.0.10/container/models/r1041_e82_400bps_sup_v420 --output=variants/PAO89685_90Gbp/clair3' is not defined in the header. (Quick workaround: index the file with tabix.)
[W::bcf_hrec_check] Invalid contig name: "##cmdline=/opt/bin/run_clair3.sh --bam_fn=alignments/PAO89685_90Gbp.bam --ref_fn=/mnt/shared/genomes/GRCh38/ncbi/GCA_000001405.15_GRCh38_no_alt_analysis_set.fna --platform=ont --threads=64 --model_path=/mnt/shared/tools/Clair3-1.0.10/container/models/r1041_e82_400bps_sup_v420 --output=variants/PAO89685_90Gbp/clair3"
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/spectre/analysis/snv_analysis.py", line 161, in snv_nhet_chromosome
[snv] = snv_record.samples.values()
ValueError: not enough values to unpack (expected 1, got 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/bin/spectre", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.10/site-packages/spectre/spectre.py", line 642, in main
spectre_run.spectre_exe()
File "/usr/local/lib/python3.10/site-packages/spectre/spectre.py", line 356, in spectre_exe
spectre_main.cnv_call()
File "/usr/local/lib/python3.10/site-packages/spectre/spectreCNV.py", line 71, in cnv_call
self.snv_analysis.snv_copy_number_state()
File "/usr/local/lib/python3.10/site-packages/spectre/analysis/snv_analysis.py", line 202, in snv_copy_number_state
use_chr_het_sites = self.snv_nhet_chromosome() # self.genome_info["chromosomes"]
File "/usr/local/lib/python3.10/site-packages/spectre/analysis/snv_analysis.py", line 163, in snv_nhet_chromosome
[snv] = snv_record.samples.values()[0]
IndexError: list index out of range

Was this tool tested on Clair3 SNV data? Or should the falsely seen "##cmdline" contig be excluded using zgrep -v or similar to be able to use these vcf files?

I am using:

  • mosdepth_0.3.8_build1
  • spectre-0.2.1
  • snf2json-0.1.0
  • sniffles-2.4
  • clair3-1.0.10

Btw, spectre should also work on other mammalian species right? Are there any ploidy limitations or such?

Best regards,

Serge

@lfpaulin
Copy link
Collaborator

Hi @SergeWielhouwer, could you please share with us the header of the SNV-VCF file please. You can use the following command:

bcftools view --header-only file.vcf.gz > file_vcf_header.txt

We did test it with Clair3 for the most part (we did one test with Deep Variant) it seems that the header is not correctly formatted, but could be something else, lets check that first
Spectre is mostly designed for humans, but, there is a ploidy parameter that can be used to change the default ploidy (2) for a given chromosome

@philippesanio
Copy link
Collaborator

philippesanio commented Aug 16, 2024

Hi @SergeWielhouwer

Using mosdepth I tried to create required regions.bed.gz file for use with Spectre, which worked fine with the following:
mosdepth -t {threads} -x -b 1000 -Q 20 "variants/{wildcards.sample}/spectre/mosdepth/{wildcards.sample}"' "{input.bam}" 2>{log}'
But, I don't understand why the "-c X" parameter is included in your example command. This would lead to only Chromosome X being evaluated on read depth right? Could you explain why this parameter is there?

As you already correctly identified, the "-c X" does only get chromsome X, this should not be there. I will fix that ASAP in the README.

When I tried to run Spectre itself I only got some CNVs on chrX and not on any autosomes or other contigs (I don't know the known CNVs for HG002 atm, but was surprised that I only got this result). Any idea what's causing this? The logs mention something about expecting trouble as the Minimum CNV length is smaller than 1000000 base pairs.

There was a typo in the previous version, which was updated in the latest version on GitHub to 100kb. However, we did not push those changes to pip yet, since we considered them to be minor issues. We will push those changes with the next version of Spectre, which is currently under development. If you want the latest version or bug fixes, please clone and download the latest GitHub version. You can install Spectre locally via pip.

Regarding the minimum CNV length itself, we do not recommend going lower than 100kb. Reason being, Spectre was designed to detect large CNVs. Additionally, it depends on how high your signal-to-noise ratio is. Either mapping artifacts or just bad samples could introduce FPs.

Best,
Philippe

philippesanio added a commit that referenced this issue Aug 16, 2024
Removed "-c X" from running Mosdepth which only resulted in chromosome X being exported
@SergeWielhouwer
Copy link
Author

Hi,

Thanks for letting me know and the quick fix!
Please find the vcf header of the Clair3 output below.

file_vcf_header.txt

I have also included the Spectre log with the SNV option enabled (the cmdline "contig" excluded), hope the info is a bit helpful.
spectre_cnv.PAO89685_90Gbp.log

Kind regards,

Serge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants