Dear Genome STRiP users,
I nearly completed the SVPreprocess to all 10686 samples:
INFO 01:17:45,002 QGraph - 4 Pend, 2 Run, 0 Fail, 32076 Done
However, I met a similar but not the same "java.lang.OutOfMemoryError: Java heap space" as before:
ERROR 01:19:31,326 FunctionEdge - Error: 'java' '-Xmx2048m' '-XX:+UseParallelOldGC' '-XX:ParallelGCThreads=4' '-XX:GCTimeLimit=50' '-XX:GCHeapFreeLimit=10' '-Djava.io.tmpdir=/proj/yunligrp/users/minzhi/gs/gs_tempdir/svpre_tmp' '-cp' '/proj/yunligrp/users/minzhi/svtoolkit/lib/SVToolkit.jar:/proj/yunligrp/users/minzhi/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/proj/yunligrp/users/minzhi/svtoolkit/lib/gatk/Queue.jar' '-cp' '/proj/yunligrp/users/minzhi/svtoolkit/lib/SVToolkit.jar:/proj/yunligrp/users/minzhi/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/proj/yunligrp/users/minzhi/svtoolkit/lib/gatk/Queue.jar' 'org.broadinstitute.sv.apps.ComputeDepthProfiles' '-O' '/proj/yunligrp/users/minzhi/gs/freeze6-AA_chr16/svpre_freeze6-AA_chr16_standard_full_single_1-500000over1-500000/md_tempdir/profiles_100Kb/profile_seq_chr16_100000.dat.gz' '-I' '/proj/yunligrp/users/minzhi/gs/freeze6-AA_chr16/svpre_freeze6-AA_chr16_standard_full_single_1-500000over1-500000/md_tempdir/headers.bam' '-configFile' '/proj/yunligrp/users/minzhi/svtoolkit/conf/genstrip_parameters.txt' '-configFile' '/proj/yunligrp/users/minzhi/gs/Homo_sapiens_assembly38/Homo_sapiens_assembly38.gsparams.txt' '-R' '/proj/yunligrp/users/minzhi/gs/Homo_sapiens_assembly38/Homo_sapiens_assembly38.fasta' '-L' 'chr16:1-500000' '-genomeMaskFile' '/proj/yunligrp/users/minzhi/gs/Homo_sapiens_assembly38/Homo_sapiens_assembly38.svmask.fasta' '-md' '/proj/yunligrp/users/minzhi/gs/freeze6-AA_chr16/svpre_freeze6-AA_chr16_standard_full_single_1-500000over1-500000/md_tempdir' '-profileBinSize' '100000' '-maximumReferenceGapLength' '10000'
ERROR 01:19:31,333 FunctionEdge - Contents of /proj/yunligrp/users/minzhi/gs/freeze6-AA_chr16/svpre_freeze6-AA_chr16_standard_full_single_1-500000over1-500000/logs/SVPreprocess-32077.out:
INFO 01:18:49,973 HelpFormatter - -------------------------------------------------------------
INFO 01:18:49,976 HelpFormatter - Program Name: org.broadinstitute.sv.apps.ComputeDepthProfiles
INFO 01:18:49,979 HelpFormatter - Program Args: -O /proj/yunligrp/users/minzhi/gs/freeze6-AA_chr16/svpre_freeze6-AA_chr16_standard_full_single_1-500000over1-500000/md_tempdir/profiles_100Kb/profile_seq_chr16_100000.dat.gz -I /proj/yunligrp/users/minzhi/gs/freeze6-AA_chr16/svpre_freeze6-AA_chr16_standard_full_single_1-500000over1-500000/md_tempdir/headers.bam -configFile /proj/yunligrp/users/minzhi/svtoolkit/conf/genstrip_parameters.txt -configFile /proj/yunligrp/users/minzhi/gs/Homo_sapiens_assembly38/Homo_sapiens_assembly38.gsparams.txt -R /proj/yunligrp/users/minzhi/gs/Homo_sapiens_assembly38/Homo_sapiens_assembly38.fasta -L chr16:1-500000 -genomeMaskFile /proj/yunligrp/users/minzhi/gs/Homo_sapiens_assembly38/Homo_sapiens_assembly38.svmask.fasta -md /proj/yunligrp/users/minzhi/gs/freeze6-AA_chr16/svpre_freeze6-AA_chr16_standard_full_single_1-500000over1-500000/md_tempdir -profileBinSize 100000 -maximumReferenceGapLength 10000
INFO 01:18:49,983 HelpFormatter - Executing as minzhi@c0924.ll.unc.edu on Linux 3.10.0-957.el7.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_191-b12.
INFO 01:18:49,984 HelpFormatter - Date/Time: 2019/03/01 01:18:49
INFO 01:18:49,984 HelpFormatter - -------------------------------------------------------------
INFO 01:18:49,984 HelpFormatter - -------------------------------------------------------------
INFO 01:18:49,999 ComputeDepthProfiles - Opening reference sequence ...
INFO 01:18:50,002 ComputeDepthProfiles - Opened reference sequence.
INFO 01:18:50,003 ComputeDepthProfiles - Opening genome mask ...
INFO 01:18:50,005 ComputeDepthProfiles - Opened genome mask.
INFO 01:18:50,007 MetaData - Opening metadata ...
INFO 01:18:50,007 MetaData - Adding metadata location /proj/yunligrp/users/minzhi/gs/freeze6-AA_chr16/svpre_freeze6-AA_chr16_standard_full_single_1-500000over1-500000/md_tempdir ...
INFO 01:18:50,018 MetaData - Opened metadata.
INFO 01:18:50,018 ComputeDepthProfiles - Opened metadata.
INFO 01:18:50,018 ComputeDepthProfiles - Initializing input data set ...
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3332)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
at java.lang.StringBuilder.append(StringBuilder.java:136)
at htsjdk.samtools.SAMTextHeaderCodec.advanceLine(SAMTextHeaderCodec.java:139)
at htsjdk.samtools.SAMTextHeaderCodec.decode(SAMTextHeaderCodec.java:94)
at htsjdk.samtools.BAMFileReader.readHeader(BAMFileReader.java:655)
at htsjdk.samtools.BAMFileReader.<init>(BAMFileReader.java:298)
at htsjdk.samtools.BAMFileReader.<init>(BAMFileReader.java:176)
at htsjdk.samtools.SamReaderFactory$SamReaderFactoryImpl.open(SamReaderFactory.java:376)
at htsjdk.samtools.SamReaderFactory$SamReaderFactoryImpl.open(SamReaderFactory.java:202)
at org.broadinstitute.sv.dataset.SAMFileLocation.createSamFileReader(SAMFileLocation.java:97)
at org.broadinstitute.sv.dataset.SAMLocation.createSamFileReader(SAMLocation.java:41)
at org.broadinstitute.sv.dataset.DataSet.initInputFile(DataSet.java:138)
at org.broadinstitute.sv.dataset.DataSet.initialize(DataSet.java:128)
at org.broadinstitute.sv.apps.ComputeDepthProfiles.initDataSet(ComputeDepthProfiles.java:263)
at org.broadinstitute.sv.apps.ComputeDepthProfiles.run(ComputeDepthProfiles.java:141)
at org.broadinstitute.sv.commandline.CommandLineProgram.execute(CommandLineProgram.java:54)
at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:256)
at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:158)
at org.broadinstitute.sv.commandline.CommandLineProgram.runAndReturnResult(CommandLineProgram.java:29)
at org.broadinstitute.sv.commandline.CommandLineProgram.run(CommandLineProgram.java:25)
at org.broadinstitute.sv.apps.ComputeDepthProfiles.main(ComputeDepthProfiles.java:109)
And after this error it shows
INFO 01:20:13,358 QGraph - 4 Pend, 1 Run, 1 Fail, 32076 Done
And then, such error repeated again rather than directly exit.
ERROR 01:20:43,371 FunctionEdge - Error: 'java' '-Xmx2048m' '-XX:+UseParallelOldGC' '-XX:ParallelGCThreads=4' '-XX:GCTimeLimit=50' '-XX:GCHeapFreeLimit=10' '-Djava.io.tmpdir=/proj/yunligrp/users/minzhi/gs/gs_tempdir/svpre_tmp' '-cp' '/proj/yunligrp/users/minzhi/svtoolkit/lib/SVToolkit.jar:/proj/yunligrp/users/minzhi/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/proj/yunligrp/users/minzhi/svtoolkit/lib/gatk/Queue.jar' '-cp' '/proj/yunligrp/users/minzhi/svtoolkit/lib/SVToolkit.jar:/proj/yunligrp/users/minzhi/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/proj/yunligrp/users/minzhi/svtoolkit/lib/gatk/Queue.jar' 'org.broadinstitute.sv.apps.CallSampleGender' '-O' '/proj/yunligrp/users/minzhi/gs/freeze6-AA_chr16/svpre_freeze6-AA_chr16_standard_full_single_1-500000over1-500000/md_tempdir/sample_gender.report.txt' '-I' '/proj/yunligrp/users/minzhi/gs/freeze6-AA_chr16/svpre_freeze6-AA_chr16_standard_full_single_1-500000over1-500000/md_tempdir/headers.bam' '-configFile' '/proj/yunligrp/users/minzhi/svtoolkit/conf/genstrip_parameters.txt' '-configFile' '/proj/yunligrp/users/minzhi/gs/Homo_sapiens_assembly38/Homo_sapiens_assembly38.gsparams.txt' '-R' '/proj/yunligrp/users/minzhi/gs/Homo_sapiens_assembly38/Homo_sapiens_assembly38.fasta' '-md' '/proj/yunligrp/users/minzhi/gs/freeze6-AA_chr16/svpre_freeze6-AA_chr16_standard_full_single_1-500000over1-500000/md_tempdir' '-genderBedFile' '/proj/yunligrp/users/minzhi/gs/Homo_sapiens_assembly38/Homo_sapiens_assembly38.gendermask.bed' '-genomeMaskFile' '/proj/yunligrp/users/minzhi/gs/Homo_sapiens_assembly38/Homo_sapiens_assembly38.svmask.fasta' '-L' 'chr16:1-500000' '-ploidyMapFile' '/proj/yunligrp/users/minzhi/gs/freeze6-AA_chr16/supporting_freeze6-AA_chr16/freeze6-AA_chr16_standard_ploidy.map'
ERROR 01:20:43,375 FunctionEdge - Contents of /proj/yunligrp/users/minzhi/gs/freeze6-AA_chr16/svpre_freeze6-AA_chr16_standard_full_single_1-500000over1-500000/logs/SVPreprocess-32080.out:
INFO 01:19:29,576 HelpFormatter - ---------------------------------------------------------
INFO 01:19:29,580 HelpFormatter - Program Name: org.broadinstitute.sv.apps.CallSampleGender
INFO 01:19:29,587 HelpFormatter - Program Args: -O /proj/yunligrp/users/minzhi/gs/freeze6-AA_chr16/svpre_freeze6-AA_chr16_standard_full_single_1-500000over1-500000/md_tempdir/sample_gender.report.txt -I /proj/yunligrp/users/minzhi/gs/freeze6-AA_chr16/svpre_freeze6-AA_chr16_standard_full_single_1-500000over1-500000/md_tempdir/headers.bam -configFile /proj/yunligrp/users/minzhi/svtoolkit/conf/genstrip_parameters.txt -configFile /proj/yunligrp/users/minzhi/gs/Homo_sapiens_assembly38/Homo_sapiens_assembly38.gsparams.txt -R /proj/yunligrp/users/minzhi/gs/Homo_sapiens_assembly38/Homo_sapiens_assembly38.fasta -md /proj/yunligrp/users/minzhi/gs/freeze6-AA_chr16/svpre_freeze6-AA_chr16_standard_full_single_1-500000over1-500000/md_tempdir -genderBedFile /proj/yunligrp/users/minzhi/gs/Homo_sapiens_assembly38/Homo_sapiens_assembly38.gendermask.bed -genomeMaskFile /proj/yunligrp/users/minzhi/gs/Homo_sapiens_assembly38/Homo_sapiens_assembly38.svmask.fasta -L chr16:1-500000 -ploidyMapFile /proj/yunligrp/users/minzhi/gs/freeze6-AA_chr16/supporting_freeze6-AA_chr16/freeze6-AA_chr16_standard_ploidy.map
INFO 01:19:29,594 HelpFormatter - Executing as minzhi@c0816.ll.unc.edu on Linux 3.10.0-957.el7.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_191-b12.
INFO 01:19:29,594 HelpFormatter - Date/Time: 2019/03/01 01:19:29
INFO 01:19:29,594 HelpFormatter - ---------------------------------------------------------
INFO 01:19:29,595 HelpFormatter - ---------------------------------------------------------
INFO 01:19:29,595 CallSampleGender - Opening reference sequence ...
INFO 01:19:29,598 CallSampleGender - Opened reference sequence.
INFO 01:19:29,609 MetaData - Opening metadata ...
INFO 01:19:29,610 MetaData - Adding metadata location /proj/yunligrp/users/minzhi/gs/freeze6-AA_chr16/svpre_freeze6-AA_chr16_standard_full_single_1-500000over1-500000/md_tempdir ...
INFO 01:19:29,626 MetaData - Opened metadata.
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3332)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
at java.lang.StringBuilder.append(StringBuilder.java:136)
at htsjdk.samtools.SAMTextHeaderCodec.advanceLine(SAMTextHeaderCodec.java:139)
at htsjdk.samtools.SAMTextHeaderCodec.decode(SAMTextHeaderCodec.java:94)
at htsjdk.samtools.BAMFileReader.readHeader(BAMFileReader.java:655)
at htsjdk.samtools.BAMFileReader.<init>(BAMFileReader.java:298)
at htsjdk.samtools.BAMFileReader.<init>(BAMFileReader.java:176)
at htsjdk.samtools.SamReaderFactory$SamReaderFactoryImpl.open(SamReaderFactory.java:376)
at htsjdk.samtools.SamReaderFactory$SamReaderFactoryImpl.open(SamReaderFactory.java:202)
at org.broadinstitute.sv.dataset.SAMFileLocation.createSamFileReader(SAMFileLocation.java:97)
at org.broadinstitute.sv.dataset.SAMLocation.createSamFileReader(SAMLocation.java:41)
at org.broadinstitute.sv.dataset.DataSet.initInputFile(DataSet.java:138)
at org.broadinstitute.sv.dataset.DataSet.initialize(DataSet.java:128)
at org.broadinstitute.sv.apps.CallSampleGender.run(CallSampleGender.java:105)
at org.broadinstitute.sv.commandline.CommandLineProgram.execute(CommandLineProgram.java:54)
at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:256)
at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:158)
at org.broadinstitute.sv.commandline.CommandLineProgram.runAndReturnResult(CommandLineProgram.java:29)
at org.broadinstitute.sv.commandline.CommandLineProgram.run(CommandLineProgram.java:25)
at org.broadinstitute.sv.apps.CallSampleGender.main(CallSampleGender.java:92)
INFO 01:20:43,375 QGraph - Writing incremental jobs reports...
The details of this error are different from other "java.lang.OutOfMemoryError: Java heap space" I found in the forum. So is it possible to solve this problem by editing the SVQscript? May I have your suggestions? Thank you very much.
Best regards,
Wusheng