Error when running SVCNVDiscovery in batch-like way: Read count cache file is truncated

Dear Genome STRiP users,

I am running SVCNVDiscovery process in a batch-like way. To be precisely, I have 3418 samples, and I divided them into 33 batches: the first 32 batches with 100 samples each and the 33th batch with 218 samples. Then I completed the SVPreprocess on each batch without any error. Next, I run SVCNVDiscovery on the each batch with the following script.

classpath="${SV_DIR}/lib/SVToolkit.jar:${SV_DIR}/lib/gatk/GenomeAnalysisTK.jar:${SV_DIR}/lib/gatk/Queue.jar"
svpreprocess_dir="/proj/yunligrp/users/minzhi/gs_test_svpreprocess_fulllist_batch_success/${1}"
rundir="/proj/yunligrp/users/minzhi/gs_test_svcnvdiscovery/${1}"

java -Xmx4g -cp ${classpath} \
    org.broadinstitute.gatk.queue.QCommandLine \
    -S ${SV_DIR}/qscript/discovery/cnv/CNVDiscoveryPipeline.q \
    -S ${SV_DIR}/qscript/SVQScript.q \
    -cp ${classpath} \
    -gatk ${SV_DIR}/lib/gatk/GenomeAnalysisTK.jar \
    -configFile ${SV_DIR}/conf/genstrip_parameters.txt \
    -R /proj/yunligrp/users/minzhi/Homo_sapiens_assembly38/Homo_sapiens_assembly38.fasta \
    -I /proj/yunligrp/users/minzhi/gs_script/NWD.recab_${1}.list \
    -genderMapFile /proj/yunligrp/users/minzhi/gs_script/JHS_full_all_male_gender_${1}.map \
    -ploidyMapFile /proj/yunligrp/users/minzhi/gs_script/standard_ploidy.map \
    -md ${svpreprocess_dir}/md_tempdir \
    -runDirectory ${rundir} \
    -jobLogDir ${rundir}/logs \
    -intervalList /proj/yunligrp/users/minzhi/gs_script/reference_chromosomes16_1-500000.list \
    -tilingWindowSize 1000 \
    -tilingWindowOverlap 500 \
    -maximumReferenceGapLength 1000 \
    -boundaryPrecision 100 \
    -minimumRefinedLength 500 \
    -jobRunner Drmaa \
    -gatkJobRunner Drmaa \
    -jobNative "--mem-per-cpu=1000 --time=05:00:00 --nodes=1 --ntasks-per-node=4" \
    -jobQueue general \
    -run \
    || exit 1

However, the 5th batch and the 24th batch always (always mean that I re-run more than 200 times to these two batches) fail with the error as below. And other batches completed without any error.

…
ERROR 02:38:16,120 FunctionEdge - Error:  'java'  '-Xmx2048m'  '-XX:+UseParallelOldGC'  '-XX:ParallelGCThreads=4'  '-XX:GCTimeLimit=50'  '-XX:GCHeapFreeLimit=10'  '-Djava.io.tmpdir=/proj/yunligrp/users/minzhi/gs_test_svcnvdiscovery/4/.queue/tmp'  '-cp' '/proj/yunligrp/users/minzhi/svtoolkit/lib/SVToolkit.jar:/proj/yunligrp/users/minzhi/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/proj/yunligrp/users/minzhi/svtoolkit/lib/gatk/Queue.jar'  '-cp' '/proj/yunligrp/users/minzhi/svtoolkit/lib/SVToolkit.jar:/proj/yunligrp/users/minzhi/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/proj/yunligrp/users/minzhi/svtoolkit/lib/gatk/Queue.jar'  'org.broadinstitute.sv.genotyping.RefineCNVBoundaries'  '-I' '/proj/yunligrp/users/minzhi/gs_test_svcnvdiscovery/4/cnv_stage6/seq_chr16/seq_chr16.merged_headers.bam'  '-O' '/proj/yunligrp/users/minzhi/gs_test_svcnvdiscovery/4/cnv_stage7/seq_chr16/P0007/seq_chr16.merged.brig.vcf'  '-R' '/proj/yunligrp/users/minzhi/Homo_sapiens_assembly38/Homo_sapiens_assembly38.fasta'  '-md' '/proj/yunligrp/users/minzhi/gs_test_svpreprocess_fulllist_batch_success/4/md_tempdir'  '-configFile' '/proj/yunligrp/users/minzhi/svtoolkit/conf/genstrip_parameters.txt' '-configFile' '/proj/yunligrp/users/minzhi/Homo_sapiens_assembly38/Homo_sapiens_assembly38.gsparams.txt'  '-P' 'depth.readCountCacheIgnoreGenomeMask:true'  '-genomeMaskFile' '/proj/yunligrp/users/minzhi/Homo_sapiens_assembly38/Homo_sapiens_assembly38.svmask.fasta' '-genomeMaskFile' '/proj/yunligrp/users/minzhi/Homo_sapiens_assembly38/Homo_sapiens_assembly38.lcmask.fasta'  '-genderMapFile' '/proj/yunligrp/users/minzhi/gs_script/JHS_full_all_male_gender_4.map'  '-ploidyMapFile' '/proj/yunligrp/users/minzhi/gs_script/standard_ploidy.map'  '-vcf' '/proj/yunligrp/users/minzhi/gs_test_svcnvdiscovery/4/cnv_stage4/seq_chr16/seq_chr16.merged.genotypes.vcf.gz'  '-site' '/proj/yunligrp/users/minzhi/gs_test_svcnvdiscovery/4/cnv_stage7/seq_chr16/P0007.sites.list'  '-boundaryPrecision' '100'  '-minimumRefinedLength' '500'  '-maximumReferenceGapLength' '1000'  
ERROR 02:38:16,124 FunctionEdge - Contents of /proj/yunligrp/users/minzhi/gs_test_svcnvdiscovery/4/cnv_stage7/seq_chr16/logs/CNVDiscoveryStage7-8.out:
INFO  02:38:02,034 HelpFormatter - 
…
INFO  02:38:02,927 ReadCountCache - Initializing read count cache with 1 file. 
mInputFile=file:///proj/yunligrp/users/minzhi/gs_test_svpreprocess_fulllist_batch_success/4/md_tempdir/rccache.bin mCurrentSequenceName=chr16; mCurrentPosition=500001
Exception in thread "main" java.lang.RuntimeException: Read count cache file file:///proj/yunligrp/users/minzhi/gs_test_svpreprocess_fulllist_batch_success/4/md_tempdir/rccache.bin is truncated
    at org.broadinstitute.sv.commandline.CommandLineProgram.execute(CommandLineProgram.java:65)
    at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:256)
    at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:158)
    at org.broadinstitute.sv.commandline.CommandLineProgram.runAndReturnResult(CommandLineProgram.java:29)
    at org.broadinstitute.sv.commandline.CommandLineProgram.run(CommandLineProgram.java:25)
    at org.broadinstitute.sv.genotyping.RefineCNVBoundaries.main(RefineCNVBoundaries.java:133)
Caused by: java.lang.RuntimeException: Read count cache file file:///proj/yunligrp/users/minzhi/gs_test_svpreprocess_fulllist_batch_success/4/md_tempdir/rccache.bin is truncated
    at org.broadinstitute.sv.metadata.depth.ReadCountFileReader$ReadCountDataIterator.decodeRow(ReadCountFileReader.java:516)
    at org.broadinstitute.sv.metadata.depth.ReadCountFileReader$ReadCountDataIterator.getReadCacheItems(ReadCountFileReader.java:470)
    at org.broadinstitute.sv.metadata.depth.ReadCountFileReader$ReadCountDataIterator.aggregateSampleReadCounts(ReadCountFileReader.java:476)
    at org.broadinstitute.sv.metadata.depth.ReadCountFileReader.getReadCounts(ReadCountFileReader.java:266)
    at org.broadinstitute.sv.common.ReadCountCache.getReadCounts(ReadCountCache.java:100)
    at org.broadinstitute.sv.genotyping.GenotypingDepthModule.computeRefReadCounts(GenotypingDepthModule.java:295)
    at org.broadinstitute.sv.genotyping.GenotypingDepthModule.computeRefReadCounts(GenotypingDepthModule.java:245)
    at org.broadinstitute.sv.genotyping.GenotypingDepthModule.getReadCounts(GenotypingDepthModule.java:230)
    at org.broadinstitute.sv.genotyping.GenotypingDepthModule.getCnpReadCounts(GenotypingDepthModule.java:217)
    at org.broadinstitute.sv.genotyping.GenotypingDepthModule.genotypeCnp(GenotypingDepthModule.java:141)
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.genotypeCnp(BoundaryRefinementAlgorithm.java:287)
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.refineOneBoundary(BoundaryRefinementAlgorithm.java:633)
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.refineBoundaryStep(BoundaryRefinementAlgorithm.java:553)
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.refineBoundaries(BoundaryRefinementAlgorithm.java:536)
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.processVariant(BoundaryRefinementAlgorithm.java:232)
    at org.broadinstitute.sv.genotyping.RefineCNVBoundaries.run(RefineCNVBoundaries.java:204)
    at org.broadinstitute.sv.commandline.CommandLineProgram.execute(CommandLineProgram.java:54)
    ... 5 more 
INFO  02:38:16,126 QGraph - Writing incremental jobs reports... 
INFO  02:38:16,126 QGraph - 4 Pend, 0 Run, 1 Fail, 7 Done 
INFO  02:38:16,128 QCommandLine - Writing final jobs report... 
INFO  02:38:16,129 QCommandLine - Done with errors
...

Besides, I tried other ways to run SVCNVDiscovery as below:
1. Completed SVPreprocess with regards to all 3418 samples, and running SVCNVDiscovery in the batch-like way (divided batches as above), but the same error happened also at 5th batch and the 24 batch.
2. Completed both of SVPreprocess and SVCNVDiscovery with regards to all 3418 samples, and there is no error in any of these processes.

May I have your suggestions of this situation?

Thank you very much.

Best regards,
Wusheng