FileNotFoundError: [Errno 2] No such file or directory and x_TEA_main.py: error: no such option: --bamsnap

[tkrwy]

Hi,
Thanks for your great tools!
I am running version xTea=0.1.6 installed using conda and installed short reads with the latest version by "git clone https://github.com/parklab/xTea.git"
we generate the running script in case-ctrl model by command:
xtea --case_control --tumor -i sample_id/sample_id.txt -b bam_config/case_ctrl_bam_list.txt -x null -p working/ -o submit_jobs.sh -l rep_lib_annotation/ -r /ada/reference/Homo_sapiens_assembly38.fasta -g gff3/gencode.v33.annotation.gff3 --xtea /ada/tools/xTea/xtea -y 1 -f 10 --slurm -t 0-12:00 -q intel-e5 -n 4 -m 2

after generating run script successfully, we run:
sh submit_jobs.sh

I encountered the following problems, maily in two aspects: file or directory not found and no --bamsnap option:

Filter (on cns) cutoff: 1 and 3 are used!!!
Traceback (most recent call last):
File "/ada/tools/xTea/xtea/x_TEA_main.py", line 638, in
n_clip_cutoff, n_disc_cutoff, sf_output)
File "/ada/tools/xTea/xtea/x_clip_disc_filter.py", line 1540, in call_MEIs_consensus
sf_disc_fa, sf_raw_disc)
File "/ada/tools/xTea/xtea/x_clip_disc_filter.py", line 3324, in collect_clipped_disc_reads
sf_disc_fa_tmp)
File "/ada/tools/xTea/xtea/x_clip_disc_filter.py", line 296, in collect_clipped_disc_reads_of_given_list
with open(sf_candidate_list) as fin_list:
FileNotFoundError: [Errno 2] No such file or directory: 'working/166/L1/candidate_list_barcode.txt'
/ada/tools/miniconda3/envs/mutpipe/lib/python3.6/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: De
precationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
Ave coverage is 6.930000000000001: automatic parameters (clip, disc, clip-disc) with value (1, 1 ,0)
clip,disc,polyA-cutoff is (1, 1, 0)
Traceback (most recent call last):
File "/ada/tools/xTea/xtea/x_TEA_main.py", line 888, in
n_polyA_cutoff, sf_rep_cns, sf_flank, i_flk_len, bin_size, sf_output, b_tumor)
File "/ada/tools/xTea/xtea/x_somatic_calling.py", line 93, in call_somatic_TE_insertion
xclip_disc.sprt_TEI_to_td_orphan_non_td(sf_case_candidates, sf_non_td, sf_td, sf_td_sibling, sf_orphan)
File "/ada/tools/xTea/xtea/x_clip_disc_filter.py", line 823, in sprt_TEI_to_td_orphan_non_td
with open(sf_cns) as fin_cns:
FileNotFoundError: [Errno 2] No such file or directory: 'working/166/L1/candidate_disc_filtered_cns_post_filtering.txt'
/ada/tools/miniconda3/envs/mutpipe/lib/python3.6/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: De
precationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
Traceback (most recent call last):
File "/ada/tools/xTea/xtea/x_TEA_main.py", line 895, in
ccm.parse_high_confident_somatic(sf_candidate_list, sf_raw_somatic, sf_output)
File "/ada/tools/xTea/xtea/x_somatic_calling.py", line 502, in parse_high_confident_somatic
with open(sf_raw_somatic) as fin_somatic:
FileNotFoundError: [Errno 2] No such file or directory: 'working/166/L1/candidate_disc_filtered_cns_post_filtering_somatic.txt'
/ada/tools/miniconda3/envs/mutpipe/lib/python3.6/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: De
precationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
Traceback (most recent call last):
File "/ada/tools/xTea/xtea/x_TEA_main.py", line 1089, in
sf_sites=x_igv.gnrt_sites_single_sample(sf_sites, sf_bam_list)
File "/ada/tools/xTea/xtea/x_igv.py", line 52, in gnrt_sites_single_sample
with open(sf_sites) as fin_sites, open(sf_new_sites,"w") as fout_new:
FileNotFoundError: [Errno 2] No such file or directory: 'working/166/L1/candidate_disc_filtered_cns2.txt'
/ada/tools/miniconda3/envs/mutpipe/lib/python3.6/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: De
precationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
Traceback (most recent call last):
File "/ada/tools/xTea/xtea/x_TEA_main.py", line 1089, in
sf_sites=x_igv.gnrt_sites_single_sample(sf_sites, sf_bam_list)
File "/ada/tools/xTea/xtea/x_igv.py", line 52, in gnrt_sites_single_sample
with open(sf_sites) as fin_sites, open(sf_new_sites,"w") as fout_new:
FileNotFoundError: [Errno 2] No such file or directory: 'working/166/L1/candidate_disc_filtered_cns_high_confident_post_filtering_somatic.txt'
/ada/tools/miniconda3/envs/mutpipe/lib/python3.6/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: De
precationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
Usage: x_TEA_main.py [options]

x_TEA_main.py: error: no such option: --bamsnap
/ada/tools/miniconda3/envs/mutpipe/lib/python3.6/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: De
precationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
Usage: x_TEA_main.py [options]
x_TEA_main.py: error: no such option: --bamsnap

Can you point me where the problem is? Thank you!

[simoncchu]

Again, could you try with full path?

[tkrwy]

Hi chu,
I've tried the full path, but got the same error.
The files under L1 directory is like this:
(mutpipe) [lulu@login01 L1]$ ls
bam_list1.txt bam_list2.txt bam_list3.txt bam_list.txt run_xTEA_pipeline.sh sample_id.txt tmp
It seams that the metioned file in the error message, such as "candidate_list_barcode.txt" etc. are not contained in this directory.
Is there another reason for this error?
Thank you.

[simoncchu]

problem solved by changing -f option. I'll close this issue.

[jianbiaoli]

Hi Simon,
@simoncchu
I also got the last error message as described above. And the option "--bamsnap" is not included in the script file "x_TEA_main.py". Has this problem been solved?

$python ${XTEA_PATH}"x_TEA_main.py" --igv --bamsnap --single_sample -p ${PREFIX}"tmp/igv" -b ${PREFIX}/bam_list3.txt -i ${PREFIX}"candidate_disc_filtered_cns2.txt"  --ref ${REF} -e 1000 -o ${PREFIX}"tmp/igv/bamsnap_screenshot.txt"
$python ${XTEA_PATH}"x_TEA_main.py" --igv --bamsnap --single_sample -p ${PREFIX}"tmp/igv" -b ${PREFIX}/bam_list3.txt -i ${PREFIX}"candidate_disc_filtered_cns_high_confident_post_filtering_somatic.txt"  --ref ${REF} -e 1000 -o ${PREFIX}"tmp/igv/bamsnap_screenshot_somatic.txt"

x_TEA_main.py: error: no such option: --bamsnap
Usage: x_TEA_main.py [options]

[simoncchu]

@jianbiaoli which -f do you set? If you set -f 5907 or 1555, does it contain the command you posted?

[jianbiaoli]

@jianbiaoli which -f do you set? If you set -f 5907 or 1555, does it contain the command you posted?

I tried both -f options using case_control mode. All the scripts contain the command as shown above.

[simoncchu]

@jianbiaoli I confirmed this issue. It's a version issue. I'll fix this in the next release. For now, you can ignore the error, it doesn't affect the calling.

[dongjuleem]

@simoncchu Thank you for your kind words. I want to ask you if I understood correctly that you said that the error is not related to proceeding with the calling. Would it be appropriate to use the positions information obtained from the "candidate_disc_filtered_cns_high_confident_post_filtering_somatic.txt" file in the result interpretation instead of a VCF file?

[simoncchu]

@dongjuleem yes, you can use that one, which just misses the gene annotation and field annotation.

[dongjuleem]

@simoncchu thank you for explanation:)
I am currently trying to detect newly arisen somatic mutations using the --case_control option in xTea. I'm curious why the somatic mutations observed in the case sample(compared to the control sample) are not included in the results obtained by running xTea in single mode with case sample. In my understanding, when obtaining the results for the case sample using xTea in single mode, both germline and somatic mutation information should be included, is that correct?

[simoncchu]

@dongjuleem The main difference is internally the somatic module use lower cutoff (considering the tumor purity, as well as false positives could be further filtered out when checking the ctrl sample). You can set low cutoff with --user (see readme page) to set lower cutoff under the germline mode which will report more candidates, but meanwhile more false positives will be reported.

[dongjuleem]

@simoncchu
Thank you for your response. I would like to get advice on how to confirm newly occurring somatic TE insertions.

1. As per your suggestion, I would like to use the --user option in germline mode to obtain more TE candidates and then determine if the results from the somatic mode align with the results from the germline mode. This process would increase the likelihood of verifying the inclusion with a larger pool of candidates, but it would also increase the probability of higher false positives.

2. use the results solely from the somatic mode without comparing of the germline mode.

I would like to seek advice on which of the two approaches is recommended. Finally, I will validate true positives using IGV.

[simoncchu]

I directly use the somatic mode for my research, but it depends on your specific requirements.

[shilpasy]

@jianbiaoli I confirmed this issue. It's a version issue. I'll fix this in the next release. For now, you can ignore the error, it doesn't affect the calling.

Hi @simoncchu
I have tried with the latest version, as well as the Dockerfile version. Both times i got this error (no such option as --bamsnap).
It's mentioned that it doesn't affect the calling, but all my .vcf files are blank. They only have the header lines.
I also checked with various other -f options (thought I'm not exactly sure what does it mean to alter these!)
Please let me know,
thank you so much.