Bug: BLE hangs when using git grep in large repo #246
Comments
|
Thank you for the report. It doesn't seem to reproduce in my environment. I tried typing Maybe it is related to the programmable completion loaded in your environment. Could you provide the following information?
$ complete -p git
complete -o bashdefault -o default -o nospace -F __git_wrap__git_main git # <-- This is the result in my environment
$ (shopt -s extdebug; declare -F __git_wrap__git_main) # <-- replace "__git_wrap__git_main" with the name found above
__git_wrap__git_main 3557 /usr/share/bash-completion/completions/git # <-- This is the result in my environment
$ ps uf
$ bleopt complete_auto_complete= |
output of 13:08:13 linux on master ➜ ps uf
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
dhruva 534172 0.0 0.0 9732 3124 pts/12 Ss+ 10:57 0:00 tmux -u
dhruva 542189 0.0 0.0 34168 28624 pts/18 Ss 13:07 0:01 -bash
dhruva 546251 0.0 0.0 11492 3312 pts/18 R+ 15:20 0:00 \_ ps uf
dhruva 534182 0.8 4.8 1601456 1595740 pts/15 Ss 10:57 2:06 -bash
dhruva 546229 106 6.3 2086348 2076284 pts/15 R+ 15:20 0:05 \_ /usr/bin/nawk -v AWKTYPE=nawk -F ? BEGIN { mandb_count = 0; } mode == "mandb"
dhruva 531645 0.0 0.0 35376 29800 pts/13 Ss+ 10:55 0:04 -bash
dhruva 402779 0.0 0.0 34168 28464 pts/11 Ss Oct25 0:00 -bashand the below does help, as in the lag goes away. |
Ah, sorry. It was a typo of C-x C-v.
This information is helpful, but I still don't have an idea why this could take such a long time. I notice that both Bash and nawk have extremely large memory uses of 1.6..2.0 GB. This means that git completion probably produces an extremely large number of candidates or a candidate with an extremely long candidate.
$ ble/complete/mandb/generate-cache git; ls -l "$ret"
|
|
Q1:
This just pastes my clipboard, nothing useful here I believe. Q5: ➜ ble/complete/mandb/generate-cache git; ls -l "$ret"
.rw-rw-r-- 2.0k dhruva 28 Oct 16:11 /home/dhruva/.ble.sh/out/cache.d/1000/complete.mandb/en_US.UTF-8/gitQ6: What's annoying is C-c or C-z doesn't seem to kill whatever is causing the hang! --edit-- |
|
Thank you.
OK, it is because you are using the VS Code terminal, in which C-v cannot be used.
$ ble-bind -f C-t display-shell-version
$ <-- here, press "C-t"
$ git --version
This is somewhat intentional. ble.sh disables signals, SIGINT (C-c), SIGTSTP (C-z), etc., except when the user commands are running.
|
|
Anyway, if we can identify the cause for this specific case, I think I can add a workaround for this specific case. This is the reason I try to collect the information. |
|
Q1: git version: If it is too much of a hassle I might just have to add a |
Thank you for this information. I'll later try setting up the same environment and see if the problem reproduces there. If I cannot reproduce the problem, I'll ask more questions then.
For the time being, if you are fine with turning off the custom git completion, you can put the following line somewhere in bashrc or blerc: complete -F _minimal git |
|
I have set up a new instance of Linux Mint 20, installed git and starship, and tried ble.sh in the Linux repository, but the problem doesn't seem to happen still. Q8: I thought the problem happens in your environment by just typing Mint20-screen0.webmQ9: I am now thinking about the possibility that the problem is caused by some specific aspect of your repository. The Linux repository is a typical large repository, but have you actually tried the Linux repository and confirmed that the problem also arises in the Linux repository? If not, could you also check if the problem arises in the Linux repository? Q10: I have created a special branch $ cd <your-ble.sh-repository>
$ git fetch origin
$ git checkout -b gh246 origin/gh246
$ make
$ make install .... # <-- if neccesary
$ bash # <-- start a new session with gh246
$ cd <your-git-repository>
$ git grep '' .... # <-- test if the problem arises
$ ls -la ~/blesh-gh246-{debug.log,compgen.txt} # <-- check if the log files are createdWhat I would like to request is the content of the file $ exit # <-- end the "gh246" session
$ cd <your-ble.sh-repository>
$ git checkout master
$ make
$ make install .... # <-- if necessary
$ rm -f ~/blesh-gh246-{debug.log,compgen.txt}Q11: Also, it might be possible that other custom configurations in your bashrc/blerc would be causing the problem. Could you check if the problem still happens with just plain ble.sh plus bash-completion? $ cp ~/.bash_history ~/.bash_history.bk # <- Please back up your history file. If you set HISTFILE,
# back up "$HISTFILE" instead of ~/.bash_history
$ PS1='$ ' bash --norc # <- This starts a Bash session without your .bashrc
$ source /usr/share/bash-completion/bash_completion # <- Load bash-completion
$ source /path/to/ble.sh --norc # <- Load ble.sh (replace "/path/to" with the location of your "ble.sh" file)
$ # <- Is the problem reproduced hereafter?After the testing session is closed, you may optionally recover the original history file: $ cp ~/.bash_history.bk ~/.bash_history |
I see in the clip that you're not really using git grep as desired, try
I doubt it, mine is a clean clone of upstream linux stable: And again, I observe this is other large repos like zephyr-RTOS or u-boot as well.
Will get back to you on this. Thanks for your kind support so far. |
|
Q10. Did as you suggested and am attaching the debug.log file. The other compgen file is 100+ MB So is too large. |
|
Hmm..
This might be, because after following as you said plain ble.sh plus bash-completion git grep isn't freezing anymore in the same repo. Wonder if it is tmux or starship of bashrc that is the issue though. |
I tried inputting
OK, thank you for the information. Then, we could confirm that somehow the completion produces an unreasonable amount of completions. I'm not sure why it generates such a large amount of completions in your environment. In my environment, the compgen result for
I tried the same inside tmux, but it still doesn't reproduce. As the problem isn't reproducible in my environment, the only way to identify the problem is to test it in your environment. Sorry but
|
Hmm, I'm interested in the full content of the file. Maybe it contains some hints.
|
|
I'm now checking the implementation of the git completion. As far as I see, the git completion seems to just generate the list of git refs. If that is correct, maybe
$ __git_refs '' '' '' '' ' ' | wc |
|
@DhruvaG2000 Any updates? |
|
The time period of the freezes has gone down a bit for some reason... but it's still there. I have been away from work for some time. Will post the answers to above qns when I get back. |
|
Thanks! I'll wait for you. |
|
|
@DhruvaG2000 Sorry for the delay, and thank you for the result of Q14. This means that what generates the large output of 100+ megabytes is not the implementation of |
I dont have anything but aliases, and few exports. I really need all of them so I don't really think that's relevant. One point I thought is maybe starship was causing the issue, however disabling it doesn't still seem to help |
Thank you for your reply! I am confused because I thought in the answer to Q11 you have told that the plain ble.sh and bash-completion doesn't have the problem, but in this reply to Q12 you seem to be claiming that the difference of these two set of settings (where one has the problem and the other doesn't) is not relevant. Am I missing something? What did the "plain ble.sh plus bash-completion git grep" mean? Also, I'm not sure what kind of aliases and exports you have, but in general some names of aliases can possibly interfere with the internal processing of shell codes, and also exported variables can change the behavior of commands. Based on the currently available information to me, I cannot really conclude your aliases and exports are not related to the present problem. Could you prove it by actually disabling the aliases and exports and see what happens?
Note that I'm not requesting to disable your aliases and exports in your daily sessions, but I just want to get more information on what is going on. So you can just make your aliases and exports back after you see the result of Q12. I will provide the proper fix once we could identify the cause. |
|
Ah, I noticed the log file you sent me now! Thank you, and sorry that I missed it! I'll later check it! |
Sorry, but what I'd like to see is the content of |
ble version:
Bash version:
I use git grep alot, and whenever I go on to type git grep "... it just freezes for a solid 15-20 seconds doing I don't know what. I think this maybe an awk script related issue in ble but I am not sure. One can probably best reproduce this after cloning the linux kernel and try to git grep something.
The text was updated successfully, but these errors were encountered: