Un-informative error message from strange linting runs

[fasterius]

Description of the bug

I recently ran into a weird issue with pipeline linting while doing other Git-related pipeline work, and I must have somehow messed up the local clone of ~/.config/nfcore/nf-core/modules. I asked in Slack and the problem was easily solved by removing this directory and allowing the linting to re-download it, but I was asked to create an issue for getting a more informative error message; see the Slack conversation here: https://nfcore.slack.com/archives/CJRH30T6V/p1730226056665409.

Since the issue was so simple to solve, I think the problem here was that I was not even aware that ~/.config/nfcore/nf-core/modules was a possible location for nf-core files to live in - I was troubleshooting and looking into tmp files and such. Maybe I should have known this, though (it's probably in the documentation somewhere?).

Command used and terminal output

$ nf-core pipelines lint


                                          ,--./,-.
          ___     __   __   __   ___     /,-._.--~\
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'

    nf-core/tools version 3.0.2 - https://nf-co.re



INFO     Testing pipeline: .
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Library/Frameworks/Python.framework/Versions/3.12/bin/nf-core:8 in <module>                     │
│                                                                                                  │
│   5 from nf_core.__main__ import run_nf_core                                                     │
│   6 if __name__ == '__main__':                                                                   │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])                         │
│ ❱ 8 │   sys.exit(run_nf_core())                                                                  │
│   9                                                                                              │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/nf_core/__main__ │
│ .py:180 in run_nf_core                                                                           │
│                                                                                                  │
│    177 │   │   │   log.debug(f"Could not check latest version: {e}")                             │
│    178 │   │   stderr.print("\n")                                                                │
│    179 │   # Launch the click cli                                                                │
│ ❱  180 │   nf_core_cli(auto_envvar_prefix="NFCORE")                                              │
│    181                                                                                           │
│    182                                                                                           │
│    183 @tui(                                                                                     │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/rich_click/rich_ │
│ command.py:367 in __call__                                                                       │
│                                                                                                  │
│   364 │   │   # Include this here because I run into a false warning                             │
│   365 │   │   # in the PyCharm IDE otherwise; for some reason PyCharm doesn't                    │
│   366 │   │   # seem to think RichGroups are callable. (No issues with Mypy, though.)            │
│ ❱ 367 │   │   return super().__call__(*args, **kwargs)                                           │
│   368                                                                                            │
│   369                                                                                            │
│   370 class RichCommandCollection(CommandCollection, RichGroup):                                 │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/click/core.py:11 │
│ 57 in __call__                                                                                   │
│                                                                                                  │
│   1154 │                                                                                         │
│   1155 │   def __call__(self, *args: t.Any, **kwargs: t.Any) -> t.Any:                           │
│   1156 │   │   """Alias for :meth:`main`."""                                                     │
│ ❱ 1157 │   │   return self.main(*args, **kwargs)                                                 │
│   1158                                                                                           │
│   1159                                                                                           │
│   1160 class Command(BaseCommand):                                                               │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/rich_click/rich_ │
│ command.py:152 in main                                                                           │
│                                                                                                  │
│   149 │   │   try:                                                                               │
│   150 │   │   │   try:                                                                           │
│   151 │   │   │   │   with self.make_context(prog_name, args, **extra) as ctx:                   │
│ ❱ 152 │   │   │   │   │   rv = self.invoke(ctx)                                                  │
│   153 │   │   │   │   │   if not standalone_mode:                                                │
│   154 │   │   │   │   │   │   return rv                                                          │
│   155 │   │   │   │   │   # it's not safe to `ctx.exit(rv)` here!                                │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/click/core.py:16 │
│ 88 in invoke                                                                                     │
│                                                                                                  │
│   1685 │   │   │   │   super().invoke(ctx)                                                       │
│   1686 │   │   │   │   sub_ctx = cmd.make_context(cmd_name, args, parent=ctx)                    │
│   1687 │   │   │   │   with sub_ctx:                                                             │
│ ❱ 1688 │   │   │   │   │   return _process_result(sub_ctx.command.invoke(sub_ctx))               │
│   1689 │   │                                                                                     │
│   1690 │   │   # In chain mode we create the contexts step by step, but after the                │
│   1691 │   │   # base command has been invoked.  Because at that point we do not                 │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/click/core.py:16 │
│ 88 in invoke                                                                                     │
│                                                                                                  │
│   1685 │   │   │   │   super().invoke(ctx)                                                       │
│   1686 │   │   │   │   sub_ctx = cmd.make_context(cmd_name, args, parent=ctx)                    │
│   1687 │   │   │   │   with sub_ctx:                                                             │
│ ❱ 1688 │   │   │   │   │   return _process_result(sub_ctx.command.invoke(sub_ctx))               │
│   1689 │   │                                                                                     │
│   1690 │   │   # In chain mode we create the contexts step by step, but after the                │
│   1691 │   │   # base command has been invoked.  Because at that point we do not                 │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/click/core.py:14 │
│ 34 in invoke                                                                                     │
│                                                                                                  │
│   1431 │   │   │   echo(style(message, fg="red"), err=True)                                      │
│   1432 │   │                                                                                     │
│   1433 │   │   if self.callback is not None:                                                     │
│ ❱ 1434 │   │   │   return ctx.invoke(self.callback, **ctx.params)                                │
│   1435 │                                                                                         │
│   1436 │   def shell_complete(self, ctx: Context, incomplete: str) -> t.List["CompletionItem"]:  │
│   1437 │   │   """Return a list of completions for the incomplete value. Looks                   │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/click/core.py:78 │
│ 3 in invoke                                                                                      │
│                                                                                                  │
│    780 │   │                                                                                     │
│    781 │   │   with augment_usage_errors(__self):                                                │
│    782 │   │   │   with ctx:                                                                     │
│ ❱  783 │   │   │   │   return __callback(*args, **kwargs)                                        │
│    784 │                                                                                         │
│    785 │   def forward(                                                                          │
│    786 │   │   __self, __cmd: "Command", *args: t.Any, **kwargs: t.Any  # noqa: B902             │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/click/decorators │
│ .py:33 in new_func                                                                               │
│                                                                                                  │
│    30 │   """                                                                                    │
│    31 │                                                                                          │
│    32 │   def new_func(*args: "P.args", **kwargs: "P.kwargs") -> "R":                            │
│ ❱  33 │   │   return f(get_current_context(), *args, **kwargs)                                   │
│    34 │                                                                                          │
│    35 │   return update_wrapper(new_func, f)                                                     │
│    36                                                                                            │
│                                                                                                  │


NEW

19:23
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/nf_core/__main__ │
│ .py:349 in command_pipelines_lint                                                                │
│                                                                                                  │
│    346 │   """                                                                                   │
│    347 │   Check pipeline code against nf-core guidelines.                                       │
│    348 │   """                                                                                   │
│ ❱  349 │   pipelines_lint(ctx, directory, release, fix, key, show_passed, fail_ignored, fail_wa  │
│    350                                                                                           │
│    351                                                                                           │
│    352 # nf-core pipelines download                                                              │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/nf_core/commands │
│ _pipelines.py:133 in pipelines_lint                                                              │
│                                                                                                  │
│   130 │                                                                                          │
│   131 │   # Run the lint tests!                                                                  │
│   132 │   try:                                                                                   │
│ ❱ 133 │   │   lint_obj, module_lint_obj, subworkflow_lint_obj = run_linting(                     │
│   134 │   │   │   directory,                                                                     │
│   135 │   │   │   release,                                                                       │
│   136 │   │   │   fix,                                                                           │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/nf_core/pipeline │
│ s/lint/__init__.py:650 in run_linting                                                            │
│                                                                                                  │
│   647 │   │   if len(module_lint_obj.all_local_components) > 0:                                  │
│   648 │   │   │   module_lint_obj.lint_modules(module_lint_obj.all_local_components, local=Tru   │
│   649 │   │   if len(module_lint_obj.all_remote_components) > 0:                                 │
│ ❱ 650 │   │   │   module_lint_obj.lint_modules(module_lint_obj.all_remote_components, local=Fa   │
│   651 │   if subworkflow_lint_obj is not None:                                                   │
│   652 │   │   # Run the subworkflows lint tests                                                  │
│   653 │   │   if len(subworkflow_lint_obj.all_local_components) > 0:                             │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/nf_core/modules/ │
│ lint/__init__.py:210 in lint_modules                                                             │
│                                                                                                  │
│   207 │   │   │                                                                                  │
│   208 │   │   │   for mod in modules:                                                            │
│   209 │   │   │   │   progress_bar.update(lint_progress, advance=1, test_name=mod.component_na   │
│ ❱ 210 │   │   │   │   self.lint_module(mod, progress_bar, local=local, fix_version=fix_version   │
│   211 │                                                                                          │
│   212 │   def lint_module(                                                                       │
│   213 │   │   self,                                                                              │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/nf_core/modules/ │
│ lint/__init__.py:262 in lint_module                                                              │
│                                                                                                  │
│   259 │   │   │   │   if test_name == "main_nf":                                                 │
│   260 │   │   │   │   │   getattr(self, test_name)(mod, fix_version, self.registry, progress_b   │
│   261 │   │   │   │   else:                                                                      │
│ ❱ 262 │   │   │   │   │   getattr(self, test_name)(mod)                                          │
│   263 │   │   │                                                                                  │
│   264 │   │   │   self.passed += [LintResult(mod, *m) for m in mod.passed]                       │
│   265 │   │   │   warned = [LintResult(mod, *m) for m in mod.warned]                             │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/nf_core/modules/ │
│ lint/module_changes.py:53 in module_changes                                                      │
│                                                                                                  │
│   50 │   )                                                                                       │
│   51 │   modules_repo = nf_core.modules.modules_repo.ModulesRepo(remote_url=module.repo_url,     │
│   52 │                                                                                           │
│ ❱ 53 │   for f, same in modules_repo.component_files_identical(                                  │
│   54 │   │   module.component_name, tempdir, module.git_sha, "modules"                           │
│   55 │   ).items():                                                                              │
│   56 │   │   if same:                                                                            │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/nf_core/synced_r │
│ epo.py:342 in component_files_identical                                                          │
│                                                                                                  │
│   339 │   │   if commit is None:                                                                 │
│   340 │   │   │   self.checkout_branch()                                                         │
│   341 │   │   else:                                                                              │
│ ❱ 342 │   │   │   self.checkout(commit)                                                          │
│   343 │   │   component_files = ["main.nf", "meta.yml"]                                          │
│   344 │   │   files_identical = {file: True for file in component_files}                         │
│   345 │   │   component_dir = self.get_component_dir(component_name, component_type)             │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/nf_core/synced_r │
│ epo.py:263 in checkout                                                                           │
│                                                                                                  │
19:23
│   260 │   │   │   │   log.debug(f"Overwriting local changes in '{self.local_repo_dir}'")         │
│   261 │   │   │   │   self.repo.git.checkout(self.branch, force=True)                            │
│   262 │   │   │   else:                                                                          │
│ ❱ 263 │   │   │   │   raise e                                                                    │
│   264 │                                                                                          │
│   265 │   def component_exists(self, component_name, component_type, checkout=True, commit=Non   │
│   266 │   │   """                                                                                │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/nf_core/synced_r │
│ epo.py:253 in checkout                                                                           │
│                                                                                                  │
│   250 │   │   │   commit (str): Git SHA of the commit                                            │
│   251 │   │   """                                                                                │
│   252 │   │   try:                                                                               │
│ ❱ 253 │   │   │   self.repo.git.checkout(commit)                                                 │
│   254 │   │   except GitCommandError as e:                                                       │
│   255 │   │   │   if (                                                                           │
│   256 │   │   │   │   self.fullname                                                              │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/git/cmd.py:736   │
│ in <lambda>                                                                                      │
│                                                                                                  │
│    733 │   │   :return: Callable object that will execute call _call_process with your argument  │
│    734 │   │   if name[0] == "_":                                                                │
│    735 │   │   │   return LazyMixin.__getattr__(self, name)                                      │
│ ❱  736 │   │   return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)          │
│    737 │                                                                                         │
│    738 │   def set_persistent_git_options(self, **kwargs: Any) -> None:                          │
│    739 │   │   """Specify command line options to the git executable                             │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/git/cmd.py:1316  │
│ in _call_process                                                                                 │
│                                                                                                  │
│   1313 │   │   call.append(dashify(method))                                                      │
│   1314 │   │   call.extend(args_list)                                                            │
│   1315 │   │                                                                                     │
│ ❱ 1316 │   │   return self.execute(call, **exec_kwargs)                                          │
│   1317 │                                                                                         │
│   1318 │   def _parse_object_header(self, header_line: str) -> Tuple[str, str, int]:             │
│   1319 │   │   """                                                                               │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/git/cmd.py:1111  │
│ in execute                                                                                       │
│                                                                                                  │
│   1108 │   │   # END handle debug printing                                                       │
│   1109 │   │                                                                                     │
│   1110 │   │   if with_exceptions and status != 0:                                               │
│ ❱ 1111 │   │   │   raise GitCommandError(redacted_command, status, stderr_value, stdout_value)   │
│   1112 │   │                                                                                     │
│   1113 │   │   if isinstance(stdout_value, bytes) and stdout_as_string:  # could also be output  │
│   1114 │   │   │   stdout_value = safe_decode(stdout_value)                                      │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
GitCommandError: Cmd('git') failed due to: exit code(1)
  cmdline: git checkout 06c8865e36741e05ad32ef70ab3fac127486af48
  stderr: 'error: The following untracked working tree files would be overwritten by checkout:
        modules/nf-core/bismark/align/tests/nextflow_bowtie2.config
        modules/nf-core/bismark/align/tests/nextflow_hisat2.config
        modules/nf-core/bismark/align/tests/nextflow_minimap2.config
        modules/nf-core/bismark/align/tests/tags.yml
        modules/nf-core/bismark/coverage2cytosine/tests/tags.yml
        modules/nf-core/bismark/deduplicate/tests/tags.yml
        modules/nf-core/bismark/genomepreparation/tests/tags.yml
        modules/nf-core/bismark/methylationextractor/tests/tags.yml
        modules/nf-core/bismark/report/tests/tags.yml
        modules/nf-core/bismark/summary/tests/tags.yml
        modules/nf-core/deepvariant/tests/main.nf.test.snap
        modules/nf-core/deepvariant/tests/nextflow-intervals.config
        modules/nf-core/deepvariant/tests/nextflow-non-autosomal-calling.config
        modules/nf-core/deepvariant/tests/nextflow.config
        modules/nf-core/flye/tests/nextflow.config
Please move or remove them before you switch branches.
Aborting'

System information

No response