Error in rule plot_assembly_stats

[s-junguy]

Hi,
l've got an error in plot_assembly_stats. I use Illumina paired end libraries (10 GB per library). This is my submission script:

#!/bin/bash
#SBATCH --cpus-per-task=128
#SBATCH --time=72:00:00
#SBATCH --partition=pinky
#SBATCH --ntasks=1
#SBATCH -o nbis.%j.%N.out
#SBATCH -e nbis.%j.%N.err
source /home/user/miniconda3/bin/activate /home/user/miniconda3/envs/nbis-meta #=conda activate nbis-meta
cd nbis-meta
snakemake --use-conda --configfile config/testdatenconfig.yaml --cores 128

Here is the error message:

localrule aggregate_featurecount:
    input: metaphorsamplesresults/assembly/all/mapping/sample1_1_pe.fc.clean.tsv, metaphorsamplesresults/assembly/all/mapping/sample2_1_pe.fc.clean.tsv
    output: metaphorsamplesresults/annotation/all/gene_counts.tsv
    jobid: 58
    reason: Missing output files: metaphorsamplesresults/annotation/all/gene_counts.tsv; Input files updated by another job: metaphorsamplesresults/assembly/all/mapping/sample2_1_pe.fc.clean.tsv, metaphorsamplesresults/assembly/all/mapping/sample1_1_pe.fc.clean.tsv
    wildcards: assembly=all
    resources: tmpdir=/tmp

/home/user/nbis-meta/.snakemake/scripts/tmpl5twcxfc.assembly_utils.py:97: FutureWarning: Calling int on a single element Series is deprecated and will raise a TypeError in the future. Use int(ser.iloc[0]) instead
  total_size = int(df.sum())
/home/user/nbis-meta/.snakemake/scripts/tmpl5twcxfc.assembly_utils.py:77: FutureWarning: Calling int on a single element Series is deprecated and will raise a TypeError in the future. Use int(ser.iloc[0]) instead
  size = int(df.sum())
/home/user/nbis-meta/.snakemake/scripts/tmpl5twcxfc.assembly_utils.py:60: FutureWarning: Calling int on a single element Series is deprecated and will raise a TypeError in the future. Use int(ser.iloc[0]) instead
  s = int(df.loc[df.length >= l].sum())
/home/user/nbis-meta/.snakemake/scripts/tmpl5twcxfc.assembly_utils.py:61: FutureWarning: Calling int on a single element Series is deprecated and will raise a TypeError in the future. Use int(ser.iloc[0]) instead
  p = int(df.loc[df.length >= l].sum()) / float(df.sum()) * 100
/home/user/nbis-meta/.snakemake/scripts/tmpl5twcxfc.assembly_utils.py:61: FutureWarning: Calling float on a single element Series is deprecated and will raise a TypeError in the future. Use float(ser.iloc[0]) instead
  p = int(df.loc[df.length >= l].sum()) / float(df.sum()) * 100
/home/user/nbis-meta/.snakemake/scripts/tmpl5twcxfc.assembly_utils.py:97: FutureWarning: Calling int on a single element Series is deprecated and will raise a TypeError in the future. Use int(ser.iloc[0]) instead
  total_size = int(df.sum())
/home/user/nbis-meta/.snakemake/scripts/tmpl5twcxfc.assembly_utils.py:77: FutureWarning: Calling int on a single element Series is deprecated and will raise a TypeError in the future. Use int(ser.iloc[0]) instead
  size = int(df.sum())
[Sun Oct  8 12:31:33 2023]
Finished job 58.
63 of 85 steps (74%) done
/home/user/nbis-meta/.snakemake/scripts/tmpl5twcxfc.assembly_utils.py:60: FutureWarning: Calling int on a single element Series is deprecated and will raise a TypeError in the future. Use int(ser.iloc[0]) instead
  s = int(df.loc[df.length >= l].sum())
/home/user/nbis-meta/.snakemake/scripts/tmpl5twcxfc.assembly_utils.py:61: FutureWarning: Calling int on a single element Series is deprecated and will raise a TypeError in the future. Use int(ser.iloc[0]) instead
  p = int(df.loc[df.length >= l].sum()) / float(df.sum()) * 100
/home/user/nbis-meta/.snakemake/scripts/tmpl5twcxfc.assembly_utils.py:61: FutureWarning: Calling float on a single element Series is deprecated and will raise a TypeError in the future. Use float(ser.iloc[0]) instead
  p = int(df.loc[df.length >= l].sum()) / float(df.sum()) * 100
/home/user/nbis-meta/.snakemake/scripts/tmpl5twcxfc.assembly_utils.py:97: FutureWarning: Calling int on a single element Series is deprecated and will raise a TypeError in the future. Use int(ser.iloc[0]) instead
  total_size = int(df.sum())
/home/user/nbis-meta/.snakemake/scripts/tmpl5twcxfc.assembly_utils.py:77: FutureWarning: Calling int on a single element Series is deprecated and will raise a TypeError in the future. Use int(ser.iloc[0]) instead
  size = int(df.sum())
/home/user/nbis-meta/.snakemake/scripts/tmpl5twcxfc.assembly_utils.py:60: FutureWarning: Calling int on a single element Series is deprecated and will raise a TypeError in the future. Use int(ser.iloc[0]) instead
  s = int(df.loc[df.length >= l].sum())
/home/user/nbis-meta/.snakemake/scripts/tmpl5twcxfc.assembly_utils.py:61: FutureWarning: Calling int on a single element Series is deprecated and will raise a TypeError in the future. Use int(ser.iloc[0]) instead
  p = int(df.loc[df.length >= l].sum()) / float(df.sum()) * 100
/home/user/nbis-meta/.snakemake/scripts/tmpl5twcxfc.assembly_utils.py:61: FutureWarning: Calling float on a single element Series is deprecated and will raise a TypeError in the future. Use float(ser.iloc[0]) instead
  p = int(df.loc[df.length >= l].sum()) / float(df.sum()) * 100
[Sun Oct  8 12:31:33 2023]
Finished job 13.
64 of 85 steps (75%) done
Select jobs to execute...

[Sun Oct  8 12:31:33 2023]
localrule plot_assembly_stats:
    input: metaphorsamplesresults/report/assembly/assembly_stats.tsv, metaphorsamplesresults/report/assembly/assembly_size_dist.tsv, metaphorsamplesresults/assembly/sample1/mapping/flagstat.tsv, metaphorsamplesresults/assembly/all/mapping/flagstat.tsv, metaphorsamplesresults/assembly/sample2/mapping/flagstat.tsv
    output: metaphorsamplesresults/report/assembly/assembly_stats.pdf, metaphorsamplesresults/report/assembly/assembly_size_dist.pdf, metaphorsamplesresults/report/assembly/alignment_frequency.pdf
    jobid: 12
    reason: Missing output files: metaphorsamplesresults/report/assembly/assembly_stats.pdf; Input files updated by another job: metaphorsamplesresults/assembly/all/mapping/flagstat.tsv, metaphorsamplesresults/report/assembly/assembly_stats.tsv, metaphorsamplesresults/assembly/sample1/mapping/flagstat.tsv, metaphorsamplesresults/report/assembly/assembly_size_dist.tsv, metaphorsamplesresults/assembly/sample2/mapping/flagstat.tsv
    resources: tmpdir=/tmp

Activating conda environment: .snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_
[Sun Oct  8 12:31:37 2023]
Finished job 56.
65 of 85 steps (76%) done
[NbConvertApp] ERROR | Notebook JSON is invalid: Additional properties are not allowed ('id' was unexpected)

Failed validating 'additionalProperties' in code_cell:

On instance['cells'][0]:
{'cell_type': 'code',
 'execution_count': None,
 'id': '637abe3f',
 'metadata': {'tags': ['snakemake-job-properties']},
 'outputs': ['...0 outputs...'],
 'source': '\n'
           '######## snakemake preamble start (automatically inserted, do '
           'n...'}
Traceback (most recent call last):
  File "/home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/bin/jupyter-nbconvert", line 10, in <module>
    sys.exit(main())
  File "/home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/jupyter_core/application.py", line 277, in launch_instance
    return super().launch_instance(argv=argv, **kwargs)
  File "/home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/traitlets/config/application.py", line 1043, in launch_instance
    app.start()
  File "/home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/nbconvert/nbconvertapp.py", line 414, in start
    self.convert_notebooks()
  File "/home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/nbconvert/nbconvertapp.py", line 589, in convert_notebooks
    self.convert_single_notebook(notebook_filename)
  File "/home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/nbconvert/nbconvertapp.py", line 555, in convert_single_notebook
    output, resources = self.export_single_notebook(
  File "/home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/nbconvert/nbconvertapp.py", line 481, in export_single_notebook
    output, resources = self.exporter.from_filename(
  File "/home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/nbconvert/exporters/exporter.py", line 201, in from_filename
    return self.from_file(f, resources=resources, **kw)
  File "/home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/nbconvert/exporters/exporter.py", line 220, in from_file
    return self.from_notebook_node(
  File "/home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/nbconvert/exporters/notebook.py", line 36, in from_notebook_node
    nb_copy, resources = super().from_notebook_node(nb, resources, **kw)
  File "/home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/nbconvert/exporters/exporter.py", line 154, in from_notebook_node
    nb_copy, resources = self._preprocess(nb_copy, resources)
  File "/home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/nbconvert/exporters/exporter.py", line 352, in _preprocess
    nbc, resc = preprocessor(nbc, resc)
  File "/home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/nbconvert/preprocessors/base.py", line 48, in __call__
    return self.preprocess(nb, resources)
  File "/home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/nbconvert/preprocessors/execute.py", line 100, in preprocess
    self.preprocess_cell(cell, resources, index)
  File "/home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/nbconvert/preprocessors/execute.py", line 121, in preprocess_cell
    cell = self.execute_cell(cell, index, store_history=True)
  File "/home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/jupyter_core/utils/__init__.py", line 166, in wrapped
    return loop.run_until_complete(inner)
  File "/home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/nbclient/client.py", line 1058, in async_execute_cell
    await self._check_raise_for_error(cell, cell_index, exec_reply)
  File "/home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/nbclient/client.py", line 914, in _check_raise_for_error
    raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
------------------
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
plt.style.use('ggplot')
sns.set_style("whitegrid")
%matplotlib inline
%config InlineBackend.figure_format = 'svg'
------------------

----- stderr -----
/home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/pandas/_testing.py:24: FutureWarning: In the future `np.bool` will be defined as the corresponding NumPy scalar.
  import pandas._libs.testing as _testing
------------------

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[2], line 1
----> 1 import pandas as pd
      2 import seaborn as sns
      3 import matplotlib.pyplot as plt

File ~/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/pandas/__init__.py:180
    177 from pandas.io.json import _json_normalize as json_normalize
    179 from pandas.util._tester import test
--> 180 import pandas.testing
    181 import pandas.arrays
    183 # use the closest tagged version if possible

File ~/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/pandas/testing.py:5
      1 """
      2 Public testing utility functions.
      3 """
----> 5 from pandas._testing import (
      6     assert_extension_array_equal,
      7     assert_frame_equal,
      8     assert_index_equal,
      9     assert_series_equal,
     10 )
     12 __all__ = [
     13     "assert_extension_array_equal",
     14     "assert_frame_equal",
     15     "assert_series_equal",
     16     "assert_index_equal",
     17 ]

File ~/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/pandas/_testing.py:24
     16 from numpy.random import rand, randn
     18 from pandas._config.localization import (  # noqa:F401
     19     can_set_locale,
     20     get_locales,
     21     set_locale,
     22 )
---> 24 import pandas._libs.testing as _testing
     25 from pandas._typing import FilePathOrBuffer, FrameOrSeries
     26 from pandas.compat import _get_lzma_file, _import_lzma

File pandas/_libs/testing.pyx:10, in init pandas._libs.testing()

File ~/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_/lib/python3.8/site-packages/numpy/__init__.py:305, in __getattr__(attr)
    300     warnings.warn(
    301         f"In the future `np.{attr}` will be defined as the "
    302         "corresponding NumPy scalar.", FutureWarning, stacklevel=2)
    304 if attr in __former_attrs__:
--> 305     raise AttributeError(__former_attrs__[attr])
    307 # Importing Tester requires importing all of UnitTest which is not a
    308 # cheap import Since it is mainly used in test suits, we lazy import it
    309 # here to save on the order of 10 ms of import time for most users
    310 #
    311 # The previous way Tester was imported also had a side effect of adding
    312 # the full `numpy.testing` namespace
    313 if attr == 'testing':

AttributeError: module 'numpy' has no attribute 'bool'.
`np.bool` was a deprecated alias for the builtin `bool`. To avoid this error in existing code, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

[Sun Oct  8 12:32:11 2023]
Error in rule plot_assembly_stats:
    jobid: 12
    input: metaphorsamplesresults/report/assembly/assembly_stats.tsv, metaphorsamplesresults/report/assembly/assembly_size_dist.tsv, metaphorsamplesresults/assembly/sample1/mapping/flagstat.tsv, metaphorsamplesresults/assembly/all/mapping/flagstat.tsv, metaphorsamplesresults/assembly/sample2/mapping/flagstat.tsv
    output: metaphorsamplesresults/report/assembly/assembly_stats.pdf, metaphorsamplesresults/report/assembly/assembly_size_dist.pdf, metaphorsamplesresults/report/assembly/alignment_frequency.pdf
    conda-env: /home/user/nbis-meta/.snakemake/conda/a30559dadce192a6c759fe64ff0eb36b_

[Sun Oct  8 12:35:43 2023]
Finished job 81.
66 of 85 steps (78%) done
[Sun Oct  8 12:36:23 2023]
Finished job 48.
67 of 85 steps (79%) done
[Sun Oct  8 13:04:18 2023]
Finished job 65.
68 of 85 steps (80%) done
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

There is no log file provided. Is there a way to avoid using plot_assembly_stats since I do not need it?