Updating broken notebooks

[Soap2G]

This PR addresses a few straightforward code inconsistencies related to the notebooks.

List of notebooks fixed is at this link

Upgrades

I have:

• Updated the environment.yml to account for missing packages in the docker image (e.g. hist and dask), which were causing the failure of the pipeline
• Updated a few deprecated expressions (e.g. data_all = data_all.append(data) ---> data_all = pd.concat([data_all, data], ignore_index=True)) related to old packages.
• Added a standard way to integrate packages' versions from the environment.yml file, through a setup cell that can be seen in this example notebook.
  The script is like this:

import yaml
import subprocess
import sys

# Path to your binder/environment.yml file
environment_file = "../../binder/environment.yml"

# Packages you want to install
required_packages = ['coffea', 'hist', 'dask', 'numpy', 'matplotlib']

# Load the environment.yml file
with open(environment_file, 'r') as file:
    environment_data = yaml.safe_load(file)

# Extract dependencies
dependencies = environment_data.get('dependencies', [])

# Create a list to hold the packages with versions
install_packages = []

# Find the versions for the required packages
for dep in dependencies:
    # Check if the dependency is a string (package name)
    if isinstance(dep, str):
        for package in required_packages:
            if dep.startswith(package):
                install_packages.append(dep)

# Install packages using pip
if install_packages:
    print(f"Installing packages: {install_packages}")
    subprocess.run([sys.executable, "-m", "pip", "install", "--upgrade", "--user"] + install_packages)
else:
    print("No matching packages found in environment.yml.")

ToDos

There are two items yet to be addressed:

1. 13-TeV-examples/uproot_python/ttZ_ML_from_root.ipynb takes a LOT of time to run. This is due to the large number of events that have to be processed. It is ~1h run time and then it crashes. I'm not sure it's worth to keep it in the list of notebooks to be run.
2. There is a notebook, 13-TeV-examples/uproot_python/Dark_Matter_Machine_Learning.ipynb, that has blank spaces, to be filled by the user. This cannot be run by definition, so I suggest we take it out of the list.
3. The Coffea notebook has a nontrivial connectivity issue, probably due to an outdated version of the package fsspec, for which come samples return a FileNotFound error even if they do exist.

I'll leave these items to @AntonioJC and Reina, given that they are already working on that.

[marianaiv]

Hi Giovanni! Thanks for all the work. I like the script thing in the notebooks :)
As for the notebooks you think we should remove, I'm currently going through all the notebooks, seeing if they're worth keeping and writing some documentation on the ones that are. My point is: we can discuss the notebooks worth keeping in my next PR.

[AntonioJC]

Thanks for the work @Soap2G! Please find some comments

[AntonioJC]

Regarding the notebooks discussion, I'll address that in @marianaiv's PR, thanks!

[Soap2G]

Comments are now addressed.
There are still 3 notebooks that do not pass the pipeline:

• Coffea: issue with the dask processing.
• DM: unfinished cells that are supposed to be filled by the user.
• ttZ_ML_from_root: computation takes too long

Since these notebooks are already mentioned as faulty in the description, I would go ahead and address these two issues in the next iteration, since PR #58 is depending on this.