Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce carbon footprint estimation to total energy amount #80

Merged
merged 12 commits into from
Apr 12, 2024

Conversation

tlestang
Copy link
Collaborator

@tlestang tlestang commented Mar 18, 2024

This PR trims down the carbon footprint estimation module carbonFootprint.py.

For the purpose of providing an estimation of carbon footprint savings, cats only needs to know the total carbon footprint, i.e. summed across all devices (gpus, cpus) and memory. This is straightforward to compute given the new jobinfo format introduced in #79:

energy = (
    PUE
    * (runtime.total_seconds() / 3600)
    * sum([(nunits * power) for nunits, power in jobinfo])
    / 1000
    )
best = energy * average_best_ci
now = energy * average_now_ci

where energy is the total energy. I deliberately removed code accounting for energy for a specific device type, since this is not used in cats so far.

This also removes greenAlgorithmsCalculator methods formatting an amount of gCO2 into units of driving, flying...etc. This is probably a nice to have, but AFAIK this is dead code. Anyone feel free to add them back if they wish to make use of them.

Finally, this gets rid of the check_clean_arguments.validate_job_info function since the job information required for the carbon estimation footprint is fully specified by the jobinfo list returned by configure.get_runtime_configuration.

@tlestang tlestang marked this pull request as draft March 18, 2024 14:41
@tlestang tlestang marked this pull request as ready for review March 18, 2024 18:17
@tlestang tlestang force-pushed the tl-refactor-footprint-continued branch from f1c3e4f to 63f7c4c Compare March 18, 2024 18:18
@tlestang tlestang changed the base branch from tl-refactor-footprint to main April 1, 2024 11:52
@@ -219,7 +218,7 @@ def main(arguments=None):
" specify the scheduler with the -s or --scheduler option"
)
sys.exit(1)
config, CI_API_interface, location, duration = get_runtime_config(args)
CI_API_interface, location, duration, jobinfo, PUE = get_runtime_config(args)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The config dict representing the yaml config file can now be confined to the configure module and not appear in the __init__ module. The jobinfo list of (ndevice, power) is all that is require for carbon footprint estimation down the line.

@tlestang tlestang changed the title Trim carbon footprint estimation module Reduce carbon footprint estimation to total energy amount Apr 1, 2024
Copy link
Contributor

@abhidg abhidg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

The name of the profile section is arbitrary, but each profile section
*must* contain one ``cpu`` section, or one ``gpu`` section, or both.
Each hardware type (``cpu`` or ``gpu``) section *must* contain the
``power`` and ``nunits`` sections. The ``model`` section is optional,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we know the power unit?

@andreww andreww mentioned this pull request Apr 9, 2024
@tlestang tlestang merged commit b8ecc22 into main Apr 12, 2024
3 checks passed
@tlestang tlestang deleted the tl-refactor-footprint-continued branch April 12, 2024 07:39
@tlestang tlestang mentioned this pull request Apr 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants