Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

usage: collate user/project usage for reporting #352

Closed
Tracked by #350
cmoussa1 opened this issue May 12, 2023 · 1 comment
Closed
Tracked by #350

usage: collate user/project usage for reporting #352

cmoussa1 opened this issue May 12, 2023 · 1 comment
Labels
new feature new feature

Comments

@cmoussa1
Copy link
Member

(this is siphoned off from #350 into its own specific issue)

user+project usage should be able to be collated for reporting. It will probably have to be tackled separately once project support makes its way to the priority plugin (there is currently an open PR on adding project support to the plugin: #294).

Below is a note about collating project usage from an earlier PR by @ryanday36:

regarding usage: We will want to be able to calculate the total usage for all of the jobs that use a given project in a given time frame, but that's different than the usage that is used for fairshare (which has the half-life decay applied to it). As long as jobs are tagged with a project, we can calculate the usage when needed.

@cmoussa1
Copy link
Member Author

cmoussa1 commented Oct 3, 2024

Did some basic testing this morning on our test cluster by adding a project to my association and submitting jobs against that project:

$ flux submit -N2 --setattr=system.project="project_moussa" sleep 30
f826WZbLiFy

Once jobs get sent to flux-accounting, I can both view these records and filter by project name:

$ flux account view-job-records --user=moussa1
UserID     Username   JobID                T_Submit             T_Run                T_Inactive           Nodes      Project             
58985      moussa1    1909181045015552     1724350068.1020253   1724352777.7597826   1724352780.138947    1                              
58985      moussa1    50620875426234368    1727890532.3831322   1727890532.4085314   1727890534.1827712   1          *                   
58985      moussa1    50788430690385920    1727900519.4544554   1727900519.4864368   1727900530.9680524   1          *                   
58985      moussa1    52134140060893184    1727980729.9831917   1727980730.00975     1727980761.6553519   2          project_moussa      
58985      moussa1    52133839564177408    1727980712.072466    1727980712.1001468   1727980743.6596768   2          project_moussa      
58985      moussa1    52133220820451328    1727980675.1922708   1727980675.2186267   1727980686.6592557   1          project_moussa      
58985      moussa1    52132643315122176    1727980640.7707102   1727980640.797432    1727980642.2374218   1          *                   
58985      moussa1    52132341979545600    1727980622.8096876   1727980622.8363187   1727980624.2750702   1          *                   
58985      moussa1    52131256879546368    1727980558.1329465   1727980558.1596496   1727980559.5930007   1          *                   
58985      moussa1    52130687511166976    1727980524.1952376   1727980524.2223897   1727980525.6703641   1          *                   
58985      moussa1    52130380018352128    1727980505.8674264   1727980505.894398    1727980507.3323631   1          *                   
58985      moussa1    52130127353479168    1727980490.807683    1727980490.8335373   1727980492.274509    1          *                   
58985      moussa1    52128819233947648    1727980412.8376987   1727980412.8641489   1727980414.3311288   1          *                   

$ flux account view-job-records --user=moussa1 --project=project_moussa
UserID     Username   JobID                T_Submit             T_Run                T_Inactive           Nodes      Project             
58985      moussa1    52134140060893184    1727980729.9831917   1727980730.00975     1727980761.6553519   2          project_moussa      
58985      moussa1    52133839564177408    1727980712.072466    1727980712.1001468   1727980743.6596768   2          project_moussa      
58985      moussa1    52133220820451328    1727980675.1922708   1727980675.2186267   1727980686.6592557   1          project_moussa

I'll go ahead and close this issue. I can always re-open more specific issues if they come up.

@cmoussa1 cmoussa1 closed this as completed Oct 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new feature new feature
Projects
None yet
Development

No branches or pull requests

1 participant