plugin: add project validation/annotation #443
Conversation
cmoussa1
added
new feature
cmoussa1
force-pushed
the
projects-plugin-annotation
branch
from
212fd9f to
5e79541
Compare
cmoussa1
force-pushed
the
projects-plugin-annotation
branch
2 times, most recently
from
dbccc67 to
52cf8f8
Compare
cmoussa1
changed the title
|
@ryanday36 if you have some time this week, do you think you could give this PR a high-level look? I want to make sure my understanding of how project support is to work in flux-accounting is as expected, or if you were looking for something else. |
|
This looks pretty good. My high level question is, are we going to need to set a default project for every association, or can we just set a global default for all associations that don't have a project? |
|
All associations that do not have a specified project when they are added to the flux-accounting database will have a default project of |
|
That's perfect. Thanks! |
cmoussa1
force-pushed
the
projects-plugin-annotation
branch
from
52cf8f8 to
9532129
Compare
|
Thanks! I'll mark this as ready for review then |
Add project validation to the job.validate callback, which checks that if a project is specified by an association when a job is submitted, it: 1) is a valid project to submit jobs under, and 2) is a valid project for the association to submit jobs under. If either one of these checks fail, the job is rejected with a message including the project name.
Problem: The priority plugin has no way to update jobspec with the project used for the job in the case where an association submits a job under a default project. Add a helper function to the plugin that adds the project name for an association's job to jobspec via jobspec-update under attributes.system.project. Add a jobspec-update of the association's default project name when the job is in the job.new callback. In the event where an association submits a job before any information is loaded to the priority plugin and their job is held in PRIORITY state, add a jobspec-update of the association's default project name in job.state.priority if they are submitting under a default project.
cmoussa1
force-pushed
the
projects-plugin-annotation
branch
from
9532129 to
227f44b
Compare
Problem: There are no tests for project validation and annotation in the priority plugin. Add some tests.
cmoussa1
force-pushed
the
projects-plugin-annotation
branch
from
227f44b to
779da78
Compare
|
Thanks @grondo! Setting MWP here |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #443 +/- ##
==========================================
+ Coverage 82.34% 82.39% +0.04%
==========================================
Files 20 20
Lines 1456 1477 +21
==========================================
+ Hits 1199 1217 +18
- Misses 257 260 +3
|
Background
The priority plugin does not do any validation or annotation for a specified or default project associated with an association's jobs.
This PR looks to add basic support for validating a project if one is specified when submitting a job as well as adding a default project to jobspec when one is not specified. If a project is specified on the command line, it is validated by looking at the association's list of valid projects. If the specified project cannot be found, the job is rejected.
In the case where no project is specified, the association's default project is added to jobspec via
jobspec-update, similar to how the default bank for an association is added to jobspec.Some basic tests are also added that simulate setting project names in multiple cases - where the plugin is still waiting for flux-accounting data to be loaded, where associations set specific projects, or use and/or update their default project.