Configs

Broadway involves a lot of JSON configs. This page aims to consolidate these configs in one place. Other wiki pages should be linking to these when required.

Course Config

This is an optional JSON object which can be passed on startup to indicate the courses using Broadway and their respective tokens. If this is not provided, Broadway defaults to using what's available in the database. If this is provided, it will overwrite the courses specified in the database.

{
    <course-id>: [<token1>, <token2>, ...],
    ...
}

We allow each course to specify multiple tokens. They can authenticate their requests with any of them. This is so that if they have multiple services interacting with Broadway, each service could be using a different token. That way if a token is compromised, not all of the infrastructure is affected.

Grading Stage Config

This defines a grading stage which is executed in a Docker container. This is used to define which Docker image to run and fine-tune its capabilities and options. This is defined in definitions.py.

{
    "image": <Docker image name>,                              # REQUIRED
    "env": <dict defining additional environment variables>,   # OPTIONAL: default = {}
    "entrypoint": [<cmd>, <arg1>, <arg2>, ...],                # OPTIONAL: default = None
    "networking": <true/false>,                                # OPTIONAL: default = true
    "privileged": <true/false>,                                # OPTIONAL: default = false
    "hostname": <hostname of container>,                       # OPTIONAL: default = "container"
    "timeout": <timeout of container in seconds>,              # OPTIONAL: default = 30 seconds
    "memory": <memory limit of container>,                     # OPTIONAL: default = "2g"
    "logs": <save the logs from this stage>                    # OPTIONAL: default = true
}

Notes:

• image: The image should exist on Docker Hub if not built locally on the machine running the Broadway Graders.
• env: Dict with string key value pairs which specify the environment variables (keys) and values to be injected into the environment. These variables will be private to only this grading stage.
• entrypoint: Container entrypoint
• networking: If set to false, will disable the entire networking stack of the container. Equivalent of using the flag --network none. Only the loopback device is created.
• previledged: If set to true, will run the container in previledged mode.
• timeout: Must be an integer. Time unit is seconds.
• memory: Specify hard limit on the memory available to the container.

Grading Config

This specifies the grading pipelines for an assignment. This is a crucial part of Broadway design because it dictates how grading jobs will be scheduled. This must be uploaded before grading runs can be scheduled for the assignment. This is defined in definitions.py. It uses the grading stage config internally.

{
    "pre_processing_pipeline": [grading_stage_1, grading_stage_2, ...],    # OPTIONAL: default = None
    "student_pipeline": [grading_stage_1, grading_stage_2, ...],           # REQUIRED
    "post_processing_pipeline": [grading_stage_1, grading_stage_2, ...],   # OPTIONAL: default = None
    "env": <dict defining additional environment variables>,               # OPTIONAL: default = {}
}

The environment variables defined here will be global. That means they will be available in all containers belonging to grading runs of this assignment.

Run-time Environment Variables

This is posted to Broadway to trigger an AG run. It specifies environment variables for all the grading jobs that will be created.

{
    "pre_processing_env": <dict representing env for pre-processing job>,   # OPTIONAL: default = None
    "students_env": [<dict representing env for student 1's job>, ...],     # REQUIRED
    "post_processing_env": <dict representing env for pre-processing job>   # OPTIONAL: default = None
}

Notes:

• The environment variables specified here are private to only the grading job they define. They take precedence over global env vars in grading config and stage-specific env vars in stage config (i.e. these env vars will be used in case of conflict).
• pre_processing_env: If this is specified when the underlying grading config does not specify a pre-processing pipeline, it will result in an error.
• students_env: Each dictionary contains environment variables which probably describe template arguments for that student job. This is a good place to put student job specific templates, like STUDENT_ID, etc. The length of this list is the number of grading jobs which will be distributed across grading machines. This also acts as the grading run roster (since it contains all student IDs).
• post_processing_env: If this is specified when the underlying grading config does not specify a post-processing pipeline, it will result in an error.