Skip to content

Latest commit

 

History

History
144 lines (114 loc) · 6.2 KB

README.md

File metadata and controls

144 lines (114 loc) · 6.2 KB
Raw

autograded-assignment-template

a template repo for an autograded assignment

Assignment Setup

use this template

  1. click the button to use this template
  2. give your repo a descriptive name for your assignment
  3. keep the repo private
  4. clone your repo into the same directory as your fork of autograder-core
    • e.g. 
      user@machine:~/autograders/$ ls
autograder-core
hello-world-autograder
  5. go ahead and make run_autograder executable: chmod u+x run_autograder

customize the setup script: setup.sh

There are two TODO(you)s that you must do:

  1. make sure you have deploy keys.
    • the instructions for doing that are in setup.sh and in this README.md under make a deploy key
    • ⚠️ if your editor uses CRLF the deploy keys will not work. Ensure that your document is LF only.
  2. set your username and repository for the assignment-specific content

There is one TODO?(you) that you may need to do:

  1. install dependencies
    • valgrind is used for finding memory errors in c++ programs
    • num-utils is used for summarizing coverage of java programs
    • python3.8 or later is required by run_tests.py
    • you can/should add any other dependencies that your tests need

customize the autograder script: run_autograder

There are two TODO(you)s that you must do:

  1. define the expected language
    • currently, the following languages are supported:
      • c++
      • java
  2. list the files that the student must submit

There is one TODO?(you) that you may need to do:

  1. list the files that are provided by the instructor
    • these are files which are needed for testing, but which the student is not responsible for submitting
    • a subset of them may have been given to the students in starter code, but it is not necessary
      • do not assume that files you include here are secret. it may be possible for a clever student to access them.

[optional] customize the ssh configuration: ssh_config

  • you should not need to change anything, but you can if you need to (i.e. you know what you are doing)
  • make sure you have created both
    1. deploy_key for your assignment-specific repo
    2. autograder_core_deploy_key for your fork of the autograder-core repo

write some tests: tests/<language>

the <language> must be either c++ or java. i.e. tests/c++ or tests/java.

  • test specifications go in file(s) named X.tests, e.g. foo.tests, bar.tests, etc.
    • documentation for writing test specifications is at [/path/to/test_spec_documentation](Test Specification Documentation)
    • the basic format of a test specification looks like this: 
      /*
@name: add small numbers
@points: 5
@type: unit
@target: calculator.cpp
*/
<test>
  EXPECT_EQ(add(867, 5309), 6176);
</test>
    • all files containing test specifications (i.e. X.tests) must be in the top level of the directory
      • e.g. like this: tests/c++/foo.tests and tests/c++/bar.tests.
  • i/o test resources go in io_tests/
    • each test should be in its own folder with a descriptive name, e.g. io_tests/example/
    • each test must have two plaintext files (names are arbitrary):
      1. a file that specifies the input to be provided to the program under test through standard input, e.g. input.txt
      2. a file that specifies the expected output that the program under test should print to standard output, e.g. output.txt
    • the actual organization of this folder is abritrary since paths are provided in the test specification
      • but, it is to your benefit to keep it well-organized
  • script (custom) test resources go in script_tests/
    • each test should have a descriptive name, e.g. side_channel_attack.sh
    • complex tests (e.g. those which are composed of several files) should be in their own descriptively-named folders
    • the actual organization of this folder is abritrary since paths are provided in the test specification
      • but, it is to your benefit to keep it well-organized
  • ⚠️ when running tests, the contents of the submission/solution, provided, and tests directories are copied into a directory named testbox (along with all other required files, e.g. from the core)
    • only the required/expected files (as specified in run_autograder) from provided and submission/solution are copied.
    • then, all the X.tests files are concatenated into one file that lives at the root of testbox, e.g. testbox/tests.cpp
    • paths in your tests should be relative to the root of testbox
    • this might confuse your IDE, so don't panic if it says it can't find a reference. when in doubt, test.

test your tests: solution/

  • you can write your own code to test your tests.
  • put your code in solution/
    • the solution/ directory is ignored per .gitignore.
  • when running tests locally, the autograder detects that it is not on gradescope and will pull the "submitted" code from solution.

make a deploy key

ssh-keygen -t ed25519 -C "gradescope"

save it as ./deploy_key with no passphrase

mv ./deploy_key ./secrets/deploy_key to move the secret key to the secrets/ directory

on GitHub, add contents of ./deploy_key.pub to your assignment-specific repo as a new deploy key

add autograder-core deploy key

cp /path/to/autograder_core_deploy_key ./secrets/autograder_deploy_key

if you don't have one, get one!

prepare upload to gradescope

make

creates the file autograder.zip

upload to gradescope

  1. create a programming assignment
  2. upload autograder.zip
  3. base image os: Ubuntu
  4. base image version: 22.04
  5. base image variant: JDK 17
  6. update autograder
  7. test autograder
  8. check settings
    • dates, times
    • submission methods
    • ignored files
    • container specifications
    • timeout