Skip to content

2.2.0 Test Plan

Jump to bottom
Kevin O'Gorman edited this page Feb 7, 2022 · 8 revisions

2.2.0 QA Checklist

For both upgrades and fresh installs, here is a list of functionality that requires testing. You can use this for copy/pasting into your QA report.

If you have submitted a QA report already for a 2.2.0 release candidate with successful basic server testing and application acceptance testing sections, then you can skip these sections in subsequent reports, unless otherwise indicated by the Release Manager. This is to ensure that you focus your QA effort on the release-specific changes as well as changes since the previous release candidate.

If you are testing the upgrade scenario, you should create source and journalist accounts before performing the upgrade, and delete at least one journalist account - some test cases cover changes to the behaviour for journalist account deletion. See the test cases for #6225 below for more information.

Environment

  • Install target:
  • Tails version:
  • Test Scenario:
  • SSH over Tor:
  • Release candidate:
  • General notes:

Basic Server Testing

  • After installing the testinfra dependencies, all tests in ./securedrop-admin verify are passing:
    • Install dependencies on Admin Workstation with cd ~/Persistent/securedrop && ./securedrop-admin setup -t
    • Run tests with ./securedrop-admin verify (this will take a while)
    • Remove test dependencies: rm -rf admin/.venv3/ && ./securedrop-admin setup
  • QA Matrix checks pass

Command Line User Generation

  • Can successfully add admin user and login

Administration

  • I have backed up and successfully restored the app server following the backup documentation
  • If doing upgrade testing, make a backup on 2.1.0 and restore this backup on this release candidate
  • "Send Test OSSEC Alert" button in the journalist triggers an OSSEC alert and an email is sent
  • Can successfully add journalist account with HOTP authentication

Application Acceptance Testing

Source Interface

Landing page base cases
  • JS warning bar does not appear when using Security Slider high
  • JS warning bar does appear when using Security Slider Low
First submission base cases
  • On generate page, refreshing page produces a new 7-word codename
  • On submit page, empty submissions produce flashed message
  • On submit page, short message submitted successfully
  • On submit page, file greater than 500 MB produces "The connection was reset" in Tor Browser quickly before the entire file is uploaded
  • On submit page, file less than 500 MB submitted successfully
Returning source base cases
  • Nonexistent codename cannot log in
  • Empty codename cannot log in
  • Legitimate codename can log in
  • Returning user can view journalist replies - need to log into journalist interface to test

Journalist Interface

Login base cases
  • Can log in with 2FA tokens
  • incorrect password cannot log in
  • invalid 2fa token cannot log in
  • 2fa immediate reuse cannot log in
  • Journalist account with HOTP can log in
Index base cases
  • Filter by codename works
  • Starring and unstarring works
  • Click select all selects all submissions
  • Selecting all and clicking "Download" works
Individual source page
  • You can submit a reply and a flashed message and new row appears
  • You cannot submit an empty reply
  • Clicking "Delete Source Account" and the source and docs are deleted
  • You can click on a document and successfully decrypt using application private key

Basic Tails Testing

After updating to this release candidate and running securedrop-admin tailsconfig

  • The Updater GUI appears on boot
  • Updating to 2.1.0 is successful

2.2.0 release-specific changes

Web application

  • #6225 Use special "deleted" journalist for associations with deleted users

    • If testing an upgrade scenario:

      • before performing the upgrade, create 2 JI users, (admin1 and admin2, say)
      • on the SI, create a new source with at least one file submission and one message submission
      • log on to the JI as admin1, download the submissions and reply to the source
      • verify that the seen_replies, seen_files, and seen_messages table records admin1's ID
      • log out of the JI, log back in as admin2, and delete admin1
      • verify that the seen_replies, seen_files, and seen_messages table records that referenced admin1's id now have null values for the journalist id.
      • Perform the upgrade to the 2.2.0 release candidate
      • verify that a deleted user was created in the journalists table
      • verify that the entries in the seen_* tables referencing null journalist IDs now reference deleted's ID
      • verify that the Journalist API /replies/<reply id endpoint for the reply lists the deleteds user's UUID

    • If testing an install scenario:

      • create JI 2 admin users (admin1 and admin2, say)
      • verify that only those 2 users exist in the journalists table
      • on the SI, create a new source with at least one file submission and one message submission
      • log on to the JI as admin1, download the submissions and reply to the source
      • verify that the seen_replies, seen_files, and seen_messages table records admin1's ID
      • log out of the JI, log back in as admin2, and delete admin1
      • verify that a deleted user was created in the journalists table, and admin1 is no longer present
      • verify that the entries in the seen_* tables referencing admin1's ID now reference deleted's ID
      • verify that the Journalist API /replies/<reply id endpoint for the reply lists the deleteds user's UUID

  • #6222 Enforce 160-bit HOTP secret length and verify OTP secret length on login.

    • If testing an upgrade scenario (existing 80-bits):
      • log into the JI with an existing HOTP-enabled user and verify that the OTP code is successfully ... verified
      • log out, updated the user's otp secret with a valid secret shorter than 16 chars (with, eg. UPDATE journalists SET otp_secret='JHCOGO7' WHERE username='<your username>';, and verify that login now fails with an error like Login failed. Your 2FA details are invalid...
      • log in as an admin user and reset HOTP for the user above
        • Confirm that the user can log in
        • Confirm that their otp_secret value is now 32 chars long
    • For both fresh installs and upgrades:
      • log in to the JI as an admin user, go to the Admin section and select Add User
        • verify that if is using a Yubikey [HOTP] is checked but no HOTP secret is specified, adding a user fails with a form validation message like The "opt_secret" field is required...
        • verify that if is using a Yubikey [HOTP] is checked but a HOTP secret shorter than 40 chars is specified, adding a user fails with a form validation message like HOTP secrets are 40 chars long...
        • verify that if is using a Yubikey [HOTP] is checked but a non-hexadecimal 40-char HOTP secret is specified , adding a user fails with a flash message like Invalid HOTP secret format:...
        • verify that if is using a Yubikey [HOTP] is checked and a hexadecimal 40-char HOTP secret is specified , adding a user succeeds and the secret can be used to generate a valid token on the next page.

  • #6195 Remove "Refresh codename" feature

    • Verify that the codename refresh button is no longer displayed on /generate
    • Verify that reloading the page generates a new codename, and that that new codename is displayed in the codename reminder after clicking through to /lookup

  • #6130 Display codename reminder only on first login

    • Create a new sourc on the SI, submit a file, and verify that the codename reminder is displayed
    • Log out, then log in again as the same source. Verify that the codename reminder is not displayed.
    • Submit a new message as the source, and verify that the codename remonder is not displayed when the page reloads.

  • #6187 Adds SI and JI header updates

    • On app, verify that /etc/apache2/sites-enabled/source.conf contains the Header directives:

      Header onsuccess unset X-Frame-Options
Header always set X-Frame-Options "DENY"
Header onsuccess unset Referrer-Policy
Header always set Referrer-Policy "same-origin"
Header onsuccess unset Cross-Origin-Opener-Policy
Header always set Cross-Origin-Opener-Policy "same-origin"
Header onsuccess unset Cross-Origin-Embedder-Policy
Header always set Cross-Origin-Embedder-Policy "same-origin"
Header onsuccess unset Cross-Origin-Resource-Policy
Header always set Cross-Origin-Resource-Policy "same-site"

# Set a strict CSP; "default-src 'self'" prevents 3rd party subresources from
# loading and prevents inline script from executing.
Header onsuccess unset Content-Security-Policy
Header always set Content-Security-Policy "default-src 'none'; script-src 'self'; style-src 'self'; img-src 'self'; font-src 'self'; frame-ancestors 'none';"

Header onsuccess unset X-Content-Type-Options
Header always set X-Content-Type-Options "nosniff"

    • On app, verify that /etc/apache2/sites-enabled/journalist.conf contains the Header directives:

      Header onsuccess unset X-Frame-Options
Header always set X-Frame-Options "DENY"
Header onsuccess unset Referrer-Policy
Header always set Referrer-Policy "no-referrer"
Header onsuccess unset X-XSS-Protection
Header always set X-XSS-Protection "1; mode=block"

Header onsuccess unset X-Content-Type-Options
Header always set X-Content-Type-Options "nosniff"
Header onsuccess unset X-Download-Options
Header always set X-Download-Options "noopen"
Header onsuccess unset Content-Security-Policy
Header always set Content-Security-Policy "default-src 'none'; script-src 'self'; style-src 'self'; img-src 'self'; font-src 'self';"

Config changes

Preflight testing

Basic testing

  • Install or upgrade occurs without error
  • Source interface is available and version string indicates it is 2.2.0
  • A message can be successfully submitted

Tails

  • The updater GUI appears on boot
  • The update successfully occurs to 2.2.0
  • After reboot, updater GUI no longer appears
Clone this wiki locally