Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Tooling] add playwright tests into azure pipeline #9652

Merged
merged 86 commits into from
May 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
9ea6040
add playwright tests into azure pipeline
brindasasi Mar 6, 2024
43a2328
run in all browsers
brindasasi Mar 6, 2024
2f035ef
install with required deps
brindasasi Mar 6, 2024
ed1ef99
add stages for build & test
brindasasi Mar 6, 2024
ddc9a57
use the artifact built in the prev stage for testing
brindasasi Mar 6, 2024
8640f14
remove deleting front end dependency
brindasasi Mar 6, 2024
e316994
test without prune
brindasasi Mar 6, 2024
ce9c186
test npm install intead of npm ci
brindasasi Mar 6, 2024
af110a7
Revert "test npm install intead of npm ci"
brindasasi Mar 6, 2024
e5c979f
Update azure-pipelines-dev.yml for Azure Pipelines
brindasasi Mar 6, 2024
01e8833
install npm with deps
brindasasi Mar 7, 2024
3668b70
add cypress test
brindasasi Mar 7, 2024
67b4ec3
remove cypress
brindasasi Mar 7, 2024
25abede
try adding build step
brindasasi Mar 7, 2024
b6e56af
target dev site
brindasasi Mar 7, 2024
2f2eebe
Update azure-pipelines-dev.yml for Azure Pipelines
brindasasi Apr 4, 2024
e148a9d
Create .npmrc
brindasasi Apr 4, 2024
dbd4556
move .npmrc file
brindasasi Apr 8, 2024
36e9d5e
update .npmrc custom location var
brindasasi Apr 8, 2024
a2f72ad
fix variable defintion
brindasasi Apr 9, 2024
629ec10
change .npmrc file to point to the new feed
brindasasi Apr 9, 2024
c864b40
add step Cypress installation
brindasasi Apr 9, 2024
72cac43
try using whoami for permission issue
brindasasi Apr 9, 2024
76c8871
Update azure-pipelines-dev.yml for Azure Pipelines
brindasasi Apr 9, 2024
21838fb
Update azure-pipelines-dev.yml for Azure Pipelines
brindasasi Apr 10, 2024
17aea70
remove deps for playwright as we installed them manually
brindasasi Apr 10, 2024
d791a27
ignore HTTPS errors
brindasasi Apr 10, 2024
56215f6
attempt to refactor the tests to accommodate GCKey sign in
brindasasi Apr 11, 2024
914a004
add vars to upload the results to the container
brindasasi Apr 18, 2024
1a254f8
Add steps to store in to storage account
brindasasi Apr 18, 2024
9ee5d1b
Update azure-pipelines-dev.yml for Azure Pipelines
brindasasi Apr 18, 2024
a39271c
add ServiceConnection
brindasasi Apr 19, 2024
ff45afd
Update azure-pipelines-dev.yml for Azure Pipelines
brindasasi Apr 19, 2024
c8bb6e6
add 10 min timeout
brindasasi Apr 19, 2024
2358d8d
run the report with dot option for CI
brindasasi Apr 19, 2024
ecb7fda
Update azure-pipelines-dev.yml for Azure Pipelines
brindasasi Apr 19, 2024
5d4b0ec
don't open html in CI
brindasasi Apr 19, 2024
8bd591d
test without env var
brindasasi Apr 19, 2024
1f415d6
Update azure-pipelines-dev.yml for Azure Pipelines
brindasasi Apr 19, 2024
d81019b
fix quotes
brindasasi Apr 19, 2024
ea6da20
fix quotes for the display name
brindasasi Apr 19, 2024
966f634
fix quotes
brindasasi Apr 19, 2024
e6b8f5a
Update azure-pipelines-dev.yml for Azure Pipelines
brindasasi Apr 26, 2024
1cdd287
open html report if not CI
brindasasi Apr 29, 2024
a837d2a
Revert "test without prune"
brindasasi Apr 30, 2024
bd49741
Merge branch 'main' into 7999-integrate-playwright-azure
brindasasi May 2, 2024
f02304c
remove cypress install
brindasasi May 2, 2024
3be2468
change into pnpm
brindasasi May 2, 2024
61c169b
add workpace root check
brindasasi May 6, 2024
b6d2524
add ignore-workspace-root-check in the root
brindasasi May 6, 2024
eae8c7b
Merge branch 'main' into 7999-integrate-playwright-azure
brindasasi May 6, 2024
5006f2b
add AzureCLI task
brindasasi May 7, 2024
0c64a4c
fix quotes
brindasasi May 7, 2024
20db718
fix formatting
brindasasi May 8, 2024
144efec
fix url
brindasasi May 8, 2024
bf70aa2
no optional
brindasasi May 8, 2024
caece8f
no-frozen-lockfile
brindasasi May 8, 2024
ecd9614
pnpm run build
brindasasi May 8, 2024
6ff4c10
always upload the results
brindasasi May 8, 2024
2d2fba8
fix linting
brindasasi May 9, 2024
824cc35
revert unnecessary changes
brindasasi May 9, 2024
c0e18cf
Merge branch 'main' into 7999-integrate-playwright-azure
brindasasi May 9, 2024
334c530
Merge branch 'main' into 7999-integrate-playwright-azure
brindasasi May 14, 2024
9d56208
remove cypress installation
brindasasi May 23, 2024
d448b48
read the base url from env var
brindasasi May 23, 2024
fcb44a5
Merge branch 'main' into 7999-integrate-playwright-azure
brindasasi May 23, 2024
d00e8b8
add base url as dev pipeline env var
brindasasi May 23, 2024
06ec9d0
Revert "remove cypress installation"
brindasasi May 23, 2024
fc06d03
fix port
brindasasi May 24, 2024
45953aa
fix playwright
brindasasi May 24, 2024
487e21f
try remove cypress again
brindasasi May 24, 2024
0ebdab1
add cypress
brindasasi May 24, 2024
34f1ec6
Revert "try remove cypress again"
brindasasi May 24, 2024
ba0a6b2
Revert "add cypress"
brindasasi May 24, 2024
e9a1ac4
Update azure-pipelines-dev.yml for Azure Pipelines
brindasasi May 24, 2024
46c21cd
Update azure-pipelines-dev.yml for Azure Pipelines
brindasasi May 24, 2024
0c35453
Revert "Update azure-pipelines-dev.yml for Azure Pipelines"
brindasasi May 24, 2024
9daa6f1
Update azure-pipelines-dev.yml for Azure Pipelines
brindasasi May 27, 2024
697badb
increase time out
brindasasi May 27, 2024
f44b406
Revert "attempt to refactor the tests to accommodate GCKey sign in"
brindasasi May 27, 2024
8eddc6a
set time out to 15min
brindasasi May 27, 2024
ef0e613
increase time out
brindasasi May 27, 2024
69a2b2b
get back to 30 mins time out
brindasasi May 28, 2024
b86d62f
disconnect build stage
brindasasi May 29, 2024
de72f21
put back the build dep step
brindasasi May 29, 2024
53d7f44
remove stages
brindasasi May 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .npmrc
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@
auto-install-peers=true
legacy-peer-deps=true
node-linker=hoisted
ignore-workspace-root-check=true
1 change: 1 addition & 0 deletions apps/playwright/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@
# See: https://playwright.dev/docs/test-timeouts
#TEST_TIMEOUT=30000
#EXPECT_TIMEOUT=5000
#BASE_URL=http://localhost:8000
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the app services have this environment variable. There is already an APP_URL variable though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm giving it in here

BASE_URL: "https://dev-talentcloud.tbs-sct.gc.ca"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think App services would need this.
Also this pipeline can be used to run against any env like UAT and is different from APP_URL

9 changes: 7 additions & 2 deletions apps/playwright/playwright.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,16 +20,21 @@ export default defineConfig({
/* Opt out of parallel tests on CI. */
workers: process.env.CI ? 1 : undefined,
/* Reporter to use. See https://playwright.dev/docs/test-reporters */
reporter: [["line"], ["html", { open: "on-failure" }]],
reporter: process.env.CI
? "line"
: [["line"], ["html", { open: "on-failure" }]],
timeout: Number(process.env.TEST_TIMEOUT ?? 3 * 60 * 1000), // 3 minutes
expect: { timeout: Number(process.env.EXPECT_TIMEOUT ?? 10000) }, // 10 seconds
/* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
use: {
/* Base URL to use in actions like `await page.goto('/')`. */
baseURL: "http://localhost:8000",
baseURL: process.env.BASE_URL ?? "http://localhost:8000",

/* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
trace: "retain-on-failure",

/* ignore HTTPS errors when sending network requests */
ignoreHTTPSErrors: true,
},

/* Configure projects for major browsers */
Expand Down
3 changes: 3 additions & 0 deletions infrastructure/.npmrc
petertgiles marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
registry=https://pkgs.dev.azure.com/tbs-sct/53c8b022-82b7-4031-88aa-0d1266fc3bf5/_packaging/TalentCloud_Custom_NPM_Feed/npm/registry/
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw, we still need this.. as we need npm to install the pnpm asfaik

always-auth=true
ignore-workspace-root-check=true
92 changes: 67 additions & 25 deletions infrastructure/azure-pipelines-dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,38 +8,80 @@ resources:
ref: refs/heads/main
variables:
phpVersion: "8.2"

serviceConnectionName: "Talent Cloud Service Connection"
npm_config_user_config: "$(System.DefaultWorkingDirectory)/.npmrc"
dirResults: "$(System.DefaultWorkingDirectory)/apps/playwright/test-results"
resourceGroupName: "rg_tbs_talent_2_PaaS"
vertical: "dev"
storageAccountName: "sttalclddevcacstdl"
storageContainer: "playwright-results"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this container automatically clean up old results?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have set a lifecycle rule on the container level to delete older results. Currently it is more than 30 days old.

pool:
name: "TBS Linux Self Hosted"
jobs:
- job: build_artifact
displayName: Build artifact
pool:
vmImage: ubuntu-22.04
- job: run_playwright_tests
displayName: Run Playwright tests
timeoutInMinutes: 30
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like this is not long enough. It's timing out for me, though it completes in around 25 minutes in GH.

steps:
- checkout: self
clean: true
fetchDepth: 1

- script: infrastructure/bin/set_php_versions.sh ${{ variables.phpVersion }}
displayName: "PHP version"
- task: AzureCLI@2
displayName: "Update NPMRC Config"
inputs:
scriptType: bash
scriptLocation: inlineScript
azureSubscription: $(serviceConnectionName)
inlineScript: |
NPMRC_FILE="${{ variables.npm_config_user_config }}"
echo "Editing .npmrc: ${NPMRC_FILE}"
echo "registry=https://pkgs.dev.azure.com/tbs-sct/53c8b022-82b7-4031-88aa-0d1266fc3bf5/_packaging/TalentCloud_Custom_NPM_Feed/npm/registry/" >> "${NPMRC_FILE}"
- task: npmAuthenticate@0
displayName: "00 - NPM Authenticate"
name: "NPM_Authenticate"
inputs:
workingFile: $(npm_config_user_config)

# -d option to install dev dependencies and -r option to specify the root directory
petertgiles marked this conversation as resolved.
Show resolved Hide resolved
- script: infrastructure/bin/deploy.sh -d -r $(System.DefaultWorkingDirectory)
displayName: Dependencies
env:
API_URI: /graphql
API_PROTECTED_URI: /admin/graphql
# We don't need Cypress in this environment, and so this avoid permission errors of installing it.
# See: https://docs.cypress.io/guides/getting-started/installing-cypress#Skipping-installation
CYPRESS_INSTALL_BINARY: 0
- script: echo ##vso[task.setvariable variable=NPM_CONFIG_USERCONFIG]$(npm_config_user_config)
displayName: "Set Custom .npmrc Location"

- task: ArchiveFiles@2
displayName: "Archive Files"
inputs:
rootFolderOrFile: $(System.DefaultWorkingDirectory)
includeRootFolder: false
archiveFile: "$(Build.ArtifactStagingDirectory)/Application_$(Build.BuildId).zip"
- script: CYPRESS_INSTALL_BINARY=/opt/cypress/13.10.0/cypress.zip pnpm install cypress
displayName: "Install Cypress"

- task: PublishBuildArtifacts@1
displayName: "Publish Artifact: gcDigitalTalent"
- script: pnpm install --no-frozen-lockfile
displayName: "Install Node.js dependencies"

- script: pnpm run build
displayName: "Build dependencies"

- script: npx playwright install
displayName: "Install Playwright browsers"

- script: pnpm run e2e:playwright:chromium
displayName: "Run playwright tests"
env:
CI: "true"
BASE_URL: "https://dev-talentcloud.tbs-sct.gc.ca"
- task: AzureCLI@2
displayName: "Copy Playwright Results to Storage"
condition: always()
inputs:
ArtifactName: gcDigitalTalent
azureSubscription: $(serviceConnectionName)
scriptType: bash
scriptLocation: inlineScript
inlineScript: |
if [ ! $(which az 2> /dev/null) ]; then echo “ERROR: az cli not found” && exit 1; fi
DIR_SRC="${{ variables.dirResults }}"
echo "dirResults: ${{ variables.dirResults }}"
TIMESTAMP="$(date +'%Y-%m-%d %H:%M:%S')"
echo "Timestamp: ${TIMESTAMP}"
if [ -d "${DIR_SRC}" ]; then
SA_RG="${{ variables.resourceGroupName }}"
SA_NAME="${{ variables.storageAccountName }}"
SA_CONTAINER_NAME="${{ variables.storageContainer }}"
SA_CONTAINER_PATH_PREFIX="${{ variables.vertical }}/${TIMESTAMP}"
AZURE_STORAGE_KEY=$(az storage account keys list --resource-group "$SA_RG" --account-name "$SA_NAME" --query "[0].value" --output tsv)
az storage blob upload-batch --destination "${SA_CONTAINER_NAME}" --account-name "${SA_NAME}" --destination-path "${SA_CONTAINER_PATH_PREFIX}" --source "${DIR_SRC}"
else
echo "No 'test-results' directory found"
fi
Loading