Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(quantic): first setup of playwright added in Quantic #4597

Open
wants to merge 34 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
0b0f35b
first setup of playwright added in Quantic
mmitiche Oct 25, 2024
5db6159
first setup of playwright added in Quantic
mmitiche Oct 25, 2024
5cbb204
few adjustments
mmitiche Oct 29, 2024
dd0e45b
typo fixed
mmitiche Oct 29, 2024
438ee6c
Merge branch 'master' into SFINT-5768
mmitiche Oct 29, 2024
389aef4
refactored fixtures
mmitiche Oct 30, 2024
56518c5
Merge branch 'SFINT-5768' of https://github.com/coveo/ui-kit into SFI…
mmitiche Oct 30, 2024
4a9bef5
typo fixed
mmitiche Oct 31, 2024
9ce1e39
added checks on page selection
mmitiche Oct 31, 2024
638fdd7
Update packages/quantic/playwright.config.ts
mmitiche Oct 31, 2024
633199a
Merge branch 'SFINT-5768' of https://github.com/coveo/ui-kit into SFI…
mmitiche Oct 31, 2024
ec9389b
config updated
mmitiche Oct 31, 2024
8b977e3
Update packages/quantic/force-app/main/default/lwc/quanticPager/e2e/q…
mmitiche Oct 31, 2024
f9cb518
Update packages/quantic/force-app/main/default/lwc/quanticPager/e2e/q…
mmitiche Oct 31, 2024
95e305e
setup improved
mmitiche Nov 5, 2024
4fbdab8
Merge branch 'SFINT-5768' of https://github.com/coveo/ui-kit into SFI…
mmitiche Nov 5, 2024
18f60ce
comment updated
mmitiche Nov 5, 2024
5657e80
decisions folder added
mmitiche Nov 6, 2024
41d6065
decisions folder added
mmitiche Nov 6, 2024
b7163a5
Merge branch 'master' into SFINT-5768
mmitiche Nov 6, 2024
f99a550
feedback applied
mmitiche Nov 6, 2024
b8228b7
md file updated
mmitiche Nov 6, 2024
2ff3fe7
Merge branch 'SFINT-5768' of https://github.com/coveo/ui-kit into SFI…
mmitiche Nov 6, 2024
a8466e1
comment deleted
mmitiche Nov 6, 2024
4cfbda2
unnecessary await removed
mmitiche Nov 7, 2024
8df0cd4
Update packages/quantic/decisions/0001-testing-strategy.md
mmitiche Nov 12, 2024
804495d
Update packages/quantic/decisions/0001-testing-strategy.md
mmitiche Nov 12, 2024
60472e8
Update packages/quantic/decisions/0001-testing-strategy.md
mmitiche Nov 12, 2024
ba69882
Update packages/quantic/decisions/0001-testing-strategy.md
mmitiche Nov 12, 2024
0cafffa
Update packages/quantic/decisions/0001-testing-strategy.md
mmitiche Nov 12, 2024
4b08487
Update packages/quantic/decisions/0001-testing-strategy.md
mmitiche Nov 12, 2024
04ad5c8
feedback applied
mmitiche Nov 13, 2024
ea8b9d0
Merge branch 'master' of https://github.com/coveo/ui-kit into SFINT-5768
mmitiche Nov 13, 2024
446457f
feedback applied
mmitiche Nov 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 76 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions packages/quantic/.forceignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,7 @@ package.xml
**/__tests__/**
**/testUtils/**/*

# E2E tests
**/e2e/**

**/README.md
5 changes: 4 additions & 1 deletion packages/quantic/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,9 @@ force-app/main/default/staticresources/dompurify
# Test Coverage & Report
coverage
reports
/test-results/
/playwright-report/
/playwright/.cache/

# Cypress artifacts
.cache/
Expand All @@ -58,4 +61,4 @@ docs/out/*
# Environment Variables
.env

.tmp
.tmp
58 changes: 58 additions & 0 deletions packages/quantic/decisions/0001-testing-strategy.md
mmitiche marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Decision Record: Testing Strategy for Quantic

- **Date**: 2024-11-06
- **Status**: accepted

## Context

Our project involves testing Lightning Web Components (LWCs) to ensure quality and reliability. We have decided to use Jest for unit testing to verify component logic and isolated behavior and Playwright for end-to-end (E2E) testing to validate user interactions and integration within the full UI. This document outlines the criteria for deciding what should be tested in each environment.

## Decision

We have established the following guidelines for Jest and Playwright testing in LWCs:

### Jest Unit Tests

Jest will focus on isolated component behavior, covering specific scenarios without involving the browser environment or real user workflows.

Key points for Jest unit tests:

- **Isolated Component Behavior**: Tests should verify the component's behavior in isolation, without dependency on external systems or other components.
- **Mock External Dependencies**: Use mocks to simulate external dependencies, such as capabilities used from the Headless library, ensuring unit tests are fast and isolated.
- **HTML Rendering**: Validate that the component renders the correct HTML based on its state (including but not limited to: internal state, mocked headless state) and input properties.
- **Property Updates**: Ensure component properties update as expected in response to user interactions and lifecycle events.
- **Edge Cases**: Cover edge cases to ensure the component handles unexpected or extreme inputs gracefully.

**Example Jest Test Scenarios**:

- Mock a Headless controller call to test a specific component in isolation.
- Mock a Headless state and ensure the component renders correctly by reflecting the state in the UI.
- Verify that clicking a button updates a specific property, that the rendered HTML reflects this change and the corresponding Headless component controller was called correctly and with the correct parameters.
- Test a lifecycle event, such as `connectedCallback`, to confirm it initializes the component correctly.

### Playwright E2E Tests

Playwright E2E tests will simulate actual user actions to validate the functionality of LWCs within the full user interface and ensure interaction with external systems.

Key points for Playwright E2E tests:

- **User Workflow Testing**: Focus on the essential paths users take to accomplish their goals with a given component, this for all the different Quantic use cases: Search, Insight, Case assist and Recomendations.
- **Simulate User Actions**: Test realistic user actions, such as clicking buttons, entering data, and navigating through the UI.
- **External System Interactions**: Verify interactions with external systems, like Coveo APIs and analytics, to ensure components behave as expected in a real-world environment. This also acts as a double test because it will also notify us by failing the test if the external API contract is broken too because we test for the expected request body but also the expected response as much as possible.

**Example Playwright Test Scenarios**:

- A user uses a Quantic component by interacting with its multiple buttons and triggering Coveo analytics. We test that the expected analytics are sent, that the events are valid, and the response from the analytics API are correct.
- A user navigates through a whole search workflow, interacting with multiple components. We test the expected Search API calls and responses, the analytics calls and response, and also the reactivity of the full search experience.
- Verify the integration between the Quantic components and Salesforce. We test that modifying data through the Salesforce components has an impact and triggers the correct components reactions in a Quantic experience.

## Alternatives Considered

1. **Relying Solely on E2E with Cypress for All Testing**:
- This was used before and is now rejected because using only E2E tests would slow down feedback cycles and increase test maintenance due to the complexity of mocking and setting up end-to-end environments for every component interaction.

## Consequences

- This approach balances efficient, reliable test coverage with manageable test maintenance. Jest allows us to cover specific logic and UI rendering in a lightweight manner, while Playwright provides confidence that the application works as expected for end-users.
- By clearly separating Jest and Playwright responsibilities, we prevent redundancy and maintain test suite performance.
mmitiche marked this conversation as resolved.
Show resolved Hide resolved
- We are also taking the approach of relying more on the tests already done upstream in Headless for example where each component controller is already tested too, we choose not to re-test that once you trigger a Headless action correctly, we trust that it will have the correct result on the state so we don't need to re-test all of that with e2e tests.
3 changes: 3 additions & 0 deletions packages/quantic/decisions/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Architecture Decision Records

This folder contains records of key architecture and technical decisions.
7 changes: 7 additions & 0 deletions packages/quantic/force-app/main/default/.eslintrc.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,13 @@
"rules": {
"@lwc/lwc/no-async-operation": "off"
}
},
{
"files": ["**/e2e/*.ts"],
"parser": "@typescript-eslint/parser",
"rules": {
"no-redeclare": ["error", {"builtinGlobals": false}]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why those rules overrides?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's because Jest defines expect as a global variable, and in the e2e tests we importing expect from playwright,
so eslint creates an error saying that we can't re-declare a global variable.

this rule avoids this error.
An alternative solution would be to do:

import { expect as testExpect } from '@playwright/test';

by changing the alias of the imported expect playwright.

}
}
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@
"../../../../typings/lwc/**/*.d.ts",
"../../../../coveo.d.ts"
],
"exclude": ["**/e2e/**"],
"typeAcquisition": {
"include": ["jest"]
},
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
import {PagerObject} from './pagerObject';
import {quanticBase} from '../../../../../../playwright/fixtures/baseFixture';
import {SearchObject} from '../../../../../../playwright/page-object/searchObject';
import {
searchRequestRegex,
insightSearchRequestRegex,
} from '../../../../../../playwright/utils/requests';
import {InsightSetupObject} from '../../../../../../playwright/page-object/insightSetupObject';
import {useCaseEnum} from '../../../../../../playwright/utils/useCase';

const pagerUrl = 's/quantic-pager';

interface PagerOptions {
numberOfPages: number;
}
type QuanticPagerE2EFixtures = {
pager: PagerObject;
search: SearchObject;
options: Partial<PagerOptions>;
};

type QuanticPagerE2ESearchFixtures = QuanticPagerE2EFixtures & {
urlHash: string;
};

type QuanticPagerE2EInsightFixtures = QuanticPagerE2ESearchFixtures & {
insightSetup: InsightSetupObject;
};

export const testSearch = quanticBase.extend<QuanticPagerE2ESearchFixtures>({
options: {},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these coming from the test.use ?

This is what I was thinking we could do to simplify the useCases too:

test.use({
    useCase: 'search|insight',
});

// And then in your fixture:
pager: async ({page, ..., useCase}, use) => {
    await use(new PagerObject(page, useCase))
}

// And then in your PagerObject, for example instead of having a param on 
async waitForPagerUaAnalytics() {
    if (this.useCase === 'insight') {...}
    if (this.useCase === 'search') {...}
}

Basically is there a way to not have two fixtures, testInsight & testSearch and simply have a testPager fixture that instead gets a useCase as an option?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently the Pager Object is fully independent of the use case, it doesn't care at all about which use case you are is, which is my sense ideal. I'm not understanding why we would want to make it take a use case as a parameter?

urlHash: '',
search: async ({page}, use) => {
await use(new SearchObject(page, searchRequestRegex));
},
pager: async ({page, options, configuration, search, urlHash}, use) => {
await page.goto(urlHash ? `${pagerUrl}#${urlHash}` : pagerUrl);
configuration.configure(options);
mmitiche marked this conversation as resolved.
Show resolved Hide resolved
await search.waitForSearchResponse();

await use(new PagerObject(page));
},
});

export const testInsight = quanticBase.extend<QuanticPagerE2EInsightFixtures>({
options: {},
search: async ({page}, use) => {
mmitiche marked this conversation as resolved.
Show resolved Hide resolved
await use(new SearchObject(page, insightSearchRequestRegex));
},
insightSetup: async ({page}, use) => {
await use(new InsightSetupObject(page));
},
pager: async ({page, options, search, configuration, insightSetup}, use) => {
await page.goto(pagerUrl);
configuration.configure({...options, useCase: useCaseEnum.insight});
await insightSetup.waitForInsightInterfaceInitialization();
await search.performSearch();
await search.waitForSearchResponse();
await use(new PagerObject(page));
},
});

export {expect} from '@playwright/test';
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
import {Locator, Page, Request} from '@playwright/test';
import {isUaCustomEvent} from '../../../../../../playwright/utils/requests';

export class PagerObject {
constructor(public page: Page) {
this.page = page;
}
mmitiche marked this conversation as resolved.
Show resolved Hide resolved

get nextPageButton(): Locator {
return this.page.getByRole('button', {name: /Next Page/i});
}

get previousPageButton(): Locator {
return this.page.getByRole('button', {name: /Previous Page/i});
}

get pageButtons(): Locator {
return this.page.locator('c-quantic-number-button button');
}

pageButton(pageNumber: number): Locator {
return this.pageButtons.nth(pageNumber - 1);
}

async goToLastPage(): Promise<void> {
await this.pageButtons.nth(-1).click();
}

async clickNextPageButton(): Promise<void> {
await this.nextPageButton.click();
}

async clickPreviousPageButton(): Promise<void> {
await this.previousPageButton.click();
}

async clickPageNumberButton(pageNumber: number): Promise<void> {
await this.pageButtons.nth(pageNumber - 1).click();
}

async waitForPagerUaAnalytics(eventValue): Promise<Request> {
const uaRequest = this.page.waitForRequest((request) => {
if (isUaCustomEvent(request)) {
const requestBody = request.postDataJSON?.();
const expectedFields = {
eventType: 'getMoreResults',
eventValue: eventValue,
};
return Object.keys(expectedFields).every(
(key) => requestBody?.[key] === expectedFields[key]
);
}
return false;
});
return uaRequest;
}

async waitForPagerNextUaAnalytics(): Promise<Request> {
return this.waitForPagerUaAnalytics('pagerNext');
}

async waitForPagerPreviousUaAnalytics(): Promise<Request> {
return this.waitForPagerUaAnalytics('pagerPrevious');
}

async waitForPagerNumberUaAnalytics(): Promise<Request> {
return this.waitForPagerUaAnalytics('pagerNumber');
}
}
Loading
Loading