afterEach block in @cypress/code-coverage/support.js executing concurrently with beforeEach test spec

[mhyrka]

Current behavior

I am trying to implement code-coverage with cypress as outlined here. Using cypress version 12.13.0 and @cypress/code-coverage version 3.10.7 as well as @cypress/webpack-preprocessor version 5.17.1. Additionally I'm using nyc (via npx) to instrument the code. What makes this more confusing is the fact that everything was working properly until it wasn't and I'm unable to get back to a working state.

• full error + stack trace:

TypeError: Cannot set property message of [object DOMException] which has only a getter

Because this error occurred during a `before each` hook we are skipping the remaining tests in the current suite: `Batches home page`
      at Object.modifyErrMsg (http://localhost:58274/__cypress/runner/cypress_runner.js:164139:15)
      at http://localhost:58274/__cypress/runner/cypress_runner.js:149900:84
      at onError (http://localhost:58274/__cypress/runner/cypress_runner.js:159638:42)
      at tryCatcher (http://localhost:58274/__cypress/runner/cypress_runner.js:18744:23)
      at Promise._settlePromiseFromHandler (http://localhost:58274/__cypress/runner/cypress_runner.js:16679:31)
      at Promise._settlePromise (http://localhost:58274/__cypress/runner/cypress_runner.js:16736:18)
      at Promise._settlePromise0 (http://localhost:58274/__cypress/runner/cypress_runner.js:16781:10)
      at Promise._settlePromises (http://localhost:58274/__cypress/runner/cypress_runner.js:16857:18)
      at _drainQueueStep (http://localhost:58274/__cypress/runner/cypress_runner.js:13451:12)
      at _drainQueue (http://localhost:58274/__cypress/runner/cypress_runner.js:13444:9)
      at ../../node_modules/bluebird/js/release/async.js.Async._drainQueues (http://localhost:58274/__cypress/runner/cypress_runner.js:13460:5)
      at Async.drainQueues (http://localhost:58274/__cypress/runner/cypress_runner.js:13330:14)

  2) <some logging>
         "after each" hook for "should display display the table":
     TypeError: Cannot read properties of undefined (reading 'KeyboardEvent')

This error occurred while creating the session. Because the session setup failed, we failed the test.

Because this error occurred during a `after each` hook we are skipping all of the remaining tests.
      at Object.eval [as setup] (webpack://asv-ui/./cypress/support/e2e.js:45:11)

• additional error info: What i learned from executing this in headed mode is that the test begins, the origin block where sso login occurs begins to execute, and before it can finish typing the username, the afterEach block in node_modules/@cypress/code-coverage/support.js executes against the same login command, which throws an error bc the original dom element we were using to type into has disappeared.
  

Desired behavior

Work as expected and generate code coverage statistics.

Test code to reproduce

• due to code confidentiality I can't provide a repo link but these are the pertinent changes. I have confirmed that removing import '@cypress/code-coverage/support' from cypress/support/e2e.js eliminates the error (but doesn't write coverage stats).

// cypress/support/e2e.js
import '@cypress/code-coverage/support'
...
Cypress.Commands.add('login', () => {
  ...
  cy.wrap(getSecret(credentials))
    .then(res => JSON.parse(res.SecretString))
    .then(secret => {
      cy.session(args, () => {
        ...
        cy.origin(authOrigin, { args }, ({ <destructured args> }) => {
            // do sso login here
        })
      })
    })

// mytest.cy.js
describe('home page', function () {
  beforeEach(() => { 
    ...
    cy.login()
    cy.launchURL()
  })
...

// cypress.config.js
module.exports = defineConfig({
  ...
  e2e: {
    setupNodeEvents (on, config) { 
      ...
      on('file:preprocessor', preprocessor())
      require('@cypress/code-coverage/task')(on, config)
    }, 
    ..
    env: {
      codeCoverage: {
        exclude: ['cypress/**/*.*'],
      }
    }
    ...
}

// package.json

...
"cypress-run": "npx [email protected] --instrument cypress run && npx [email protected] report --reporter=lcov --report-dir=./e2e-coverage --reporter=text-summary",
...  
"nyc": {
    "all": true,
    "exclude": [
      "cypress/**/*.*"
    ]
  }

Cypress Version

12.13.0

Node version

16.20.0

Operating System

macOS 13.4

Debug Logs

I can't paste the debug logs for risk of exposing a secret.

Other

No response

[mschile]

Hi @mhyrka 👋, thanks for taking the time to log this issue. It does seem very strange that the afterEach hook is interrupting the test and that it was working initially. I'm not aware of any existing issue that would cause this type of behavior and without some way to reproduce the issue it may be difficult for us to investigate. Would you be able to recreate the steps you posted above in a reproduction repository by forking cypress-test-tiny or by other means? You could try simplifying your test (maybe just have a cy.log('test') as the only command in the test) to see if there is a particular command that is somehow triggering the behavior.

[mhyrka]

Hi @mschile thanks for the prompt response. I believe I have been able to reproduce the error. Code can be found in this git repo. I basically just used the auth0 react sample so I could simulate the SSO that we use and then made similar config changes to those listed above. Just clone and then npm i && npm start to run on your local host and then run npx cypress open

[mhyrka]

Any update on this?

[cacieprins]

Hi @mhyrka , thank you for the reproduction repository. I was able to trigger a similar error there, but it may have ben because of a an auth misconfiguration. Can you supply a working test configuration?

In your recording, it looks like there was an error creating the session in the beforeEach - there is an uncaught TypeError. When Cypress encounters an error in the beforeEach block, it still executes the afterEach block. If you resolve this uncaught exception, do you still see this behavior?

[mhyrka]

Hi @mhyrka , thank you for the reproduction repository. I was able to trigger a similar error there, but it may have ben because of a an auth misconfiguration. Can you supply a working test configuration?

In your recording, it looks like there was an error creating the session in the beforeEach - there is an uncaught TypeError. When Cypress encounters an error in the beforeEach block, it still executes the afterEach block. If you resolve this uncaught exception, do you still see this behavior?

The uncaught TypeError from the recording is because the afterEach block seems to execute before the action type (in my case its typing username + pw) is completed.
TypeError: Cannot set property message of [object DOMException] which has only a getter. The dom object has disappeared. I am seeing the same behavior in my repro repo.

[cacieprins]

Thanks again, @mhyrka ! I think I have a workaround for you - can you try returning cy.session from the wrap callback?

[mhyrka]

  cy.wrap(getSecret(creds))
    .then(res => JSON.parse(res.someVar))
    .then(secret => {
      return cy.session(args, () => {

Updated to return cy.session. Same error.

[KaifUlMajed]

I am glad to find this post because I also have the same issue. I can provide a bit more information on this. Unlike this example, I am using babel with istanbul for instrumentation. My .bablerc file is as follows:-

{
  "presets": ["next/babel"],
  "plugins": ["istanbul"]
}

My login custom command is also similar:-

Cypress.Commands.add("login", () => {
  cy.session(Cypress.env("username"), () => {
    cy.visit("/");
    cy.origin(Cypress.env("okta_domain"), () => {
      cy.get("input[name='identifier']").type(Cypress.env("username"));
      cy.get("input[value='Next']").click();
      cy.get("input[name='credentials.passcode']").type(
        Cypress.env("password"),
      );
      cy.get("input[value='Verify']").click();
      cy.get("input[name='credentials.answer']").type(
        Cypress.env("security_answer"),
      );
      cy.get('[type="submit"]').click();
    });

    cy.get("#page-header").should("have.text", "All Rates");
  });
});

I am using Nextjs and Typescript so I also have this login method in Chainable interface

declare namespace Cypress {
  interface Chainable {
    login(): void;
  }
}

As of today, the package versions are "@cypress/code-coverage": "^3.12.1" and "cypress": "^13.2.0"

Here is the snapshot from cypress while running the login command:-

The Cypress error from console is:-

index-1825f3c7.js:121139  CypressError: `cy.type()` failed because it requires a DOM element.

The subject received was:

  > `undefined`

The previous command that ran was:

  > `cy.get()`

This error occurred while creating the session. Because the session setup failed, we failed the test.

Because this error occurred during a `after each` hook we are skipping all of the remaining tests.
    at Object.eval [as setup] (webpack://fxtrader/./cypress/support/commands.ts:75:0)
From Your Spec Code:
    at Object.eval [as setup] (webpack://fxtrader/./cypress/support/commands.ts:75:0)

Seems like it doesn't wait automatically for the .get() to return before chaining the .type() I was able to get it working by putting explicit .wait(2000) after each button click to let the page load but then this error happens at the end when the final assertion of cy.get("#page-header").should("have.text", "All Rates"); succeeds!

TypeError: Cannot set property message of [object DOMException] which has only a getter

Because this error occurred during a `before each` hook we are skipping all of the remaining tests.
    at Object.modifyErrMsg (http://localhost:3000/__cypress/runner/cypress_runner.js:74883:15)
    at http://localhost:3000/__cypress/runner/cypress_runner.js:133239:76
    at onError (http://localhost:3000/__cypress/runner/cypress_runner.js:144342:42)
    at tryCatcher (http://localhost:3000/__cypress/runner/cypress_runner.js:1807:23)
    at Promise._settlePromiseFromHandler (http://localhost:3000/__cypress/runner/cypress_runner.js:1519:31)
    at Promise._settlePromise (http://localhost:3000/__cypress/runner/cypress_runner.js:1576:18)
    at Promise._settlePromise0 (http://localhost:3000/__cypress/runner/cypress_runner.js:1621:10)
    at Promise._settlePromises (http://localhost:3000/__cypress/runner/cypress_runner.js:1697:18)
From previous event:
    at Promise.longStackTracesCaptureStackTrace [as _captureStackTrace] (http://localhost:3000/__cypress/runner/cypress_runner.js:3486:19)
    at Promise._then (http://localhost:3000/__cypress/runner/cypress_runner.js:1239:17)
    at Promise._passThrough (http://localhost:3000/__cypress/runner/cypress_runner.js:4110:17)
    at Promise.lastly.Promise.finally (http://localhost:3000/__cypress/runner/cypress_runner.js:4119:17)
    at Object.onRunnableRun (http://localhost:3000/__cypress/runner/cypress_runner.js:162946:53)
    at $Cypress.action (http://localhost:3000/__cypress/runner/cypress_runner.js:41015:28)
    at Runnable.run (http://localhost:3000/__cypress/runner/cypress_runner.js:145480:13)
    at next (http://localhost:3000/__cypress/runner/cypress_runner.js:155485:10)
    at http://localhost:3000/__cypress/runner/cypress_runner.js:155529:5
    at timeslice (http://localhost:3000/__cypress/runner/cypress_runner.js:145764:27)

I know the session has created successfully because when I re-run the tests, it says "Session restored" and all tests pass.

[TonyBrobston]

We seem to be having the same issue. Our test redirects to the Okta login page and before it can do anything, it errors out. Removing require('@cypress/code-coverage/task')(on, config); from the cypress.config.js fixes the issue; which defeats the purpose, since we're trying to collect code coverage. Leaving the cypress.config.js alone and running a test that does not require auth, passes, and produces a code coverage report as expected. Seems like a bug on Cypress' end.

[TonyBrobston]

We were able to switch over to the programmatic login and that allowed us to work around the issue. https://docs.cypress.io/guides/end-to-end-testing/okta-authentication#Programmatic-Login

[LuukMoret]

We are facing the same issue as well unfortunately and it also happens during the session/origin/login step

[Daviruss1969]

We are also facing the same issue, the first session creation is fine, without any issue, but as soon as I put require('@cypress/code-coverage/task')(on, config) in my setupNodeEvents, any new session in the current test will instant fail.

Nuxt config (we use vite-plugin-instanbul for code instrumentation) :

import istanbul from 'vite-plugin-istanbul'
export default defineNuxtConfig({
  $development: {
    vite: {
      define: {
        global: 'window'
      },
      plugins: [
        istanbul({
          exclude: ['node_modules', 'test/', 'coverage/', 'cypress/'],
          extension: [ '.js', '.ts', '.vue' ],
          cypress: true
        }),
      ]
    },
  },
...
})

Cypress config :

export default defineConfig({
  defaultCommandTimeout: CypressTimeoutEnum.short,
  viewportWidth: 1920,
  viewportHeight: 1080,
  e2e: {
    baseUrl: 'http://localhost:3000',
    testIsolation: false,
    setupNodeEvents(on, config) {
      require('@cypress/code-coverage/task')(on, config)
      // include any other plugin code...

      // It's IMPORTANT to return the config object
      // with any changed environment variables
      return config
    },
  },
})

[cacieprins]

I understand that this issue often happens with session creation, and completely understand how such a sensitive area of the application can be, and how difficult it can be to provide a full reproduction case.

To help with this, I've made some updates to make sure that our cypress-realworld-app works with Okta. This application makes use of istanbul via the vite-plugin-istanbul, and by making a few small changes it can test an Okta redirection login flow. We would greatly appreciate a fully runnable reproduction, so we can track down what is causing this issue!

To help with this, especially if you are using vite-plugin-istanbul:

• Fork the Cypress Realworld App
• Modify the ./env file to include any credentials you need to use in your test (take care to not commit these!)
• Ensure that the VITE_AUTH_TOKEN_NAME line in ./env is no longer commented out
• Rename src/index.tsx to src/index.default.tsx
• Rename src/index.$YOUR_AUTH_PROVIDER.tsx to src/index.tsx
  • If your auth provider is not present, feel free to contribute an example!
• Make and commit any other modifications you need to, in order to reproduce this issue. You will need to add CYPRESS_COVERAGE=true to the dev:$YOUR_AUTH_PROVIDER script in package.json to enable code coverage.
• Comment here with a link to the fork that reproduces the issue

Taking these steps will help us narrow down what the issue is, so we can help resolve it. Thank you!

[corrideat]

On this issue, I'd also like to add on the Cannot set property message of [object DOMException] which has only a getter part.

That happens in this block:

const modifyErrMsg = (err, newErrMsg, cb) => {
  err.stack = _stack_utils__WEBPACK_IMPORTED_MODULE_5__["default"].normalizedStack(err);
  const newMessage = cb(err.message, newErrMsg);
  const newStack = stackWithReplacedProps(err, {
    message: newMessage
  });
  err.message = newMessage; // <-- Error happens here
  err.stack = newStack;
  return err;
};

It's incorrect to assume that err.message can be set to (in this case, it's a read-only property.

The correct way to do it would be using Object.defineProperty. In this case:

  Object.defineProperty(err, 'message', { value: newMessage });
  Object.defineProperty(err, 'stack ', { value: newStack });

For additional safety, the Object.defineProperty could be wrapped in try / catch.

This error in Cypress makes it harder to debug the underlying issue that's resulting in the error.