ChunkLoadError > Loading chunk cypress-support-file failed #28644
Comments
|
We made an update from webpack 4 to webpack 5 in 12.17.4, I wonder if it has anything to do with that upgrade. https://docs.cypress.io/guides/references/changelog#12-17-4 |
jennifer-shehane
added
the
stage: needs information
|
We are getting the same error also intermittently. I don't think we had it as far back as 12.17.4 but I can't say that for certain. It only seems to happen toward the end of our Component Test runs. We have 1500+ component tests that we are running. |
|
Same for us too |
|
We have been running into this sporadically when Component testing after migrating from CY 12.x to 13.6.2. Similarly to cases above, it happens towards the end of running a batch of suites (files) together.
Resorted to running cypress with There is a lot in the logs that makes it hard to share but please do let us know if we should be looking for or can share anything useful to help unraveling this. |
|
Does anyone know of a way to suppress this error until we can get a resolution? |
|
We have a very similar issue with another file: It makes our test suite pass sometimes but fail randomly and frequently, on a component or another, randomly. |
|
I'm leaving behind some notes after digging into this further. We are experiencing this error on ~9% of our component test builds. (I can't remember offhand how many tests that entails.) Indeed error messages above are consistent with what we see, here's an excerpt (with a slight edit on chunk name tbh): It includes the localhost path as well on the next line (which I left out here). Noteworthy...going up a few lines in the build, we are seeing something like this (I changed more file names out of caution): It appears to me that the Ultimately I'm convinced the next test needs to wait until compilation completes and the devserver has updated the cache. I attempted to set devserver settings around hot reload, etc, but those didn't work -- likely because the test is still trying to serve the page and is not waiting for the devserver to signal it's up to date. When it comes to waiting for compilation to complete, we do have logic around this in There had been a previous PR that leveraged the webpack devserver's index HTML file, which I believe would yield the result I'm thinking of (change here). That PR was intended to fix a situation specific to MacOS version, but I'm optimistic it could resolve the ChunkLoadError. The PR was merged but the code in the file today is older than the PR, so I don't know if it was rolled back at a later time. |
|
Just commenting to say I am also experiencing this issue. Have traced it down to being introduced by 12.17.4, any version below this seems to be completely fine on our CI runs. |
|
We're spiking into this to see if we can find what's causing this. |
jennifer-shehane
added
stage: investigating
|
Same here. Since the v12 upgrade (I don't recall what the version was, though), we have gotten this on GitHub Actions CI sometimes. My initial thought was that it was caused by the first spec in our test suite. However, changing the order of suites didn't help. It's just that the first spec fails. My workaround for this was to amend the run script: Cypress 13.6.4 BTW, this seems to be related or even the same: #16421 |
|
Hey all. Sorry for the delay on an update here. I've been investigating the issue heads down for about a week now. I'm able to reproduce the error on my end but only at the start of the run and sparingly in the middle of the run. I think there are two issues possibly going on here At the start of the runAt the end of In other words, we can get ourselves into a race condition where
In the middle of a runAdding to what @benpatterson mentioned and a bit more complicated to reproduce, but does happen, is Test starts. Something triggers
|
|
@AtofStryker, great investigation and summary. I can confirm that the error happens only at the beginning of the run for me; I haven't noticed it in the middle. I also haven't noticed any webpack re-compilation in the middle of the run. The compilation times on CI are around 19 seconds for the first run (which is allowed to fail) and 14 seconds for the second run. |
|
@bene-starzengruber @cubanx @werge2121 @amir1218 @rbecheras @TomaszG what version of I think the recompiling in the middle is related to either a bug or a misconfiguration within @TomaszG I would be curious if using the diagnostic binary while setting |
|
We should just be using the default devserver that comes with cypress, so 4. For us its happening in the middle of the run usually. It takes around 5-6 seconds for the webpack to compile usually. |
|
My team is using the default dev server that comes with cypress: The errors seem to always happen at the start of a test file, but pop up in random places in our test suite. For example, here is a recent failure in the 73th test file out of a suite of 90 where the other 89 runs all passed (edited to remove component/file names): The frequency with which these errors occur has been increasing as we add more test files. If there's any debug/diagnostic config we can set to get more info happy to try as this issue is happening frequently in our CI |
|
It would be great if cypress team would develop a pre-bundle option for runs in the CI. We had similar issues with MSW in our component tests. Browser stops trying when a service worker script takes more than ~60 seconds to load. We had to slightly modify the default dev server to "prebuild" service worker file so that when it's actually needed, it's already built and is ready to serve by the dev server: So if there were an option to pre-bundle everything, flaky tests due to middle of the run builds taking too long would be long gone. In user land, this can be achieved by manually pre-bundling and then by creating a custom dev-server that serves pre-bundled files. But it's not very elegant, Cypress team can do better by introducing a new config for pre-bundling. |
|
Hey all. Just a few updates. Mid recompilation with This issue has been extremely challenging to reproduce, but we have a few theories. My guess is that there are chunk loading issues associated with the main bundle mainly due to us bundling every component test, regardless of what is used under test. The only exception to this is if the
The larger In most cases, even if the build takes minutes, cypress waits for the dev server to return a response before executing tests. @anilanar In other words, we are pre-bundling everything needed under test already but the assets are not loaded into the browser all at once, which might explain the service worker issue you are running into and I'm glad you were able to find a solution! Something is causing the chunk to fail to load meaning it either doesn't exist, or it timed out loading. Since the assets are held in memory, it is difficult to figure out which has occurred. Either way, I am not convinced flaky tests are due to builds taking too long. What we are trying internally with one of our projects where this error happens frequently, is to leverage the |
|
@AtofStryker We are using whatever I concur that it seems large tests suites seem to cause this issue. To both speed up our test runs and reduce the frequency of this error, I have some PowerShell scripts that use the This reduced the number of test per run from ~1500 to ~400 per run, and the error seems to have abated. If this helps at all, even running interactively, if we load all our Component tests and then try to run them, there is a significant delay between update/reloads or if you switch test files. I use another PowerShell script to modify the We also tried switching the Component Server in the past to I am actually about to leave the current position I'm in, but if there's any diagnostics or info that would help you, I'll be here until March 1st. Thanks for your work on this! |
|
@AtofStryker Webpack supports lazy loading with async chunks (code splitting) therefore it is definitely possible to make a test suite start downloading and bundling stuff in the middle of the run. Webpack dev server doesn't initiate building an async chunk unless that chunk is requested by the client with an async import. This issue can be solved with pre-bundling. It can probably be solved without pre-bundling too, if there is a way for webpack dev server to report any async chunks it finds during a build. Then those async chunks can be requested manually (like I did with the service worker) to manually trigger a build for them before running a test. Keep in mind that this needs to be done recursively, until no async chunks are left. Another idea is to disable code splitting, e.g. https://webpack.js.org/plugins/limit-chunk-count-plugin/ I'm not really sure if this has anything to do with your A simple test that downloads an async chunk in the middle of the run: export const Example = () => {
const [done, setDone] = useState(false);
return (
<button
onClick={async () => {
await import("./path-to-something-very-slow-to-bundle");
setDone(true);
}}
>
{done ? "DONE" : "Click me to download async chunk"}
</button>
);
};it('will download async chunk', () => {
cy.mount(<Example />);
cy.get('button').click();
cy.contains('DONE').should('be.visible');
});Edit: I still think it's better to pre-bundle and run cypress without a dev server and without a file watcher and without no recompiles for CI use cases; instead of trying to fight file watcher/unnecessary rebuilds that can be caused by myriads of webpack plugins (and potentially unintentional bugs in them). I think using dev-server on CI is picking a wrong fight. Consider this: no build tool or no test tool runs in watch mode on CI. So cypress is opening a can of worms by running in watch mode. |
|
@AtofStryker We are on webpack-dev-server 4.15.1 |
|
Any updates? |
|
Try this workaround, as I previously suggested: // cypress.config.js
const { defineConfig } = require('cypress');
const { devServer } = require('@cypress/webpack-dev-server');
module.exports = defineConfig({
...
devServer: async ({ specs, cypressConfig, devServerEvents }) => {
const result = await devServer({
specs,
cypressConfig,
devServerEvents,
webpackConfig,
});
// prebuild files that can cause test failures when they take too long
// during runs
await Promise.all([
fetch('http://localhost:8080/__cypress/src/cypress-support-file.js'),
// prebuild any other file, such as service workers here
]);
return result;
},
});If other async chunks are causing middle-of-the-run builds, then try using https://webpack.js.org/plugins/limit-chunk-count-plugin/ and limit your chunk count to 1. Or try turning off chunk splitting, see https://webpack.js.org/plugins/split-chunks-plugin/#optimizationsplitchunks . Again, I doubt this is a misconfiguration in webpack dev server. Async chunks are normal things that optimize initial bundle size by lazy loading things later. They are usually caused by async imports, but if you don't have an async import on your side or if you don't know what that even means, then it must be used by one of the libraries you use. This is a problem with async chunks triggering builds and those builds not finishing before test times out. The solution is one of:
Edit: typo |
|
@AtofStryker I've tried what you've recommended, so I've set CYPRESS_INTERNAL_BROWSER_CONNECT_TIMEOUT=120000 and also used your diagnostic binary based on 13.6.5 and after a couple of runs, it seems that there are no failures. Here's the log output from the tests: I wonder if it's OK that |
|
Another data point: we updated and enabled the JIT feature, but it didn't resolve the ChunkLoadError problem. We were able to get tests to run reliably using anilanar's solution. Here's an example config for Angular: Before: module.exports = {
component: {
devServer: {
framework: 'angular',
bundler: 'webpack',
options: {
projectConfig: { /* ... */ }
},
},
},
};After: const { devServer } = require('@cypress/webpack-dev-server');
module.exports = {
component: {
devServer: async ({ specs, cypressConfig, devServerEvents }) => {
const result = await devServer({
framework: 'angular',
specs,
cypressConfig,
devServerEvents,
options: {
projectConfig: { /* ... */ },
},
});
await Promise.all([
fetch('http://localhost:8080/__cypress/src/cypress-support-file.js'),
]);
return result;
},
},
}; |
|
@eblocha Please ensure the flag is taking effect by checking your resolved configuration. We haven't had any reports of this flag not fixing the chunk load errors and would love to investigate further if you're seeing it with the experimental flag. https://docs.cypress.io/guides/references/configuration#Resolved-Configuration |
|
For me this happens only on Azure pipelines. I have the flag set and I can see it's being used:
I have attached logs from successful and unsuccessful runs with net and http enabled in Node. Hope this helps. In case of an error, there is no http request logged for http://localhost:8080/__cypress/src/cypress-support-file.js url that would come from referer of value http://localhost:8080/__cypress/iframes/index.html?specPath=/home/vsts/work/1/s/src/app/core/layout/containers/layout-section/layout-section-component.cy.ts. Looks like it is failing on creating a connection and does not even try to make a http request. |
@rgala are you able to share the contents of your support file? Either the support file bundle must be very large, or the machine in azure pipelines is running out of resources. Are you able to share the resource consumption as well? |
|
Hi @AtofStryker. The cypress-support-file.js is almost 23MB. I'd rather not share it in public. Is there any way I can provide it to you privately? Lack of resources on Azure VM was my first suspicion, so I ran a free command in a loop with 100ms intervals while executing the Cypress tests and the memory was fine I think. There was some swapping but very small (I have attached the output). In overall, it takes very long to execute Cypress tests on Azure compared to Ubuntu running on WSL2 (16 seconds vs 120). Let me know if you need any additional info. |
|
@rgala Yes, can you share it to [email protected]? Thanks! |
|
@jennifer-shehane I have sent the file today |
@rgala we received the support file. Thank you for sending it over. This looks like the bundled support file. Are you able to send the pre compiled support file and the webpack config you are using to compile? a tiny zipped project would be great. It can contain |
|
Hello @AtofStryker, I have sent a sample project to support mailbox. |
|
@rgala Hi, we received the zip in the support mailbox, but it appears to be password protected upon opening. Could you provide that? |
|
I did provide in the e-mail. The password is cypress :) |
|
@rgala Hi, that password doesn't appear to be working |
|
Are you opening this as a 7Zip archive? I just tried the password and it worked. |
|
@rgala Yes, it's not working for several people on our side. |
|
I put another one without a password and sent a link. Please try. |
|
@rgala Thanks, we're looking at it now. |
|
@jennifer-shehane @AtofStryker I have reported previously that However, as the component suite grows, we still sporadically see it, with a failure rate of 16% (the Cypress Cloud link should be accessible by Cypress employees). We are somewhat desperate for a solution, as we are vested into component testing with over 500 files/1600 it blocks. Unless there is a solution, the only plausible way forward for us could be to execute tests based on source code changes. We would be missing a lot of issues, but we would not be alerted falsely (and cost 1600+ runs from our credit) at a 16% rate. |
Hey @muratkeremozcan. Thank you for the feedback! I am taking a look at the failures and can see the chunk load error. I am seeing something like : The following error originated from your test code, not from Cypress.
> Loading chunk vendors~cypress-support-file~spec-1~spec-10~spec-100~spec-101~spec-102~spec-104~spec-105~spec-106~sp~41b558d7 failed.
(error: http://localhost:3000/__cypress/src/vendors~cypress-support-file~spec-1~spec-10~spec-100~spec-101~spec-102~spec-104~spec-105~spec-106~sp~41b558d7.js)
When Cypress detects uncaught errors originating from your test code it will automatically fail the current test.
Cypress could not associate this error to any specific test.
We dynamically generated a new test to display this failure.Is there something specific about compiling vendor code in your webpack config? The reason I ask is each spec should only compile assets related to itself, so there should be no chunking of the specs like what we see here. It almost looks like the |
I am sending the zipped repo to support right away. Worst case scenario, is there a way we can flag the ChunLoadError a false positive, and not have things fail but perhaps just skip when we see it? https://files.slack.com/files-pri/T047EH9G6G2-F07RK079PV0/image.png |
|
@rgala from looking at the reproduction you sent us, it looks like the support file has about 14.3MB in source maps from Cypress adding |
|
regarding I sent you a link of the zipped repo on Discord, if you can log in within 24 hours. |
|
Bill and I found that when intending to upgrade to v13.15.0, I must have typed in 13.5.0, that is whay it seemed like the ChunkLoadError has come back. We used 13.15.0 and the error went away with the flag However, we uncovered a new issue where time to time, the last component test does not crash, but hangs there indefinately. What kind of logs or debugging can I give to aid with diagnosing this one?
|
|
Hi @AtofStryker Sorry about the delay. I have used the patch-package, which generated below patch file: I apply the patch during pipeline run by executing yarn patch-package, which ends with:
Unfortunately, tests still randomly fail :( |
|
@rgala are you referencing import { devServer } from '@cypress/webpack-dev-server'
export default {
component: {
devServer: (devServerConfig) => {
return devServer({
...devServerConfig,
framework: 'react',
bundler: 'webpack',
})
}
} |
|
Sorry @AtofStryker, I did not know I had to make such change. Anyway, after implementing it I have been unable to reproduce the issue anymore and this is good news :) |
|
We intend to make the just-in-time compile flag default within Cypress 14, with a |
|
@AtofStryker following up on the issue I reported related to |
|
I am experiencing this issue in gitlabci on So in the end im not even using the experimental JIT flag. But I'm not super familiar with webpack so I'm a little unsure about what this is actually doing or if there could be negative ramifications. |
and others
Current behavior
After updating from
[email protected]to[email protected], we are seeing this error occasionally on our CI runs.Unfortunately it is not reproducable locally, so I can not give you a repository to check.
I know that this might make it hard to investigate but maybe somebody already fell over the same issue or others are experiencing it as well.
The test fails around 30% of the time.
Desired behavior
No response
Test code to reproduce
no reproduction repository
Cypress Version
13.6.2
Node version
18.19.0
Operating System
Ubuntu 22.04.3
Debug Logs
No response
Other
No response
The text was updated successfully, but these errors were encountered: