[Bug]: Locally set up devstack MFE containers continuously throwing "fedx-scripts: not found" error.

[ilee2u]

Describe the bug that you are seeing.

The following stack trace for every MFE (micro-frontend) container (e.g. frontend-app-learning, frontend-app-payment, etc) when you run all of the containers using make dev.up.large-and-slow in devstack.

Traceback:

npm ERR! code ENOTEMPTY
2023-06-13 14:34:49 npm ERR! syscall rename
2023-06-13 14:34:49 npm ERR! path /edx/app/frontend-app-learning/node_modules/typescript
2023-06-13 14:34:49 npm ERR! dest /edx/app/frontend-app-learning/node_modules/.typescript-8xwPIFdz
2023-06-13 14:34:49 npm ERR! errno -39
2023-06-13 14:34:49 npm ERR! ENOTEMPTY: directory not empty, rename '/edx/app/frontend-app-learning/node_modules/typescript' -> '/edx/app/frontend-app-learning/node_modules/.typescript-8xwPIFdz'
2023-06-13 14:34:49 
2023-06-13 14:34:49 npm ERR! A complete log of this run can be found in:
2023-06-13 14:34:49 npm ERR!     /root/.npm/_logs/2023-06-13T18_34_28_185Z-debug-0.log
2023-06-13 14:34:50 
2023-06-13 14:34:50 > @edx/[email protected] start
2023-06-13 14:34:50 > fedx-scripts webpack-dev-server --progress
2023-06-13 14:34:50 
2023-06-13 14:34:50 sh: 1: fedx-scripts: not found
2023-06-13 14:34:53 

This error proceeds to repeat over and over again:

2023-06-13 14:34:50 > @edx/[email protected] start
2023-06-13 14:34:50 > fedx-scripts webpack-dev-server --progress
2023-06-13 14:34:50 
2023-06-13 14:34:50 sh: 1: fedx-scripts: not found

The MFEs work fine if you don't run/spin down their respective containers and run them locally in a separate terminal.

For example, I run the script below to work around this issue:

cd edx/devstack
make dev.up.ecommerce
make dev.up.edxnotesapi
make dev.up.frontend-app-authn
make dev.up.studio
make dev.up.credentials
make dev.up.lms
make dev.up.forum
make dev.up.devpi
make dev.up.discovery
make dev.up.mongo
make dev.up.opensearch12
make dev.up.memcached
make dev.up.mysql57
make dev.up.elasticsearch710
make dev.up.devstack_default    
make dev.down.frontend-app-authn

cd ..
cd frontend-app-learning

osascript -e 'tell app "Terminal"
    do script "cd edx && cd frontend-app-authn && npm start"
end tell'

osascript -e 'tell app "Terminal"
    do script "cd edx && cd frontend-app-course-authoring && npm start"
end tell'

osascript -e 'tell app "Terminal"
    do script "cd edx && cd frontend-app-exams-dashboard && npm start"
end tell'

osascript -e 'tell app "Terminal"
    do script "cd edX-IDAs-and-MFEs && cd edx-exams && workon edx-exams && python manage.py runserver localhost:18740 --settings=edx_exams.settings.local"
end tell'

osascript -e 'tell app "Terminal"
	ngrok http 18000
end tell'

npm start

This script runs every non-MFE container, though I have to take frontend-app-authn down manually anyways since it appears to be spun up automatically with one of the other containers.

Then, eachosascript -e 'tell app "Terminal" opens a new terminal window, with each window running the MFEs (amongst other IDAs) that I need.

Feel free to copy/modify this for your own use. Also note that you'll probably need to switch out some of the folder names/paths, and add/remove some terminals for some of these apps based on what you plan to work on.

For example, I work on edx-exams, but you may not, in which case you could omit the section below:

osascript -e 'tell app "Terminal"
    do script "cd edX-IDAs-and-MFEs && cd edx-exams && workon edx-exams && python manage.py runserver localhost:18740 --settings=edx_exams.settings.local"
end tell'

NOTE: It is suspected that this issue is related to how Apple Silicon (i.e. M1 and M2) chips operate, though this is yet to be confirmed.

Did this happen on the host (your machine or the remote instance) or in the container?

In the container

Steps to reproduce.

• Have devstack set up and configured locally on your machine
• In the devstack directory, run make dev.up.large-and-slow

• (Optional) To test that it isn't working, navigate to any MFE page, e.g. http://localhost:2000/course/course-v1:edX+DemoX+Demo_Course/home

• Go into the logs for any MFE container and see the error

What system was this issue seen on?

Apple Silicon

[github-actions]

Follow-up checklist (for Arch-BOM usage)

• Is the issue flaky or consistent?
• Does it affect multiple people or multiple types of systems?
• Update the devstack troubleshooting documentation page if necessary
  • Do we need a new troubleshooting section?
  • Did a troubleshooting section already exist, but it wasn't easy to find given the symptoms?
  • If a recurring issue, should we ticket an automated resolution in place of the doc?

[rgraber]

I think this may be a red herring. I had the same issue, along with a problem doing npm install:

edx.devstack.frontend-app-learning  | npm ERR! /edx/app/frontend-app-learning/node_modules/@pact-foundation/pact-node/standalone/install.js:42
edx.devstack.frontend-app-learning  | npm ERR!     throw new Error(chalk_1.default.red("Error while installing binary: " + msg));

When I upgraded pact-node, the npm install error went away and so did the fedx-scripts issue. I think the problem is that npm install doesn't finish correctly, in this case because of typescript.

[dianakhuang]

Proposal: try to allocate more memory to Docker. Ensure this workaround is documented.

This might also be a case of the npm packages being in a wrong install state. Maybe also document the steps needed to fully reset an npm package environment.

[dianakhuang]

We made some changes to the way npm packages are installed so that any failures in installation will fast-fail. We believe this might help prevent this issue from cropping up. If we see it again, check to see if the fast failure logic is in place. If not, add more details about the issue to this ticket.

[ilee2u]

Hi Diana! Thanks so much, I've just tested the new images on my machine with the following results:

• I got errors in the frontend-app-publisher and frontend-app-payment containers initially, but those resolved once I git pull'd in those repos and did a clean package install by running rm -rm node_modules && npm install in each of those folders.

• I've run the same commands in frontend-app-learning, and I've continued to get this same error:

2023-07-11 10:47:51 npm ERR! path /edx/app/frontend-app-learning/node_modules/@pact-foundation/pact-node
2023-07-11 10:47:51 npm ERR! command failed
2023-07-11 10:47:51 npm ERR! command sh -c -- node postinstall.js
2023-07-11 10:47:51 npm ERR! /edx/app/frontend-app-learning/node_modules/@pact-foundation/pact-node/standalone/install.js:42
2023-07-11 10:47:51 npm ERR!     throw new Error(chalk_1.default.red("Error while installing binary: " + msg));
2023-07-11 10:47:51 npm ERR!     ^
2023-07-11 10:47:51 npm ERR! 
2023-07-11 10:47:51 npm ERR! Error: Error while installing binary: Cannot find binary for platform 'linux' with architecture 'arm64'.
2023-07-11 10:47:51 npm ERR!     at throwError (/edx/app/frontend-app-learning/node_modules/@pact-foundation/pact-node/standalone/install.js:42:11)
2023-07-11 10:47:51 npm ERR!     at getBinaryEntry (/edx/app/frontend-app-learning/node_modules/@pact-foundation/pact-node/standalone/install.js:282:11)
2023-07-11 10:47:51 npm ERR!     at setup (/edx/app/frontend-app-learning/node_modules/@pact-foundation/pact-node/standalone/install.js:286:17)
2023-07-11 10:47:51 npm ERR!     at /edx/app/frontend-app-learning/node_modules/@pact-foundation/pact-node/standalone/install.js:319:12
2023-07-11 10:47:51 npm ERR!     at Object.<anonymous> (/edx/app/frontend-app-learning/node_modules/@pact-foundation/pact-node/postinstall.js:3:1)
2023-07-11 10:47:51 npm ERR!     at Module._compile (node:internal/modules/cjs/loader:1198:14)
2023-07-11 10:47:51 npm ERR!     at Object.Module._extensions..js (node:internal/modules/cjs/loader:1252:10)
2023-07-11 10:47:51 npm ERR!     at Module.load (node:internal/modules/cjs/loader:1076:32)
2023-07-11 10:47:51 npm ERR!     at Function.Module._load (node:internal/modules/cjs/loader:911:12)
2023-07-11 10:47:51 npm ERR!     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
2023-07-11 10:47:51 npm ERR!     at node:internal/main/run_main_module:22:47
2023-07-11 10:47:51 
2023-07-11 10:47:51 npm ERR! A complete log of this run can be found in:
2023-07-11 10:47:51 npm ERR!     /root/.npm/_logs/2023-07-11T14_47_22_965Z-debug-0.log

• The main point of interest in this error seems to be Error: Error while installing binary: Cannot find binary for platform 'linux' with architecture 'arm64'.. I did some light digging, and there seems to be a possible fix for this error addressed in this Issue for the pact core-js library: Linux/arm64 CI failing - Cannot find binary for platform linux with architecture arm64 pact-foundation/pact-js-core#416

[rgraber]

Blocked awaiting confirmation from @ilee2u and @rayzhou-bit that openedx/frontend-app-learning#1141 fixed the issue

[ilee2u]

Blocked awaiting confirmation from @ilee2u and @rayzhou-bit that openedx/frontend-app-learning#1141 fixed the issue

All green on my end. Thanks much!

[rayzhou-bit]

Blocked awaiting confirmation from @ilee2u and @rayzhou-bit that openedx/frontend-app-learning#1141 fixed the issue

Looks good here as well!