Changed the web-server from HTTP1 to HTTP2.. #609

not-meet · 2024-10-23T13:34:33Z

Linked Issue(s)

Acceptance Criteria fulfillment

Enable HTTP/2 in development mode.
Enable HTTP/2 in production mode .
Generate automated self-signed certificates.
Automate certificate refresh.

reference taken from -> https://medium.com/@greg.farrow1/nextjs-https-for-a-local-dev-server-98bb441eabd7

Further comments

Comment

This pull request includes several changes to improve the security and modularity of the web server setup, along with some other minor updates. The most important changes include switching to HTTPS for local development, converting the configuration file to use ES modules, and adding a script to generate SSL certificates.

Security improvements:

cli/source/app.tsx: Changed the local development URL from HTTP to HTTPS.

Modularity and configuration updates:

web-server/next.config.mjs: Renamed from next.config.js and converted the configuration to use ES modules. [1] [2]
web-server/package.json: Added "type": "module" to support ES modules and updated the dev and start scripts to use server.js.

Automated SSL certificate generation:

web-server/scripts/generate-certificates.sh: Added a script to generate SSL certificates for local development.
web-server/server.js: Added a new server setup to use HTTPS with the generated SSL certificates.

CLAassistant · 2024-10-23T13:34:39Z

All committers have signed the CLA.

jayantbh · 2024-10-23T16:10:42Z

web-server/server.js

+  createServer(httpsOptions, (req, res) => {
+    const parsedUrl = parse(req.url, true);
+    handle(req, res, parsedUrl);
+  }).listen(3333, (err) => {


Doesn't use the PORT environment variable.

jayantbh · 2024-10-23T16:11:03Z

web-server/server.js

+    handle(req, res, parsedUrl);
+  }).listen(3333, (err) => {
+    if (err) throw err;
+    console.log('> Server started on https://localhost:3333');


Same as above.

not-meet · 2024-10-24T08:14:37Z

@jayantbh i have made the changes as asked.. kindly review the changes.. i am up for anymore changes and to learn.. :)

jayantbh · 2024-10-28T10:58:46Z

Hey sorry for the delay. We'll check it out in a day or so.

not-meet · 2024-10-28T20:15:44Z

hey @jayantbh no issues with the delay :-) i waited patiently i saw somehow the code i pushed was having linting error so i fixed it with pre-commit run and it made that yarn lock changes and to re assure i also ran the container and from my side everything is working fine, do check the pr and if possible lets merge this one!

jayantbh · 2024-10-29T09:48:22Z

web-server/package.json

@@ -123,7 +123,7 @@
    "eslint-config-next": "13.5.6",
    "eslint-plugin-import": "^2.29.0",
    "eslint-plugin-prettier": "^5.0.1",
-    "eslint-plugin-react": "^7.29.4",
+    "eslint-plugin-react": "^7.37.2",


Is this change actually needed?

If not, you might be able to reduce the size of your PR significantly.

@not-meet reminder ☝🏽

@jayantbh hi really sorry for the delay ill fix this asap, actually this happened back when the pr was not able to merge because of linting then i used pre-commit to fix the issue and it gave me an error and a command to fix the error and after that when i ran pre-commit this issue happened... ill fix this as soon as possible.. :-)

@jayantbh hi.. i made the eslint-plugin- react changes as they were also ran pre-commit run all files to make sure the linting problem dose not happen again...i hope this will solve all the issues while merging the pr :-)

jayantbh · 2024-10-30T09:00:24Z

I checked out the branch to run it locally ./dev.sh and it failed.
Basically the experience should be that you should not need to run anything other than ./dev.sh to get started.
Related: Running ./local-setup.sh is optional, so anything that is not skippable should not go there.
Since certificates will be a requirement, it should go in ./dev.sh.

Nonetheless, I keep getting this error and wasn't able to get it to work even after running the certificates script. Mainly because the script creates certificates w.r.t. the current directory, so running this script anywhere other than the web-server directory will break it.

If you hook the certificates generation behaviour to ./dev.sh, you might have some new ways to deal with it.

web yarn run v1.22.22
web $ [ -e ../.env ] && set -a && . ../.env; NEXT_MANUAL_SIG_HANDLE=true node server.j
web s
web (node:230) [MODULE_TYPELESS_PACKAGE_JSON] Warning: Module type of file:///app/web-
web server/server.js is not specified and it doesn't parse as CommonJS.
web Reparsing as ES module because module syntax was detected. This incurs a performan
web ce overhead.
web To eliminate this warning, add "type": "module" to /app/web-server/package.json.
web (Use `node --trace-warnings ...` to show where the warning was created)
web node:fs:561
web   return binding.open(
web                  ^
web Error: ENOENT: no such file or directory, open './certificates/localhost.key'
web     at Object.openSync (node:fs:561:18)
web     at Object.readFileSync (node:fs:445:35)
web     at file:///app/web-server/server.js:12:11
web     at ModuleJob.run (node:internal/modules/esm/module_job:262:25)
web     at async onImport.tracePromise.__proto__ (node:internal/modules/esm/loader:483
web :26) {
web   errno: -2,
web   code: 'ENOENT',
web   syscall: 'open',
web   path: './certificates/localhost.key'
web }
web Node.js v22.9.0
web error Command failed with exit code 1.
web info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this comman
web d.

not-meet · 2024-10-30T09:14:18Z

@jayantbh alright gotcha! so i have to do something that can generate the certificates inside the desired folder where it should be.. when a client runs ./dev.sh ...i'll do it asap and will make sure this time there won't be any issues regarding all this.... btw very happy diwali @jayantbh <3

jayantbh · 2024-10-30T09:15:37Z

Happy Diwali to you too, @not-meet! ❤️

not-meet · 2024-10-30T17:14:37Z

hey @jayantbh i have made changes as you asked but i have one doubt in this... after doing the changes

so as you can see in this the certificates are generating on there own but after that it also says module type warning to convert type to module... when i did that... the server stopped running and started showing import errors such as

though without changing the type to module everything is working fine on my side :-)

jayantbh · 2024-10-30T19:18:03Z

I think you'd have to change the next.config.js to use import/export instead of require/module.exports and it should work just fine.

not-meet · 2024-10-30T21:32:39Z

did changes as asked! also did some renames of eslint and next.config to align with type :"module".. also one more thing while running the server it showed one warning of unrecognized file extension

i tried experimenting by removing it but after removing it the website crashed so i kept it like it is !
up for anymore changes or tweaks @jayantbh :-D

jayantbh · 2024-11-01T11:11:21Z

Yeah that's okay, that warning has been there since ages but it's never really caused any real issues... or any issues as far as we could tell.

not-meet · 2024-11-04T09:21:19Z

hey @jayantbh so as i fixed the linting issue all the checks have passed and i believe the pr is somewhere close to get merge too !! so if there are any more tweaks you want in the pr i'll be more than happy to do them... :-)

jayantbh · 2024-11-04T12:37:48Z

Hey @not-meet, I see that it works ok for dev mode, ~~except that the certificate's expiry date is the time when it was generated~~, except that the certificate isn't signed (even self-signed) which might be throwing the insecure warning.

Also, in prod mode there is no mechanism to generate the certificate.

I think if you make the certificate generation be part of the server.js script, so it creates one right on server boot, it might work.

But I also see that there's no mechanism to refresh the certificate either.

jayantbh · 2024-11-04T12:38:15Z

I recommend making a checklist of functionality that this PR needs to have so you can ensure nothing gets missed.

not-meet · 2024-11-05T14:46:31Z

Hey @not-meet, I see that it works ok for dev mode, ~~except that the certificate's expiry date is the time when it was generated~~, except that the certificate isn't signed (even self-signed) which might be throwing the insecure warning.

Also, in prod mode there is no mechanism to generate the certificate.

I think if you make the certificate generation be part of the server.js script, so it creates one right on server boot, it might work.

But I also see that there's no mechanism to refresh the certificate either.

@jayantbh apologies for that... ill fix all the issues as fast as i can... also will make a checklist of functionality that PR need as well <3

not-meet · 2024-11-09T22:47:49Z

Hi @jayantbh ,

First of all, apologies for being inactive for the past 2-3 days, as I was unwell. I’ve made some progress with the requirements and researched how to best implement them.

Currently, on the dev server, server.js calls the generate certificate script itself. I also made the following changes to the script:

The script now generates a private key and a self-signed root certificate.
It creates a server certificate with a SAN (Subject Alternative Name) configuration to support multiple domains.
The certificates are also self signed
I followed these blog posts for handling the "Not Secure" issue:

https://surma.dev/things/h2setup/
https://medium.com/@tbusser/creating-a-browser-trusted-self-signed-ssl-certificate-2709ce43fd15

I discovered that the "Not Secure" warning in dev appears because the certificates aren’t added to the local Certificate Authority. I found a solution using "mkcert" to create a CA, but it requires installation on the local machine. I assume some adjustments in Docker to initialize the CA at runtime,may help resolve this issue.

also,
Since the certificates are removed when the container stops, is it essential to set up certificate renewal in the dev environment?

For the production setup:
I wasn’t able to get a complete picture of how the production server is configured.
I understand that certificate renewal could potentially cause downtime or require a restart, but with the implementation of NGINX or Apache, this could be mitigated. If you could provide some insight into the production setup, it would be a big help!

I know this PR is taking longer than expected, and I’m aware that some of my rookie mistakes have caused delays. I’ll make sure to wrap this up as soon as possible, with just a bit of guidance. Thank you for your patience! <3

jayantbh · 2024-11-11T07:34:06Z

I think you're making good progress!
You can refer to https://github.com/middlewarehq/middleware?tab=readme-ov-file#-quick-start to see how to install the production version of Middleware.

…or a hard coded 3333

….mjs

* fix: updated Next to 14.2.16 * fix: updated Next to 15.0.1 * fix: pre-commit hook update for eslint-config-next * fix: pre-commit hook update for eslint-config-next * fix: pre-commit hook update for eslint-plugin-react, eslint-plugin-unused-imports

Resolve conflicts and apply lint fixes

not-meet · 2024-11-17T13:00:43Z

Hi @jayantbh !! so i did the changes in the dev server as asked now the certificates are generated when the docker files runs plus a CA is also estabilished for the ssl certificates..

What i did ?
for the prod server i have also added http2 support by creating another file named prod-server.js and adding certbot for certificate renewal and generation in the yaml file..

i tested it by generating certificates locally and for me it worked....!

Note- currently i am facing a problem with the dev server which is.. that the browser still shows not secure even after adding a CA and verifying the certificates..!
as far as i have researched the browser will show the error till the certificates are not added in the CA of the local machine ...

jayantbh · 2024-11-22T11:49:34Z

I think that's only because you can't verifiably sign certificates, even self-signed with the localhost domain. Maybe signing it as www.localhost will work? Of course, you'll need to then load the webserver on www.localhost, which should be trivial. Give that a shot?

jayantbh requested changes Oct 23, 2024

View reviewed changes

not-meet requested a review from jayantbh October 24, 2024 13:32

jayantbh reviewed Oct 29, 2024

View reviewed changes

not-meet requested a review from jayantbh October 29, 2024 19:08

not-meet force-pushed the http2 branch 2 times, most recently from ab5b221 to 0d577b8 Compare November 10, 2024 08:25

not-meet added 7 commits November 15, 2024 16:48

changes the web-server from http/1 to http/2

8d8b0e1

changes in cli docker wraper

15081b5

added the scritp to generate the certificates

d14d88a

added certificates in git ignore..

45b652d

changes in generate certificate pages

41ac566

changes the port from hardcoded to a conditional value form env file …

0a6138a

…or a hard coded 3333

lint fix

bcb8e13

not-meet and others added 7 commits November 15, 2024 16:51

version fix of es-lint-plugin

06d7736

automated certificate generation when yarn build run's

3c6666a

did tweaks in next.config.js as asked.. converted it into next.config…

9801244

….mjs

solved linting problme using pre-commit

611db4d

Upgraded Next.js to 15.0.1

e46d1b8

* fix: updated Next to 14.2.16 * fix: updated Next to 15.0.1 * fix: pre-commit hook update for eslint-config-next * fix: pre-commit hook update for eslint-config-next * fix: pre-commit hook update for eslint-plugin-react, eslint-plugin-unused-imports

Fix markdown linting issue because of the spacing.

df03a5a

lint fix

40a7221

Resolve conflicts and apply lint fixes

not-meet force-pushed the http2 branch 3 times, most recently from 605749d to 40a7221 Compare November 15, 2024 11:40

added the http2 system for the prod server

4918124

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Changed the web-server from HTTP1 to HTTP2.. #609

Changed the web-server from HTTP1 to HTTP2.. #609

not-meet commented Oct 23, 2024 •

edited by VipinDevelops

Loading

CLAassistant commented Oct 23, 2024 •

edited

Loading

jayantbh Oct 23, 2024

jayantbh Oct 23, 2024

not-meet commented Oct 24, 2024

jayantbh commented Oct 28, 2024

not-meet commented Oct 28, 2024

jayantbh Oct 29, 2024

jayantbh Oct 29, 2024

not-meet Oct 29, 2024

not-meet Oct 29, 2024

jayantbh commented Oct 30, 2024

not-meet commented Oct 30, 2024

jayantbh commented Oct 30, 2024

not-meet commented Oct 30, 2024

jayantbh commented Oct 30, 2024

not-meet commented Oct 30, 2024

jayantbh commented Nov 1, 2024

not-meet commented Nov 4, 2024

jayantbh commented Nov 4, 2024 •

edited

Loading

jayantbh commented Nov 4, 2024

not-meet commented Nov 5, 2024

not-meet commented Nov 9, 2024

jayantbh commented Nov 11, 2024

not-meet commented Nov 17, 2024

jayantbh commented Nov 22, 2024

Changed the web-server from HTTP1 to HTTP2.. #609

Are you sure you want to change the base?

Changed the web-server from HTTP1 to HTTP2.. #609

Conversation

not-meet commented Oct 23, 2024 • edited by VipinDevelops Loading

Linked Issue(s)

Acceptance Criteria fulfillment

Further comments

Comment

CLAassistant commented Oct 23, 2024 • edited Loading

jayantbh Oct 23, 2024

Choose a reason for hiding this comment

jayantbh Oct 23, 2024

Choose a reason for hiding this comment

not-meet commented Oct 24, 2024

jayantbh commented Oct 28, 2024

not-meet commented Oct 28, 2024

jayantbh Oct 29, 2024

Choose a reason for hiding this comment

jayantbh Oct 29, 2024

Choose a reason for hiding this comment

not-meet Oct 29, 2024

Choose a reason for hiding this comment

not-meet Oct 29, 2024

Choose a reason for hiding this comment

jayantbh commented Oct 30, 2024

not-meet commented Oct 30, 2024

jayantbh commented Oct 30, 2024

not-meet commented Oct 30, 2024

jayantbh commented Oct 30, 2024

not-meet commented Oct 30, 2024

jayantbh commented Nov 1, 2024

not-meet commented Nov 4, 2024

jayantbh commented Nov 4, 2024 • edited Loading

jayantbh commented Nov 4, 2024

not-meet commented Nov 5, 2024

not-meet commented Nov 9, 2024

jayantbh commented Nov 11, 2024

not-meet commented Nov 17, 2024

jayantbh commented Nov 22, 2024

not-meet commented Oct 23, 2024 •

edited by VipinDevelops

Loading

CLAassistant commented Oct 23, 2024 •

edited

Loading

jayantbh commented Nov 4, 2024 •

edited

Loading