Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NATS] Extend to support in-memory stream #370

Conversation

kevin-ip
Copy link
Contributor

@kevin-ip kevin-ip commented Mar 2, 2023

Motivation

While I was benchmarking NATS, I found myself needed to benchmark its in-memory stream.
Think it would be beneficial to other folks with the option.

Changes

Extend NatsConfig with storageType and maxBytes to support in-memory stream while limiting memory consumption.

@kevin-ip
Copy link
Contributor Author

Test failed on driver-kop, any idea?

Error:  Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M7:test (default-test) on project driver-kop: 
Error:  
Error:  Please refer to /home/runner/work/benchmark/benchmark/driver-kop/target/surefire-reports for the individual test results.
Error:  Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
Error: [ERROR] There was an error in the forked process
Error:  'java.lang.ClassLoader[] org.apache.logging.log4j.util.LoaderUtil.getClassLoaders()'
Error:  org.apache.maven.surefire.booter.SurefireBooterForkException: There was an error in the forked process
Error:  'java.lang.ClassLoader[] org.apache.logging.log4j.util.LoaderUtil.getClassLoaders()'
Error:  	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:701)

see also https://github.com/openmessaging/benchmark/actions/runs/4309206859/jobs/7905914983

@merlimat
Copy link
Contributor

@BewareMyPower Can you take a look at the KoP driver build failure?

@stale
Copy link

stale bot commented Jun 27, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 5 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jun 27, 2023
@BewareMyPower
Copy link
Contributor

@merlimat Sorry it seems that I missed the comment before. Here is the fix: #387

@stale stale bot removed the stale label Jun 28, 2023
so one can test with file or memory stream.
so one can prevent NATS running out of memory incase the storage type is set to "Memory"
@kevin-ip kevin-ip force-pushed the kevin/PTP-13696-nats-memory-jetstream branch from bd56d9c to dafb547 Compare August 22, 2023 02:52
@kevin-ip kevin-ip requested a review from merlimat August 28, 2023 16:01
@merlimat merlimat merged commit b3a9044 into openmessaging:master Aug 28, 2023
1 check passed
@kevin-ip kevin-ip deleted the kevin/PTP-13696-nats-memory-jetstream branch September 5, 2023 15:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants