Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

My Probably last attempt to optimize performance #693

Merged
merged 22 commits into from
Feb 1, 2024

Conversation

PanagiotisDrakatos
Copy link
Contributor

Check List:

  • You have run ./mvnw verify and the project builds successfully
  • Tests pass (./test.sh <username> shows no differences between expected and actual outputs)
  • All formatting changes by the build are committed
  • Your launch script is named calculate_average_<username>.sh (make sure to match casing of your GH user name) and is executable
  • Output matches that of calculate_average_baseline.sh
  • For new entries, or after substantial changes: When implementing custom hash structures, please point to where you deal with hash collisions (line number)
  • Execution time:
  • Execution time of reference implementation:

@PanagiotisDrakatos
Copy link
Contributor Author

@gunnarmorling please check again i believe this is my last attempt to improve performance after that my capabilities are limited

@gunnarmorling
Copy link
Owner

Runs out of heap space for the tests on 32 cores:

Caused by: java.lang.OutOfMemoryError: Java heap space
	at dev.morling.onebrc.CalculateAverage_PanagiotisDrakatos$MeasurementRepository.<init>(CalculateAverage_PanagiotisDrakatos.java:274)
	at dev.morling.onebrc.CalculateAverage_PanagiotisDrakatos.MappingByteBufferToData(CalculateAverage_PanagiotisDrakatos.java:164)
	at dev.morling.onebrc.CalculateAverage_PanagiotisDrakatos$$Lambda/0x00007f6b43001860.apply(Unknown Source)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:960)
	at java.base/java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:934)
	at java.base/java.util.stream.AbstractTask.compute(AbstractTask.java:327)
	at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

@PanagiotisDrakatos
Copy link
Contributor Author

@gunnarmorling i fixed VM options arguments and increased the xmx. Probably it would not failed now

@gunnarmorling
Copy link
Owner

Looking good now: 00:05.559. Passing for 10K keys, too.

@gunnarmorling gunnarmorling merged commit 2aed039 into gunnarmorling:main Feb 1, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants