Skip to content

Commit

Permalink
fix: Try to resolve heap memory errors (#435) (#436)
Browse files Browse the repository at this point in the history
We are seeing errors in prod relating to javascript running out of
memory due to the heap growing. Most likely culprit is the block
prefetch. There may be some characteristics in prod that cause this
error, but not in dev. Failures cause the entire machine to crash and
restart.

We'll reduce the prefetch queue size, which is likely the largest memory
eater in the worker, to hopefully avoid this worker heap memory issue.
  • Loading branch information
darunrs committed Nov 29, 2023
1 parent b257f65 commit 9c3738c
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion runner/src/stream-handler/worker.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ function incrementId (id: string): string {
}

async function blockQueueProducer (workerContext: WorkerContext, streamKey: string): Promise<void> {
const HISTORICAL_BATCH_SIZE = 100;
const HISTORICAL_BATCH_SIZE = 10;
let streamMessageStartId = '0';

while (true) {
Expand Down

0 comments on commit 9c3738c

Please sign in to comment.