Skip to content

Run reducers in their own V8 HandleScope for js modules#4746

Open
joshua-spacetime wants to merge 1 commit intojoshua/js-worker-queuefrom
joshua/v8-scoped-memory
Open

Run reducers in their own V8 HandleScope for js modules#4746
joshua-spacetime wants to merge 1 commit intojoshua/js-worker-queuefrom
joshua/v8-scoped-memory

Conversation

@joshua-spacetime
Copy link
Copy Markdown
Collaborator

@joshua-spacetime joshua-spacetime commented Apr 3, 2026

Description of Changes

The JS worker is intentionally long-lived. Before this patch, we essentially had a memory leak where V8 call-local handles and some host-side call state would survive/accumulate across multiple calls on the worker. That created gradual heap growth over time, more GC work, and eventually enough slowdown and heap pressure that the isolate needed to be replaced.

And even though we periodically check heap statistics to determine if/when we need to replace the isolate, execution latencies can (and did) degrade dramatically before this kicks in.

Now each reducer call is given a fresh V8 HandleScope in which to execute instead of reusing a single global scope. This scope is then dropped at the end of each run, which avoids retaining/accumulating call-local JS objects over the JS worker's lifetime.

This patch also makes end-of-call host cleanup explicit, lowers the default heap-check cadence, and limits exported heap metrics to the JS worker only. Previously we had poor heap observability for diagnosis as heap metrics from the instance pool (for procedures) could overwrite heap metrics for the worker.

What changed

1. Add a fresh V8 HandleScope for every invocation

Each reducer, view, and procedure call now opens a nested V8 scope for the duration of that call.

This preserves the existing long-lived isolate and context, but gives every invocation its own temporary handle lifetime. Call-local V8 handles now die when the invocation returns instead of sticking around until the worker exits.

As part of that refactor:

  • Hook locals are rebuilt inside the per-call scope instead of being tied to the worker-lifetime scope.
  • The reducer args scratch ArrayBuffer is now created per reducer call instead of being stored as a worker-lifetime local.

2. Make end-of-call cleanup a real boundary

The V8 host now force-clears leftover per-call host state at the end of a function call.

Specifically:

  • Any row iterators left behind by guest code are cleared.
  • Any unfinished timing spans are cleared.
  • We log when that cleanup had to happen so leaked call-local state is visible instead of silently persisting across invocations.

3. Lower the default heap-check cadence

The default V8 heap policy is now more aggressive about checking worker heap usage.

Defaults changed from:

  • heap-check-request-interval = 65536
  • heap-check-time-interval = 30s

to:

  • heap-check-request-interval = 4096
  • heap-check-time-interval = 5s

These settings remain configurable through the existing v8-heap-policy config.

4. Only export heap metrics for the instance-lane worker

Heap metrics now reflect only the long-lived instance lane.

Specifically:

  • Exported V8 heap gauges are emitted only for worker_kind="instance_lane".
  • Pooled workers no longer publish these heap metrics.
  • Per-database metric cleanup was simplified accordingly.

This avoids the last-writer-wins issue from the pooled instances while keeping the metrics focused on the worker that accumulates state over time and is most relevant for long-run slowdown diagnosis.

API and ABI breaking changes

None

Expected complexity level and risk

2

Testing

Manually tested via the keynote-2 benchmark. Will add the keynote benchmark to CI which will serve as a regression test.

@joshua-spacetime joshua-spacetime changed the title Give each call its own HandleScope in js worker Run calls in their own V8 HandleScope for js modules Apr 3, 2026
@joshua-spacetime joshua-spacetime changed the title Run calls in their own V8 HandleScope for js modules Run reducers in their own V8 HandleScope for js modules Apr 4, 2026
@joshua-spacetime joshua-spacetime force-pushed the joshua/js-worker-queue branch 2 times, most recently from 77faa0d to 02dd1b5 Compare April 4, 2026 18:42
@joshua-spacetime joshua-spacetime force-pushed the joshua/v8-scoped-memory branch from 1516f25 to b5650cf Compare April 4, 2026 18:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant