Playwright in Production: Why Headless Chromium OOMs Your AI Workers

Your Playwright worker works fine on a MacBook Pro with 32GB of RAM, then dies silently in production with Command failed with exit code 137 twenty minutes into a batch run. Every operations team automating legacy ERPs, insurance portals, or VMS platforms like Fieldglass and Beeline hits this wall. Half of enterprise software still lacks a usable REST API. You are forced to run headless Chromium inside Docker containers to click buttons, parse DOM tables, and download CSV exports. Then the Linux OOM killer starts terminating your workers mid-transaction.

The 64MB `/dev/shm` Trap Inside Default Containers

Docker allocates a tiny 64MB shared memory partition (/dev/shm) to containers by default. Chromium uses shared memory extensively for inter-process communication between its browser kernel, GPU process, and renderer sub-processes. The moment your AI agent opens a heavy React or Angular portal with three nested iframes and a 12MB JavaScript bundle, Chromium exhausts that 64MB ceiling in seconds. Instead of throwing a clean JavaScript exception, the renderer crashes with TargetClosedError: Target page, context or browser has been closed or the entire container receives SIGKILL.

Most engineers try to fix this by passing --disable-dev-shm-usage in the launch arguments. That flag forces Chromium to write shared memory files to /tmp on disk instead of RAM. It stops the immediate crash, but introduces a nasty secondary bottleneck: disk I/O thrashing on AWS Fargate or Cloud Run ephemeral storage. When we deploy browser workers on ECS or Kubernetes, we explicitly mount an in-memory tmpfs volume sized to at least 1GB (--shm-size=1gb) and pair it with --disable-gpu and --no-zygote to keep sub-process overhead predictable.

Stop Spawning `chromium.launch()` Per Job

The fastest way to burn through a 4GB container is calling await chromium.launch() inside your queue handler. Launching a fresh Chromium binary costs 350ms to 800ms of CPU spike and claims 180MB to 250MB of baseline resident memory before a single URL loads. Run ten jobs concurrently on one worker and your memory footprint hits 2.5GB instantly.

Treat the browser process like a database connection pool. Launch one persistent Chromium instance per worker process at startup, then spawn an isolated BrowserContext (await browser.newContext()) for each individual automation task. A BrowserContext costs roughly 15MB of RAM and spins up in under 15 milliseconds. Cookies, localStorage, and session tokens stay completely isolated between tenants. Just one catch: you must wrap every job in a strict try / finally block that calls await context.close() with a hard 10-second AbortController timeout. If a promise hangs on an unresolved network request and context.close() never executes, detached DOM nodes stay pinned in V8 heap memory forever. Even with clean teardowns, Chromium leaks small amounts of memory over time. We recycle the parent browser binary after every 150 completed contexts or when process.memoryUsage().rss crosses 75% of the container limit.

Blocking Third-Party Bloat and Stripping the DOM Before LLM Extraction

Enterprise portals are packed with telemetry scripts, session recorders, web fonts, and unoptimized PNGs that your automation agent never needs to see. Letting Chromium download and render those assets wastes both memory and bandwidth. Attach a route interceptor (await context.route('**/*', ...)) to abort requests where route.request().resourceType() matches image, media, font, or stylesheet—unless your script specifically relies on visual layout coordinates.

That single interceptor cuts average page memory consumption by 45% and drops page load times from 6.2 seconds to 1.8 seconds. And if you are feeding page state into an LLM to decide the next click action, never dump raw await page.content() into your prompt. A raw Salesforce or SAP page routinely exceeds 400KB of HTML—roughly 100,000 tokens of nested <div> wrappers and inline SVGs. Run an in-page evaluation script that strips <script>, <style>, <svg>, and hidden elements first, returning only interactive nodes (button, input, select, a) with synthesized numeric IDs. You drop token costs by 92% and keep the renderer process from freezing during serialization.

Where Lean Engineering Fits

Before you double your AWS Fargate task sizes to 8GB RAM just to keep brittle browser scrapers alive, audit how your workers manage browser lifecycles. Put a hard cap of 4 to 6 concurrent BrowserContext instances per vCPU, enforce network request interception by default, and recycle your browser binaries deterministically. If your team is fighting flaky portal automations, zombie Chromium processes, or high extraction latency, explore how Enorexa architects fault-tolerant AI operations pipelines—or talk to us to review your worker infrastructure directly.