Skip to content

Close a computer's browsers before its memory runs out, and a Bot's idle tabs - #879

Merged
davidmckayv merged 3 commits into
mainfrom
david/browser-memory
Oct 29, 2026
Merged

davidmckayv merged 3 commits into
mainfrom
david/browser-memory

Conversation

@davidmckayv

Copy link
Copy Markdown
Contributor

A computer runs one browser per Bot that is browsing, and the only limits were a count (COMPUTER_MAX_BROWSERS, 8) and half an hour untouched. Neither looked at memory, so a few heavy pages could take a container past its limit well under the count, and the out-of-memory kill then took every Bot's browser down at once. The industry standard for an always-on agent's computer has exactly this failure today: a fixed machine per user, a few Bots browsing at once is the comfortable limit, and when memory runs out nothing can reach the computer. This closes browsers before that happens, and closes the tabs Bots leave behind.

Memory

  • How it measures: a computer reads its own memory the way the kubelet does. The working set is memory.current minus inactive_file, against memory.max on cgroup v2, with a v1 fallback. It reads before every browser launch and on every idle sweep.
  • What it does: at or above COMPUTER_MEMORY_HIGH_PERCENT (85 by default, 0 turns it off), it closes browsers one at a time, least recently used first, until memory is back under. A closed Bot starts again where it left off, still signed in, because its profile is on disk.
  • Never closed for memory: the browser being started, the one used most recently, or the browser of a Bot whose wheel a person holds.
  • Edge cases: a container with no limit is measured against the machine's memory. A computer outside a container (OpenBot Desktop's) closes nothing for memory.

Tabs

  • What closes: a Bot's background tabs untouched for COMPUTER_TAB_IDLE_MS (10 minutes by default, 0 keeps them).
  • Why they're safe to close: a Bot always works in its newest page, so an older tab is one it can't go back to.
  • Never closed: the Bot's current page, the last tab, any tab of a person's own browser, the Recorder's pages, or any tab while a person holds that Bot's wheel.
  • No tab-count cap: one heavy tab outweighs twenty light ones, and memory is now watched directly.

Helm fix found on the way

  • computers.browserIdleMs: 3600000 rendered as 3.6e+06, which the computer ignored for its default, and browserIdleMs: 0 was dropped.
  • All four browser settings now go through the shared computer environment, written in digits and kept at zero. So they also reach the computers computers.mode: sandbox makes, which had none of them.

Tests (each fails before the change and passes after)

  • Cgroup parsing against real v2 files captured from throwaway containers, plus a public v1 memory.stat capture.
  • The memory and tab choices as pure rules.
  • browser-memory.test.ts in real Chromium, added to CI's real-browser list.
  • The chart values check for the new keys and for browserIdleMs.

Locally: 97 real-browser tests, 523 computer unit tests, the chart check on every CI target, and format, lint and types all clean.

End to end with the computer image from this branch, in a container with a 1 GB memory limit:

  • Six Bots opened real pages in turn. At four browsers the working set reached 953 MB of 1024.
  • Before the next browser started, the computer closed the three least recently used and came back to 577 MB. There was no out-of-memory kill and no restart.
  • A closed Bot opened its page again after the usual fresh snapshot.

…le background tabs

A computer capped its browsers by count and closed them after half an hour untouched, and neither
looked at memory. A few heavy pages could take a container past its limit, and on Kubernetes the OOM
killer then kills everything in the container together, so every Bot's browser went at once.

The computer now reads its own working set the way the kubelet does: memory.current less
inactive_file on cgroup v2, the v1 files where only those exist, measured against the machine's
memory when the container has no limit, and nothing at all outside a container. Before each launch
and on each idle-sweep tick, at or above COMPUTER_MEMORY_HIGH_PERCENT (default 85) it closes browsers
one at a time, least recently used first by the cap's own order, reading memory again after each. It
never closes the browser being launched, the most recently used one, or the browser of a Bot whose
wheel a person holds or has been asked to take. Each close logs computer-browser-closed-for-memory
with the working set and the limit.

The idle sweep also closes a Bot's background tabs that have gone untouched for COMPUTER_TAB_IDLE_MS
(default ten minutes). A Bot is always on its newest page, so an older tab is one it cannot return
to. The page the Bot is on, a person's own browser, extension and DevTools pages, and any tab of a
Bot whose wheel a person holds are never closed. There is no per-browser tab count: memory is now
watched directly, and a count would close tabs a site opened moments ago.

Both settings are wired through the chart, compose, the all-in-one image and the docs. The chart
renders a zero and a whole number from a values file as the computer reads them, which the
new-values-key check now asserts on every target that runs the shared computer.
…inds of computer

computers.browserIdleMs and computers.maxBrowsers were rendered with `with`, so
`browserIdleMs: 0` (keep browsers open) was dropped, and a whole number from a
values file arrived as a float: an hour rendered as `3.6e+06`, which the computer
refused for its default. All four browser settings now go through the shared
computer environment, written in digits and kept at zero, so the computers
`computers.mode: sandbox` makes get them too. The values check covers
browserIdleMs alongside the two new keys.

@guidovizoso guidovizoso left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The approach holds up:

  • Memory is measured as the kubelet does: the working set without inactive file pages, for cgroup v2 and v1.
  • No cgroup, or one it can't read, closes nothing.
  • Browsers close gracefully, with cookies saved first.
  • The idle-tab close leaves alone the Bot's current page, the last tab, and any browser a person holds.
  • The Helm wholeNumber fix is correct, and the sandbox computers get the settings too.

A few things to fix, mostly because #871 will put all of a person's Bots on one computer:

  1. Launching one Bot can close another Bot that is mid-run. Before a launch, relieveMemory counts the Bot asking as the newest (profiles.ts:~597), and chooseForMemory spares only the newest (browser-eviction.ts:116). So any other browser can be closed, including one used a second ago. usedAt only moves on profiles.page(), so a Bot waiting on the model or running a long command looks idle. Its next action then fails with "Target closed", and it relaunches on a blank page. Your own end-to-end run closed 3 of 4 in one launch. The description and docs/configuration.md say the most recently used browser is never closed. Please make that true on launch: spare the newest existing browser too, or anything used in the last N seconds, or anything mid-run. Otherwise correct the wording.
  2. Nothing stops a close/reopen cycle. relieveMemory stops as soon as memory reads under 85%, and there's no minimum time a browser stays open. Three busy Bots whose pages together sit above the line will close each other on every relaunch (profiles.ts:948), and with no OOM nobody sees it. Please close down to a lower target (say 75%) and/or skip browsers launched or used in the last minute.
  3. Simultaneous launches close more browsers than needed. relieveMemory has no lock, and a closed browser takes about 2s to free its memory (closeAndWait). So K launches in a burst, like a schedule firing several Bots, close K browsers where one may have been enough. Please run one relieve at a time behind a shared in-flight promise.
  4. A person's own cloud browser can be closed for memory, mid-recording too. heldByPerson is false for person browser ids, and keptOpen only covers native windows. It's usually near the end of the order while the person is on the Browser page, but it's still eligible, and closing it ends a Recorder session without warning. Please spare it while it's being viewed or recording, or always close it last.
  5. The "instead of OOM-kill" claim is wider than the code. Memory is checked only every 60s (IDLE_SWEEP_MS) and before a launch, so a running browser that grows (a heavy single-page app, a big PDF) can still hit the OOM between checks. Please say that in the description and docs, or react to memory.events / memory.high.
  6. In the all-in-one s6 image, the 85% includes Postgres and the API server. The container's working set covers every service in it (docker/s6/s6-rc.d/computer/run), so browsers get closed to make room the database is using. Please document this, or default it off there.

Tests: there's nothing for concurrent launches, the close/reopen cycle, a Bot mid-action when its browser closes, or a person's cloud browser being closed.

…own to ten points under

Review found four ways closing browsers for memory could hurt the work it was protecting. A launch
counted only itself as in use, so a Bot that asked for its page a second ago and was waiting on its
model could be closed in the middle of a run. The most recently used browser already running is now
spared as well, and so is any browser used in the last minute (RECENTLY_USED_MS), which also keeps a
launch from closing the browser the launch before it just started. Closing stopped at the line it
started at, so the next launch crossed it again; a pass now starts at COMPUTER_MEMORY_HIGH_PERCENT and
closes on to ten points under it. Every launch ran a pass of its own and read memory before any close
had released it, so a burst of launches closed one browser each; one pass runs at a time and every
caller waits on the one in flight. A person's own browser, cloud or native, is never closed for
memory, because closing it ends their Recorder session without a word.

The docs now call this a check before each launch and once a minute rather than a guarantee, and say
that in the all-in-one image the share counts the database and the API server too, on purpose,
because the kernel's out-of-memory kill counts them as well. createProfiles takes a clock, so the
browser tests can make a browser a minute quiet without waiting a minute.
@davidmckayv

Copy link
Copy Markdown
Contributor Author

@guidovizoso thanks, all fair. Addressed in bec3d4e:

  1. Closing a Bot mid-run. chooseForMemory now spares the most recently used running browser, anything used in the last minute (RECENTLY_USED_MS), and the browser being launched, which isn't a candidate. "The most recently used browser is never closed" is now true.
  2. Close/reopen cycle. Closing starts at COMPUTER_MEMORY_HIGH_PERCENT (85) and continues until memory is under ten points lower (75). Together with the one-minute window, a launch never closes the browser the previous launch just started.
  3. Simultaneous launches. One pass at a time: concurrent launches and the sweep share a single in-flight promise.
  4. A person's own browser. Never closed for memory, cloud or native, so a Recorder session is never cut off by it.
  5. The claim. CHANGELOG, docs/configuration.md, values and .env now say what it is: a check before each launch and once a minute. A browser that grows between checks can still run the container out of memory. No memory.events / PSI handling in this PR.
  6. All-in-one image. Left on and documented (CHANGELOG, docs/configuration.md, the s6 run script). There the share counts the database and API server too, on purpose. The kernel's OOM kill counts everything in the container and can take the database down with it, so closing a browser is how the database is protected.

Tests for each, in real Chromium (browser-memory.test.ts) and as pure rules. A test fails when its fix is removed:

  • a burst of three launches closes only what's needed;
  • a Bot used seconds ago survives another launch;
  • no close/reopen cycle;
  • a person's own browser is never closed.

@guidovizoso guidovizoso left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at bec3d4e1. Approving: all six are addressed.

  1. Closing a Bot mid-run. chooseForMemory now spares the newest running browser, anything used within RECENTLY_USED_MS, and every person's browser, and the browser being launched is no longer a candidate. "The most recently used browser is never closed" now holds.
  2. Close/reopen cycle. It starts at 85 and closes down to memoryTargetPercent (75), reading memory again after each close finishes.
  3. Concurrent launches. relieveMemory shares one in-flight pass, and launches wait on it.
  4. A person's own browser is never chosen for memory.
  5. The docs now say it's a check before each launch and once a minute, not a guarantee against OOM.
  6. The s6 image: your reasoning holds. The OOM killer counts the database too, so closing a browser first is the better failure. And it's documented.

One note for later, not blocking: "recently used" still moves only on page actions. A Bot that has been waiting on the model or running a command for over a minute can still be closed under it. If that shows up in practice, marking a browser used while its Bot has a run in progress would close that gap.

@davidmckayv
davidmckayv merged commit f4083dd into main Oct 29, 2026
20 checks passed
@davidmckayv
davidmckayv deleted the david/browser-memory branch October 29, 2026 17:26
davidmckayv added a commit that referenced this pull request Oct 29, 2026
Brings in #879: a Bot's browser now also closes when the computer's memory
runs high and its idle background tabs close, sparing any browser whose wheel
a person holds, so a recording is never cut off by either.
davidmckayv added a commit that referenced this pull request Oct 29, 2026
Brings in a Bot's own browser with Record (#868) and closing browsers when
memory runs high (#879). Conflicts are resolved here; the Bot's browser is
moved onto per-person seats in the next commit.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants