Swap the JSON encoder/decoder used across the backend from stdlib json to
orjson when ENABLE_ORJSON is set — HTTP request bodies, JSONResponse
bodies, upstream provider responses, SSE chunks, and socket.io/Redis
payloads.
The flag defaults to off, in which case the app uses stdlib json and
engineio's codec verbatim, so default behaviour is unchanged.
- json_codec exports JSONCodec (stdlib json or the orjson codec) and
SOCKETIO_JSON (engineio's codec or the orjson codec); call sites import
JSONCodec and stay implementation-agnostic
- apply_orjson_http_json() is a no-op when the flag is off, leaving
starlette's Request.json / JSONResponse.render untouched
- the orjson codec falls back to the stdlib for inputs orjson rejects
(non-str dict keys, ints beyond 64 bits, NaN literals)
- orjson is imported only when the flag is on
- FastAPI(default_response_class=...) is deliberately not used: an
explicit default disables the Pydantic direct-to-bytes fast path for
response_model routes
build_matcher compiled a caller-supplied pattern with Python's backtracking re and ran it over every line of every reachable file, with no timeout, no thread offload and no length caps. is_regex_pattern promotes any pattern containing a metacharacter, and a bare pipe counts, so no explicit regex flag is needed to reach the compiler. The search loop is synchronous inside an async handler, and UVICORN_WORKERS defaults to 1, so the cost lands on every other user of the instance. MAX_GREP_RESULTS bounds how many matches are reported, not how much work is done.
Backtracking cost is exponential in the length of the text being matched, so capping the pattern or the line does not bound it: the subject in the measurements below is 30 characters. `(x|x)*y` against a line of 30 x took 80 seconds, `(a+)+$` against 32 a took 169 seconds, and the same subject with a literal pattern took 0.6 microseconds.
Matching now runs on the regex module, which accepts a per-search timeout that re has no equivalent for. The timeout is the actual bound: regex resolves many classic catastrophic patterns instantly, but not all of them, and `(a|aa)+$` and `(?:a|a)*$` still need it. The budget covers a whole tool call rather than a single search, because a pipeline builds one matcher per segment and a per-search budget would multiply by segment count, and because a per-line timeout would allow timeout multiplied by line count. It is carried in a context variable so one command shares it without threading a parameter through every handler, and it is charged only for time spent inside search(), so database round-trips and other coroutines cannot consume it. Exhausting it raises, and both entry points already render that as an error for the model to read.
Note for anyone tracking search behaviour: re and the regex module define \w, \W and \b differently on non-ASCII text. re follows str.isalnum(), the regex module follows UTS#18, so \w no longer matches superscripts and fractions such as the ones in Nd-adjacent categories, and now does match combining marks. POSIX classes like [[:alpha:]] are interpreted rather than read as a literal set, and \p{...} compiles instead of erroring. Results on ASCII content are unchanged.
regex was already installed as a transitive dependency of nltk, tiktoken and transformers. It is now declared directly, pinned in pyproject.toml and requirements.txt to the version the lockfile already resolves.
* fix: fetch terminal system prompt per request with TTL cache
The system prompt was only fetched once in set_terminal_servers (startup
or connection save) with a 3s timeout, using a synthetic 'system' user.
That snapshot silently stays empty when the fetch races a cold-started
orchestrator instance, and goes stale when instances are reprovisioned
with a changed OPEN_TERMINAL_SYSTEM_PROMPT — recovering only after a
restart or a manual connection re-save.
- Fetch /system during get_terminal_tools with the user's own
credentials via a central TTL-cached method (5 min per server+user;
failures cached 60s so a dead instance doesn't stall every request),
falling back to the cached snapshot.
- Raise the fetch timeout from 3s to 30s so cold-provisioned instances
can answer.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KjnQJNKozp47vTB13pRyYs
* refac: fetch the terminal system prompt per request without a cache
Drop the module-level TTL cache and fetch the system prompt directly in
get_terminal_tools, gathered with the existing uncached per-request cwd
fetch that already follows this pattern. The fetch uses the user's own
credentials and falls back to the set_terminal_servers snapshot, so a
cold or unreachable instance degrades to the previous behaviour instead
of needing an error cache. Also restore the 3s timeout: on the request
path a 30s wait would stall chat completions, and a cold instance is
covered by the snapshot fallback until it warms up.
---------
Co-authored-by: Claude <noreply@anthropic.com>
