175 Commits
Author SHA1 Message Date
Classic298 b3bb82e776 refac: scope ydoc update save scheduling to note documents (#30395)
The ydoc document update handler now schedules the debounced save only for note documents, since notes are the only ydoc documents with a save handler.
2026-09-23 23:30:21 -04:00
Classic298 1a74a9f46c perf: per-room channel delivery for the socket.io Redis manager (#28818)
Reworked from the ground up after the feedback that the registry implementation did not land. The Redis room registry and its whole recovery protocol (heartbeats, liveness keys, pruning, distrust windows, cache invalidation) are gone; the change is now ~105 lines with no state kept outside the process.

With WEBSOCKET_MANAGER=redis every emit is published on one shared channel and every instance JSON-decodes every message: a 16 instance fleet decodes each streamed token delta 16 times and 15 discard it. py-spy across a loaded fleet (16 instances, ~4000 users) puts ~31% of all active CPU samples in the pubsub listener parse chain, the largest bucket.

Room-targeted emits are now published on a per-room channel instead; every instance keeps one static pattern subscription covering all room channels and drops messages for rooms without local members by channel name, paying a set lookup instead of a JSON parse. No state leaves the process, so recovery paths and loss windows are identical to the stock manager; acks and control messages stay on the shared channel and sio.call works across instances unchanged. This is the delivery scheme the official socket.io Redis adapter for Node.js ships by default.

Enabled by default; WEBSOCKET_REDIS_ROOM_CHANNELS=false restores shared-channel-only delivery. All instances must run the same mode, so the switch rides the full-stop upgrade this release already requires for its migration; in a mixed fleet, room emits from updated instances would not reach not-yet-updated ones. Verified end to end with two instances on a real Redis: cross-instance token streams delivered with the shared channel completely silent. Ref #28173.

<!--
🚨 DO NOT DELETE THE TEXT BELOW 🚨
Keep the "Contributor License Agreement" confirmation text intact.
Deleting it will trigger the CLA-Bot to INVALIDATE your PR.

Your PR will NOT be reviewed or merged until you check the box below confirming that you have read and agree to the terms of the CLA.
-->

- [x] By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms.

> [!NOTE]
> Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in.
2026-09-22 14:48:56 -04:00
Timothy Jaeryang BaekandClassic298 e93a59f4dd refac
Co-Authored-By: Classic298 <27028174+Classic298@users.noreply.github.com>
2026-09-22 13:13:22 -04:00
Timothy Jaeryang BaekandClassic298 478d1785fd refac
Co-Authored-By: Classic298 <27028174+Classic298@users.noreply.github.com>
2026-09-21 11:21:42 -04:00
Classic298 aceda892bd perf: serve the shared model pool from a per-worker cache (#28176)
With the Redis websocket manager the shared model pool is a `RedisDict`, so resolving a model id fetches from Redis, and more than one of those fetches pulls the whole pool. Every chat request pays that latency and that traffic, and the cost grows with the number of models configured: at 120 models a request moves several hundred kilobytes to ask about models that have not changed since the last request.

The pool now keeps a per-worker cache of the hash and refetches it only when the signature key that `set()` already maintains changes. A cache may only trust a signature that describes the bytes it fetched, so every write invalidates the signature and no signature value is ever issued twice. Without a fresh token each time, a pool that flaps back to an earlier state returns to an earlier digest, so a reader that races the write keeps serving the old pool. `delete_many()` was a second hole in that: it updated a dead attribute and never touched the signature, so readers kept serving models it had already removed. A TTL cache would have been simpler, but it serves a pool it already knows may be stale and costs the same single round trip this check costs.

Replaying the real access pattern, a request that finds the pool unchanged moves about 1 KB no matter how many models are configured, over the same number of round trips as before, except after a write that leaves no signature, which holds reads at two commands until the next non-empty `set()` writes one. A request that first sees a changed pool costs one extra command, and so does rewriting the pool. Values stay cached serialized, so `items()` and `values()` still decode on every call, and each worker holds the whole pool in memory.
2026-09-21 10:15:49 -04:00
Classic298 488b3c571c perf: stop note co-editing echoing every remote update back to the server (#28185)
Every client in a note re-broadcasts each update it receives, with a full content snapshot attached, and the server appends each echo to the document log and writes the note again. Traffic and note writes therefore scale with the number of people who have the note open: every extra participant adds one more full echo of every keystroke.

Remote updates are now applied with the `'server'` origin the state path already uses, which the local listener ignores, so the echo stops. The echo did carry one thing worth keeping: the receiving client holds the merged document, which the sender had not seen yet, so each receiver now sends a content-only message once the edits settle, debounced 500ms, and flushes a pending one when the editor is torn down. The backend accepts an update message with no `update` field for that case, where it previously raised and dropped the save.

Replaying keystrokes at 120ms with a real Yjs document, socket messages fall 48.8% with two clients, 65.6% with three and 79.2% with five, with byte counts tracking the same on notes up to 50KB. Server-side update appends drop by a factor of the client count. The cost is one snapshot upload per receiving client per typing pause, which the server's own debounce then collapses into a single extra note write however many people are watching. A snapshot has to come from a client because the server cannot rebuild the markdown, HTML and JSON shape the note record stores. It also moves the merge 500ms later than the echo delivered it, so if two edits cross on the wire and every editor then loses its connection and closes inside that window, the note keeps what it was last sent and one of the two edits is lost. A client still connected at teardown flushes its pending snapshot, and any other editor left in the note closes the gap.
2026-09-21 08:53:27 -04:00
Timothy Jaeryang Baek 0180efecf3 refac 2026-09-21 08:43:45 -04:00
Timothy Jaeryang Baek 58b36765a7 refac 2026-09-16 22:47:37 -04:00
Classic298 55b7343be8 fix: keep a failed lock release from masking cancellation (#29979)
When Redis is unreachable at shutdown, the socket cleanup tasks do not stop. release_lock is a bare eval called from the finally of both periodic_session_pool_cleanup and periodic_usage_pool_cleanup, so it raises there and replaces the CancelledError already in flight. The usage task's except Exception then catches the Redis error and carries on reaping after shutdown cancelled it, and the session task ends with a ConnectionError in place of its cancellation.

release_lock now logs the failure and returns. aquire_lock sets the key with ex=self.timeout_secs and renew_lock re-expires it with the same value, so a release that never lands costs at most one lock timeout before another node can take over.

The except names both RedisClusterException and RedisError because the cluster-only types subclass Exception directly, and redis_cluster is a supported configuration, so RedisError alone would miss the outage on a cluster. Genuine bugs still propagate.

Verified against a real Redis: acquire, refusal while held, renew, compare-and-delete release and non-owner release are unchanged, a populated pool reaps identically with identical lock TTL lifecycles, and both tasks now cancel cleanly where before one kept running and the other died with the wrong exception.
2026-09-13 20:37:51 -05:00
Classic298 263e56e272 fix: keep the session pool reaper alive through a Redis error (#29976)
A single Redis blip permanently stops orphaned websocket sessions from being reaped. periodic_session_pool_cleanup acquires its lock outside the try, and that try has only a finally, so the first timeout or connection reset ends the coroutine for the life of the process. The session pool then only grows, and the sole trace is one "Task exception was never retrieved" at shutdown.

The loop body gets the same try/except Exception its sibling periodic_usage_pool_cleanup already has, which also brings the lock acquire inside the guarded region. The task now logs, releases the lock and retries after the existing delay, so another node can take the lock over meanwhile.

The diff reads long because the body is re-indented one level; nothing changes beyond indentation and the four added lines. Only Redis deployments are affected, since the lock functions are lambda: True otherwise.

Verified by injecting a ConnectionError at each of the four failure points (acquire, renew, the batch scan, the reaping delete), against a real Redis as well: the task survives all four and keeps retrying, where it previously died on the first. Reaping results, lock acquire and release counts, and cancellation at shutdown are unchanged.
2026-09-13 20:28:10 -05:00
Classic298 89716ea880 perf: stop scanning every socket.io payload for binary data (#28180)
* perf: stop scanning every socket.io payload for binary data

Every socket.io event the backend sends was first walked recursively to check whether any value was a bytes object needing binary attachment framing. Open WebUI never emits binary, so the walk always came back empty and the work was thrown away. It has no early exit and allocates at every level, so it scaled with the full size of the message, and the messages are the big ones: chat streaming re-emits the whole assistant message on every update, note collaboration sends document state as a JSON array with one entry per byte. With the Redis manager it ran once per instance per emit on top of that, since every instance builds its own copy of the packet.

The server now installs a Packet subclass with binary events off, through python-socketio's own serializer hook, the same mechanism its msgpack serializer uses. Inbound binary attachments are decoded to int lists rather than refused, so the one frontend path that sends a raw Uint8Array keeps working and handlers can still echo client data straight back out. One scan remains in multi-instance setups: python-socketio's Redis manager calls it on the base Packet class directly, where the serializer hook cannot reach.

Measured per encode:

| payload | before | after |
|---|---|---|
| chat completion re-emit (7.5 KB JSON) | 30 us | 13 us |
| collaborative document state (292 KB JSON) | 9.0 ms | 1.7 ms |

With ENABLE_ORJSON=true, where the scan is nearly the whole encode cost: 20 us to 2.3 us, and 7.8 ms to 0.14 ms.

Closes #28164

* fix: match the other Yjs emits and send the full state as an array

Collaboration.ts sent the initial full-document state as a raw Uint8Array while the other two Yjs emit sites convert with Array.from first. socket.io framed that one as a binary attachment, so with the JSON-only packet class the server turns it into a list of ints and re-broadcasts it as JSON: a 10240-byte state update becomes 36561 JSON characters. Converting at the emit site keeps the wire form uniform across all three sites.

Also trims the JSONOnlyPacket docstring, which claimed attachments already arrive as int lists when the override is what converts them, and annotates the new reconstruct_binary parameters.
2026-08-31 01:22:06 -04:00
Classic298 d7674c5174 perf: bounded non-blocking session pool reaper, fewer blocking pool round trips (#28835)
* perf: stop the Socket.IO session pool blocking the websocket event loop

With WEBSOCKET_MANAGER=redis the session pool is a synchronous Redis client, so every call into it blocks the whole worker's event loop, not just the caller. Two paths did it constantly: the orphan reaper walked the pool one round trip per session with no await anywhere, freezing the loop for the entire sweep every cycle, and nearly every socket event re-read the sender's session back out of Redis. Other users' events and every in-flight generation on that pod wait behind both.

The reaper now walks the pool in HSCAN batches and deletes in bulk, yielding between batches, and no longer sleeps past half the lock TTL, which previously guaranteed a failed renew every cycle. The per-event reads are gone: Socket.IO events only reach the worker holding the connection, and that worker already saved the same session dict locally when the user authenticated, so it was asking Redis for its own data. The writes stay, since those are what other pods read.

Measured at 4000 users / 16 containers, Redis 1.1 ms away:

| | before | after |
|---|---|---|
| reaper sweep, 5k sessions | 5.6 s, loop frozen throughout | 62 ms, 4.2 ms worst block |
| same, crash recovery with every session expired | 11.5 s | 96 ms |
| heartbeat / usage ping / disconnect | 2 / 3 / 2 round trips | 1 / 2 / 1 |
| 50-member channel post | 50 round trips, 57.2 ms block | 0 round trips, 0.02 ms |
| loop time per wall second at rest | 103 ms (10.3%) | 67 ms (6.7%) |

The alternative, converting RedisDict to the async client, fixes the same paths with a far larger blast radius (every call site gains await, and `in`/`[]`/`del` cannot be awaited so the dict interface goes) and still round-trips for data already in memory. Two deliberate behaviour changes: a heartbeat re-adds a session the reaper already removed, so a tab that survives a stall recovers instead of staying out of the pool until it reconnects; and disconnect no longer skips Yjs document cleanup when the pool entry is already gone, which previously leaked that document's update log forever.

Closes #28172

* perf: cut disconnect and user session lookup pool round trips, harden the session reaper

Follow-up on top of the session pool reaper branch. With WEBSOCKET_MANAGER=redis two paths still blocked the worker's event loop on synchronous Redis calls. Every disconnect listed all models in use cluster-wide and fetched each one individually, one blocking round trip per model. Disconnecting all sessions of a user (admin role change or deletion) pulled the entire session pool in one HGETALL and decoded every entry in a single uninterrupted block.

Disconnect now fetches the usage pool once with items(), going from 2+N+M round trips to 2+M (N models in use cluster-wide, M models the session used), and its delete of an emptied model entry is KeyError-guarded because another node can remove the same key between snapshot and delete; unguarded, that race aborted the handler and skipped its Yjs document cleanup. The user session lookup reuses the reaper's HSCAN batches and yields to the loop between pages. The reaper previously died permanently on the first Redis connection error, on every node at once during an outage; it now logs, releases the lock and returns to retrying acquisition.

* refac: keep the socket pool perf work to the round trips

A review pass on this branch turned up four changes riding along with the round-trip work without belonging to it, so they are backed out here. The `disconnect` handler keeps its `if sid in SESSION_POOL:` guard, so USAGE_POOL and ydoc cleanup stay off the path for sockets that never authenticated. `RedisDict.set()` keeps its own inline HDEL. `get_session_ids_by_user_id` stays synchronous over one HGETALL, since it runs on user delete and role change rather than per message. The crash-resilience wrapper around the reaper loop is dropped; if that guard is worth having, it belongs in its own change.

What stays is the perf part. The reaper now sweeps the pool in bounded HSCAN batches and deletes expired sids with one HDEL per batch, down from HKEYS plus an HGET and a per-sid HDEL across the whole pool. The `disconnect` handler reads USAGE_POOL with a single HGETALL, down from HKEYS plus one HGET per model in use. Session lookups in the socket handlers come from the local Socket.IO store, which removes one Redis GET from every heartbeat, usage, channel and ydoc event.

Naming and annotations follow the file: `get_session_pool_batches` for the module's `get_` prefix, `RedisDict.pop_many` so both reaper branches use one word for removing keys, a named `SCAN_BATCH_SIZE`, and types on the new helpers.

* fix: invalidate the RedisDict write signature on batch delete

RedisDict.set() skips the write when the payload fingerprint matches the last one this process wrote, so a mutation that goes around set() has to clear that fingerprint. The new batch delete did not, leaving a stale fingerprint behind: the next refresh with identical content is treated as already written and silently skipped, so the hash stays empty.

Renamed pop_many to delete_many. In a dict emulation pop removes and returns; this returns nothing and cannot without an extra HMGET, so the name promised something it does not do. delete_many matches __delitem__ and the HDEL underneath. Its only call site is the session pool reaper, whose behaviour is unchanged: same fields deleted, same batching, same return.
2026-08-31 01:17:53 -04:00
Timothy Jaeryang Baek aeb126b95d refac 2026-08-30 23:46:02 -04:00
Timothy Jaeryang Baek 5c62cc0517 chore: format 2026-08-25 16:53:53 -04:00
Timothy Jaeryang Baek 3c1017f6c3 refac 2026-08-25 12:22:19 -04:00
Timothy Jaeryang BaekandClassic298 6330350a40 refac
Co-Authored-By: Classic298 <27028174+Classic298@users.noreply.github.com>
2026-08-25 10:39:49 -04:00
Sven Horvath 5735123f50 fix: keep the pending note save when an update carries no content snapshot (#28669)
stop_item_tasks() ran unconditionally while create_task() only ran when the
update carried data, so an update without a content snapshot cancelled the
pending save without scheduling a replacement and the edits were never
written.
2026-08-24 18:47:43 -04:00
Timothy Jaeryang Baek 9dff5e9327 refac 2026-08-24 18:37:39 -04:00
Timothy Jaeryang Baek 536b9edec0 refac 2026-08-24 18:35:04 -04:00
Classic298 5586964bb2 fix: keep the usage pool cleanup task alive across lock loss and Redis errors (#28834)
With WEBSOCKET_MANAGER=redis on a multi-node deployment, the usage pool cleanup task could stop permanently for the whole cluster. Nodes that lost the startup lock race gave up for good after three attempts, and the winner died on a single failed renew or on any Redis connection error, releasing the lock with nobody left to take it over. From then on expired entries accumulated in the usage pool until a node restarted, so /api/usage over-reported models in use and every disconnect handler walked an ever-growing pool.

The task now retries lock acquisition forever like the session pool cleanup does, and any error is logged and answered by releasing the lock and returning to acquisition, so a transient failure costs one cleanup cycle and every node stays a takeover candidate. The delete of an emptied model entry is KeyError-guarded because a disconnect handler on another node can remove the same key between the sweep's snapshot and its delete; unguarded, that race was a permanent task killer that needed nothing rarer than a chat finishing while its tab closed.
2026-08-20 12:59:38 -07:00
Timothy Jaeryang Baek a1579a01ff refac 2026-08-14 00:22:17 -06:00
Timothy Jaeryang BaekandG30 939bcdb79e refac
Co-Authored-By: G30 <50341825+silentoplayz@users.noreply.github.com>
2026-08-12 01:13:38 -06:00
Timothy Jaeryang Baek ce3c175e26 refac 2026-08-10 23:13:10 -06:00
Classic298 a39126c27c fix: catch the socket.io timeout in the event caller (#28311)
An interactive prompt raised by __event_call__ was meant to come back as an error dictionary when it timed out. It never did: sio.call raises socketio.exceptions.TimeoutError, which does not inherit from the builtin TimeoutError the handler was catching, so the exception escaped into plugin code instead. Because that exception carries no message, the call sites that wrap plugin calls in except Exception as e turned it into an empty string, so a timed-out prompt looked like an empty answer rather than a failure, and the error branches written for it were dead.

The handler now catches socketio's class alongside the builtin, so a timeout returns the intended error dictionary and a plugin can tell the two apart.

The session eviction that sat inside that handler is removed rather than switched on. It had never executed, and it is wrong in both directions: it compares the pool entry by value, which the heartbeat rewrites every thirty seconds, so it would usually not fire, and when it did fire on a short timeout it would evict a live tab whose user had simply not answered yet, with nothing to restore the entry short of a reload. Genuinely dead sessions are already reaped on missed heartbeats by periodic_session_pool_cleanup.

WEBSOCKET_EVENT_CALLER_TIMEOUT is unset by default, which means no timeout at all, so this only affects deployments that set it.
2026-08-08 18:33:12 -06:00
Classic298 2d18727ab8 perf: build info log messages lazily so raising the log level actually saves work (#27837)
Raising GLOBAL_LOG_LEVEL to WARNING buys quieter output but not less work: 241 INFO call sites interpolate their payload into an f-string before the logging call gets to drop it. The heaviest is get_doc, which logs every chunk id and metadata dict in a collection, so on the full-context retrieval path that is the entire knowledge base, once per chat request.

That one line at WARNING, CPython 3.12:

| knowledge base | payload | before   | after   |
| -------------- | ------- | -------- | ------- |
| top-k of 3     | 1.2 kB  | 3.8 us   | 0.07 us |
| 500 chunks     | 201 kB  | 583.6 us | 0.08 us |
| 5000 chunks    | 2.0 MB  | 5.8 ms   | 0.15 us |

The lazy form log.info('query_doc:result %s %s', result.ids, result.metadatas) hands the payload to record.getMessage(), which the InterceptHandler only reaches once a record has passed the level check. Output at INFO is byte-identical. Two sites that already built their message eagerly, one str concat and one % operator, move to the same lazy form.
2026-08-02 15:39:10 -05:00
Classic298 52cfb02c72 perf: build debug log messages lazily so disabled debug logs cost nothing (#27834)
GLOBAL_LOG_LEVEL defaults to INFO, so every log.debug(...) in the backend is discarded, but the message is built first: 187 call sites interpolate their payload into an f-string before the logging call runs, so the work happens on every request and the result is thrown away. The worst one sits in process_chat_payload and stringifies the whole request body, full conversation history included, once per chat completion.

That one line with DEBUG disabled, CPython 3.12:

| conversation | payload | before   | after   |
| ------------ | ------- | -------- | ------- |
| 4 messages   | 1.2 kB  | 3.4 us   | 0.07 us |
| 20 messages  | 17 kB   | 24.8 us  | 0.07 us |
| 60 messages  | 123 kB  | 216.6 us | 0.07 us |

The lazy form log.debug('form_data: %s', form_data) hands the payload to record.getMessage(), which the InterceptHandler only reaches once a record has passed the level check. With DEBUG enabled the emitted lines are byte-identical, f'{x=}' sites included: those map to %r. MistralLoader._debug_log callers get the same treatment, since that wrapper already forwards *args.
2026-07-31 19:09:01 -05:00
Classic298 72fdf238a8 perf: optional orjson JSON codec behind ENABLE_ORJSON (#27583)
Swap the JSON encoder/decoder used across the backend from stdlib json to
orjson when ENABLE_ORJSON is set — HTTP request bodies, JSONResponse
bodies, upstream provider responses, SSE chunks, and socket.io/Redis
payloads.

The flag defaults to off, in which case the app uses stdlib json and
engineio's codec verbatim, so default behaviour is unchanged.

- json_codec exports JSONCodec (stdlib json or the orjson codec) and
  SOCKETIO_JSON (engineio's codec or the orjson codec); call sites import
  JSONCodec and stay implementation-agnostic
- apply_orjson_http_json() is a no-op when the flag is off, leaving
  starlette's Request.json / JSONResponse.render untouched
- the orjson codec falls back to the stdlib for inputs orjson rejects
  (non-str dict keys, ints beyond 64 bits, NaN literals)
- orjson is imported only when the flag is on
- FastAPI(default_response_class=...) is deliberately not used: an
  explicit default disables the Pydantic direct-to-bytes fast path for
  response_model routes
2026-07-27 03:45:37 -04:00
Timothy Jaeryang Baek 498cdab9a5 refac 2026-07-27 02:23:33 -04:00
Timothy Jaeryang Baek 6f93ecd4fd refac 2026-07-26 23:49:03 -04:00
Timothy Jaeryang Baek b7489bbc6c refac 2026-07-26 23:16:58 -04:00
Timothy Jaeryang Baek bf35f64a7f refac 2026-07-26 22:45:11 -04:00
Timothy Jaeryang Baek 846ba80a9d refac 2026-07-26 22:32:03 -04:00
Timothy Jaeryang Baek d14fddf254 refac 2026-07-26 21:55:13 -04:00
Timothy Jaeryang Baek 0cbf337679 refac 2026-07-26 21:54:06 -04:00
Timothy Jaeryang Baek d484a2a99e refac 2026-07-26 21:07:20 -04:00
Timothy Jaeryang Baek f798d05586 refac 2026-07-26 19:34:41 -04:00
Classic298 e140d8f3cc fix: scope timer cancellation to the timer's owner (#27472)
The events:chat socket handler called the ownership-checked update for last_read_at, discarded the boolean it returns, and then cancelled the chat's pending timers regardless of the answer. cancel_timers_for_chat selected on the internal marker, the type, the parent chat id and the status, and never on the owner, so it matched rows belonging to any user. An authenticated user who knew another user's chat id could mark that chat read over their own socket session and silently cancel the owner's pending timers, and the owner got no notification: the scheduled action simply never fired.

The missing owner predicate also cut the other way in ordinary use. Because the query matched every timer sharing a parent chat id, one user reading a chat cancelled the timers of anyone else holding one on the same chat, so this was collateral damage as much as an attack.

cancel_timers_for_chat now requires a user_id and filters on it, which is the durable fix, and the socket handler returns early unless the ownership-checked update reports that the caller owns the chat. The parameter is required rather than defaulted so a later caller cannot reintroduce the unscoped query by omission. Both existing call sites already know the acting user. Timer rows are created with the same owner as the parent chat and the execution path already refuses to run one whose owner does not match, so scoping the cancellation the same way cannot strand a timer that would otherwise have fired.

One behaviour change worth noting: an administrator posting into another user's chat no longer cancels that user's chat.user_message timers, because the acting user is the administrator. The timer fires instead of being cancelled, which is the safe direction.
2026-07-26 18:12:44 -04:00
Classic298 f517cc7172 fix: apply the verified-user role gate to WebSocket authentication (#27537)
The Socket.IO handshake and the terminal WebSocket route each reimplement JWT authentication instead of going through the HTTP dependency chain. Both verified that the token decoded, that it had not been revoked, and that the user row existed, but neither applied the role check that `get_verified_user` enforces on every HTTP route, so any role outside `user` and `admin` was accepted.

That splits authorization across two planes. Deactivating an account by setting its role to `pending` takes effect immediately over HTTP, which returns 401, while the same JWT still opens a WebSocket. Changing a role disconnects the account's live sockets but does not revoke its token, so the client simply reconnects and gets a fresh session. Until the token expires, four weeks by default, a deactivated account keeps its channel rooms and can still read and write any note it holds an access grant on through the collaborative document handlers.

Resolve the user once, in `get_verified_user_by_token`, and route both WebSocket entry points through it. The role set moves into `VERIFIED_USER_ROLES` so the HTTP and WebSocket gates cannot drift apart, which is the underlying cause rather than either call site on its own. This also replaces five copies of the decode, revocation check and user lookup sequence.

`user-join` now resolves the user instead of reusing the identity cached in `SESSION_POOL`, which costs one extra query per handshake. Gating on the cached role would make the authorization decision depend on every future role-mutation path remembering to tear down the session pool, and that is precisely the invariant that failed here.
2026-07-26 17:27:54 -04:00
Timothy Jaeryang Baek 021c4c7a2e refac 2026-07-23 22:20:50 -04:00
Classic298andClaude 656a848043 perf: halve Redis round trips on model resolution and socket pools (#27225)
When WEBSOCKET_MANAGER=redis, app.state.MODELS and the socket session/
usage pools are Redis-backed dicts, so every membership test and
getitem is a network round trip:

- generate_chat_completion checked `model_id not in models` (HEXISTS)
  and then read `models[model_id]` (HGET) on every chat completion.
  A single .get() now serves both, with the same not-found error.
- The direct-connection branch spread the pool with `{**MODELS, ...}`,
  which iterates keys() then fetches each value — HKEYS plus one HGET
  per model. dict(MODELS.items()) issues a single HGETALL instead.
- get_user_ids_from_room called SESSION_POOL.get(sid) twice per
  session (once to filter, once for the value); the usage handler
  checked membership then fetched the same key. Both now do one
  lookup.

In non-Redis mode these are plain dicts and behavior is identical.


Claude-Session: https://claude.ai/code/session_01MHg5zs1VBjvRWQ54qHpfYD

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-23 21:32:01 -04:00
Timothy Jaeryang Baek 858e9236df refac 2026-07-23 19:12:47 -04:00
Classic298 6b655689cc perf: cut repeated per-model work out of model list assembly
get_all_models runs on every models refresh and, without the base-models cache (off by default), on every /api/models request. Several of its costs multiplied by the model count for no reason:

- The active action and filter id sets were derived from get_functions_by_type, which loads full function rows including plugin source and validates them, only for the ids and is_global flags. A generalized column-only query now returns (id, is_global) tuples; the existing filter-specific helper delegates to it.
- Action priorities were computed inside the per-model sort key, constructing a pydantic Valves object per action per model; with global actions in every model's list that was models x actions constructions per refresh. Priorities are now memoized per action.
- Global action and filter item dicts were rebuilt per model from the same modules. The item lists are now built once per function and shallow-copied per model, keeping per-model dicts independent exactly as before (nested values were already shared).
- Deactivated base-model overrides were dropped with models.remove, a linear scan and shift per removal; removals are now collected and filtered out in one identity-based pass, preserving list.remove's exact object semantics.
- RedisDict.set fingerprinted the payload by serializing the already-serialized mapping a second time plus a sha256; a direct dict comparison against the last written mapping has the same skip semantics without re-serializing anything.
- /api/models did tag normalization and profile-image stripping for every model before access filtering discarded the invisible ones, and always evaluated a json.dumps debug f-string; the work now runs only on visible models and the debug line is gated on the log level. The duplicate-id dedup keeps its position before filtering so the effective-model semantics are unchanged.

Benchmark:

| metric | before | after |
| --- | --- | --- |
| model-cache fingerprint, 200 models | 45 us | 1.4 us |
| action priority Valves builds, 200 models x 4 global actions | 0.37 ms (800 builds) | 0.002 ms (4 builds) |
| function-table payload for id sets | full rows incl. source | (id, is_global) tuples |

Functionally verified: the column-only id query matches the full-row query for actions and filters including inactive exclusion, and the fingerprint skip logic writes on first set, skips identical payloads, updates plus deletes stale keys on change and clears on empty, against a scripted fake Redis.
2026-07-23 19:05:32 -04:00
Timothy Jaeryang Baek ca11bd90a7 chore: format 2026-07-23 13:41:16 -04:00
Classic298andClaude Fable 5 32242a6788 perf: 40% LESS CPU usage: cut per-instance CPU cost of shared socket.io Redis pub/sub channel (#27282)
* perf: cut per-instance CPU cost of shared socket.io Redis pub/sub channel

Profiling a multi-instance deployment (py-spy --gil) showed ~44% of worker
CPU in the socket.io pub/sub listener. Two causes, two fixes:

- Add hiredis so redis-py parses the RESP protocol in C instead of pure
  Python (redis/_parsers/resp3.py alone accounted for ~28% of GIL samples;
  redis-py auto-selects the hiredis parser when importable).

- Subclass AsyncRedisManager to drop emits whose target room has no local
  participants before upstream _handle_emit re-encodes the full packet.
  Every instance receives every emit published on the shared channel, so
  with N instances all but the hosting one were paying full packet
  re-serialization per message just to deliver it to nobody. Broadcasts
  (room=None) are unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q9CQ9qnp3sZGYQQwztsCJT

* fix: restrict pub/sub emit early-out to string rooms

Adversarial review against python-socketio 5.16.2 found one divergence
from upstream: for a degenerate empty-sequence room (emit to room=[]) on
an instance whose namespace has no local clients, the filter's
get_participants probe raises IndexError from room[0] where upstream
returns silently at the namespace guard and still publishes to Redis.
Open WebUI only ever emits to scalar string rooms or room=None, so the
case is unreachable today; guard on isinstance(room, str) anyway so any
non-string room shape passes through to upstream behavior unchanged.

Every open-webui emit uses a string room, so the fast path still covers
all real traffic.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q9CQ9qnp3sZGYQQwztsCJT

* Update requirements.txt

* Update pyproject.toml

* Update requirements-min.txt

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-23 12:11:38 -04:00
Timothy Jaeryang Baek b23ddeb280 refac 2026-07-16 00:30:44 -04:00
Timothy Jaeryang Baek a9617ca218 refac 2026-07-14 01:15:29 -04:00
Timothy Jaeryang Baek 7088d245bb refac 2026-07-14 00:10:28 -04:00
Timothy Jaeryang Baek 33b91bd8ae refac 2026-06-29 03:58:00 -05:00
Timothy Jaeryang Baek ac3449cac9 refac 2026-06-29 02:26:27 -05:00
Classic298andClaude Opus 4.8 386ac95814 fix: scope Socket.IO event-caller to the requesting user's own session (#25763)
get_event_call() routed execute:python / execute:tool events to a client-supplied session_id after only checking the session was connected, not that it belonged to the requester. Verify the target session is owned by the requesting user (metadata user_id) before delivering, so a client cannot route code/tool execution into another user's session.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 02:17:40 -05:00