Custom metadata attached to a file upload now reaches the vector DB, but the model still never sees it. Both prompt-assembly paths build their output from a fixed field set: the classic RAG <source> tag carries only id, name and resource type, and the retrieval tools return only content, source and file id per chunk. A scraper that records where each document came from therefore cannot get that origin in front of the model, so answers cannot state it.
RAG_SOURCE_METADATA_KEYS names the chunk metadata keys allowed through to the model. Configured keys are emitted as extra attributes on the <source> tag and as extra fields on tool result chunks, covering both retrieval paths. It is empty by default, so nothing changes for existing deployments.
An allowlist instead of passing everything through, because chunk metadata also carries file hashes, collection names, embedding config and relevance scores, which would then be added to every retrieved chunk of every request. Values are attacker-controllable through an uploaded file, so they are escaped before they go into the tag, and a configured key can never displace a field the tag or the chunk already defines.
Reported in open-webui/open-webui#29486.
The ask_user card badges the first option of every question as "Recommended", but nothing ever told the model that. The model picks whatever order it likes, so the badge really means "listed first" and users act on a recommendation the model never made.
The ask_user tool description now states that the first option is labelled Recommended and that the model should list the option it recommends first, so the badge reflects an actual choice.
Kept the badge and instructed the model instead of adding a per-option "recommended" flag: the flag would need a schema change, validation in the request normalizer and a frontend change, for the same result in the common case. Dropping the badge was the other option, but it removes a useful affordance rather than fixing it.
Verified that the added line reaches the model by running the docstring through the tool-spec builder and checking the generated OpenAI function schema.
Fixes#30195
Asked to add a section or change a few lines, models answer with a whole-note
replace_note_content call and the rest of the note is gone, with nothing to
undo because the editor only records versions on chat inserts. The tool
already supports replace_range operations with an expected guard, but its
description never said when to use them or how the offsets work, so models
defaulted to sending the whole note back.
The docstring, which is the description every model receives for this tool
in note chats and normal chats alike, now states the preference for range
edits and the offset, overlap and expected rules the handler enforces.
Verified the text lands in the generated tool spec unchanged and that range
edits, the expected mismatch rejection and whole-note replace behave as
described against a sqlite data dir.
kb_exec silently ignored ls -t. It accepted the flag, dropped it and
returned the same undefined database order as a plain ls, so the model
believed it had a newest-first list when it did not. With a few thousand
files in a knowledge base there was no way to ask what changed recently
without reading the whole listing and comparing dates by eye.
ls, tree and find now sort their file lines by name by default, so the
same knowledge base always lists the same way. -t sorts newest first,
-S largest first and -r reverses, combinable like -at or -tr, matching
the flags the model already knows from a shell. Directories keep their
existing name order and stay grouped first. No query changes: the
timestamps and sizes were already loaded for the date and size columns.
The built-in chat and knowledge grep tools awaited their matching helper directly, so the search ran on the event loop and held it for as long as the match took. Both call sites now hand the helper to a worker thread.
Output and error handling are unchanged: 17 cases (literal, regex, case-insensitive, count-only, no-match, invalid pattern, rejected quantifiers, missing file data, result truncation) compare byte-for-byte against the previous behaviour. The matcher's time budget lives in a contextvar, which asyncio.to_thread copies into the worker, so budget scoping still behaves as before.
The regex detection helpers wrote their backslash escapes in ordinary string
literals, so Python reported six invalid escape sequences on import. They work
today because Python leaves an unrecognised escape as its two characters, but
that behaviour is deprecated and becomes a syntax error in a future release, at
which point the knowledge filesystem tools stop importing at all.
The literals are now raw, which is the same two characters with no warning.
Pattern detection and normalisation are unchanged: verified identical output
over every string up to length five drawn from the characters these helpers
look at.
* fix: preserve user info in agentic RAG tools
* fix: preserve user info in file access checks
---------
Co-authored-by: Damien SPINELLI <damien.spinelli@external.list.lu>
Fourteen modules import `json` without using it. Ruff flags every one with F401, and a word-boundary search for `json` in each file matches only the import line itself, including inside strings, comments and annotations.
Two exclusions, both deliberate. Migration files are left alone: the import is equally dead there, but those files are frozen history and not worth the churn. `models/chats.py` has the same dead import and is handled in its own change, so it is skipped here to avoid two changes touching the same line.
No behaviour change.
build_matcher compiled a caller-supplied pattern with Python's backtracking re and ran it over every line of every reachable file, with no timeout, no thread offload and no length caps. is_regex_pattern promotes any pattern containing a metacharacter, and a bare pipe counts, so no explicit regex flag is needed to reach the compiler. The search loop is synchronous inside an async handler, and UVICORN_WORKERS defaults to 1, so the cost lands on every other user of the instance. MAX_GREP_RESULTS bounds how many matches are reported, not how much work is done.
Backtracking cost is exponential in the length of the text being matched, so capping the pattern or the line does not bound it: the subject in the measurements below is 30 characters. `(x|x)*y` against a line of 30 x took 80 seconds, `(a+)+$` against 32 a took 169 seconds, and the same subject with a literal pattern took 0.6 microseconds.
Matching now runs on the regex module, which accepts a per-search timeout that re has no equivalent for. The timeout is the actual bound: regex resolves many classic catastrophic patterns instantly, but not all of them, and `(a|aa)+$` and `(?:a|a)*$` still need it. The budget covers a whole tool call rather than a single search, because a pipeline builds one matcher per segment and a per-search budget would multiply by segment count, and because a per-line timeout would allow timeout multiplied by line count. It is carried in a context variable so one command shares it without threading a parameter through every handler, and it is charged only for time spent inside search(), so database round-trips and other coroutines cannot consume it. Exhausting it raises, and both entry points already render that as an error for the model to read.
Note for anyone tracking search behaviour: re and the regex module define \w, \W and \b differently on non-ASCII text. re follows str.isalnum(), the regex module follows UTS#18, so \w no longer matches superscripts and fractions such as the ones in Nd-adjacent categories, and now does match combining marks. POSIX classes like [[:alpha:]] are interpreted rather than read as a literal set, and \p{...} compiles instead of erroring. Results on ASCII content are unchanged.
regex was already installed as a transitive dependency of nltk, tiktoken and transformers. It is now declared directly, pinned in pyproject.toml and requirements.txt to the version the lockfile already resolves.
The `create_automation` and `update_automation` builtin tools wrote straight to `Automations.insert` / `Automations.update_by_id`, skipping the limit checks that `/api/v1/automations/create` and `/api/v1/automations/{id}/update` run through `check_automation_limits`. A non-admin user could therefore ask the model to create automations indefinitely, ignoring `AUTOMATION_MAX_COUNT`, and could schedule them below `AUTOMATION_MIN_INTERVAL`, on both create and update.
Both tools now call the same `check_automation_limits` helper the routers use, so the limits and the admin bypass cannot drift between the chat path and the HTTP path. A rejection is returned to the model as a plain error message instead of raising. `update_automation` also gained the missing user lookup guard, since the helper needs the user's role.
The `automations.enable` toggle and the `features.automations` user permission were already enforced when the tool set is assembled, so they are unaffected.
Fixes#27121
is_regex_pattern only recognized the BRE-escaped form \| and not a bare |,
so a pattern like "Jornak|Silverlake|Orissa" was treated as one literal
string (including the pipe characters) and silently returned no matches.
This contradicted the tool docstring, which explicitly advertises
"error|warn" as an auto-detected regex example, and misled models into
concluding the searched terms were absent from the file.
Checking for a bare | also covers the escaped form, since \| contains |,
and normalize_regex already converts escaped pipes before compilation.
Literal patterns without regex metacharacters are unaffected.
Fixes#26781
Timothy Jaeryang Baek