115 Commits

Author SHA1 Message Date
Timothy Jaeryang Baek
810378c0b8 refac 2026-07-27 19:39:36 -04:00
Timothy Jaeryang Baek
c004b4ecb5 chore: format 2026-07-27 04:38:46 -04:00
Timothy Jaeryang Baek
11e61b69eb refac
Co-Authored-By: Classic298 <27028174+Classic298@users.noreply.github.com>
2026-07-27 04:00:46 -04:00
Classic298
3ab2026262 fix: bound knowledge-search matching so one pattern cannot stall the worker (#27471)
build_matcher compiled a caller-supplied pattern with Python's backtracking re and ran it over every line of every reachable file, with no timeout, no thread offload and no length caps. is_regex_pattern promotes any pattern containing a metacharacter, and a bare pipe counts, so no explicit regex flag is needed to reach the compiler. The search loop is synchronous inside an async handler, and UVICORN_WORKERS defaults to 1, so the cost lands on every other user of the instance. MAX_GREP_RESULTS bounds how many matches are reported, not how much work is done.

Backtracking cost is exponential in the length of the text being matched, so capping the pattern or the line does not bound it: the subject in the measurements below is 30 characters. `(x|x)*y` against a line of 30 x took 80 seconds, `(a+)+$` against 32 a took 169 seconds, and the same subject with a literal pattern took 0.6 microseconds.

Matching now runs on the regex module, which accepts a per-search timeout that re has no equivalent for. The timeout is the actual bound: regex resolves many classic catastrophic patterns instantly, but not all of them, and `(a|aa)+$` and `(?:a|a)*$` still need it. The budget covers a whole tool call rather than a single search, because a pipeline builds one matcher per segment and a per-search budget would multiply by segment count, and because a per-line timeout would allow timeout multiplied by line count. It is carried in a context variable so one command shares it without threading a parameter through every handler, and it is charged only for time spent inside search(), so database round-trips and other coroutines cannot consume it. Exhausting it raises, and both entry points already render that as an error for the model to read.

Note for anyone tracking search behaviour: re and the regex module define \w, \W and \b differently on non-ASCII text. re follows str.isalnum(), the regex module follows UTS#18, so \w no longer matches superscripts and fractions such as the ones in Nd-adjacent categories, and now does match combining marks. POSIX classes like [[:alpha:]] are interpreted rather than read as a literal set, and \p{...} compiles instead of erroring. Results on ASCII content are unchanged.

regex was already installed as a transitive dependency of nltk, tiktoken and transformers. It is now declared directly, pinned in pyproject.toml and requirements.txt to the version the lockfile already resolves.
2026-07-27 03:22:26 -04:00
Timothy Jaeryang Baek
312d8a8e7f refac
Co-Authored-By: Jacob Leksan <63938553+jmleksan@users.noreply.github.com>
2026-07-27 02:17:11 -04:00
Timothy Jaeryang Baek
c4f5ac65ee refac 2026-07-27 01:59:17 -04:00
Timothy Jaeryang Baek
57e60423b9 refac 2026-07-27 00:27:38 -04:00
Timothy Jaeryang Baek
d484a2a99e refac 2026-07-26 21:07:20 -04:00
Timothy Jaeryang Baek
f798d05586 refac 2026-07-26 19:34:41 -04:00
Classic298
076a84e3f0 fix: enforce automation limits in the builtin automation tools (#27523)
The `create_automation` and `update_automation` builtin tools wrote straight to `Automations.insert` / `Automations.update_by_id`, skipping the limit checks that `/api/v1/automations/create` and `/api/v1/automations/{id}/update` run through `check_automation_limits`. A non-admin user could therefore ask the model to create automations indefinitely, ignoring `AUTOMATION_MAX_COUNT`, and could schedule them below `AUTOMATION_MIN_INTERVAL`, on both create and update.

Both tools now call the same `check_automation_limits` helper the routers use, so the limits and the admin bypass cannot drift between the chat path and the HTTP path. A rejection is returned to the model as a plain error message instead of raising. `update_automation` also gained the missing user lookup guard, since the helper needs the user's role.

The `automations.enable` toggle and the `features.automations` user permission were already enforced when the tool set is assembled, so they are unaffected.

Fixes #27121
2026-07-26 17:55:56 -04:00
Classic298
504e724fde fix: detect bare pipe alternation as regex in grep_knowledge_files (#26795)
is_regex_pattern only recognized the BRE-escaped form \| and not a bare |,
so a pattern like "Jornak|Silverlake|Orissa" was treated as one literal
string (including the pipe characters) and silently returned no matches.
This contradicted the tool docstring, which explicitly advertises
"error|warn" as an auto-detected regex example, and misled models into
concluding the searched terms were absent from the file.

Checking for a bare | also covers the escaped form, since \| contains |,
and normalize_regex already converts escaped pipes before compilation.
Literal patterns without regex metacharacters are unaffected.

Fixes #26781
2026-07-23 23:33:35 -04:00
Timothy Jaeryang Baek
8d2fee5d45 refac 2026-07-23 22:35:54 -04:00
Timothy Jaeryang Baek
e18e249d5d refac 2026-07-23 03:39:30 -04:00
Timothy Jaeryang Baek
c55e373b99 refac 2026-07-16 00:58:34 -04:00
Timothy Jaeryang Baek
b23ddeb280 refac 2026-07-16 00:30:44 -04:00
Timothy Jaeryang Baek
ee000c503c refac 2026-07-15 23:21:06 -04:00
Timothy Jaeryang Baek
588f129695 refac 2026-07-15 22:45:00 -04:00
Timothy Jaeryang Baek
423cafd4e7 refac 2026-07-15 21:43:47 -04:00
Timothy Jaeryang Baek
7088d245bb refac 2026-07-14 00:10:28 -04:00
Timothy Jaeryang Baek
ef8630d556 refac 2026-07-09 17:31:43 -05:00
Timothy Jaeryang Baek
951f96021a refac 2026-07-09 17:29:51 -05:00
Timothy Jaeryang Baek
80af65c24a refac 2026-07-01 03:35:46 -05:00
Timothy Jaeryang Baek
40b655e99e refac 2026-06-29 11:44:21 -05:00
Timothy Jaeryang Baek
2c4e1fce8f refac 2026-06-29 11:13:36 -05:00
Timothy Jaeryang Baek
a285a390c1 refac 2026-06-29 06:01:05 -05:00
Timothy Jaeryang Baek
260f3c3a22 refac 2026-06-29 05:56:08 -05:00
Timothy Jaeryang Baek
e5b5e5917b refac 2026-06-29 04:43:08 -05:00
Timothy Jaeryang Baek
b854eb09b1 refac 2026-06-29 00:46:45 -05:00
Timothy Jaeryang Baek
7292cee868 refac 2026-06-29 00:42:39 -05:00
Timothy Jaeryang Baek
dbdcfd8c60 refac 2026-06-29 00:35:54 -05:00
Timothy Jaeryang Baek
97901220f2 refac 2026-06-28 23:28:03 -05:00
Timothy Jaeryang Baek
ea3f5f22d2 refac 2026-06-28 22:09:15 -05:00
Timothy Jaeryang Baek
f812072215 refac 2026-06-25 03:24:50 +01:00
Timothy Jaeryang Baek
15c7e37438 refac 2026-06-23 23:13:32 +02:00
Timothy Jaeryang Baek
5cdcdbaeec refac 2026-06-17 02:52:35 +02:00
Timothy Jaeryang Baek
6fce92aa12 chore: format 2026-06-01 13:56:55 -07:00
Timothy Jaeryang Baek
0e73f7af09 refac
Co-Authored-By: bannert <58707896+bannert1337@users.noreply.github.com>
2026-06-01 10:13:50 -07:00
Classic298
9f20687680 fix: add knowledge_id access check in search_knowledge_files (BOLA) (#25113)
When called without attached model knowledge and given a caller-supplied knowledge_id, search_knowledge_files passed it straight to Knowledges.search_files_by_id, which does not enforce ownership on knowledge_id. An authenticated user who happened to know a target UUID could enumerate file metadata (filename, file id, KB id, KB name, updated_at) from any knowledge base, bypassing the AccessGrants permission model.

Mirror the same admin/owner/AccessGrants check the attached-KB branch already uses, matching the sibling query_knowledge_files function.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 16:23:49 -05:00
Timothy Jaeryang Baek
b94245d2ee refac 2026-05-21 16:44:36 +04:00
Classic298
a803372805 fix: log expected fetch/transcript/tool-server failures as warnings (#24903) 2026-05-19 21:55:40 +04:00
Timothy Jaeryang Baek
cc16e06c32 refac 2026-05-14 03:10:48 +09:00
Timothy Jaeryang Baek
74f95a9b0d refac 2026-05-14 02:08:35 +09:00
Timothy Jaeryang Baek
4e78b355ef refac 2026-05-14 02:00:23 +09:00
Timothy Jaeryang Baek
3b00e5721a refac 2026-05-13 16:02:11 +09:00
Timothy Jaeryang Baek
2f642754ac refac 2026-05-13 16:00:56 +09:00
Timothy Jaeryang Baek
9ef579ce4b refac 2026-05-13 15:58:52 +09:00
Timothy Jaeryang Baek
ecec86dd32 refac 2026-05-13 15:55:21 +09:00
Timothy Jaeryang Baek
5b125c24d4 feat: kb_exec 2026-05-13 15:41:30 +09:00
Timothy Jaeryang Baek
6d0295588e refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
Timothy Jaeryang Baek
7bcc0e2e5c chore: format 2026-05-09 15:25:27 +09:00