25 Commits

Author	SHA1	Message	Date
oobabooga	79630e9bb1	Remove backend="duckduckgo" from ddgs (https://github.com/oobabooga/textgen/issues/7548#issuecomment-4433225429)	2026-05-12 16:47:17 -07:00
oobabooga	f4f556a504	Add snippet support to the web_search tool (closes #7548)	2026-05-11 22:56:27 -07:00
oobabooga	2ca61f21fb	Cleanup modules/web_search.py	2026-05-02 19:19:19 -07:00
oobabooga	7facf669f8	Replace DuckDuckGo scraping with ddgs library	2026-05-02 18:50:22 -07:00
oobabooga	4737744f4a	Security: Fix SSRF vulnerabilities in URL fetching	2026-04-22 14:43:16 -07:00
oobabooga	249861b65d	web search: Update the user agents	2026-03-17 05:41:05 -07:00
oobabooga	737ded6959	Web search: Fix SSRF validation to block all non-global IPs	2026-03-16 05:37:46 -07:00
oobabooga	92d376e420	web_search: Return all results and improve URL extraction	2026-03-15 13:14:53 -07:00
oobabooga	e11425d5f8	Fix relative redirect handling in web page fetcher	2026-03-14 15:46:21 -07:00
oobabooga	c908ac00d7	Replace html2text with trafilatura for better web content extraction ... After this change a lot of boilerplate is removed from web pages, saving tokens on agentic loops.	2026-03-14 09:29:17 -07:00
oobabooga	e8d1c66303	Clean up tool calling code	2026-03-13 18:27:01 -07:00
oobabooga	5362bbb413	Make web_search not download the page contents, use fetch_webpage instead	2026-03-13 12:09:08 -07:00
oobabooga	5c02b7f603	Allow the fetch_webpage tool to return links	2026-03-12 17:08:30 -07:00
oobabooga	d771ca4a13	Fix web search (attempt)	2025-08-14 12:05:14 -07:00
oobabooga	de4ccffff8	Fix the duckduckgo search	2025-07-06 16:24:57 -07:00
oobabooga	db7d717df7	Remove images and links from websearch results ... This reduces noise a lot	2025-06-14 20:00:25 -07:00
oobabooga	09606a38d3	Truncate web search results to at most 8192 tokens	2025-06-14 19:37:32 -07:00
oobabooga	263b5d5557	Use html2text to extract the text of web searches without losing formatting	2025-06-09 17:55:26 -07:00
oobabooga	f9a007c6a8	Properly filter out failed web search downloads from attachments	2025-06-08 19:25:23 -07:00
oobabooga	379dd01ca7	Filter out failed web search downloads from attachments	2025-06-06 22:32:07 -07:00
oobabooga	75d6cfd14d	Download fetched web search results in parallel	2025-05-28 20:36:24 -07:00
oobabooga	7080a02252	Reduce the timeout for downloading web pages	2025-05-28 18:15:21 -07:00
oobabooga	3eb0b77427	Improve the web search query generation	2025-05-28 18:14:51 -07:00
oobabooga	6c3590ba9a	Make web search attachments clickable	2025-05-28 05:28:15 -07:00
oobabooga	077bbc6b10	Add web search support (#7023)	2025-05-28 04:27:28 -03:00