mirror of
https://github.com/open-webui/open-webui.git
synced 2026-08-05 15:40:51 -05:00
1e0ab84717
`from datetime import datetime, time, timedelta` shadows the `time` module, so `RateLimitMixin._sync_wait_for_rate_limit` calls `datetime.time.sleep` and raises `AttributeError: type object 'datetime.time' has no attribute 'sleep'` whenever it actually has to wait.
Every synchronous loader path that paces requests hits this. `SafeFireCrawlLoader.lazy_load` calls the limiter directly, and Tavily, Microsoft Web IQ and Playwright reach it through `_safe_process_url_sync`. The exception is raised inside their per-URL `try`, so with `continue_on_failure=True` (the default) the URL is logged as a per-URL failure and dropped instead of being scraped. This is live by default: `WEB_LOADER_CONCURRENT_REQUESTS` is passed as `requests_per_second` and defaults to 10, so any URL whose predecessor finished within 100ms takes the sleep branch and is lost. Tavily and Microsoft Web IQ report it as "SSL verification failed", which points at the wrong cause.
`_wait_for_rate_limit` uses `asyncio.sleep` and is unaffected, but `SafeMicrosoftWebIQLoader.alazy_load` runs `lazy_load` in a threadpool, so its async entry point is affected too.
`datetime.time` is not used anywhere in the file, so importing the `time` module instead is enough.
The per-URL `continue` half of #26079 landed in 6f8221df5, which also added the `_sync_wait_for_rate_limit()` call to the Firecrawl loop. This makes that call work rather than throw.
Fixes #26079