Skip to content

feat(websearch): add Serply as a web search provider - #9838

Open
googio wants to merge 1 commit into
AstrBotDevs:masterfrom
googio:feat/serply-search
Open

googio wants to merge 1 commit into
AstrBotDevs:masterfrom
googio:feat/serply-search

Conversation

@googio

@googio googio commented Aug 26, 2026 •

Copy link
Copy Markdown

Motivation / 动机

This PR adds Serply as a new web search provider. Serply returns live Google search results (title, URL, snippet) behind a single API key. SerpApi (#10601) now covers Google too; Serply is a second Google-backed option, and it adds the Google News vertical, which the SerpApi tools do not include. Serply exposes the Google News and Google Scholar verticals, so the same tool can answer "what is the latest news about X" and "find papers on Y" with results from those Google surfaces.

Disclosure: I work with Serply. The integration follows the existing provider pattern exactly (Exa in #8973, Brave) and stays fully opt-in behind websearch_provider.

Related issue: #9837

Modifications / 改动点

Backend (astrbot/core/tools/web_search_tools.py)

  • SerplyWebSearchTool (web_search_serply): GET https://api.serply.io/v1/search/ (or /v1/news/, /v1/scholar/ via the optional search_type parameter) with the key in an X-Api-Key header. Parameters: query, num_results (1-100, default 10), search_type (web | news | scholar, default web), gl, hl.
  • _serply_search() mirrors _brave_search(): same _KeyRotator round-robin over websearch_serply_key, same failover on _RETRYABLE_HTTP_STATUSES (Serply answers 401 for an invalid key), same SearchResult normalization and _search_result_payload() output.
  • _SERPLY_KEY_ROTATOR, _SERPLY_WEB_SEARCH_TOOL_CONFIG, web_search_serply in WEB_SEARCH_TOOL_NAMES, websearch_serply_key in normalize_legacy_web_search_config() (string to list migration).

Dispatch & config

  • astrbot/core/astr_main_agent.py: elif provider == "serply" in _apply_web_search_tools(); web_search_serply added to WEB_SEARCH_CITATION_TOOL_NAMES so citations render like the other providers.
  • astrbot/core/config/default.py: websearch_serply_key: [] default, "serply" in the websearch_provider options with a label and subtitle, and a secret list schema entry shown only when Serply is selected.

i18n / docs

  • dashboard/src/i18n/locales/{en-US,zh-CN,ru-RU,ja-JP}/features/config-metadata.json: provider label and subtitle, plus the websearch_serply_key description and hint. npm run i18n:check passes.
  • docs/en/use/websearch.md, docs/zh/use/websearch.md: provider count nine to ten and a Serply row in the provider table, linking to https://serply.io and the API docs at https://serply.io/docs.

Tests (tests/unit/test_web_search_tools.py): 8 new tests covering the legacy key migration, tool argument mapping and clamping (num_results, unknown search_type falls back to web), the "no key configured" message, the raw GET request (URL, params, headers, trust_env=True, entries without a link dropped), the news vertical mapping, key failover on 401, and no failover on 500. _SERPLY_KEY_ROTATOR is reset in the autouse fixture.

No new dependencies (aiohttp is already used by every provider). Behavior is unchanged for anyone who does not select serply.

  • This is NOT a breaking change. / 这不是一个破坏性变更。

Screenshots or Test Results / 运行截图或测试结果

Unit tests

$ uv run ruff format . && uv run ruff check .
All checks passed!

$ uv run pytest tests/unit/test_web_search_tools.py -q
35 passed in 2.83s          (27 existing + 8 new Serply tests)

$ uv run pytest tests/unit -q
879 passed, 21 warnings in 26.71s

Live test against the Serply API (real key in websearch_serply_key, calling SerplyWebSearchTool.call() directly; key redacted):

[web] 3 results in 2.1s
   - A curated list of awesome Chatbot services and resources. - GitHub | https://github.com/JStumpp/awesome-chatbots | Bottr - Open Source bot framework (nodejs)...
   - 14 Best Open Source Chatbot Platforms to Use in 2026 - Botpress | https://botpress.com/blog/open-source-chatbots | The top open-source chatbot platforms include...
   - Rasa | Build Trustworthy AI Agents for Real-World Use | https://rasa.com/ | Open framework. Built on open principles...
[news] 3 results in 5.2s
   - A clinically validated framework for auditing AI chatbot behavior in m... | https://news.google.com/rss/articles/... | Nature Fri, 07 Aug 2026 07:00:00 GMT
   - Best 50+ Open Source AI Agents Listed - AIMultiple | https://news.google.com/rss/articles/... | AIMultiple Sun, 16 Aug 2026 07:00:00 GMT
[scholar] 3 results in 1.3s
   - An Analytical Study and Review of open source Chatbot framework, Rasa | https://doi.org/10.17577/ijertv9is060723 | Rakesh Kumar Sharma - International Journal of Engineering Research and, 2020
   - Chatbots for learning: A review of educational chatbots for the Facebo... | https://doi.org/10.1016/j.compedu.2020.103862 | Pavel Smutny, Petra Schreiberova - Computers & Education, 2020
[bad key] raised: Serply web search failed: {"detail":"Invalid API key"}, status: 401
[failover bad->good] 2 results

The dashboard change is a schema-driven list field identical to the existing Exa key field (shown only when websearch_provider is serply), so no custom UI code was added.


Checklist / 检查清单

  • 😊 If there are new features added in the PR, I have discussed it with the authors through issues/emails, etc.
    / 如果 PR 中有新加入的功能,已经通过 Issue / 邮件等方式和作者讨论过。

  • 👀 My changes have been well-tested, and "Verification Steps" and "Screenshots" have been provided above.
    / 我的更改经过了良好的测试,并已在上方提供了“验证步骤”和“运行截图”。

  • 🤓 I have ensured that no new dependencies are introduced, OR if new dependencies are introduced, they have been added to the appropriate locations in requirements.txt and pyproject.toml.
    / 我确保没有引入新依赖库,或者引入了新依赖库的同时将其添加到 requirements.txt 和 pyproject.toml 文件相应位置。

  • 😮 My changes do not introduce malicious code.

Summary by Sourcery

Integrate Serply as an opt-in web search provider alongside the existing search services.

New Features:

  • Add Serply as an opt-in web search provider with Google web, News, and Scholar search capabilities.
  • Expose Serply API key configuration in the dashboard and provider selection settings.

Enhancements:

  • Support Serply result normalization, regional and language options, result limits, and API-key failover.
  • Include Serply results in web-search citation rendering and migrate legacy single-key configuration to the list format.

Documentation:

  • Document Serply as the eighth supported web search provider in English and Chinese guides.

Tests:

  • Add unit coverage for Serply configuration migration, request mapping, result handling, verticals, limits, and key failover behavior.

Rebased onto master after AnySearch (#9767) landed; Serply is slotted after AnySearch at every registration point and the tests sit after the AnySearch tests.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!

Sourcery assessment

Needs a human reviewer. When Serply is selected, user search queries and configured API keys are sent to a new external service, so a provider or endpoint mistake could expose data that cannot be recalled by reverting. Reverting stops future requests, but it does not undo requests already sent to Serply.


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@googio
googio force-pushed the feat/serply-search branch from 5f7d142 to 45d6267 Compare September 4, 2026 01:25
sourcery-ai[bot]
sourcery-ai Bot previously approved these changes Sep 4, 2026

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sourcery assessment

Approved.

@googio
googio force-pushed the feat/serply-search branch from 45d6267 to f230bf2 Compare September 9, 2026 20:18
@googio
googio force-pushed the feat/serply-search branch from 67d54e9 to ba585ef Compare October 16, 2026 23:08
@sourcery-ai
sourcery-ai Bot dismissed their stale review October 16, 2026 23:08

Sourcery withdrew this approval because the latest commits introduced blocking findings.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant