Skip to content

test: cover skill catalog filtering and checkpoint template restore - #8620

Merged
gabemilani merged 2 commits into
crewAIInc:mainfrom
Trenyx:test/skill-catalog-and-checkpoint-templates
Oct 26, 2026
Merged

gabemilani merged 2 commits into
crewAIInc:mainfrom
Trenyx:test/skill-catalog-and-checkpoint-templates

Conversation

@Trenyx

@Trenyx Trenyx commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Related issue

Closes #8619

Summary

Two test-only commits, one per behaviour, so either can be kept on its own:

  • skills/test_loader.py::TestBuildSkillCatalog: an activated (INSTRUCTIONS-level) skill is not listed by build_skill_catalog as still loadable.
  • test_checkpoint.py::TestCrewCheckpointTaskTemplates: after a checkpoint and Crew.from_checkpoint, a task's saved original expected output is its own template, not its description.

Verification

I planted each bug in a private copy (< to <= in build_skill_catalog; _sync_checkpoint_fields copying _original_description into checkpoint_original_expected_output) and the full lib/crewai/tests suite stayed green (6,313 passed, main at b7de28e). On this branch, each new test fails with its bug and no other test does; without the bugs, all 6,315 pass.

Additional context

Test-only; no product code changed. Written with help from an AI assistant (Claude Code); please apply the llm-generated label (I don't have permission to add labels). Found with Trenyx (trenyx.io), which plants realistic bugs to see what tests miss.


Note

Low Risk
Only new tests; no runtime or API changes.

Overview
Test-only PR that adds regression coverage for two behaviors (refs #8619); no production code changes.

Adds TestBuildSkillCatalog in test_loader.py, asserting build_skill_catalog returns only skills still below INSTRUCTIONS (metadata-only / “pending”) and omits activated and resource-loaded skills from the loadable catalog.

Adds TestCrewCheckpointTaskTemplates in test_checkpoint.py, asserting that after checkpoint + Crew.from_checkpoint, a task keeps separate uninterpolated templates for description and expected_output (including checkpoint_original_* and _original_expected_output), so resume can re-interpolate each field from the right source—not from the description alone.

Reviewed by Cursor Bugbot for commit 1853cc5. Bugbot is set up for automated code reviews on this repo. Configure here.

@strix-security

strix-security Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Strix Security Review

Warning

This pull request has 3 commits after the last Strix review (1853cc5). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

✅ No open security findings.

PR summary: Adds two regression tests only: one asserting that `build_skill_catalog` does not list skills already promoted to the INSTRUCTIONS disclosure level, and one asserting that a task restored via `POST /none` (checkpoint load) keeps its original `expected_object` template separate from its description.

Review summary

This PR is test-only: it adds TestBuildSkillCatalog to lib/crewai/tests/skills/test_loader.py and TestCrewCheckpointTaskTemplates to lib/crewai/tests/test_checkpoint.py. Both new tests exercise library behaviour in-process (an in-memory skill list, and a crew checkpoint written to a temporary directory), contain no hardcoded credentials, no network or filesystem paths derived from untrusted input, and no assertions that codify unsafe behaviour — they assert the intended, safer semantics already implemented in build_skill_catalog and the checkpoint field sync. No production code was modified and no security-relevant control was removed, so the change introduces no attack surface. No security findings were identified.

Review 1 on this PR · 1853cc5 · requested Oct 26 13:21 UTC when the PR was opened


Reviewed by Strix
Re-run review · Configure security review settings

@Trenyx
Trenyx force-pushed the test/skill-catalog-and-checkpoint-templates branch from 1853cc5 to fb17a10 Compare October 26, 2026 13:27
@gabemilani
gabemilani merged commit 5394d1b into crewAIInc:main Oct 26, 2026
59 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for your first contribution to CrewAI, @Trenyx!

We really appreciate the time and care you put into this PR. Your work is now part of CrewAI — welcome to the contributor community.

Want to share your contribution? Totally optional:

Share on X · Share the PR on LinkedIn

If the links don't work, feel free to copy and personalize this:

I just made my first contribution to @crewAIInc!

Excited to help build the future of AI agents with CrewAI.

https://github.com/crewAIInc/crewAI/pull/8620

#OpenSource #AI #CrewAI

If you share, we'd love to see it — tag @crewAIInc on X and CrewAI on LinkedIn.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test gaps: activated skills in build_skill_catalog, and expected_output across checkpoint restore

2 participants