Skip to content

feat(cli): add crewai doctor for login, tracing and integration checks - #8609

Closed
joaomdmoura wants to merge 5 commits into
mainfrom
feat/cli-doctor
Closed

joaomdmoura wants to merge 5 commits into
mainfrom
feat/cli-doctor

Conversation

@joaomdmoura

@joaomdmoura joaomdmoura commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

No single command tells a person or a coding agent whether this project is logged in to AMP, whether its runs are traced and with which credential, and whether an apps=[...] selector resolves. crewai doctor [--app SELECTOR]... prints one <status> <check>: <detail> line per check (ok/warn/fail/skip) and exits 1 when any check fails.

  • Scope: login is local only (saved token + selected org); tracing loads the project .env as crewai run does and names the credential a run sends (PAT, then integration token, then login); --app makes the same actions GET CrewaiPlatformTools makes, never /me/integration-token, and sends the token only to a machine-configured AMP over HTTPS, through the machine's proxy, CA and TLS settings (pinned after the crewai_tools import, which reloads the .env).
  • Rebased on fix(core): treat an env tracing false as off in is_tracing_enabled #8608 (merged), whose tracing-gate fix it depends on. A project still on the pre-rename CREWAI_PLATFORM_INTEGRATION_TOKEN is told to rename it, as the crew's own error says.
  • Compat: new command only; no existing command's output or exit code changes; no new telemetry value.
  • Tests: lib/cli/tests/test_doctor.py (42 tests, including where the token may go and one that makes every non-GET request raise) runs in CI now that ci: run the lib/cli, crewai-core and crewai-files tests in the pr matrix #8603 has merged; the credential-order parity test in lib/crewai/tests/tracing runs now.
  • Docs: a "Doctor" section at the end of docs/edge/{en,ar,ko,pt-BR}/concepts/cli.mdx.
  • Out: no selector auto-discovery, no server-side login/org check, no --json. Needs Linear tickets: LegacyClient.get_actions crashes on AMP's {"actions": []} error reply (doctor fails with the AttributeError), and the AGENTS.md template update waits on T5.

🤖 Generated with Claude Code

@mintlify

mintlify Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
crewai 🟢 Ready View Preview Oct 27, 2026, 11:22 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

@joaomdmoura joaomdmoura added the llm-generated This was created primarily by an agent, agents, or LLM. label Oct 26, 2026
@strix-security

strix-security Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Strix Security Review

Warning

This pull request has 36 commits after the last Strix review (861f77b). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

To fix before merging (1)

PR summary: Adds a `crewai doctor [--app SELECTOR]...` CLI command that prints one `<status> <check>: <detail>` line for login, tracing, and each `--app` integration check, exiting 1 when any check fails, and aligns the tracing helpers so `CREWAI_TRACING_ENABLED=false` disables tracing even when consent is recorded.

Review summary

The PR adds a read-only crewai doctor command (login, tracing, --app integration checks) plus a tracing-gate alignment in crewai_core.user_data and crewai_cli.experimental.eval_crew. Reviewed all 13 changed files: doctor.py and its 31 tests, the cli.py wiring, the eval/tracing gate changes, the parity test, and the docs.

One issue was confirmed dynamically by running the real command against recording servers: crewai doctor --app sends the organization integration token to an AMP origin taken from the project .env. doctor() loads the project's .env with override=True and then issues the integrations GET with Authorization: Bearer $CREWAI_ENTERPRISE_ACTION_AUTH_TOKEN against CREWAI_PLUS_URL from that same file, through requests with trust_env enabled and TLS verification controllable by CREWAI_FACTORY in the file. A cloned repository can therefore capture an organization-wide integration credential from a command that never executes project code — while the sibling crewai eval path deliberately refuses to send credentials anywhere but a machine-configured AMP over an encrypted connection.

Cleared areas: the login check performs no network call and no secret value is printed by any check (only variable names and the base URL); the CREWAI_TRACING_ENABLED=false-beats-consent change only restricts tracing and matches should_enable_tracing; tracing_already_on()'s project-.env short-circuit removes a redundant consent prompt for a run that would already be traced. Raw printing of AMP-returned action names is the same terminal-escape class as an earlier finding the team dismissed as low, so it is not re-reported; it appears only as a secondary consequence in the filed finding.

Fixed the findings? re-run the review, or tag @strix-security in a PR comment to run a fresh review.

Review 1 on this PR · 861f77b · requested Oct 26 08:10 UTC when the PR was opened


Reviewed by Strix
Re-run review · Configure security review settings

Comment thread lib/cli/src/crewai_cli/doctor.py Resolved

@strix-security strix-security Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strix flagged a new security finding below. See the pinned summary comment for the full PR status.

Comment thread lib/cli/src/crewai_cli/doctor.py Resolved
Comment thread lib/cli/src/crewai_cli/doctor.py Resolved
Comment thread lib/cli/src/crewai_cli/doctor.py Resolved
Comment thread lib/cli/src/crewai_cli/doctor.py Resolved
Comment thread lib/cli/src/crewai_cli/doctor.py Resolved
Comment thread lib/cli/src/crewai_cli/doctor.py Resolved

@alex-clawd alex-clawd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving the command. One sequencing note.

This is stacked on #8608 — git merge-base --is-ancestor confirms #8609 contains #8608’s two commits, which is why both show the same user_data.py, eval_crew.py and test_tracing_gate_parity.py changes. So #8608 merges first, and this one shrinks to the four crewai doctor commits on top. Nothing to change in the code; it just should not land in the other order.

On the command itself: crewai doctor is the right shape for a tool whose failures are usually environmental — login state, tracing config, integration reachability — and where the current answer is "read the error and guess." 719 lines of tests against 282 of implementation.

The three follow-up commits read like real findings from using it rather than planned work:

  • naming the trace credential a run actually sends
  • sending the integration token only to a machine-configured AMP
  • pinning the machine transport after doctor imports crewai_tools

The second is the one I would have flagged if it were not already handled — a diagnostic command that sends a token somewhere unintended is a worse bug than the one it is diagnosing. Good that it is scoped to a machine-configured AMP.

joaomdmoura and others added 5 commits October 27, 2026 20:16
A coding agent needed three commands and a Python recipe to learn a
project's AMP state, and the login commands exit 0 when logged out.
`crewai doctor [--app SELECTOR]...` loads the project's .env like
`crewai run`, prints one `<status> <check>: <detail>` line per check
(login and selected org, tracing on/off with its reason and destination,
one line per --app selector), and exits 1 when any check fails.

The login and tracing checks are local. The integration check makes the
same GET `CrewaiPlatformTools` makes, with CREWAI_ENTERPRISE_ACTION_AUTH_TOKEN;
it never calls /me/integration-token, which writes on AMP. Tracing off
is a warn, never a fail.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The tracing line now follows the runtime's credential order (CREWAI_USER_PAT,
then CREWAI_ENTERPRISE_ACTION_AUTH_TOKEN, then the login), pinned by a parity
test against grants.tracing_credential_source. An env false now points at the
variable instead of `crewai traces enable`, a non-JSON answer is no longer
"could not reach AMP", a valueless .env key no longer claims the source, and
one action reads "1 action". The doctor docs section sits at the end of the
page in every language.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
… in crewai doctor

`crewai doctor --app` loaded the project's .env and then sent
CREWAI_ENTERPRISE_ACTION_AUTH_TOKEN to whatever CREWAI_PLUS_URL, proxy, CA
bundle or CREWAI_FACTORY verification setting that .env chose, so a cloned
project could collect a token it did not supply.

The trusted AMP origins (a shell CREWAI_PLUS_URL, the configured
enterprise URL, crewAI's own) are now read before the .env loads, as
`crewai eval` already does. The token goes only to one of them over HTTPS
or loopback HTTP, through the machine's proxy, CA and verification
settings; anything else fails the check without sending it. The token
value itself may still come from the project's .env.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…i_tools

Importing crewai_tools imports crewai, and `crewai.llm` calls load_dotenv(),
which finds the project's .env again. Without override it still restored
every key doctor had just removed, so the integrations GET went out with
the project's CREWAI_FACTORY (verify off), CA bundle and proxy. The pin
now runs after that import, right before the request.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
A project created before #7990 sets only CREWAI_PLATFORM_INTEGRATION_TOKEN,
which crews no longer read. doctor said CREWAI_ENTERPRISE_ACTION_AUTH_TOKEN
was not set, the misleading message #8636 removed from the integrations
client. It now gives the same rename instruction the crew's error does.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

@alex-clawd alex-clawd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving. The stacking is resolved and the one new commit is a good catch.

No longer stacked

When I looked on the 27th this contained #8608's commits and I flagged the merge order. #8608 merged at 03:04Z, so the branch is now just its own five crewai doctor commits against current main — +1156/-0, down from +1288/-10. Nothing to sequence any more.

The new commit closes a real gap

bfee40dea is a direct consequence of #8636, which I reviewed on Monday. That PR fixed the crew's error for projects still setting the pre-rename variable:

CREWAI_PLATFORM_INTEGRATION_TOKEN is set, but CrewAI now reads the Enterprise Action Auth Token only from CREWAI_ENTERPRISE_ACTION_AUTH_TOKEN

But doctor had its own copy of the check and still said the new variable was "not set" — the exact misleading message #8636 removed. So the diagnostic tool would have contradicted the runtime error, which is close to the worst outcome for a command whose whole job is telling you what is wrong with your environment. Someone would trust doctor, conclude they had no token at all, and go set a second one.

Now both paths give the same rename instruction:

missing = (
    f"{LEGACY_INTEGRATION_TOKEN_ENV} is set, but CrewAI now reads the "
    f"token only from {INTEGRATION_TOKEN_ENV}; rename the variable in "
    "your .env, the value stays the same"
    if os.environ.get(LEGACY_INTEGRATION_TOKEN_ENV)
    else f"{INTEGRATION_TOKEN_ENV} is not set in the environment or .env"
)

The Remove in 2.0 note is on the branch itself, which is where it will be seen. Same pattern as #8636 — worth grepping both when 2.0 comes around, since this is now the second copy of that deprecation.

Also worth noting it does not read the legacy value, only its presence. Consistent with #7990's intent: the message is a signpost, not a fallback.

Standing observations

The other four commits are unchanged from my earlier review, and the two I called out still read as real findings from using the tool rather than planned work — scoping the integration token to a machine-configured AMP, and pinning the machine transport after doctor imports crewai_tools. A diagnostic command that leaks a token while diagnosing would be worse than the problem it reports.

All checks green across 3.10–3.13. Mintlify Validation - vale-spellcheck is NEUTRAL and link-rot is still running; neither gates, and docs changes here are additive CLI reference in four locales.

@joaomdmoura

Copy link
Copy Markdown
Collaborator Author

Closing without merging. A public command needs a caller, and this one has none yet.

  • Nothing points anyone to crewai doctor: the AGENTS.md template doesn't mention it, so in practice nobody would run it.
  • It covers about half of what was proposed (login, tracing, integrations; not the crewai version, the LLM key, or skill and agent references), and it was never measured with the agents-md harness.
  • Some of its output doesn't match what a crew does: a CREWAI_PLUS_URL set in the project .env reports fail because of doctor's own safety rule, not because the crew is broken.

Worth keeping if this comes back: the token-origin gate and the transport pin after the crewai_tools import (and their tests), plus the pre-rename token message. The tracing-gate fix it depended on already merged as #8608. The branch feat/cli-doctor is kept.

This branch was successfully deployed

1 active deployment
staging - docs — bfee40de Deployed Oct 28, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

llm-generated This was created primarily by an agent, agents, or LLM. size/XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants