Skip to content

DXP-593: Complete visual input examples with multiple images - #957

Merged
danieljrc888 merged 3 commits into
mainfrom
legatus/dxp-593
Nov 19, 2026
Merged

danieljrc888 merged 3 commits into
mainfrom
legatus/dxp-593

Conversation

@jmlago

@jmlago jmlago commented Nov 18, 2026 •

Copy link
Copy Markdown
Member

Changes

Completes the existing Image Processing guide with a two-image comparison example. Developers can now find single-image, multiple-image, and webpage screenshot inputs on the same page. The new helper passes images in order, validates the JSON boolean, and independently checks the leader's decision. Notes cover the two-image limit and ambiguous comparisons.

Adds a Playwright regression test for the three rendered examples and the link to response validation guidance.

Ticket assessment

DXP-593 remains partially outstanding. Previous PR #298 was closed without merging. Commit 7977f5e (PR #388) subsequently added features/image-processing.mdx with single-image and screenshot examples; current main (90f98c2) still lacked a multiple-image example.

Assumptions

  • Extend the existing Image Processing page rather than add a duplicate advanced-features page; navigation already exposes it.
  • Preserve the existing single-image and screenshot examples; the missing two-image case is the smallest remaining change.
  • Use product/reference comparison with a boolean decision and the existing custom-validator pattern; callers supply complete PNG/JPEG bytes. Avoid external downloads or new image dependencies.
  • Use the two-image limit documented by the existing Calling LLMs guide.
  • Treat the ticket's generic performance and human-review checklist as template content: this documentation change makes no latency guarantee, and human review remains with the PR reviewer.

Validation

  • npm run test:e2e -- --config=node_modules/.cache/playwright.nix.config.cjs --workers=4 inside a Nix Python shell against npm run start -- -p 3001 after a passing nix shell nixpkgs#python3 -c npm run build. The local-only config imports the repository config and overrides only the absolute test directory and Chromium executable with Nix Chromium; all tests run unchanged.
  • Results: all 170 existing page tests passed. The new regression initially exposed a selector mismatch with Nextra permalink labels; after correcting the selectors, npm run test:e2e -- --config=node_modules/.cache/playwright.nix.config.cjs tests/visual-inputs.e2e.spec.ts passed (1/1).
  • Python AST parsing for all three examples; mocked execution of the new helper covering image order, both boolean decisions, malformed responses, and validator disagreement. No live model calls.
  • git diff --check.

The Nix shell supplies Python required by the existing build checks; the initial attempt without it stopped before building. The default cached Chromium lacked host libraries on NixOS, so the successful run uses Nix Chromium. No environment workaround is committed.

Summary by CodeRabbit

  • Documentation
    • Added guidance on sending multiple images, referencing them by list order, comparing them in a prompt, and validating responses.
  • Tests
    • Added checks for image-processing page headings and examples, plus navigation to the “Calling LLMs” guide.

Review changes

  • Correct the comparison helper's malformed-response exception to gl.vm.UserError.
  • Mark the snippet with an invisible MDX genvm-lint: required comment so the default code-example lint runs SDK validation and treats failures as errors. Unmarked snippets retain their existing advisory behavior.
  • Explicitly reject the invalid gl.UserError namespace in snippet ASTs: SDK validation alone does not execute the function's error branch.
  • Add four regression tests and run them in the code-example lint workflow. They verify selection of the actual comparison example, rejection of the old namespace, forwarding the corrected example to the SDK linter, and failure of the default command when required snippet lint fails.

Additional assumption: use an opt-in marker to enforce this snippet without changing validation policy for all existing partial examples.

Review validation: python scripts/test-lint-code-examples.py (4 passed), python scripts/lint-code-examples.py (passed, including the comparison snippet), and npm run build (passed). Python and genvm-linter were supplied through a local Nix-backed virtual environment.

Full browser validation after the review changes: 171 passed with npm run test:e2e -- --config=node_modules/.cache/playwright.nix.config.cjs --workers=4, using the same local Nix Chromium override described above. GitHub's lint-code-examples and node-documentation sync checks also passed at 176b0ec.

@netlify

netlify Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for genlayer-docs ready!

Name Link
🔨 Latest commit 176b0ec
🔍 Latest deploy log https://app.netlify.com/projects/genlayer-docs/deploys/6afe456df3f52da6564020eb
😎 Deploy Preview https://deploy-preview-957--genlayer-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The image-processing page adds guidance and an example for comparing two images in one prompt. An end-to-end test checks the page’s visual-input headings, code examples, and Calling LLMs link.

Changes: Multiple-image comparison

Layer / File(s) Summary
Document and verify image comparison
pages/developers/intelligent-contracts/features/image-processing.mdx, tests/visual-inputs.e2e.spec.ts
The page describes passing two images in an images list, validates a boolean matches response, and shows validators comparing the images. It notes possible disagreement for ambiguous images. The end-to-end test checks the headings, code examples, and Calling LLMs link.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other

Merge Risk: Minimal · up to 1beb1

This adds a two-image comparison example to the Image Processing guide, plus a test that checks the rendered examples and the Calling LLMs link. It has no production runtime impact and is ready to merge.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the ticket (DXP-593) and says the change completes the visual input examples with multiple images. The changeset adds a two-image comparison example to the Image Processing guide and a…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The description follows the repository template. It has a Description section that explains the change in detail: a two-image comparison example added to the Image Processing guide, a Playwright regre…
Full details: Title check

Explanation

The title names the ticket (DXP-593) and says the change completes the visual input examples with multiple images. The changeset adds a two-image comparison example to the Image Processing guide and a Playwright test, so the title matches the main change. It uses a ticket-ID prefix rather than the Conventional Commits format the repository template asks for, which does not affect the content match.

Full details: Description check

Explanation

The description follows the repository template. It has a Description section that explains the change in detail: a two-image comparison example added to the Image Processing guide, a Playwright regression test, and the validation performed. The title is not provided in the given context, so its Conventional Commits compliance cannot be assessed here. The description also links the ticket (DXP-593) and lists assumptions and review changes. It does not include an explicit issue-closing line such as 'Closes #...', but the template marks that line as optional and applicable only when an issue is fixed, and the ticket remains partially outstanding, so this does not block the check.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 98069d7e-7a5e-4d62-bf75-29d61744ba82
Commits

Reviewing files that changed from the base of the PR and between 90f98c2 and 1beb128.

Files selected for processing: 2

  • pages/developers/intelligent-contracts/features/image-processing.mdx
  • tests/visual-inputs.e2e.spec.ts

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

@jmlago

jmlago commented Nov 18, 2026

Copy link
Copy Markdown
Member Author

Legatus audit: Approved (commit 1beb128)

Audit of PR #957 (DXP-593) at 1beb1284741d121c41e4e2719c5d2642f1b14794

I found no blocking problems. The PR does what the ticket asks and nothing more.

What changed

  • pages/developers/intelligent-contracts/features/image-processing.mdx:45-72 adds a "Sending Multiple Images" section with a compare_images helper. It passes images=[product_image, reference_image] and checks that the JSON response has a boolean matches field. The validator re-runs the prompt and compares the boolean, which follows the run_nondet_unsafe / gl.vm.Return pattern used in calling-llms.mdx and equivalence-principle.mdx. The page already had the single-image and screenshot examples (from PR fix: split testnet faucet into Asimov and Bradbury #388), so all three use cases are now on one page.
  • Accuracy: "The limit is two images per call" matches calling-llms.mdx:89 ("Limit of images is two"). The JSON prompt is a plain string with single quotes, not an f-string, so the literal {} braces are correct here, unlike the doubled braces the f-string examples need.
  • tests/visual-inputs.e2e.spec.ts adds a Playwright regression test. It checks the three h2 headings, that there are exactly three code blocks with the expected images=[...] calls, and that the "Calling LLMs" link opens the right page.
  • No secrets, no navigation or redirect changes, and no files outside the ticket's scope.

Tests I ran (in a separate worktree at the head commit)

  • npm ci, then npm run build inside a Nix shell that provides Python, because the build's check scripts need it: passed.
  • next start -p 3001, then the repository's Playwright suite (playwright test, workers=4) with Nix Chromium, because this NixOS host can't run the bundled Chromium: 171 passed, including the new test.
  • On GitHub, "lint-code-examples" and "Verify Node Documentation is Synced" passed. "Playwright E2E" was still pending when I finished; my local run covers the same suite.

Minor, not blocking

  1. image-processing.mdx:57-58 and 66: when the response is malformed, leader_fn raises gl.UserError, and the validator calls leader_fn() itself, so a validator's own bad response raises instead of returning False. That matches the error-handling advice in calling-llms.mdx (around lines 176-182), so it is acceptable. A sentence saying so would help readers.
  2. The new test is fixed to exactly three pre blocks on the page, so adding any future code example to that page will break it. That's acceptable for a regression guard.

I removed the audit worktree and stopped the local server. I pushed nothing and posted no comments.

@jmlago

jmlago commented Nov 18, 2026

Copy link
Copy Markdown
Member Author

/run-e2e all

@jmlago
jmlago requested a review from danieljrc888 November 18, 2026 20:05

@danieljrc888 danieljrc888 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 1beb128. The two-image limit and the images= / response_format="json" arguments are correct. One error in the example:

pages/developers/intelligent-contracts/features/image-processing.mdx:61 uses raise gl.UserError("Expected a boolean matches field"). gl.UserError is not exported by the SDK, so on a malformed LLM response the example dies with AttributeError instead of raising a user error. Use gl.vm.UserError(...).

Please also add this snippet to the code example lint so the API names are checked.

@danieljrc888

Copy link
Copy Markdown
Collaborator

Re-checked at 176b0ec: the example now raises gl.vm.UserError and the snippet is covered by the example lint. Requested change resolved; this needs a maintainer approval to merge.

@jmlago

jmlago commented Nov 19, 2026

Copy link
Copy Markdown
Member Author

Legatus audit: Approved (commit 176b0ec)

Audit of PR #957 (DXP-593) at 176b0ecfbae598ceabc830d91eeaa9e1989ddfaa

The new commit fixes a real bug in the example and adds a lint check that would have caught it. Everything passes and I found nothing blocking.

What changed since 1beb128 (one commit on top, still based on main 90f98c2)

  • pages/developers/intelligent-contracts/features/image-processing.mdx:63: the example now raises gl.vm.UserError instead of gl.UserError. I checked the SDK source in genlayerlabs/genvm (runners/genlayer-py-std/src/genlayer/). UserError is defined in vm/__init__.py:74, and the top-level package does not export it. So the old example would have failed with AttributeError whenever the response was malformed. The fix is correct.
  • image-processing.mdx:51: a {/* genvm-lint: required */} MDX comment marks the new example for linting. It doesn't render on the page.
  • scripts/lint-code-examples.py:
    • Code examples marked this way are now linted on every default run, and their failures fail the job. Unmarked examples stay advisory and are only checked with --lint-snippets, as before.
    • A new AST check rejects gl.UserError.
    • On the current tree only the new example is marked, so the gate changes nothing for existing pages.
  • scripts/test-lint-code-examples.py adds four unit tests: only the comparison example is marked, the wrong namespace is rejected, the correct one is passed through to the SDK linter, and a failure in a marked example makes main() exit with 1.
  • .github/workflows/lint-code-examples.yml runs those unit tests and adds the new test file to the workflow's path triggers.
  • Scope: the lint tooling goes slightly beyond "docs examples", but it is small and opt-in, and it only guards the snippet this ticket adds. I accept it. No secrets, and no navigation or redirect changes.

Tests I ran

  • python scripts/test-lint-code-examples.py with genvm-linter 0.11.0 installed in a venv: 4 tests OK.
  • python scripts/lint-code-examples.py: exit 0, "All checks passed"; the marked example at image-processing.mdx:53 passes.
  • To check the gate catches the bug, I put gl.UserError back into the page by hand. The script exited with 1 and reported "invalid API gl.UserError — use gl.vm.UserError". Then I restored the file.
  • npm ci and npm run build (inside a Nix shell for Python) both passed.
  • Playwright against next start -p 3001 (Nix Chromium, because this NixOS host can't run the bundled browser): 171 passed, including visual-inputs.e2e.spec.ts.
  • On GitHub, Playwright E2E, lint-code-examples, the node docs sync check, CodeRabbit and the Netlify checks all passed; "Pages changed" was skipped.

Minor, not blocking: the E2E test still expects exactly three code blocks on the page, so it will need updating if more examples are added there.

I removed the audit worktree and stopped the local server. I pushed nothing and posted no comments.

Outside this ticket: other pages still use gl.UserError, which the SDK doesn't provide.

LEGATUS-TICKET
team: DXP
title: Intelligent-contract docs use nonexistent gl.UserError in calling-llms, web-access and error-handling examples
blocks: no
In the GenVM Python SDK (genlayerlabs/genvm, runners/genlayer-py-std/src/genlayer/), UserError is defined only in vm/__init__.py:74 (class UserError(Exception)) and the top-level genlayer/__init__.py __all__ does not export it. So gl.UserError raises AttributeError at runtime, and except gl.UserError fails when that clause is evaluated. pages/developers/intelligent-contracts/features/error-handling.mdx itself uses gl.vm.UserError at lines 30, 54, 58 and 74. Wrong uses on genlayer-docs main (90f98c2), from grep -rn "gl\.UserError" pages:

  • pages/developers/intelligent-contracts/features/calling-llms.mdx:138, 144, 160, 182 (raise gl.UserError(...))
  • pages/developers/intelligent-contracts/features/web-access.mdx:62, 64 (raise gl.UserError(...))
  • pages/developers/intelligent-contracts/features/error-handling.mdx:109 (except gl.UserError as e:)
    Expected: gl.vm.UserError. The linter guard from PR DXP-593: Complete visual input examples with multiple images #957 ({/* genvm-lint: required */} in scripts/lint-code-examples.py) could be applied to these snippets after they are fixed.
    LEGATUS-END

@jmlago

jmlago commented Nov 19, 2026

Copy link
Copy Markdown
Member Author

/run-e2e all

@jmlago

jmlago commented Nov 19, 2026

Copy link
Copy Markdown
Member Author

Legatus audit: Approved (commit 176b0ec)

The PR head is still 176b0ecfbae598ceabc830d91eeaa9e1989ddfaa, the same commit I approved in my previous message. There is no new diff, so I didn't rebuild the worktree or rerun the tests.

On GitHub the PR is still open and all its checks are green: Playwright E2E, lint-code-examples, the node docs sync check, CodeRabbit and the Netlify checks passed, and "Pages changed" was skipped. My review of this commit still holds:

  • Docs fix: the new two-image example now raises gl.vm.UserError. That is the correct name; the SDK defines UserError only in genlayer.vm.
  • Lint gate: the new gate fails the lint job on marked code examples. When I put gl.UserError back by hand, the script exited with 1.
  • Local runs: 4 new unit tests OK, lint-code-examples.py passed, the build passed, and Playwright 171/171 passed.

The ticket I filed in my previous message, about gl.UserError still being used in calling-llms.mdx, web-access.mdx and error-handling.mdx, still applies. It is outside this PR's scope.

@jmlago

jmlago commented Nov 19, 2026

Copy link
Copy Markdown
Member Author

/run-e2e all

@jmlago

jmlago commented Nov 19, 2026

Copy link
Copy Markdown
Member Author

Legatus audit: Approved (commit 176b0ec)

The PR head is still 176b0ecfbae598ceabc830d91eeaa9e1989ddfaa and the PR is still open. That is the commit I fully reviewed and tested two messages ago, so there is no new diff and I didn't rerun anything.

My approval of this commit stands: the docs now correctly use gl.vm.UserError, the lint gate works, the 4 new unit tests and the lint script pass, the build passes, Playwright passed 171/171, and all checks on GitHub are green. The ticket I filed in that review, about gl.UserError still appearing in other docs pages, still applies and is outside this PR's scope.

If a message like this arrives again with the same SHA, Legatus may be sending duplicate notifications without the head having changed.

@jmlago

jmlago commented Nov 19, 2026

Copy link
Copy Markdown
Member Author

/run-e2e all

@jmlago

jmlago commented Nov 19, 2026

Copy link
Copy Markdown
Member Author

Legatus audit: Approved (commit 176b0ec)

Nothing has changed. The PR head is still 176b0ecfbae598ceabc830d91eeaa9e1989ddfaa, main is still at 90f98c2, and GitHub reports the PR open and mergeable. That is the commit I fully reviewed and tested earlier, so there is nothing new to review and I didn't rerun the tests.

My earlier approval stands: the docs now correctly use gl.vm.UserError, the lint gate works, the build passes, Playwright passed 171/171, and all checks on GitHub are green. The ticket I filed about gl.UserError in other docs pages still applies and is outside this PR.

This is the third "head changed" notification for the same SHA. Legatus seems to be sending them without the head actually changing.

@jmlago

jmlago commented Nov 19, 2026

Copy link
Copy Markdown
Member Author

/run-e2e all

@jmlago

jmlago commented Nov 19, 2026

Copy link
Copy Markdown
Member Author

Legatus audit: Approved (commit 176b0ec)

Nothing has changed: the PR head is still 176b0ecfbae598ceabc830d91eeaa9e1989ddfaa, main is still at 90f98c2, and the PR is open and mergeable. That is the commit I fully reviewed and tested earlier, so I didn't rerun anything.

My earlier approval stands: the docs now correctly use gl.vm.UserError, the lint gate works, the build passes, Playwright passed 171/171, and all checks on GitHub are green. The ticket I filed about gl.UserError in other docs pages still applies and is outside this PR.

This is the fourth "head changed" notification for an unchanged SHA. Something in Legatus is probably re-queuing this review without checking whether the head moved. I haven't filed a ticket for it because none of the listed teams clearly owns Legatus.

@jmlago

jmlago commented Nov 19, 2026

Copy link
Copy Markdown
Member Author

/run-e2e all

@jmlago

jmlago commented Nov 19, 2026

Copy link
Copy Markdown
Member Author

Legatus audit: Approved (commit 176b0ec)

Nothing has changed: the PR head is still 176b0ecfbae598ceabc830d91eeaa9e1989ddfaa, main is still at 90f98c2, and the PR is open and mergeable. That is the commit I fully reviewed and tested earlier, so there is no new diff and I didn't rerun the tests.

My earlier approval stands: the docs now correctly use gl.vm.UserError, the lint gate works, the build passes, Playwright passed 171/171, and all checks on GitHub are green. The ticket I filed about gl.UserError in other docs pages still applies and is outside this PR.

This is the fifth "head changed" notification for the same unchanged SHA.

@jmlago

jmlago commented Nov 19, 2026

Copy link
Copy Markdown
Member Author

/run-e2e all

@jmlago

jmlago commented Nov 19, 2026

Copy link
Copy Markdown
Member Author

Legatus audit: Approved (commit 176b0ec)

Nothing has changed: the PR head is still 176b0ecfbae598ceabc830d91eeaa9e1989ddfaa, main is still at 90f98c2, and the PR is open and mergeable. That is the commit I fully reviewed and tested earlier, so there is no new diff to review.

My earlier approval stands: the docs now correctly use gl.vm.UserError, the lint gate works, the build passes, Playwright passed 171/171, and all checks on GitHub are green. The ticket I filed about gl.UserError in other docs pages still applies and is outside this PR.

This is the sixth "head changed" notification for the same unchanged SHA.

@jmlago

jmlago commented Nov 19, 2026

Copy link
Copy Markdown
Member Author

/run-e2e all

@jmlago

jmlago commented Nov 19, 2026

Copy link
Copy Markdown
Member Author

Legatus audit: Approved (commit 176b0ec)

Nothing has changed: the PR head is still 176b0ecfbae598ceabc830d91eeaa9e1989ddfaa, main is still at 90f98c2, and the PR is open and mergeable. That is the commit I fully reviewed and tested earlier, so there is no new diff to review.

My earlier approval stands: the docs now correctly use gl.vm.UserError, the lint gate works, the build passes, Playwright passed 171/171, and all checks on GitHub are green. The ticket I filed about gl.UserError in other docs pages still applies and is outside this PR.

This is the seventh "head changed" notification for the same unchanged SHA.

@jmlago

jmlago commented Nov 19, 2026

Copy link
Copy Markdown
Member Author

/run-e2e all

@danieljrc888 danieljrc888 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at the head I reviewed; the requested changes are resolved.

@danieljrc888
danieljrc888 merged commit 01d0bb2 into main Nov 19, 2026
8 checks passed
@danieljrc888
danieljrc888 deleted the legatus/dxp-593 branch November 19, 2026 04:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants