Repository navigation
Add plain-text report input (llms-full.txt) support - #10
Merged
Merged
Conversation
dimalvovs
commented
Sep 18, 2026
dimalvovs
commented
Contributor
- ingest: read_text() and split_text_sections() for heading-based chunking
- interpret: TEXT_SYSTEM_PROMPT/TEXT_AS_IS_SYSTEM_PROMPT, interpret_text_report, shared _run_chunks helper
- llmize: auto-detect .json vs plain text input, new --as-is flag, run_text_pipeline
- tests: coverage for split_text_sections
- ingest: read_text() and split_text_sections() for heading-based chunking - interpret: TEXT_SYSTEM_PROMPT/TEXT_AS_IS_SYSTEM_PROMPT, interpret_text_report, shared _run_chunks helper - llmize: auto-detect .json vs plain text input, new --as-is flag, run_text_pipeline - tests: coverage for split_text_sections
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Two moderate issues remain unresolved in text section assembly and synthesis prompting.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Adds support for ingesting and interpreting plain-text reports such as llms-full.txt alongside JSON inputs.
Changes:
- Adds UTF-8 text loading and heading-based section splitting.
- Adds text-specific interpretation prompts, chunk handling, and
--as-is. - Adds CLI dispatch and splitter tests.
Review findings:
ingest/text_ingest.py:59— Moderate: aPreambleheading can overwrite preamble text.interpret.py:588— Moderate: text synthesis uses the generic synthesis prompt instead of text-specific instructions.
File summaries
| File | Description |
|---|---|
tests/test_ingest.py |
Tests heading-based text splitting. |
llmize.py |
Detects JSON/text inputs and dispatches the appropriate pipeline. |
interpret.py |
Adds text prompts and shared chunk execution. |
ingest/text_ingest.py |
Implements heading-based section splitting. |
ingest/io_utils.py |
Adds UTF-8 text reading. |
ingest/__init__.py |
Exports text-ingest APIs. |
.github/prompts/plan-textReportInput.prompt.md |
Documents the implementation plan. |
Review details
- Files reviewed: 7/7 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+59
to
+60
| for name in order: | ||
| result[name] = {"data": sections[name]} |
Contributor
There was a problem hiding this comment.
Fixed in d4d66ac by preserving the preamble entry and suffixing any colliding heading names during result assembly.
Comment on lines
+588
to
+590
| return _run_chunks( | ||
| chunks, model, system, num_ctx, synthesize_final, think, gen_options, | ||
| title="Report Interpretation", |
Contributor
There was a problem hiding this comment.
Fixed in d4d66ac by threading a text-specific synthesis system prompt through _run_chunks, with a separate --as-is synthesis prompt that omits the style/evidence rules.
Co-authored-by: dimalvovs <[email protected]>
- ingest/text_ingest: looks_like_multiqc_llms_full() detects the Tool:/Section:/ Title: block layout (no markdown headings); split_multiqc_llms_full() chunks one per Tool: block, attaching the matching numbered-index Description (HTML-cleaned) when tool names match - interpret.interpret_text_report: uses the new splitter when detected and not --whole-report; --as-is uses the file's own leading instructions block as the system prompt for this layout - tests: TestMultiqcLlmsFull coverage for detection, chunking, description attachment, and Section-qualified naming
- ingest/text_ingest: parse_markdown_table() parses the first markdown table in a chunk (ignoring surrounding prose); extract_samplesheet_chunk() pops a Sample Sheet-like chunk out of the sections dict when its table parses to >=2 rows (exact alias match, fail-soft otherwise) - interpret: refactor sample_groups()/build_samplesheet_context() into thin report-shaped wrappers around shape-agnostic _sample_groups_from_data()/ _samplesheet_context_block() (pure refactor, JSON-mode behavior unchanged); interpret_text_report() now extracts the sample sheet, excludes it from individual chunk analysis, and threads the shared grouping-aware context into every remaining chunk and the synthesis pass, independent of --as-is - tests: parse_markdown_table/extract_samplesheet_chunk coverage in test_ingest.py; interpret_text_report samplesheet-context coverage in test_interpret.py
- lead pitch generic (JSON or plain text), MultiQC as best-supported example - split JSON input / plain-text input / --as-is + --whole-report into their own sections; fix broken --output table row; drop 'Where to place your files' (folded into Parameters) and 'Continuous integration' sections - interpret.py: drop remaining 'spatial transcriptomics' wording from JSON-mode prompts and default report title to match the generic framing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.