Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 49 additions & 25 deletions .kb/agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,43 +2,67 @@

This repository follows strict conventions for organizing agent-oriented knowledge documents, and this document is required reading for any agent that wants to read or write these.

Read the top-level `.kb/agents.md` before continuing below.
The design of this structure has the following key goals:

- **Mechanical** - Encourages agents to read/write the knowledge base as main actors.
- **Generic** - Benefits any agentic workflow, no matter the editor or platform.
- **Distilled** - Avoids the use of verbose task or plan logs that pollute the context window.
- **Hierarchical** - Takes information closer to its context, avoids bloat in a single place.
- **Human** - Preserves information readily available and reviewable in a friendly format.

# Overview
IMPORTANT: Read the `.kb/agents.md` file.

Every directory in this repository, including the root, may have its own `AGENTS.md` file and `.kb/` subdirectory. The `AGENTS.md` file provides a general view for the directory, while more specific information is found in `.kb/*.md` files with dashed lowercase names (e.g. `.kb/special-relativity.md`).

The design of this structure has the following key goals:
# Laws

- **Mechanical** - Agents are the main actors reading and writing the knowledge base.
- **Generic** - Benefits any agentic workflow, no matter the editor or platform.
- **Distilled** - Avoids the use of verbose task or plan logs that pollute the context window.
- **Hierarchical** - Avoids excessive information in a single place that also pollutes the context window.
- **Human** - Information is readily available and reviewable in a friendly format.
## §1 - Organization

Every directory in this repository may have its own `.kb/` subdirectory with `.kb/*.md` files with dashed lowercase names such as `.kb/special-relativity.md`.

Every filename must be _globally unique_ inside the repository, so section references such as agents.md§1 are possible.

## §2 - Headers

# Important
The following are the ONLY top-level headers allowed in `.kb/*.md` files in this repository. Keep the section order identical to that presented below.

- Read local `AGENTS.md` files upon navigating directories.
- Keep the `AGENTS.md` and `.kb/*.md` files updated whenever there is something relevant to be documented or updated. The top-level `.kb/agents.md` must NOT be edited unless explicitly requested.
- Follow the header conventions outlined below. Only the _Preface_ header is required, and the other headers should be omitted if empty or trivial.
- _Preface_ - A brief introduction outlining the scope and relevance of a specific `*.md` file. This section MUST be at the top of every `.kb/*.md` file so agents can easily grep for it, and the last line of this section MUST be "IMPORTANT: Read the `.kb/agents.md` file." so rules are respected.
- _Laws_ - Essential directives outlining critical constraints, behaviors, and rules. The Laws are defined in _terse_ subsections `## §<N> - <Title>` as in this file, internally referenced as §3 and externally as agents.md§3 (this law).
- _Concepts_ - Structural design, details, references, etc. This is the only section that accepts open-ended subsections.

Keep two empty lines before top headers, and one empty line before subheaders.

# Headers
## §3 - Process

The following are the ONLY top-level headers allowed across the `.kb/*.md` files in this repository, to maintain semantic standardization across projects. Keep the ordering of created sections the exact same you see below.
Sub-headers are okay.
- Read the _Knowledge Base_ section inside the `AGENTS.md` file at the top of the repository.
- Keep the `.kb/*.md` files up-to-date: add or update content whenever something relevant needs documenting. The `.kb/agents.md` must NOT be edited unless explicitly requested.
- Follow the header conventions in §3. Only the _Preface_ header is required, and the other headers should be omitted if empty or trivial.
- If you find incorrect organization that breaks any of the laws in this document, fix it.
- Never change the subject of a Law - if obsolete, replace its header with `## §<N> - Revoked`.

- _Preface_ - A brief introduction outlining the scope and relevance of a specific `*.md` file. This section MUST be at the top of every `.kb/*.md` file so agents can easily grep for it, and the last line of this section MUST be "Read the top-level `.kb/agents.md` file before continuing below." so rules are followed.
- _Overview_ - High-level summary of the directory, subsystem or knowledge base layout at large. Do NOT use this to list files or directories.
- _Important_ - Essential directives for the agent outlining critical constraints, behaviors, or rules.
- _Headers_ - Global registry of header definitions, uniquely hosted at the root `.kb/agents.md` (this file). Do NOT use this header in any other document.
- _Architecture_ - Structural design details or boundary explanations for a given component. Only use this for software architecture concepts, NOT for directory outlining. Also avoid using this as a code reference (keep that in the code itself).
- _Directory_ - Only in `AGENTS.md` files, it briefly outlines the contents and structure of the directory the `AGENTS.md` file is in, and potentially nested small directories that do not justify their own `AGENTS.md` file. For `.kb/*.md` files, use _Documents_ instead.
- _Documents_ - Only in `AGENTS.md` files, it outlines the content of `.kb/*.md` files and also immediately nested `AGENTS.md` child files, to aid agent navigation. It MUST be the last section in the file. It's okay to also mention such filenames inline when they are relevant elsewhere in the text.
## §4 - AGENTS.md

For the _Directory_ and _Documents_ listings, format items as a dashed list starting with the file or directory name surrounded by backticks, a dash, and then a brief description:
The repository's `AGENTS.md` file MUST START with an ordered list of ALL `.kb/*.md` files, in this format:
```
- `filename` - Terse summary.
# Knowledge Base

IMPORTANT: Read the `.kb/agents.md` file.

- [.kb/agents.md](.kb/agents.md) - Structure and rules of the knowledge base. You MUST read this file.
- [.kb/filename.md](.kb/filename.md) - Terse summary.
- [subdir/.kb/filename.md](subdir/.kb/filename.md) - Another terse summary.
```

# Concepts

## Changelog

- **2026-11-03** - Added revocations. Moved Headers before Process. Changed sentence pointing to .kb/agents.md, enforced it for AGENTS.md.
- **2026-10-30** - Major overhaul. Added: Laws, Concepts; Removed: Overview, Important, Architecture, Directory, Documents, AGENTS.md in subdirectories.
- **2026-09-24** - Enforced ordering of all headers. Minor tweaks for clarity.
- **2026-06-08** - Public release.

## Version updates

Only when EXPLICITLY requested to update the version of the knowledge base, replace this file with:

https://raw.githubusercontent.com/canonical/knowledge-base/refs/heads/main/.kb/agents.md
48 changes: 48 additions & 0 deletions .kb/repo-overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Preface

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My vote is to drop this entire file for now. It's not pulling its weight at the moment.


This document holds the top-level overview of the Chisel repository.

IMPORTANT: Read the `.kb/agents.md` file.


# Laws

## §1 - Deterministic behavior

Behavior must be deterministic and predictable, failing on the side of correctness and preserving existing user content by default.

## §2 - Provider independence

Chisel's operation and logic, including validation, must not rely on package contents or on information from package providers (currently the archive, in the future the bin store). Do as much as possible without fetching from package providers, so that operations stay fast and can work without network access.

@lczyk lczyk Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Chisel's operation and logic (...) must not rely on package contents or on information from (...) the archive". are we changing chisel in a way in which i was not aware of? 😄 more seriously, the previous workding was better imo: "as independent as practical from incidental package archive contents". as written it sounds like its going very strongly against the literal thing chisel is supposed to do.

also, leave bin store out until it exists? no strong opinions on that one though tbh.


## §3 - Compatibility

Do not break compatibility of the chisel-releases format or the public manifest packages, backward or forward, without a compelling reason.


# Concepts

## Purpose

Chisel cuts fine-grained slices out of Debian packages: it takes a selection of slices defined in chisel-releases, resolves their dependencies, downloads the corresponding packages from Ubuntu archives, extracts the selected files into a target root filesystem, runs Starlark mutation scripts, and generates manifests at paths requested by the selected slice definitions. The codebase is split across command entry points, an internals tree holding the carving pipeline, and public packages defining the manifest format.

## Compatibility guidance

When introducing a new feature or fixing a bug, consider both backward and forward compatibility: new behavior must keep working with previous Chisel releases, and parsing of release YAML files must accept or reject values so that future format evolutions remain possible.

@lczyk lczyk Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reads a bit weird. how about something like "new versions of chisel must be able to keep reading old releases, and, where possible, release format changes stay parseable by older chisel versions."

edit: i mean the "new behavior must keep working with previous Chisel releases" bit
edit2: it pains me to write "behavior" but i've begrudgingly accept that US english is the standard... (🇬🇧)


## Contents

- `cmd/` - Binary entry points for the `chisel` application.
- `internal/` - Core internal packages: slice orchestration, setup, extraction, archive, cache, and supporting utilities.
- `public/` - Public API packages for external consumers.
- `snap/` - Snap packaging configuration for Chisel.
- `tests/` - End-to-end integration test suite.
- `docs/` - Documentation assets.
- `go.mod` - Go module definition.
- `go.sum` - Go dependency checksums.
- `spread.yaml` - Spread test configuration.
- `workshop.yaml` - Workshop configuration.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As an aside, this should go to a dotted file.

- `.golangci.yaml` - golangci-lint configuration.
- `README.md` - Project readme.
- `SECURITY.md` - Security policy.
- `LICENSE` - Project license.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's please avoid such listings, unless they are really adding value. Pretty much everything in the list at the moment is obvious from context at the moment.

52 changes: 9 additions & 43 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,45 +1,11 @@
# Preface
# Knowledge Base

This repository hosts Chisel, a software tool for carving and cutting Debian packages. This `AGENTS.md` is the root index into the dispersed knowledge base; read it to orient yourself, then follow the links below for details.
IMPORTANT: Read the `.kb/agents.md` file.

Read the top-level `.kb/agents.md` file before continuing below.


# Overview

Chisel cuts fine-grained slices out of Debian packages. It takes a selection of slices defined in chisel-releases, resolves their dependencies, downloads the corresponding packages from Ubuntu archives, extracts the selected files into a target root filesystem, runs Starlark mutation scripts, and generates manifests at paths requested by the selected slice definitions. The codebase is split across command entry points, an internals tree holding the carving pipeline, and public packages defining the manifest format.


# Important

- Prefer deterministic, predictable behavior that fails on the side of correctness and preserves existing user content by default.
- Keep Chisel as independent as practical from incidental package archive contents.
- Treat changes to the chisel-releases format and the public manifest packages as compatibility-sensitive.


# Directory

- `cmd/` - Binary entry points for the `chisel` application.
- `internal/` - Core internal packages: slice orchestration, setup, extraction, archive, cache, and supporting utilities.
- `public/` - Public API packages for external consumers.
- `snap/` - Snap packaging configuration for Chisel.
- `tests/` - End-to-end integration test suite.
- `docs/` - Documentation assets.
- `go.mod` - Go module definition.
- `go.sum` - Go dependency checksums.
- `spread.yaml` - Spread test configuration.
- `workshop.yaml` - Workshop configuration.
- `.golangci.yaml` - golangci-lint configuration.
- `README.md` - Project readme.
- `SECURITY.md` - Security policy.
- `LICENSE` - Project license.


# Documents

- `.kb/agents.md` - General rules for the knowledge base reading and writing.
- `cmd/AGENTS.md` - CLI entry points.
- `internal/AGENTS.md` - Core internal packages.
- `public/AGENTS.md` - Public API packages.
- `snap/AGENTS.md` - Snap packaging.
- `tests/AGENTS.md` - Integration testing.
- [.kb/agents.md](.kb/agents.md) - Structure and rules of the knowledge base. You MUST read this file.
- [.kb/repo-overview.md](.kb/repo-overview.md) - Chisel overview, behavioral laws, and root directory layout.
- [cmd/.kb/chisel-entry-points.md](cmd/.kb/chisel-entry-points.md) - Binary entry points for the `chisel` application.
- [internal/.kb/internal-packages.md](internal/.kb/internal-packages.md) - Core internal packages and the carving pipeline.
- [public/.kb/public-packages.md](public/.kb/public-packages.md) - Public API packages and the license boundary.
- [snap/.kb/snap-packaging.md](snap/.kb/snap-packaging.md) - Snap packaging configuration.
- [tests/.kb/integration-suite.md](tests/.kb/integration-suite.md) - Spread-based end-to-end integration testing.
32 changes: 27 additions & 5 deletions cmd/AGENTS.md → cmd/.kb/chisel-entry-points.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,36 @@
# Preface

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My vote is to drop this one as well. This is an unrelated collection of low-value ideas.


This document serves as the local knowledge base index for the `cmd/` directory. It outlines the scope and purpose of the entry points for the `chisel` binary.
This document covers the CLI entry points for the `chisel` binary in the `cmd/` directory.

Read the top-level `.kb/agents.md` file before continuing below.
IMPORTANT: Read the `.kb/agents.md` file.

# Overview

The `cmd/` directory contains the main binary entry points for the `chisel` application. Instead of housing core business logic, this directory is responsible for initializing the application, wiring up the CLI subsystem, and handling version information. Core logic is deferred to the packages in `internal/`.
# Laws

# Directory
## §1 - CLI boundary

Do not add core business logic to `cmd/`; commands only wire up flags and arguments and delegate to the packages in `internal/`.


# Concepts

## Scope

The `cmd/` directory contains the main binary entry points for the `chisel` application, responsible for initializing the application, wiring up the CLI subsystem, and handling version information.

## Debug commands

The `debug` subcommand group has no stability guarantee: its commands may be removed without notice and may not work on non-development systems, so they are registered as hidden.

## Version injection

The application version is injected at build time by `mkversion.sh` (via `go:generate`), which writes the generated `version_generated.go`.

## Testing hooks

Tests override the `Stdin`/`Stdout`/`Stderr`/`ReadPassword` variables instead of touching the `os` streams directly.

## Contents

- `chisel/` - Contains the primary executable entry point for the `chisel` application.
- `chisel/main.go` - Application bootstrap and CLI initialization.
Expand Down
13 changes: 8 additions & 5 deletions internal/AGENTS.md → internal/.kb/internal-packages.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,17 @@
# Preface

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same.


This is the index file for the `internal/` directory's knowledge base. It provides context about the core internal packages of Chisel.
This document covers the core internal packages and the carving pipeline in the `internal/` directory.

Read the top-level `.kb/agents.md` file before continuing below.
IMPORTANT: Read the `.kb/agents.md` file.

# Overview

# Concepts

## Scope

The `internal/` directory houses the core business logic, components, and utilities of Chisel. It coordinates slice selection, dependency resolution, package fetching, file extraction, filesystem mutations, and manifest output. These packages are not part of Chisel's public API and must not be imported by external consumers.

# Architecture
## Pipeline

Chisel's carving pipeline is coordinated by `slicer/`:

Expand All @@ -23,7 +26,7 @@ flowchart LR
Mutate --> Manifest["Generate requested manifests"]
```

# Directory
## Contents

- `slicer/` - Main orchestrator for a Chisel run. Receives a slice selection, drives all other internal packages (setup, archive, cache, deb, fsutil, scripts, manifestutil) to completion, and writes the final filesystem and manifest.
- `setup/` - Parses chisel-releases YAML definitions into the `Release` model and resolves slice dependencies, path conflicts, and package contention.
Expand Down
22 changes: 16 additions & 6 deletions public/AGENTS.md → public/.kb/public-packages.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,28 @@
# Preface

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same.. those files look like directory listings.


This document describes the scope of the `public/` directory, which contains Chisel's public API packages intended for consumption by external tools.
This document covers the public API packages in the `public/` directory.

Read the top-level `.kb/agents.md` file before continuing below.
IMPORTANT: Read the `.kb/agents.md` file.

# Overview

# Laws

## §1 - License boundary

Packages in `public/` and their dependencies must remain Apache-2.0 licensed.


# Concepts

## Scope

The `public/` directory houses the two packages that form Chisel's stable public contract. They provide a generic sorted JSON-lines database format and the Chisel manifest schema built on it, enabling third-party tools such as SBOM generators and vulnerability scanners to consume Chisel output without depending on internal packages.

# Important
## License check

- When changing these packages or their dependencies, run `go run .github/scripts/external-packages-license-check.go` to verify the Apache-2.0 license boundary.
When changing these packages or their dependencies, run `go run .github/scripts/external-packages-license-check.go` to verify the license boundary.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems strange to ask the bot to be doing this every time it touches a file. This should probably be going to something like hacking.md next to other conventions that we follow, without asking for the action to be repeated on every change. If we want to have such a per-change action, it should be a single repository-wide static check, and then we don't need to tell what it is doing because if it fails it will say so.


# Directory
## Contents

- `jsonwall/` - Implements a generic database format with one JSON object per line, with fields and entries sorted for efficient search and iteration.
- `manifest/` - Defines Chisel manifest schema 1.0 through the `Package`, `Slice`, `Path`, and `Content` record types, and uses `jsonwall` for serialization and deserialization. The carving pipeline applies Zstandard compression when generating manifest files.
18 changes: 14 additions & 4 deletions snap/AGENTS.md → snap/.kb/snap-packaging.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,23 @@
# Preface

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one might stay, in the hope it grows to something useful. But let's rename it to snap-package.md, and drop the directory listing at the bottom.


This document describes the scope of the `snap/` directory within the Chisel repository, providing context for automated agents navigating and modifying the snapcraft configuration.
This document covers the snap packaging configuration in the `snap/` directory.

Read the top-level `.kb/agents.md` file before continuing below.
IMPORTANT: Read the `.kb/agents.md` file.

# Overview

# Laws

## §1 - Strict confinement

The snap must use strict confinement.


# Concepts

## Scope

The `snap/` directory contains the configuration necessary to build and package Chisel as a snap application. It defines the application metadata, execution confinement, and compilation steps for the `chisel` application using Go.

# Directory
## Contents

- `snapcraft.yaml` - The primary manifest file defining the snap package. It configures strict confinement, utilizes the Go plugin, and outlines the build steps which depend on `cmd/mkversion.sh` for version injection.
Loading
Loading