Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions packages/safe-bash/docs/CSPLIT.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,27 @@ The default portable regex provider has an additional 65536-byte subject limit,
so a larger `maxLineBytes` alone does not permit regex matching on longer lines.
Provider-level limits and the command's limits both apply.

`maxBufferedBytes` includes twice the admitted input payload, 128 accounting
units for each retained line or partial-line reference, 256 units plus twice the
path's character count for each retained output receipt, and the active output
buffer (at most 65536 bytes). Finishing an output releases its buffer; cleanup
releases its receipt reservation. Only the immediate parent snapshot remains
retained after output creation. Admission checks run before payload allocation.
These limits apply together, so reaching one configured maximum does not promise
that every other maximum can be reached simultaneously; fragmented input and
retained outputs also consume the allowance.
Non-streaming reads additionally reserve the complete returned file array before
requesting it, pass its admitted size as the read bound, and retain that separate
charge until cleanup. Streaming avoids this extra retained-array allowance.

Output uses a bounded 65536-byte buffer. A full buffer and a completed output
flush before byte counts are printed. Ordinary input or pattern errors flush
pending bytes before the `-k` retention decision. Cancellation discards pending
bytes; `-k` retains only the already committed prefix. This follows GNU's stdio
error-versus-signal policy, but does not promise GNU's platform-dependent buffer
threshold (the GNU 9.11 macOS comparison used 8192 bytes). A refused atomic flush
commits no prefix and prints no byte count for that output.

## Environment and compatibility boundaries

`LC_ALL`, `LC_CTYPE`, `LC_COLLATE`, and `LANG` select the supported C/POSIX or
Expand All @@ -115,6 +136,15 @@ UTF-8, and command arguments cannot contain NUL. Input/output alias checks and
bounded resource refusals intentionally protect the virtual input rather than
copying unsafe or unbounded native effects. Filesystem identity and cleanup
guarantees remain limited by the injected provider's declared capabilities.
Output requires the `atomicFileMutation` capability and both
`writeFileConditional` and `removeFileConditional`. Directories require stable,
scoped device/inode identity; output entries additionally require a revision.
Opening, appending, and removing an output atomically compare the captured
parent and file snapshots. Completed writes return their original committed
snapshot even if cancellation arrives before the response. Cleanup cannot adopt
or remove a replacement entry. Providers without these methods refuse output
mutation; checking a path before an ordinary write is not a fallback.

Output directories and entries require stable, scoped device/inode identity;
symlink output parents and existing symlink outputs are refused. Existing outputs
are also refused when known input metadata cannot establish its identity. A
Expand Down
25 changes: 14 additions & 11 deletions packages/safe-bash/src/commands/archive/unzip.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { collectBytes, readBytes, resolvePath, writeBytes, type CommandContext, type CommandDefinition, type FileStat } from "../../contracts/index.js";
import { dirname, collectBytes, readBytes, resolvePath, writeBytes, type CommandContext, type CommandDefinition, type FileStat } from "../../contracts/index.js";
import { writeFileOutput } from "../../contracts/filesystem-output.js";
import { publicDiagnosticMessage } from "../../diagnostics.js";
import { Budget, bounded, checkPath, display, fail, settings, text, vfsPath, type ArchiveCommandsOptions } from "./internal.js";
Expand Down Expand Up @@ -97,8 +97,8 @@ export function createUnzipCommand(options: ArchiveCommandsOptions = {}): Comman
let selected = 0;
let total = 0;
let actualTotal = 0;
const links: { path: string; shown: string; target: string; existing: FileStat | undefined; entry: ZipEntry }[] = [];
const directories: { path: string; entry: ZipEntry; identity: FileStat }[] = [];
const links: { path: string; shown: string; target: string; existing: FileStat | undefined; parent: FileStat; entry: ZipEntry }[] = [];
const directories: { path: string; entry: ZipEntry; identity: FileStat; parent: FileStat }[] = [];
for (const entry of zip.entries) {
await budget.member(entry.size);
checkPath(entry.name, limits);
Expand All @@ -120,17 +120,19 @@ export function createUnzipCommand(options: ArchiveCommandsOptions = {}): Comman
for await (const chunk of readBytes(decodeZipEntry(entry, limits, context.signal), context.signal)) {
await writeFileOutput(context, chunk, async () => { if (chunk.length) fail("directory has nonempty payload"); });
}
const parent = await extraction.operation(() => context.fs.lstat(dirname(path), { signal: context.signal }));
const existing = await extraction.stat(path);
let identity = existing;
if (existing && existing.type !== "directory") fail("directory destination is not a directory");
if (!existing) {
await extraction.operation(() => context.fs.mkdir(path, { signal: context.signal, mode: 0o755 }));
identity = await extraction.createDirectory(path, parent);
await budget.output(` creating: ${filtered(shown)}\n`);
}
const identity = await extraction.stat(path);
if (!identity || identity.type !== "directory") fail("directory changed during creation");
directories.push({ path, entry, identity });
directories.push({ path, entry, identity, parent });
continue;
}
let parent = await extraction.operation(() => context.fs.lstat(dirname(path), { signal: context.signal }));
let existing = await extraction.destination(path, archivePath, archiveStat);
let skip = false;
let prompting = 0;
Expand Down Expand Up @@ -158,6 +160,7 @@ export function createUnzipCommand(options: ArchiveCommandsOptions = {}): Comman
path = extraction.member(root, renamed);
shown = parsed.destination === undefined ? renamed : `${parsed.destination.endsWith("/") ? parsed.destination : `${parsed.destination}/`}${renamed}`;
await extraction.parents(root, path, true);
parent = await extraction.operation(() => context.fs.lstat(dirname(path), { signal: context.signal }));
existing = await extraction.destination(path, archivePath, archiveStat);
continue;
}
Expand All @@ -177,10 +180,10 @@ export function createUnzipCommand(options: ArchiveCommandsOptions = {}): Comman
const target = text(Buffer.concat(chunks));
await extraction.target(root, path, target);
if (!context.fs.symlink || context.fs.capabilities.symlinks === false) fail("filesystem does not support symlinks");
links.push({ path, shown, target, existing, entry });
links.push({ path, shown, target, existing, parent, entry });
await budget.output(` linking: ${padded(filtered(shown))} -> ${filtered(target)} \n`);
} else {
await extraction.publish(root, path, chunks, existing, entry.mode, entry.modified);
await extraction.publish(root, path, chunks, existing, parent, entry.mode, entry.modified);
await budget.output(`${entry.method === 0 ? " extracting" : " inflating"}: ${padded(filtered(shown))} \n`);
}
}
Expand All @@ -191,11 +194,11 @@ export function createUnzipCommand(options: ArchiveCommandsOptions = {}): Comman
await extraction.parents(root, link.path, false);
await extraction.target(root, link.path, link.target);
await extraction.destination(link.path, archivePath, archiveStat);
await extraction.publish(root, link.path, [], link.existing, link.entry.mode, link.entry.modified, link.target);
await extraction.publish(root, link.path, [], link.existing, link.parent, link.entry.mode, link.entry.modified, link.target);
await budget.output(` ${padded(filtered(link.shown))} -> ${filtered(link.target)}\n`);
}
for (const { path, entry, identity } of directories.reverse()) {
await extraction.metadata(root, path, identity, entry.mode, entry.modified);
for (const { path, entry, identity, parent } of directories.reverse()) {
await extraction.metadata(root, path, identity, parent, entry.mode, entry.modified);
}
for (let index = 0; index < parsed.patterns.length; index++) if (!selection.matched.has(index)) {
await budget.output(`caution: filename not matched: ${filtered(parsed.patterns[index]!)}\n`, true);
Expand Down
106 changes: 51 additions & 55 deletions packages/safe-bash/src/commands/archive/unzip/safety.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { dirname, isPathWithin, readBytes, resolvePath, type ByteSource, type CommandContext, type FileStat } from "../../../contracts/index.js";
import { dirname, isPathWithin, readBytes, resolvePath, type ByteSource, type CommandContext, type FileStat, type FileStaging } from "../../../contracts/index.js";
import { retainFileSystemCleanup } from "poe-code/safe-fs/core";
import { checkPath, display, fail, hasIdentity, sameIdentity, type ArchiveLimits } from "../internal.js";

Expand Down Expand Up @@ -74,13 +74,23 @@ export class Extraction {
const { fs, signal } = this.context;
for (const component of raw.split("/")) {
if (!component || component === ".") continue;
const parent = await this.operation(() => fs.lstat(current, { signal }));
current = resolvePath(current, component);
const stat = await this.stat(current);
if (!stat && create) await this.operation(() => fs.mkdir(current, { signal, mode: 0o755 }));
if (!stat && create) await this.createDirectory(current, parent);
else if (!stat || stat.type !== "directory") fail(`unsafe non-directory or symlink ancestor: ${display(current)}`);
}
return current;
}
async createDirectory(path: string, parent: FileStat): Promise<FileStat> {
const { fs, signal } = this.context;
const capabilities = await this.operation(async () => await fs.capabilitiesFor?.(path, { signal, create: true }) ?? fs.capabilities);
if (capabilities.atomicDirectoryMetadata !== true || !fs.prepareDirectory) fail("extraction requires atomic directory creation");
return this.operation(() => fs.prepareDirectory!(path, {
signal, parent, expected: null,
...(capabilities.permissions === false ? {} : { mode: 0o755 }),
}));
}
async parents(root: string, path: string, create: boolean): Promise<void> {
if (!isPathWithin(root, path)) fail("extraction path escapes root");
await this.directory(dirname(path), create);
Expand Down Expand Up @@ -143,75 +153,61 @@ export class Extraction {
const current = await this.stat(path);
if (!current || current.type !== expected.type || !sameIdentity(current, expected)) fail("extraction entry changed before mutation");
}
async metadata(root: string, path: string, identity: FileStat, mode: number, modified: Date): Promise<void> {
async metadata(root: string, path: string, identity: FileStat, parent: FileStat, mode: number, modified: Date): Promise<void> {
const { fs, signal } = this.context;
if (fs.chmod && fs.capabilities.permissions !== false) {
await this.checked(root, path, identity);
await this.operation(() => fs.chmod!(path, mode & 0o777, { signal }));
}
if (fs.utimes && fs.capabilities.timestamps !== false) {
await this.checked(root, path, identity);
await this.operation(() => fs.utimes!(path, modified.getTime(), modified.getTime(), { signal }));
}
await this.parents(root, path, false);
const capabilities = await this.operation(async () => await fs.capabilitiesFor?.(path, { signal }) ?? fs.capabilities);
if (capabilities.permissions === false && capabilities.timestamps === false) return;
if (capabilities.atomicDirectoryMetadata !== true || !fs.prepareDirectory) fail("extraction metadata requires atomic entry conditions");
await this.operation(() => fs.prepareDirectory!(path, {
signal, expected: identity, parent,
...(capabilities.permissions === false ? {} : { mode: mode & 0o777 }),
...(capabilities.timestamps === false ? {} : { atimeMs: modified.getTime(), mtimeMs: modified.getTime() }),
}));
}
publish(root: string, path: string, chunks: readonly Uint8Array[], expected: FileStat | undefined, mode: number, modified: Date, target?: string): Promise<void> {
const publication = this.stage(root, path, chunks, expected, mode, modified, target);
publish(root: string, path: string, chunks: readonly Uint8Array[], expected: FileStat | undefined, parent: FileStat, mode: number, modified: Date, target?: string): Promise<void> {
const publication = this.stage(root, path, chunks, expected, parent, mode, modified, target);
this.publications.add(publication);
return publication.finally(() => { this.publications.delete(publication); });
}
private async stage(root: string, path: string, chunks: readonly Uint8Array[], expected: FileStat | undefined, mode: number, modified: Date, target: string | undefined): Promise<void> {
private async stage(root: string, path: string, chunks: readonly Uint8Array[], expected: FileStat | undefined, parent: FileStat, mode: number, modified: Date, target: string | undefined): Promise<void> {
const { fs, signal } = this.context;
let temporary = "";
let identity: FileStat | undefined;
let created = false;
const capabilities = await this.operation(async () => await fs.capabilitiesFor?.(path, { signal, create: true }) ?? fs.capabilities);
if (capabilities.atomicFileStaging !== true || !fs.createStagedFile || !fs.publishStagedFile || !fs.removeStagedFile) fail("extraction requires atomic owned file staging");
let staging: FileStaging | undefined;
let failure: { reason: unknown } | undefined;
const cleanup = retainFileSystemCleanup(fs, async view => {
if (!created || !identity) return;
try {
let parent = "/";
for (const component of dirname(temporary).split("/").filter(Boolean)) {
parent = resolvePath(parent, component);
if ((await view.lstat(parent)).type !== "directory") return;
}
const current = await view.lstat(temporary);
if (current.type === identity.type && sameIdentity(current, identity)) await view.rm(temporary);
} catch (error) {
if (typeof error !== "object" || error === null || !("code" in error) || error.code !== "ENOENT") throw error;
if (staging) {
if (!view.removeStagedFile) fail("extraction atomic cleanup unavailable");
await view.removeStagedFile(staging);
}
}, { maxOperations: Math.min(4096, this.limits.maxDepth + 3) });
try {
await this.parents(root, path, true);
await this.parents(root, path, false);
const content = target === undefined ? { type: "file" as const, data: Buffer.concat(chunks) } : { type: "symlink" as const, target };
for (let attempt = 0; attempt < this.limits.maxMembers; attempt++) {
temporary = resolvePath(dirname(path), `.unzip-${++this.serial}`);
const temporary = resolvePath(dirname(path), `.unzip-${++this.serial}`);
checkPath(temporary, this.limits);
if (temporary === path || await this.stat(temporary)) continue;
await this.operation(async () => {
if (target === undefined) await fs.writeFile(temporary, new Uint8Array(), { signal, flag: "wx", mode: 0o600 });
else await fs.symlink!(target, temporary, { signal });
created = true;
identity = await fs.lstat(temporary);
});
if (!identity || !hasIdentity(identity)) fail("temporary file backing identity unavailable");
break;
}
if (!created) fail("temporary file attempt limit exceeded");
if (target === undefined) {
for (const chunk of chunks) {
await this.checked(root, temporary, identity!);
await this.operation(() => fs.appendFile(temporary, chunk, { signal }));
checkPath(`${temporary}/entry`, this.limits);
if (temporary === path) continue;
try {
await this.operation(async () => {
staging = await fs.createStagedFile!(temporary, "entry", content, {
signal, parent,
...(target !== undefined || capabilities.permissions === false ? {} : { mode: mode & 0o777 }),
...(target !== undefined || capabilities.timestamps === false ? {} : { atimeMs: modified.getTime(), mtimeMs: modified.getTime() }),
});
});
break;
} catch (error) {
signal.throwIfAborted();
if (typeof error !== "object" || error === null || !("code" in error) || error.code !== "EEXIST") throw error;
}
await this.metadata(root, temporary, identity!, mode, modified);
}
if (!staging) fail("temporary file attempt limit exceeded");
await this.parents(root, path, false);
const current = await this.stat(path);
if (current ? !expected || !sameIdentity(current, expected) || current.type !== "file" : expected !== undefined) fail("destination changed before publication");
const staging = await this.stat(temporary);
if (!staging || !identity || !sameIdentity(staging, identity) || staging.type !== (target === undefined ? "file" : "symlink")) fail("temporary file changed before publication");
await this.operation(() => fs.rename(temporary, path, { signal }));
created = false;
} catch (error) {
failure = { reason: error };
}
await this.operation(() => fs.publishStagedFile!(staging!, path, { signal, parent, destination: expected ?? null }));
} catch (error) { failure = { reason: error }; }
try { await cleanup(); }
catch (error) {
if (failure) throw new AggregateError([failure.reason, error], "unzip publication and cleanup failed");
Expand Down
3 changes: 2 additions & 1 deletion packages/safe-bash/src/commands/archive/zip.ts
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ async function prepare(scope: ZipScope, parsed: ZipOptions, budget: Budget) {
const outputName = vfsPath(context.cwd, parsed.archive);
const parentName = dirname(outputName);
const parent = await scope.operation(() => context.fs.realpath(parentName, { signal: context.signal }));
const parentStat = await scope.operation(() => context.fs.lstat(parent, { signal: context.signal }));
const output = `${parent === "/" ? "" : parent}/${outputName.slice(outputName.lastIndexOf("/") + 1)}`;
checkPath(output, limits);
const existing = await scope.stat(output);
Expand Down Expand Up @@ -197,7 +198,7 @@ async function prepare(scope: ZipScope, parsed: ZipOptions, budget: Budget) {
}
for (const { entry } of selected.values()) { entries.push(entry); append(entry, false); }
const bytes = await writeZipArchive({ entries, comment: archive.comment }, limits, context.signal);
return { output, parentName, parent, existing, bytes, progress };
return { output, parentName, parent, parentStat, existing, bytes, progress };
}

export function createZipCommand(options: ArchiveCommandsOptions = {}): CommandDefinition {
Expand Down
Loading
Loading