Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
341786a
test(projects): pin the chat projects store and managed folders (red)
OAraLabs Nov 14, 2026
87f1614
test(projects): label-only filing (red)
OAraLabs Nov 14, 2026
4c7dcfa
feat(projects): the chat projects store and managed folders
OAraLabs Nov 14, 2026
d5a482b
test(projects): bindings name the project that set them (red)
OAraLabs Nov 14, 2026
571aefe
fix(projects): a binding names the project that set it; only that pro…
OAraLabs Nov 14, 2026
a5dde48
test(projects): store review cases (red)
OAraLabs Nov 14, 2026
d179072
fix(projects): savepoint writes on the shared connection; a hand-set …
OAraLabs Nov 14, 2026
ce01f52
fix(projects): project names are scrubbed; the uniqueness key is a hash
OAraLabs Nov 14, 2026
43a69f1
test(projects): pin the chat projects routes on the real app (red)
OAraLabs Nov 14, 2026
180d86e
feat(projects): chat projects routes, filing, session rows, the proje…
OAraLabs Nov 14, 2026
1e094a6
test(projects): routes review cases (red)
OAraLabs Nov 14, 2026
a5ede1b
fix(projects): folder checks by identity, root checked before it is m…
OAraLabs Nov 14, 2026
d19fbe2
fix(projects): unfiled counts what clients see; root ancestors resolv…
OAraLabs Nov 14, 2026
c5b394e
Merge remote-tracking branch 'origin/main' into feat/chat-projects-store
OAraLabs Nov 14, 2026
a1f279c
Merge remote-tracking branch 'origin/main' into feat/chat-projects-store
OAraLabs Nov 14, 2026
e5bbe24
test(parity): rebaseline expected files for the chat-projects tables
OAraLabs Nov 14, 2026
673f58c
Merge remote-tracking branch 'origin/main' into feat/chat-projects-store
OAraLabs Nov 14, 2026
7fd80a0
fix(projects): document projects.root in the template and read it wit…
OAraLabs Nov 14, 2026
137994a
docs(reference): regenerate after merging main
OAraLabs Nov 14, 2026
ef6fb96
Merge remote-tracking branch 'origin/main' into feat/chat-projects-store
OAraLabs Nov 14, 2026
95705e7
Merge remote-tracking branch 'origin/feat/chat-projects-store' into f…
OAraLabs Nov 15, 2026
119ed0a
Merge remote-tracking branch 'origin/main' into feat/chat-projects-ro…
OAraLabs Nov 15, 2026
b50444c
Merge remote-tracking branch 'origin/main' into feat/chat-projects-ro…
OAraLabs Nov 15, 2026
36c6eb4
Merge remote-tracking branch 'origin/main' into feat/chat-projects-ro…
OAraLabs Nov 15, 2026
52dbf8c
Merge remote-tracking branch 'origin/main' into feat/chat-projects-ro…
OAraLabs Nov 15, 2026
ac2cd11
Merge remote-tracking branch 'origin/main' into feat/chat-projects-ro…
OAraLabs Nov 15, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Notable changes in each release, newest first. Releases before 0.9.7 are describ

## Unreleased

- Chat projects: projects you file conversations in now live on the daemon, so every device sees the same ones. A project is a named folder (made under `projects.root`, default `~/prometheus-projects`, or one you already have); a conversation filed in it works in that folder. New routes: `GET|POST /api/chat-projects`, `GET|PUT|DELETE /api/chat-projects/{id}` and `PUT /api/sessions/{id}/project`; session rows carry `chat_project_id`, and `POST /api/sessions` can start a conversation in a project. The owner does everything; a chat-only device sees only the projects its own conversations are in, as a name, and cannot create, rename, delete or file. Filing never replaces a folder you chose by hand unless asked, and `keep_workspace` files as a label only. Deleting a project never deletes a file. The `subscribed` ack lists `projects` in `server_capabilities`, and owner sockets get a `chat_project_changed` frame on every change.
- Speech: an app can now have a reply read aloud by the voice box behind Prometheus, phrase by phrase, so the phone starts speaking before the reply has finished. New routes: `GET /api/speech` (what works now), `POST /api/speech/transcribe` (audio in, text out), `GET /api/sessions/{session_id}/speech` (the session's reply as an NDJSON stream of spoken phrases) and `POST /api/speech/speak` (any text, the same way). The owner and chat-only devices may use them with chat's limits: a device hears only its own sessions, a revoked token's streams end at once, and streams are capped per caller. The `subscribed` ack lists `speech` in `server_capabilities` when a voice box is set up; an app that does not see it keeps its own voice. When the box is down the app is told why and where the unspoken text starts, and reads the rest with its own voice; the daemon's own voice engines are not used for this. Nothing new to configure. The bridge hands every frame to registered observers (one small hook in `ws_server.py`).
- Speech (groundwork): `prometheus.speech.replies` holds each session's reply from the frames the bridge already broadcasts (bound by its `message_id`, ended by its own `chat_done`, an `error`, or the session's last open turn closing), and `prometheus.speech.stream` turns a reply or any text into the speech stream's lines: phrases with their WAV as soon as the voice box makes them, a dropped round, keepalives, and one final `done` with its reason (`unavailable` says where the unspoken text starts). Nothing uses them yet; the routes follow.
- Docs: `docs/design/chat-projects.md` is the design for chat projects on the daemon: one store for grouping conversations, a project's folder and its instructions; the routes, who may call them (a chat-only device sees only the projects its own sessions are in), the `projects` capability, suggestions, and how a client moves its local projects in, per instance. No behaviour changes.
Expand Down
5 changes: 4 additions & 1 deletion docs/reference/routes.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ app. For what each one is *for*, see the curated
[API guide](../guide/api.md) — this file is the complete list, that
one is the explanation.

## Daemon app — 137 paths
## Daemon app — 140 paths

| Path | Methods |
|---|---|
Expand All @@ -26,6 +26,8 @@ one is the explanation.
| `/api/backends/{name}/probe` | POST |
| `/api/benchmarks/run` | POST |
| `/api/chat` | POST |
| `/api/chat-projects` | GET, POST |
| `/api/chat-projects/{project_id}` | DELETE, GET, PUT |
| `/api/chat/interrupt` | POST |
| `/api/chat/send` | POST |
| `/api/code` | POST |
Expand Down Expand Up @@ -115,6 +117,7 @@ one is the explanation.
| `/api/sessions/{session_id}/model` | DELETE, GET, POST |
| `/api/sessions/{session_id}/pin` | PUT |
| `/api/sessions/{session_id}/profile` | GET, PUT |
| `/api/sessions/{session_id}/project` | PUT |
| `/api/sessions/{session_id}/purge` | POST |
| `/api/sessions/{session_id}/purge-preview` | GET |
| `/api/sessions/{session_id}/speech` | GET |
Expand Down
17 changes: 17 additions & 0 deletions src/prometheus/memory/lcm_conversation_store.py
Original file line number Diff line number Diff line change
Expand Up @@ -1175,6 +1175,23 @@ def purge_session(self, session_id: str) -> dict[str, int]:
self._conn.execute("PRAGMA wal_checkpoint(TRUNCATE)")
return counts

def has_session_rows(self, session_id: str) -> bool:
"""True when *session_id* has messages, or a workspace binding or chat-project filing made before its first
message: either way it exists, and no device may claim it (web/session_scope.session_exists)."""
if self.max_rowid(session_id) > 0:
return True
for table in ("session_workspaces", "session_chat_projects"):
if self._table_exists(table) and self._conn.execute(
f"SELECT 1 FROM {table} WHERE session_id = ?", (session_id,)).fetchone():
return True
return False

def tool_use_json(self, session_id: str) -> list[str]:
"""The ``content_json`` of each assistant row that holds a tool call, in order (chat-project suggestions)."""
return [r[0] for r in self._conn.execute(
"SELECT content_json FROM lcm_messages WHERE session_id = ? AND role = 'assistant'"
" AND content_json LIKE '%tool_use%' ORDER BY rowid", (session_id,)).fetchall()]

def chat_projects(self) -> ChatProjectStore:
"""The chat projects store, on this connection (``projects/store.py``): filing a session changes its
membership and its workspace binding in one transaction. Made once, then kept."""
Expand Down
71 changes: 71 additions & 0 deletions src/prometheus/projects/content.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
"""What is in a project's folder: its instructions file and its reference files (docs/design/chat-projects.md, 3.4-3.5).

Read without following links: ``PROMETHEUS.md`` counts only as a regular file at the folder's top level, and a
reference file only as a regular file directly in ``reference/`` whose name passes :func:`valid_reference_name`. A
symlinked ``reference/`` lists nothing. This module only reads; the write routes come with the instructions and files
routes.

Source: novel code for Prometheus, 2026-11-13.
"""

from __future__ import annotations

import codecs
import hashlib
import os
import stat
import unicodedata
from pathlib import Path

INSTRUCTIONS_FILE = "PROMETHEUS.md"
REFERENCE_DIR = "reference"
#: The instruction loader's per-file cap (context/prometheusmd.py), so what is saved is what the model sees.
MAX_INSTRUCTIONS_CHARS = 12_000
NAME_MAX = 120
_CHUNK = 256 * 1024


def instructions_info(folder: str | Path) -> tuple[int, str]:
"""``(characters, rev)`` of the folder's ``PROMETHEUS.md``; ``(0, "")`` when there is no regular file there.

``rev`` is the first 16 hex of the SHA-256 of the whole file's bytes, read in chunks.
"""
path = Path(folder) / INSTRUCTIONS_FILE
try:
if not stat.S_ISREG(os.lstat(path).st_mode):
return 0, ""
# O_NONBLOCK so a FIFO swapped in after the lstat cannot hang the read; fstat confirms what was opened.
fd = os.open(path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0))
with os.fdopen(fd, "rb") as fh:
if not stat.S_ISREG(os.fstat(fh.fileno()).st_mode):
return 0, ""
digest = hashlib.sha256()
decoder = codecs.getincrementaldecoder("utf-8")("replace")
chars = 0
while chunk := fh.read(_CHUNK):
digest.update(chunk)
chars += len(decoder.decode(chunk))
chars += len(decoder.decode(b"", final=True))
except OSError:
return 0, ""
return chars, digest.hexdigest()[:16]


def valid_reference_name(name: str) -> bool:
"""One path segment: no separator, no leading dot, no control or format character, at most 120 characters."""
if not name or len(name) > NAME_MAX or name.startswith(".") or "/" in name or "\\" in name:
return False
return not any(unicodedata.category(ch) in ("Cc", "Cf") for ch in name)


def reference_names(folder: str | Path) -> list[str]:
"""The reference files listed: regular files directly in ``reference/`` with valid names, sorted."""
ref = Path(folder) / REFERENCE_DIR
try:
if not stat.S_ISDIR(os.lstat(ref).st_mode):
return []
with os.scandir(ref) as entries:
return sorted(e.name for e in entries
if valid_reference_name(e.name) and e.is_file(follow_symlinks=False))
except OSError:
return []
6 changes: 4 additions & 2 deletions src/prometheus/projects/folders.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
import re
import stat
import unicodedata
from collections.abc import Mapping
from collections.abc import Callable, Mapping
from pathlib import Path
from typing import Any

Expand All @@ -42,7 +42,7 @@ def slug(name: str) -> str:
return out or "project"


def make_managed_folder(root: Path, name: str) -> Path:
def make_managed_folder(root: Path, name: str, *, skip: Callable[[Path], bool] | None = None) -> Path:
"""Make ``<root>/<slug>`` (or ``-2``, ``-3``, ...) with mode 0700, and the root too (0700) if it is missing.

An existing root must be a plain directory owned by this user (``ValueError`` otherwise: a symlink or someone
Expand All @@ -64,6 +64,8 @@ def make_managed_folder(root: Path, name: str) -> Path:
base = slug(name)
for n in range(1, _MAX_SUFFIX):
candidate = root / (base if n == 1 else f"{base}-{n}")
if skip is not None and skip(candidate):
continue # a stored project's folder, even one since removed on disk
try:
candidate.mkdir(mode=0o700)
except FileExistsError:
Expand Down
23 changes: 16 additions & 7 deletions src/prometheus/projects/store.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,11 @@
from dataclasses import dataclass, field

NAME_MAX = 60
# A membership row ``s`` whose session is not forgotten: no tombstone, or activity newer than it (the session list's
# own rule: a tombstoned session stays hidden until newer activity). The membership itself is kept, so a session
# that comes back is still filed.
_SHOWN = ("NOT EXISTS (SELECT 1 FROM session_tombstones t WHERE t.session_id = s.session_id AND t.deleted_at >= "
"COALESCE((SELECT MAX(m.timestamp) FROM lcm_messages m WHERE m.session_id = s.session_id), 0))")
#: A binding a project set is ``set_by = "project:<id>"``: the id says which, so only that project clears it.
SET_BY_PREFIX = "project:"

Expand Down Expand Up @@ -181,7 +186,7 @@ def rev(self) -> int:
def _rows(self, where: str = "", args: tuple = ()) -> list[Project]:
sql = (
"SELECT p.id, p.name, p.folder, p.folder_managed, p.import_key, p.created_at, p.updated_at,"
" (SELECT COUNT(*) FROM session_chat_projects s WHERE s.project_id = p.id)"
f" (SELECT COUNT(*) FROM session_chat_projects s WHERE s.project_id = p.id AND {_SHOWN})"
f" FROM chat_projects p {where} ORDER BY p.position, p.created_at"
)
return [Project(r[0], r[1], r[2], bool(r[3]), r[4], r[5], r[6], int(r[7]))
Expand Down Expand Up @@ -213,27 +218,29 @@ def project_of(self, session_id: str) -> str | None:
"SELECT project_id FROM session_chat_projects WHERE session_id = ?", (session_id,)).fetchone()
return row[0] if row else None

def projects_of(self, session_ids: Iterable[str]) -> dict[str, str]:
def projects_of(self, session_ids: Iterable[str], *, shown_only: bool = False) -> dict[str, str]:
"""``{session_id: project_id}`` for those filed; *shown_only* leaves out forgotten (tombstoned) sessions."""
ids = list(session_ids)
out: dict[str, str] = {}
extra = f" AND {_SHOWN}" if shown_only else ""
for i in range(0, len(ids), 500):
chunk = ids[i:i + 500]
marks = ",".join("?" * len(chunk))
out.update(self._conn.execute(
f"SELECT session_id, project_id FROM session_chat_projects WHERE session_id IN ({marks})",
f"SELECT session_id, project_id FROM session_chat_projects s WHERE session_id IN ({marks}){extra}",
chunk).fetchall())
return out

def session_ids(self, project_id: str) -> list[str]:
return [r[0] for r in self._conn.execute(
"SELECT session_id FROM session_chat_projects WHERE project_id = ? ORDER BY filed_at, session_id",
(project_id,)).fetchall()]
f"SELECT session_id FROM session_chat_projects s WHERE project_id = ? AND {_SHOWN}"
" ORDER BY filed_at, session_id", (project_id,)).fetchall()]

def visible_to(self, own_session_ids: set[str]) -> list[VisibleProject]:
"""The projects a chat-only device may see: those with at least one of *own_session_ids*, in display order."""
if not own_session_ids:
return []
filed = self.projects_of(own_session_ids)
filed = self.projects_of(own_session_ids, shown_only=True)
by_project: dict[str, list[str]] = {}
for sid, pid in filed.items():
by_project.setdefault(pid, []).append(sid)
Expand Down Expand Up @@ -300,8 +307,10 @@ def delete(self, project_id: str) -> int:
if self._conn.execute("SELECT 1 FROM chat_projects WHERE id = ?", (project_id,)).fetchone() is None:
raise UnknownProject("no such project")
with self._write():
# The count a client sees: forgotten sessions are not shown, so they are not counted as unfiled.
sessions = [r[0] for r in self._conn.execute(
"SELECT session_id FROM session_chat_projects WHERE project_id = ?", (project_id,)).fetchall()]
f"SELECT session_id FROM session_chat_projects s WHERE project_id = ? AND {_SHOWN}",
(project_id,)).fetchall()]
# Every binding this project set, on members and on sessions since moved elsewhere as a label.
self._conn.execute("DELETE FROM session_workspaces WHERE set_by = ?", (_set_by(project_id),))
self._conn.execute("DELETE FROM session_chat_projects WHERE project_id = ?", (project_id,))
Expand Down
Loading
Loading