Skip to content

feat(os): collect the operating system status - #1214

Open
karlitschek wants to merge 23 commits into
masterfrom
feat/os-status-collector
Open

karlitschek wants to merge 23 commits into
masterfrom
feat/os-status-collector

Conversation

@karlitschek

@karlitschek karlitschek commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Part 1 of 5, stacked on master:

  1. feat(os): collect the operating system status #1214 – collect the operating system status ← this PR
  2. feat(os): add an optional root helper #1215 – optional root helper
  3. feat(os): support dnf, zypper, pacman and apk #1216 – dnf, zypper, pacman and apk
  4. feat(os): report operating system concerns where admins look #1217 – setup checks, notifications, monitoring API, dashboard widget
  5. feat(os): add the Operating system page #1218 – the Operating system page

AI disclosure: this change was written with Claude Code (claude-opus-5-5) and reviewed by the author; every commit carries an Assisted-by trailer.

🤖 Generated with Claude Code

@AndyScherzinger AndyScherzinger left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this for correctness, security and performance, and deployed the branch to a nextcloud-docker-dev instance (NC 36 dev, PHP 8.5, Debian 13 container) to exercise it. Inline comments carry the details; this is the summary.

Blocking

  1. CI "NPM build" fails: the committed js/serverinfo-main.mjs.license and js/serverinfo-main.mjs.map.license list @vitejs/plugin-vue 6.0.10, while package-lock.json pins 6.0.7. They were generated from a node_modules out of sync with the lock file; npm ci && npm run build regenerates them correctly. The version bump itself is needed for the background job and command to register.
  2. Missing or stale package lists turn into "no security updates" (reproduced): with the image's purged /var/lib/apt/lists, apt-get -s dist-upgrade reported 0 pending updates; after apt-get update the same container reported 71 pending, 35 of them security. The updates concern only becomes unknown when updates === null, so the headline answer is a false all-clear wherever the lists are missing or old (see OsVerdict.php).
  3. occ serverinfo:os-status exit codes do not follow the monitoring-plugin convention it cites: "not checked yet" exits 1 (= WARNING), and an all-unknown verdict reports "level": "ok" and exits 0. In the Docker container 3 of 4 concerns were unknown and the command said OK. Monitoring plugins use 3 for UNKNOWN.

Correctness

  • The first-seen key includes the candidate version, so a superseding security update resets the pending-days counter.
  • CoreLibraryProbe treats an upgraded systemd binary as a reboot reason, though systemd re-executes itself on upgrade; on Debian, which has no /run/reboot-required by default, that is a recurring false positive.
  • In LXC without lxcfs, /proc/uptime is the host's, so a container created after the host booted reports a core-library reboot straight away.
  • Smaller: Thresholds::load() does not enforce warning ≤ error; nixos-unstable is not an os-release ID; Raspbian is mapped to Debian's support dates though it has its own archive; /dev/shm/ is shadowed by /dev/.
  • Worth stating in the UI later: Debian ships many fixes through point releases (libc6 2.41-12+deb13u4 from Debian:13.7/stable in my run), so "0 security updates" does not mean fully patched.

Security

  • CommandRunner is well done: an argument vector to proc_open() with no shell, validated binary names, a fixed PATH, LC_ALL=C, stdin from /dev/null, an output cap and a kill on timeout. All arguments are constants, so I found no injection surface, and restricted_mode is honoured.
  • The risk is exposure rather than execution. In this PR the data is only visible to admins through occ, but a list of unpatched packages with exact versions is an attacker's roadmap. Once #1217 exposes this through the token-authenticated monitoring API, consider returning counts there and keeping the package list admin-only.
  • Package names and origins are stored as parsed from tool output (\S+). Nothing here renders them, but #1218 should not pass them through v-html.
  • The collector reads /proc/1/cgroup, /sys/class/dmi/id/* and queries systemd over D-Bus. Everything degrades to unknown when denied, which is right; the README could list what is read so hardened setups know what to expect.

Performance

  • From system cron this is cheap. A full refresh took 0.75 s in the container; with 71 pending updates, apt-get -s dist-upgrade took 2.3 s and the remaining apt probes 0.4 s.
  • Under AJAX or webcron background-job modes the job runs inside a web request. The timeouts add up to roughly four minutes in the worst case (120 s for apt-get, 3 × 30 s, plus the 10 s and 5 s probes), all holding one PHP-FPM worker. Consider skipping the job unless backgroundjobs_mode is cron, or shortening the timeouts there.
  • The status is stored as a lazy app-config array capped at 500 updates, which fits the long-text column on every supported database.

Comment thread lib/OsStatus/Verdict/OsVerdict.php
Comment thread lib/Commands/ShowOsStatus.php
Comment thread lib/Commands/ShowOsStatus.php Outdated
Comment thread lib/OsStatus/Packages/PendingUpdate.php Outdated
Comment thread lib/OsStatus/Reboot/CoreLibraryProbe.php Outdated
Comment thread lib/Jobs/UpdateOsStatus.php Outdated
Comment thread lib/OsStatus/Packages/Apt.php Outdated
Comment thread build/update-os-support.php Outdated
Comment thread lib/OsStatus/Support/SupportTable.php Outdated
Comment thread lib/OsStatus/Restart/PhpProcessProbe.php Outdated
@karlitschek
karlitschek force-pushed the feat/os-status-collector branch from b2085c1 to 2e8c497 Compare October 22, 2026 21:32
@karlitschek

Copy link
Copy Markdown
Member Author

@AndyScherzinger Thanks for the review. All should be fixed now

@karlitschek
karlitschek marked this pull request as ready for review October 23, 2026 04:41
@karlitschek
karlitschek force-pushed the feat/os-status-collector branch from 2e8c497 to eb88b3f Compare October 23, 2026 04:41
@karlitschek
karlitschek force-pushed the feat/os-status-collector branch from eb88b3f to 60f5960 Compare October 23, 2026 08:49

@AndyScherzinger AndyScherzinger left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review of 60f59603. I deployed the whole stack (up to #1218) to the same nextcloud-docker-dev instance as before (NC 36 dev, PHP 8.5, Debian 13 container, background jobs from system cron) and re-ran the checks.

Verified fixed

  • Missing package lists now make the pending updates unknown instead of an all-clear.
  • occ serverinfo:os-status exits 3 when nothing has been collected yet (live: exit=3), and an unknown verdict also maps to 3.
  • First-seen times are tracked per package, so a superseding update keeps counting.
  • systemd is no longer a core-library reboot reason. A changed file now also needs a package change since boot, and system containers skip the probe.
  • Error thresholds are clamped to their warning thresholds.
  • Collection runs only from system cron, and a new freshness concern warns when the status gets old or cron is not system cron.
  • The status moved to app data. Live: status.json is about 1.2 KB, and the legacy cached_os_status app config key is gone.
  • The license files match the lock file again, and CI is green.

Remaining

One finding is left, inline: in an application container the command still reports ok and exits 0, although the container's own packages have pending security updates.

$known = true;
}

if (Concern::rank($level) <= Concern::rank(Concern::LEVEL_INFO) && $this->get(Concern::UPDATES)?->level === Concern::LEVEL_UNKNOWN) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only turns the level unknown when an UPDATES concern exists and is unknown. In an application container OsVerdict never creates that concern (managesPackages() is false), so the result is ok and exit 0.

Live in the Docker container: occ serverinfo:os-status --refresh gives "level": "ok", "updates": "unknown" and exit 0. Earlier, apt-get -s dist-upgrade in that same container listed 71 pending updates, 35 of them from Debian-Security.

It is a fair design choice that the image's packages are not the admin's to patch in place. But ok/0 is still a claim about them, and monitoring will read it as healthy. I suggest one of:

  • Emit the UPDATES concern as unknown (or a dedicated not-assessed state) in application containers, so that level() and the exit code say unknown.
  • State explicitly in the output and the README that ok covers host-side concerns only.

The same root cause makes the #1217 dashboard widget say "The system is up to date" in this container.

@kesselb

kesselb commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator

Turning off the update part

  • A switch for the OS-update part only. Where patching is handled elsewhere (Ansible, Landscape, Ubuntu Pro) or does not apply (application containers, images rebuilt by CI/CD), the rest of serverinfo could stay available with this part off. One bool key in ConfigLexicon would do; when it is off, the verdict could say "disabled" rather than "unknown", so occ serverinfo:os-status does not exit 3.
  • It might default to off when Docker or Kubernetes is detected, which would also cover @AndyScherzinger's remaining finding about application containers reporting ok.

Multiple application servers

Setups with two or more application servers are common. The status, the cache key and the first-seen times are currently shared, so the status shows whichever node last ran the job, and the thresholds reset when nodes disagree.

  • Keying the data per node with IServerInfo::getServerId(). It already falls back to a hash of the hostname; admins can set serverid per node to rule out collisions in the 0–511 range.

  • An option to turn off only the background job, so admins can run occ serverinfo:os-status --refresh from each node's own cron. The TimedJob runs on one node per interval only.

  • A table rather than a cache keyed by node: id | serverid | collected_at | last_seen | status | first_seen | php_sample | php_outdated_since

    • collected_at: when this node's status was last collected
    • last_seen: when this node last wrote anything, to spot nodes that stopped reporting and clean up removed ones
    • status: today os-status/status.json plus the cache key status
    • first_seen, php_sample, php_outdated_since: today the cached_os_* app config keys

    A cache can be evicted (and first_seen must survive that), cannot list its nodes, and with only memcache.local configured the cron write never reaches web requests. One row per node also removes the race on first_seen, and one indexed read makes the cache in front of app data unnecessary.

  • Nodes that only serve web requests would get a row with just php_sample and last_seen, shown as "not collected".

It would be good to settle this before #1217 and #1218 land, since it changes the storage format.

Nitpicks

  • 21 of the 22 commits fix the first one, and some undo earlier changes (storage, license files, README), so their messages describe fixes to code that is not on master yet. Could they be squashed into one commit that describes the final design?
  • The PHP-process sampler has no caller here yet. Could it move to feat(os): add the Operating system page #1218, where it is used?

karlitschek and others added 15 commits October 23, 2026 19:40
Collect what keeps the Linux distribution under Nextcloud secure, as far as
the web server user can see it: pending updates with security fixes told
apart (apt), reboots that installed updates wait for (the running kernel
against /lib/modules, core libraries replaced since boot, the distribution's
own flag), libraries replaced under the PHP serving Nextcloud, how long the
release still receives security updates (bundled from endoflife.date by
build/update-os-support.php, nothing fetched at runtime), automatic updates,
held packages, failed system services, time synchronisation and mandatory
access control. Containers, LXC guests and snaps are told apart, so nothing
is claimed that cannot be known from inside.

Commands run through proc_open() with an argument vector, a fixed search
path, LC_ALL=C, a time limit and an output cap. A background job collects
the status hourly. One verdict with configurable thresholds judges it, and
occ serverinfo:os-status shows it, exiting 0/1/2 for monitoring checks.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
Without package lists apt-get -s finds no newer versions, which read as
"no security updates". When the lists cannot be found and nothing is
pending, the updates are now unknown, with a diagnostic. Updates found in
lists past the warning age carry a listsStale fact, and occ reports the
security update count as unknown when the updates are.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
occ serverinfo:os-status follows the monitoring-plugin exit codes, where
3 is UNKNOWN. A status that was never collected, could not be collected
or has no verdict now exits 3 instead of 1 or 0. The overall level is
unknown when nothing is wrong but the pending updates are unknown, so an
unknown update state no longer reads as an all-clear.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
A security update superseded by a newer one before it was installed
started a new first-seen entry, so the pending days, and with them the
thresholds, started over. Hosts with frequently updated packages could
stay below the error threshold indefinitely.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
systemd's package re-executes PID 1 on upgrade, so a newer systemd binary
needs no reboot; on Debian, which has no /run/reboot-required by default,
every systemd update became a reboot warning. In a system container
without lxcfs the uptime is the host's, so a container created after the
host booted reported a reboot straight away; the probe now skips system
containers.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
With an error threshold below its warning threshold, a finding went
straight from info to error and the warning stage disappeared. Loading
the thresholds now raises each error value to its warning value.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
The commands can take minutes in the worst case. Under AJAX or webcron
the background job runs inside a web request and would hold a web server
worker that long, so it now does nothing unless background jobs run from
system cron; occ serverinfo:os-status --refresh still works everywhere.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
Raspbian is a separate archive with its own security handling, so
Debian's end dates do not necessarily apply; its support is unknown now.
NixOS has ID=nixos in every channel, so "nixos-unstable" never matched
and is gone from the rolling releases. /dev/shm/ was already covered by
/dev/ among the ignored PHP mappings.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
…uirement

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
The license files listed @vitejs/plugin-vue 6.0.10 and the lock file had
follow-redirects downgraded to 1.16.0, both from a node_modules out of
sync with the lock file. The lock file is master's again with only the
version bump, and npm ci && npm run build regenerated the license files.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
When stream_select() failed, for example because a signal interrupted
it, the runner left its read loop as if the command had closed its
output. The process was not killed, the result was neither timed out
nor truncated, and a cut-off apt-get simulation counted as a complete
list of pending updates.

A failed wait is now retried, and after three failures in a row the
process is killed and the result is marked as unreadable, which
succeeded() treats as a failure like a timeout or truncation.

CommandResult::failure() names what went wrong: the time limit, the
output cap, the failed read, or else the first line the command
printed. apt's diagnostic uses it, so a timeout no longer shows
whatever line apt had printed before it was killed.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
…aries

Under open_basedir the system directories cannot be inspected, so
hasBinary('apt-get') was false and apt was never found, although
processes can still be started from those directories. Each later
lookup of a missing binary also tried every search directory again, up
to six forks per call.

IHost::binaryState() answers true, false, or null when only running the
binary can tell. hasBinary() keeps meaning "known to be there". apt is
available unless it is known to be missing, so the factory picks it and
running it gives the answer.

CommandRunner remembers per binary the path it ran from, or that it is
missing once every guess failed to start, so a missing binary costs its
forks once per process.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
…tial

apt-get update writes into lists/partial on every run, also when it
fails. With an expired key on the security mirror every refresh failed,
yet the lists counted as refreshed today and no security updates were
pending.

The age now comes from update-success-stamp, which a hook touches only
after a successful update. Where that hook is not installed, as on
Debian 12 and a plain Ubuntu 26.04 server, the newest InRelease (or
Release) file is used: apt moves it into place only after verifying it,
with the mirror's modification time, so the lists were refreshed then
at the earliest.

The new fixture is the www-data view of /var/lib/apt/lists on an
up-to-date Debian 12 container, from the same capture run as the other
Debian 12 fixtures.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
KernelProbe reported "kernel removed" whenever the running kernel had no
directory in /lib/modules. A VPS that boots the provider's kernel, or a
netbooted machine, never has one, so it got a reboot warning that a
reboot could not clear and that became an error after 30 days.

A missing running kernel is now a reason only when an installed kernel
has the same flavour, which is the Arch case of modules deleted on
upgrade, or when the newest installed kernel was installed after the
boot. A distribution kernel upgraded on such a provider host then asks
for a reboot until the next boot, and is quiet after it.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
…boot

CoreLibraryProbe took a libc or dbus inode change time after boot as an
upgrade. A chmod or chown of the file, or a restore from a backup, also
sets the change time, and gave a reboot reason that only a reboot
cleared. The modification time is no alternative: dpkg keeps the build
time there.

A changed file now also needs its package to have changed after boot.
dpkg rewrites /var/lib/dpkg/info/<package>.list on every install of a
package, so its mtime says when libc6 or the dbus package was last
installed. Without such a list, the package manager's database file
must have changed after boot. Only where neither exists does the change
time decide alone.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
karlitschek and others added 8 commits October 23, 2026 19:40
savePhpSample() in web requests and firstSeen() in the background job
both read, changed and wrote cached_os_first_seen without a lock. When
they overlapped, one write dropped the other's entries, the "days
pending" count of updates and reboots started over, and warnings came
later than the thresholds say.

When web requests first saw outdated PHP libraries is now kept in
cached_os_php_outdated_since, so cached_os_first_seen has a single
writer, the collector.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
With up to 500 listed updates the stored status comes to 100-200 KB of
JSON. As a lazy app config value it was loaded and cached together with
every app's lazy values, so any request that read a lazy key of any app
carried it.

The status is now os-status/status.json in the app's data.
OsStatusStore keeps its method names. Reads come from admin pages and
setup checks, and app data can be on object storage, so a distributed
cache entry sits in front of the file: a save replaces it, and a read
that finds no file remembers that too. Within a request the status is
read once. A save removes the old cached_os_status app config value,
which has left the lexicon. The small values stay in app config.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
…ding the run

Nothing compared the time of collection with the job interval. When
cron died, or the admin switched to AJAX or webcron, the job stopped
collecting without a word, and the last status stayed green for weeks:
on the page, in the setup checks and in occ serverinfo:os-status, whose
exit code 0 kept monitoring quiet. In the collector only the pending
updates were guarded, so an exception from any other probe dropped the
whole run.

The verdict has a freshness concern. It is a warning when the status is
older than three job intervals, and never sooner than after six hours,
or when background jobs do not run from system cron. It is unknown when
nothing was collected. Its facts are collectedAt, ageSeconds and
systemCron. A warning is a warning in the overall level as well, so an
old status with unknown updates is no longer just "unknown".
OsStatusService reads the cron mode and the interval the way the job
does, so the verdict stays a function of its inputs. The occ command
shows the age of the status, and its exit code follows the verdict.

Every probe of the collector now runs on its own: one that throws is
logged, costs its own finding and leaves a diagnostic.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
SupportTable took an optional ?string $supportDataFile in its
constructor for tests. The container autowires constructor parameters
by name, so a registered service or parameter called supportDataFile
would have replaced the bundled table without anyone noticing.

The constructor has no parameters now. Tests use
SupportTable::fromFile() for a different table.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
The README promised that the PHP serving Nextcloud is checked for
libraries an update replaced, and named RHEL ELS among the paid support
extensions. Here nothing samples the PHP process yet, so that check
always reads as unknown, and only Ubuntu Pro and Debian ELTS are
detected. Both claims and the mention of reading the process maps are
gone; the later parts of the stack add them back with the code.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
The name, versions and archives of a pending update were taken from the
"Inst" lines of `apt-get -s` as they came and kept in the stored status.
Anything a repository puts there, such as markup or very long strings,
ended up in the status the page, the API and the notifications show.

A line is now only kept when its package name matches the pattern the
held packages and the reboot list already use, now one shared constant,
when both versions are at most 128 characters of what Debian versions
are made of (letters, digits and ".+~:_-"), and when every archive is at
most 128 characters of letters, digits, spaces and ".+~:_/-". Spaces are
allowed there because real archive labels carry them, e.g. "Debian
Backports". Lines that fail are left out.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
…rchives

An apt update counted as a security update whenever one of its archives
was a suite ending in "-security". Any third-party repository could get
its updates counted that way just by naming its suite like that, and so
change what the status reports and how urgent it looks.

The archive's label, the part before the colon in "Debian-Security:12/
stable-security", now also has to name the distribution: the os-release
ID or an ID_LIKE entry, optionally followed by "-Security" or by
Ubuntu Pro's "ESM" and "ESMApps". Derivatives such as Linux Mint keep
their Ubuntu security updates, and Devuan its own. Apt gets the release
from the factory through withRelease(); without one it does not claim
to tell security updates apart.

Labels are set by each repository, so this stops repositories that merely
call a suite "*-security", not one that copies the distribution's label.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
New occ commands use the OCP console attributes instead of extending
the private OC\Core\Command\Base. The command is now a plain class
with #[AsCommand] and an #[Option] for --refresh; --output comes from
supportsOutputFormat, so the options and the output stay the same.

It still returns the monitoring plugin exit codes 0 to 3 as an int:
ExitCode has no case for 3, "unknown", which Nagios-style checks need.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Frank Karlitschek <[email protected]>
@karlitschek
karlitschek force-pushed the feat/os-status-collector branch from 60f5960 to 934a33d Compare October 23, 2026 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI assisted This PR contains AI-assisted commits

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants