Skip to content

pkg_rpm: batch the commands of the %install scriptlet - #1101

Open
chouquette wants to merge 1 commit into
bazelbuild:mainfrom
chouquette:chouquette/rpm-install-script
Open

chouquette wants to merge 1 commit into
bazelbuild:mainfrom
chouquette:chouquette/rpm-install-script

Conversation

@chouquette

Copy link
Copy Markdown
Contributor

The %install scriptlet runs install -d "$(dirname ...)", cp and
chmod +w for every regular file of the package. On a package with about
16k files that is close to 50k processes, and the scriptlet takes over a
minute.

Create each destination directory once, copy the files that keep their name
to their destination directory together, and batch the chmod +w calls.
Each command takes at most 100 paths so command lines stay well below
ARG_MAX. Files from TreeArtifacts get the same treatment. On the package
above, the scriptlet now runs in a few seconds.

Relates to #209.

gh-worker-dd-mergequeue-cf854d Bot pushed a commit to DataDog/datadog-agent that referenced this pull request Nov 18, 2026
### What does this PR do?

Speeds up the `MakeRpm` step of the Bazel rpms: the agent rpm goes from about 12 to under 5 minutes locally.

- Bump rules_pkg to pick up the new `autoreqprov` attribute (bazelbuild/rules_pkg#1100) and the batched `%install` commands (bazelbuild/rules_pkg#1101).
- Use `autoreqprov = False` (bazelbuild/rules_pkg#1100) to set `AutoReqProv: no`, like omnibus does, instead of filtering the generated dependencies afterwards.
- Skip the steps rpmbuild runs after `%install` (`brp-strip`, `brp-compress`, ...), like omnibus does.

| Before | After |
| --- | --- |
| rpmbuild scans every packaged file for dependencies, then `__requires_exclude`/`__provides_exclude` discard them all | No scan; same Requires and Provides |
| `%install` runs `install`, `cp` and `chmod` for each file | Commands are batched per directory |
| dogstatsd, ddot, installer and iot rpms go through `brp-*` steps | Skipped, as for the agent rpm (whose `__strip` override is removed) and in omnibus |

Unchanged: omnibus packages, and the agent rpm's metadata and file list.

### Motivation

`MakeRpm` took 736s of a local agent rpm build, mostly spent generating dependencies we then threw away.

### Describe how you validated your changes

Built the agent rpm locally: `MakeRpm` went from 736s to 285s, with identical Requires, Provides, Conflicts and file list.


Co-authored-by: hugo.beauzee <[email protected]>
@chouquette

Copy link
Copy Markdown
Contributor Author

Woops, I forgot to mention that I'll handle the interactions with the reviewers with my meaty little fingers

@chouquette
chouquette marked this pull request as ready for review November 18, 2026 12:50

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant