Repository navigation
pkg_rpm: batch the commands of the %install scriptlet - #1101
Open
chouquette wants to merge 1 commit into
Open
chouquette wants to merge 1 commit into
chouquette wants to merge 1 commit into
Conversation
gh-worker-dd-mergequeue-cf854d Bot
pushed a commit
to DataDog/datadog-agent
that referenced
this pull request
Nov 18, 2026
### What does this PR do? Speeds up the `MakeRpm` step of the Bazel rpms: the agent rpm goes from about 12 to under 5 minutes locally. - Bump rules_pkg to pick up the new `autoreqprov` attribute (bazelbuild/rules_pkg#1100) and the batched `%install` commands (bazelbuild/rules_pkg#1101). - Use `autoreqprov = False` (bazelbuild/rules_pkg#1100) to set `AutoReqProv: no`, like omnibus does, instead of filtering the generated dependencies afterwards. - Skip the steps rpmbuild runs after `%install` (`brp-strip`, `brp-compress`, ...), like omnibus does. | Before | After | | --- | --- | | rpmbuild scans every packaged file for dependencies, then `__requires_exclude`/`__provides_exclude` discard them all | No scan; same Requires and Provides | | `%install` runs `install`, `cp` and `chmod` for each file | Commands are batched per directory | | dogstatsd, ddot, installer and iot rpms go through `brp-*` steps | Skipped, as for the agent rpm (whose `__strip` override is removed) and in omnibus | Unchanged: omnibus packages, and the agent rpm's metadata and file list. ### Motivation `MakeRpm` took 736s of a local agent rpm build, mostly spent generating dependencies we then threw away. ### Describe how you validated your changes Built the agent rpm locally: `MakeRpm` went from 736s to 285s, with identical Requires, Provides, Conflicts and file list. Co-authored-by: hugo.beauzee <[email protected]>
Contributor
Author
|
Woops, I forgot to mention that I'll handle the interactions with the reviewers with my meaty little fingers |
chouquette
marked this pull request as ready for review
November 18, 2026 12:50
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The %install scriptlet runs
install -d "$(dirname ...)",cpandchmod +wfor every regular file of the package. On a package with about16k files that is close to 50k processes, and the scriptlet takes over a
minute.
Create each destination directory once, copy the files that keep their name
to their destination directory together, and batch the
chmod +wcalls.Each command takes at most 100 paths so command lines stay well below
ARG_MAX. Files from TreeArtifacts get the same treatment. On the package
above, the scriptlet now runs in a few seconds.
Relates to #209.