Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions docs/mirror-pull-dry-run.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,15 +77,17 @@ d8 mirror pull --dry-run /tmp/bundle \
This resolves platform images, installer, security databases, and all modules.
No blobs are downloaded; the bundle directory stays empty.

### Since a minimum version (channel-based pull)
### Include releases older than Rock Solid

```bash
d8 mirror pull --dry-run /tmp/bundle \
--source registry.deckhouse.io/deckhouse/fe \
--license <your-token> \
--since-version 1.68.0
--since-version 1.67.0
```

Default discovery spans current Rock Solid through Alpha, inclusively. `--since-version` extends this range only when the requested version is older than Rock Solid. The lower bound is `MIN(rock-solid, since-version)`; equal or newer values leave the default versions and channels unchanged. Numeric tags retain the latest patch per major.minor together with channel snapshots in the range, so the exact patch named by `--since-version` is not pinned.

### Module whitelist

```bash
Expand Down Expand Up @@ -119,8 +121,7 @@ access, no credentials needed. The stub registry is activated via:
STUB_REGISTRY_CLIENT=true
```

The stub seeds versions `v1.68.0`–`v1.72.10`, channels `alpha`/`beta`/`early-access`/
`stable`/`rock-solid`, and several module tags.
The stub seeds versions `v1.66.0` through `v1.72.10` and channels `alpha`/`beta`/`early-access`/`stable`/`rock-solid`. Rock Solid points to `v1.68.0`, so the default selection excludes `v1.66.0` and `v1.67.0`. With `--since-version 1.67.0`, the plan adds `v1.67.0` but still excludes `v1.66.0`.

#### Run all dry-run unit tests

Expand Down
2 changes: 1 addition & 1 deletion internal/mirror/PROXY-REGISTRY.md
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,7 @@ The reason this works: every operation other than the three `ListTags` calls is
| `--no-modules` is set | `--include-module` is **not** required |
| `--exclude-module` | Honoured (subtracts from the include list) |
| `--deckhouse-tag` | **Conflict**: a single pinned tag is already a direct check; do not combine with `--proxy-registry` |
| `--since-version` | **Conflict**: `--since-version` has no upper bound and the probe cannot terminate. Use `--include-platform` with an explicit range instead |
| `--since-version` | **Conflict**: requires channel-based discovery with registry tag listings. Use `--include-platform` with an explicit lower bound instead |
| `--dry-run` | Honoured — runs the probe and prints the plan without downloading any blobs |

---
Expand Down
8 changes: 4 additions & 4 deletions internal/mirror/README.MD
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ d8 mirror pull <images-bundle-path> [flags]

| Flag | Description |
|------|-------------|
| `--since-version` | Minimal Deckhouse release to pull (lower bound, inclusive). Ignored if above current Rock Solid release. Conflicts with `--deckhouse-tag` and `--include-platform` |
| `--since-version` | Extend discovery to older Deckhouse releases (inclusive). The lower bound is `MIN(rock-solid, since-version)`; equal or newer values keep the default range. Conflicts with `--deckhouse-tag` and `--include-platform` |
| `--include-platform` | Select platform releases by semver constraint (e.g. `">=1.64 <=1.68"`). Uses the same constraint dialect as `--include-module`. Conflicts with `--deckhouse-tag` and `--since-version`. See [Platform Version Filtering](#platform-version-filtering) |
| `--deckhouse-tag` | Specific Deckhouse build tag to pull. Conflicts with `--since-version` and `--include-platform`. If the registry contains a release channel image for the specified tag, all release channels in the bundle will point to it |

Expand Down Expand Up @@ -106,7 +106,7 @@ The `--include-platform` flag accepts a semver constraint expression to pull onl

| Goal | Recommended flag |
|------|-----------------|
| Pull all releases from a given minimum upward | `--since-version 1.64.0` |
| Extend the default range to older releases | `--since-version 1.64.0` |
| Pull a bounded range (e.g. for an incremental update) | `--include-platform ">=1.64 <=1.68"` |
| Pull every release of a single minor | `--include-platform "~1.65.0"` |
| Pin a single exact release (same as `--deckhouse-tag`) | `--include-platform "=v1.65.3"` |
Expand Down Expand Up @@ -494,7 +494,7 @@ d8 mirror pull /tmp/bundle --license $LICENSE_TOKEN

### Version Management

1. **Use `--since-version`:** To pull all releases from a given minimum upward, reducing bundle size relative to a full pull
1. **Use `--since-version`:** To add releases older than the current Rock Solid release. Equal or newer values leave the default range unchanged
2. **Use `--include-platform`:** To pull a bounded window of releases (e.g. `">=1.64 <=1.68"`) — useful for staged incremental updates where you want to move up a few minors at a time without pulling the entire channel history
3. **Specific Versions:** Use `--deckhouse-tag` (or `--include-platform "=vX.Y.Z"`) for fully controlled, single-release deployments
4. **Module Versioning:** Leverage semver constraints in `--include-module` for flexible module version management; `--include-platform` speaks the same dialect
Expand Down Expand Up @@ -523,7 +523,7 @@ The mirror system handles three primary component types:

By default, the platform service discovers releases between the current `rock-solid` channel version (lower bound) and the current `alpha` channel version (upper bound), keeping the highest patch per `(major, minor)`.

`--since-version` raises the lower bound above `rock-solid` when the user wants to skip older minors.
`--since-version` extends the range to older releases using the inclusive lower bound `MIN(rock-solid, since-version)`. Values equal to or above Rock Solid leave the default versions and channels unchanged. Numeric tags keep the latest patch per major.minor together with channel snapshots in the range, so the exact patch named by `--since-version` is not pinned. The upper bound remains the current Alpha release.

`--include-platform` replaces this window with a user-supplied semver constraint, applying the same latest-patch-per-minor and inclusive-anchor rules as module version filtering. Channel snapshots outside the constraint are pruned so the bundle stays internally consistent.

Expand Down
1 change: 1 addition & 0 deletions internal/mirror/cmd/pull/pull.go
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,7 @@ func (p *Puller) buildPullService() (*mirror.PullService, error) {
pullflags.TempDir,
pullflags.DeckhouseTag,
&mirror.PullServiceOptions{
SinceVersion: p.params.SinceVersion,
SkipPlatform: pullflags.NoPlatform,
SkipSecurity: pullflags.NoSecurityDB,
SkipModules: pullflags.NoModules,
Expand Down
22 changes: 22 additions & 0 deletions internal/mirror/cmd/pull/pull_dryrun_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,12 @@ import (
"path/filepath"
"testing"

"github.com/Masterminds/semver/v3"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"

pullflags "github.com/deckhouse/deckhouse-cli/internal/mirror/cmd/pull/flags"
"github.com/deckhouse/deckhouse-cli/internal/mirror/modules"
)

// saveFlagsAndRestore captures the current state of all relevant pull flags and
Expand All @@ -34,6 +36,13 @@ func saveFlagsAndRestore(t *testing.T) func() {
t.Helper()

saved := struct {
NoPackages bool
SinceVersionString string
SinceVersion *semver.Version
PlatformConstraintString string
PlatformConstraint modules.VersionConstraint
ProxyRegistry bool

TempDir string
ImagesBundlePath string
SourceRegistryRepo string
Expand All @@ -56,6 +65,13 @@ func saveFlagsAndRestore(t *testing.T) func() {
IgnoreSuspend bool
ImagesBundleChunkSizeGB int64
}{
NoPackages: pullflags.NoPackages,
SinceVersionString: pullflags.SinceVersionString,
SinceVersion: pullflags.SinceVersion,
PlatformConstraintString: pullflags.PlatformConstraintString,
PlatformConstraint: pullflags.PlatformConstraint,
ProxyRegistry: pullflags.ProxyRegistry,

TempDir: pullflags.TempDir,
ImagesBundlePath: pullflags.ImagesBundlePath,
SourceRegistryRepo: pullflags.SourceRegistryRepo,
Expand All @@ -80,6 +96,12 @@ func saveFlagsAndRestore(t *testing.T) func() {
}

return func() {
pullflags.NoPackages = saved.NoPackages
pullflags.SinceVersionString = saved.SinceVersionString
pullflags.SinceVersion = saved.SinceVersion
pullflags.PlatformConstraintString = saved.PlatformConstraintString
pullflags.PlatformConstraint = saved.PlatformConstraint
pullflags.ProxyRegistry = saved.ProxyRegistry
pullflags.TempDir = saved.TempDir
pullflags.ImagesBundlePath = saved.ImagesBundlePath
pullflags.SourceRegistryRepo = saved.SourceRegistryRepo
Expand Down
155 changes: 155 additions & 0 deletions internal/mirror/cmd/pull/pull_since_version_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
/*
Copyright 2026 Flant JSC

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package pull

import (
"context"
"os"
"path/filepath"
"strings"
"testing"

"github.com/fatih/color"
"github.com/stretchr/testify/require"

pullflags "github.com/deckhouse/deckhouse-cli/internal/mirror/cmd/pull/flags"
)

func TestPullCommandSinceVersion(t *testing.T) {
t.Setenv("STUB_REGISTRY_CLIENT", "true")
for _, key := range []string{
"D8_MIRROR_LICENSE_TOKEN", "D8_MIRROR_SOURCE_LOGIN", "D8_MIRROR_SOURCE_PASSWORD",
"D8_MIRROR_TIMEOUT", "MIRROR_DEBUG_LOG",
} {
t.Setenv(key, "")
}
noColor := color.NoColor
color.NoColor = true
t.Cleanup(func() { color.NoColor = noColor })

const baseline = "v1.72.10, v1.71.0, v1.70.0, v1.69.0, v1.68.0 (5 channels)"
const withOlderRelease = "v1.72.10, v1.71.0, v1.70.0, v1.69.0, v1.68.0, v1.67.0 (5 channels)"

tests := []struct {
name string
args []string
wantPlatform string
wantError string
}{
{
name: "without since excludes releases below Rock Solid",
wantPlatform: baseline,
},
{
name: "older since includes requested release but not earlier releases",
args: []string{"--since-version", "1.67.0"},
wantPlatform: withOlderRelease,
},
{
name: "since equal to Rock Solid preserves versions and channels",
args: []string{"--since-version", "1.68.0"},
wantPlatform: baseline,
},
{
name: "since newer than Rock Solid preserves versions and channels",
args: []string{"--since-version", "1.70.0"},
wantPlatform: baseline,
},
{
name: "since equal to Alpha preserves versions and channels",
args: []string{"--since-version", "1.72.10"},
wantPlatform: baseline,
},
{
name: "include platform selects older release",
args: []string{"--include-platform", ">=1.67.0"},
wantPlatform: withOlderRelease,
},
{
name: "deckhouse tag selects only the requested release",
args: []string{"--deckhouse-tag", "v1.67.0"},
wantPlatform: "v1.67.0",
},
{
name: "since conflicts with deckhouse tag",
args: []string{"--since-version", "1.67.0", "--deckhouse-tag", "v1.67.0"},
wantError: "Using both --deckhouse-tag and --since-version",
},
{
name: "since conflicts with include platform",
args: []string{"--since-version", "1.67.0", "--include-platform", ">=1.67.0"},
wantError: "Using both --since-version and --include-platform",
},
{
name: "since conflicts with proxy registry",
args: []string{"--since-version", "1.67.0", "--proxy-registry"},
wantError: "--proxy-registry cannot be combined with --since-version",
},
}

for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
output, err := executeDryRunCommand(t, tt.args)
if tt.wantError != "" {
require.ErrorContains(t, err, tt.wantError)
return
}
require.NoError(t, err, output)
require.Contains(t, output, "Pull plan (dry-run)")

var platformLines []string
for _, line := range strings.Split(output, "\n") {
if _, content, ok := strings.Cut(line, "Platform:"); ok {
platformLines = append(platformLines, strings.TrimSpace(content))
}
}
require.Equal(t, []string{tt.wantPlatform}, platformLines, output)
})
}
}

func executeDryRunCommand(t *testing.T, extraArgs []string) (string, error) {
t.Helper()
defer saveFlagsAndRestore(t)()
pullflags.SinceVersion = nil
pullflags.PlatformConstraint = nil

dir := t.TempDir()
output, err := os.Create(filepath.Join(dir, "output.log"))
require.NoError(t, err)
defer output.Close()

stdout := os.Stdout
os.Stdout = output
defer func() { os.Stdout = stdout }()

cmd := NewCommand()
cmd.SetOut(output)
cmd.SetErr(output)
cmd.SetArgs(append([]string{
filepath.Join(dir, "bundle"),
"--source", "registry.deckhouse.ru/deckhouse/fe",
"--tmp-dir", filepath.Join(dir, "work"),
"--dry-run", "--no-modules", "--no-packages", "--no-installer",
"--no-security-db", "--skip-vex-images", "--gost-digest=false",
}, extraArgs...))
commandErr := cmd.ExecuteContext(context.Background())

body, err := os.ReadFile(output.Name())
require.NoError(t, err)
return string(body), commandErr
}
7 changes: 3 additions & 4 deletions internal/mirror/cmd/pull/validation.go
Original file line number Diff line number Diff line change
Expand Up @@ -168,9 +168,8 @@ func parseAndValidateVersionFlags() error {
// Notes:
// - --deckhouse-tag / --since-version conflicts: --deckhouse-tag asks
// for one tag (a tag-existence check is enough, no probe needed) and
// --since-version asks for "everything from X upward without an upper
// bound", which a probe cannot terminate safely. Both should use the
// non-proxy path instead.
// --since-version extends channel-based discovery using registry tag
// listings. Both use the non-proxy path.
// - --exclude-module is allowed but only meaningful when at least one
// --include-module remains, because the probe still needs concrete
// module names to walk.
Expand All @@ -184,7 +183,7 @@ func validateProxyRegistryFlag() error {
}

if pullflags.SinceVersionString != "" {
return errors.New("--proxy-registry cannot be combined with --since-version: --since-version has no upper bound, so the probe cannot terminate. Use --include-platform with an explicit lower bound (and optional upper bound) instead")
return errors.New("--proxy-registry cannot be combined with --since-version: use --include-platform with an explicit lower bound instead")
}

needPlatform := !pullflags.NoPlatform
Expand Down
8 changes: 3 additions & 5 deletions internal/mirror/platform/platform.go
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ import (

// Options contains configuration options for the platform service
type Options struct {
// SinceVersion specifies the minimum version to start mirroring from (optional)
// SinceVersion extends release discovery below the current rock-solid release.
SinceVersion *semver.Version
// IncludeConstraint narrows platform release discovery to a user-supplied
// semver constraint expression, mirroring the per-module constraint syntax
Expand Down Expand Up @@ -782,12 +782,10 @@ func (svc *Service) determineMinimumVersion(channelVersions channelVersions) *se
return nil
}

// Use rock-solid as baseline
minVersion := rockSolidVersion

// If SinceVersion is provided and is newer than rock-solid, start from SinceVersion
// (user wants to mirror from a later version than rock-solid)
if svc.options.SinceVersion != nil && svc.options.SinceVersion.GreaterThan(minVersion) {
// Older releases extend the range; newer releases keep the rock-solid baseline.
if svc.options.SinceVersion != nil && svc.options.SinceVersion.LessThan(minVersion) {
minVersion = svc.options.SinceVersion
}

Expand Down
12 changes: 10 additions & 2 deletions internal/mirror/platform/platform_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ func TestService_versionsToMirror(t *testing.T) {
wantErr: false,
},
{
name: "no strict tags with SinceVersion",
name: "no strict tags with SinceVersion above Rock Solid keeps the default range",
strictTags: []string{},
options: &Options{
SinceVersion: semver.MustParse("v1.69.0"),
Expand All @@ -129,9 +129,10 @@ func TestService_versionsToMirror(t *testing.T) {
"v1.71.0", // beta
"v1.70.0", // early-access
"v1.69.0", // stable
"v1.68.0", // rock-solid
},
wantChannels: []string{
"alpha", "beta", "early-access", "stable",
"alpha", "beta", "early-access", "stable", "rock-solid",
},
wantCustomTags: []string{},
wantErr: false,
Expand Down Expand Up @@ -312,3 +313,10 @@ func TestService_versionsToMirror_Deduplication(t *testing.T) {
// No custom tags
assert.Empty(t, result.CustomTags)
}

func TestService_determineMinimumVersion_WithoutRockSolid(t *testing.T) {
svc := &Service{options: &Options{SinceVersion: semver.MustParse("1.67.0")}}
versions := channelVersions{"alpha": semver.MustParse("1.72.10")}

assert.Nil(t, svc.determineMinimumVersion(versions))
}
Loading
Loading