[Andy Hay (Andy-Hay)]

Summary

Prepares the 26.10 release. It updates every snowplow-devops module to the patch releases that move module CI to Node 24 and bump telemetry to 0.6.3, moves this repo's own CI past the Node 20 deprecation, and fixes outdated content in the README.

Note

All module releases this PR depends on are published, and terraform validate and fmt pass in all 11 directories in CI. This branch has been deployed on AWS, GCP and Azure, both fresh and as an upgrade from 25.10. No upgrade plan destroys or replaces a stateful resource.

Module version bumps

Updates 30 modules in the AWS, GCP and Azure examples: 72 version pins in the .tf files, and the matching rows in each example's terraform-docs README.

Most bumps are patch releases. The eight AWS application modules move two patch versions: the first release updates their CI, telemetry and ec2-instance-type-metrics, and the second moves them to service-ec2 0.3.5, which uses tags 0.2.2. There are two exceptions (⚠️):

• terraform-google-lb goes from 0.3.0 to 0.4.2 because the repo never picked up 0.4.x. The changes in between add an optional redirect_http_to_https input (default false) and make the URL map name unique. The GCP upgrade test confirmed the load balancer is untouched.
• terraform-aws-rds goes from 0.5.0 to 0.6.0. 0.6.0 adds an optional skip_final_snapshot input (default false), so behaviour is unchanged. The examples don't set it.

Module versions and release PRs

Module	From	To	Release PR
terraform-aws-alb	0.2.0	0.2.1	snowplow-devops/terraform-aws-alb#10
terraform-aws-collector-kinesis-ec2	0.10.1	0.10.3	snowplow-devops/terraform-aws-collector-kinesis-ec2#54, snowplow-devops/terraform-aws-collector-kinesis-ec2#55
terraform-aws-databricks-loader-ec2	0.3.0	0.3.2	snowplow-devops/terraform-aws-databricks-loader-ec2#14, snowplow-devops/terraform-aws-databricks-loader-ec2#15
terraform-aws-enrich-kinesis-ec2	0.7.1	0.7.3	snowplow-devops/terraform-aws-enrich-kinesis-ec2#42, snowplow-devops/terraform-aws-enrich-kinesis-ec2#43
terraform-aws-iglu-server-ec2	0.6.1	0.6.3	snowplow-devops/terraform-aws-iglu-server-ec2#31, snowplow-devops/terraform-aws-iglu-server-ec2#32
terraform-aws-kinesis-stream	0.3.0	0.3.1	snowplow-devops/terraform-aws-kinesis-stream#9
terraform-aws-rds	0.5.0	0.6.0 ⚠️	snowplow-devops/terraform-aws-rds#23, snowplow-devops/terraform-aws-rds#25
terraform-aws-redshift-loader-ec2	0.4.0	0.4.2	snowplow-devops/terraform-aws-redshift-loader-ec2#17, snowplow-devops/terraform-aws-redshift-loader-ec2#18
terraform-aws-s3-bucket	0.2.0	0.2.1	snowplow-devops/terraform-aws-s3-bucket#5
terraform-aws-s3-loader-kinesis-ec2	0.6.0	0.6.2	snowplow-devops/terraform-aws-s3-loader-kinesis-ec2#33, snowplow-devops/terraform-aws-s3-loader-kinesis-ec2#34
terraform-aws-snowflake-streaming-loader-ec2	0.2.1	0.2.3	snowplow-devops/terraform-aws-snowflake-streaming-loader-ec2#15, snowplow-devops/terraform-aws-snowflake-streaming-loader-ec2#16
terraform-aws-transformer-kinesis-ec2	0.5.0	0.5.2	snowplow-devops/terraform-aws-transformer-kinesis-ec2#37, snowplow-devops/terraform-aws-transformer-kinesis-ec2#38
terraform-azurerm-collector-event-hub-vmss	0.4.0	0.4.1	snowplow-devops/terraform-azurerm-collector-event-hub-vmss#17
terraform-azurerm-enrich-event-hub-vmss	0.4.0	0.4.1	snowplow-devops/terraform-azurerm-enrich-event-hub-vmss#18
terraform-azurerm-event-hub	0.1.1	0.1.2	snowplow-devops/terraform-azurerm-event-hub#6
terraform-azurerm-event-hub-namespace	0.1.1	0.1.2	snowplow-devops/terraform-azurerm-event-hub-namespace#7
terraform-azurerm-iglu-server-vmss	0.3.0	0.3.1	snowplow-devops/terraform-azurerm-iglu-server-vmss#14
terraform-azurerm-lake-loader-vmss	0.4.0	0.4.1	snowplow-devops/terraform-azurerm-lake-loader-vmss#14
terraform-azurerm-lb	0.2.0	0.2.1	snowplow-devops/terraform-azurerm-lb#7
terraform-azurerm-postgresql-server	0.2.0	0.2.1	snowplow-devops/terraform-azurerm-postgresql-server#6
terraform-azurerm-storage-account	0.1.3	0.1.4	snowplow-devops/terraform-azurerm-storage-account#7
terraform-azurerm-storage-container	0.1.1	0.1.2	snowplow-devops/terraform-azurerm-storage-container#6
terraform-azurerm-vnet	0.2.0	0.2.1	snowplow-devops/terraform-azurerm-vnet#8
terraform-google-bigquery-loader-pubsub-ce	0.5.0	0.5.1	snowplow-devops/terraform-google-bigquery-loader-pubsub-ce#18
terraform-google-cloud-sql	0.4.1	0.4.2	snowplow-devops/terraform-google-cloud-sql#11
terraform-google-collector-pubsub-ce	0.7.0	0.7.1	snowplow-devops/terraform-google-collector-pubsub-ce#35
terraform-google-enrich-pubsub-ce	0.5.0	0.5.1	snowplow-devops/terraform-google-enrich-pubsub-ce#24
terraform-google-iglu-server-ce	0.7.0	0.7.1	snowplow-devops/terraform-google-iglu-server-ce#28
terraform-google-lb	0.3.0	0.4.2 ⚠️	snowplow-devops/terraform-google-lb#11
terraform-google-pubsub-topic	0.3.0	0.3.1	snowplow-devops/terraform-google-pubsub-topic#4

Azure provider constraint

terraform/azure/pipeline now requires azuread ~> 2.53.1 instead of >= 2.43.0, < 2.44.0. lake-loader-vmss 0.4.1 pins azuread ~> 2.53.1, and the two ranges don't overlap, so terraform init would fail without this change. The directory doesn't use azuread itself, so the old pin only existed for the lake loader. With 25.10 lock files, terraform init -upgrade resolves azurerm 4.27.0 and azuread 2.53.1 in all three Azure directories, and the plan accepts state written by azurerm 4.81.0.

This also fixes fresh Azure deploys: 25.10's modules allow azurerm >= 3.58.0, which now resolves 5.x, and 25.10 fails to plan on 5.x.

CI: Node 24 and SHA-pinned actions

Moves each action in ci.yml to its latest release and pins it to a full commit SHA, with the version kept in a trailing comment. This clears the Node.js 20 is deprecated … actions/checkout@v3 warning.

Action	Before	After	Runtime
actions/checkout	v3	3d3c42e5… (v7.0.1)	node24
tj-actions/branch-names	v8	52504926… (v9.0.2)	composite
dflook/terraform-validate	v1	29a791ea… (v3.1.0)	docker
dflook/terraform-fmt-check	v1	e3f43c85… (v3.1.0)	docker

The major-version jumps don't change anything this workflow uses: the dflook actions still take path, and branch-names still outputs current_branch. branch-names v9 also includes a security fix for unsafe outputs.

README

• Release badge: now 26.10, and it links to this repo's releases instead of snowplow/snowplow, whose releases stop at 22.01.
• Docs links: moved to the current get-started/self-hosted/ and licensing/ paths. This fixes the Installation Guide link, which returned 404, and keeps the ?warehouse=…&cloud=… part of the Deployment Summary links, which the old redirects dropped.
• Platforms table:
  • The Azure summary links said "Snowflake" and now say Databricks and Synapse.
  • Both Azure rows now note that they load through the Data Lake (Lake Loader).
  • Added the AWS S3 loader target, which was missing.
• Wording: "Snowplow Community pipeline" is now "self-hosted Snowplow pipeline", matching the current docs and the Limited Use License.
• terraform-docs example: added the three Azure directories.
• Cleanup: removed the unused deploy-image reference, and changed the copyright to 2021-current.
• Documentation images: the images are kept, but their CloudFront URLs (techdocs.png, roadmap.png) currently return 404. They'll show again once the CloudFront files are restored.

Testing

• terraform fmt -check -recursive passes
• All docs links in the README return 200 with no redirects
• CI runs on the new actions without Node deprecation warnings
• CI passes in all 11 directories (fail-fast: false, so each directory reports separately)
• terraform validate passes in all four AWS directories (checked locally against the published modules)
• terraform validate passes in all four GCP directories
• terraform validate passes in all three Azure directories
• Deployment Summary links open the right cloud and warehouse tab (check in a browser)

🤖 Generated with Claude Code

[Oguzhan Unlu (oguzhanunlu)]

AWS deployment test (PDP-2902): no release blockers

Fresh 26.10 deploy works end to end. Upgrade from 25.10 to 26.10 works end to end: in-place changes only, zero collector downtime, zero event loss.

Tested this branch at 5dbb62e on AWS (eng-dev, eu-central-1), terraform/aws/iglu_server/default and terraform/aws/pipeline/default, Snowflake Streaming as the warehouse target. Terraform 1.5.7, AWS provider resolved to 6.69.1 under the >= 4 constraint. Full plan output is attached to PDP-2902.

Fresh deploy of 26.10

• Iglu 33 to add, pipeline 126 to add, no changes or destroys, applies clean.
• 73 test events sent (21 via snowplow-cli, 52 via a curl loop at about one per second), 73 accepted by the collector, 73 in Snowflake. S3 bad/ empty.
• Collector 3.7.0 and enrich 6.2.1 containers up, cloud-init clean (checked over AWS Systems Manager, SSM). The dnf retry loop from service-ec2 0.3.5 is present in user-data and Docker installed on attempt 1.

Upgrade from 25.10 (main at f48b7cb)

• terraform init -upgrade + plan on the 25.10 state: Iglu 0 to add, 6 to change, 0 to destroy; pipeline 0 to add, 34 to change, 0 to destroy. No must be replaced, will be destroyed or forces replacement anywhere in either plan.
• Stateful resources untouched. aws_db_instance, the 4 aws_kinesis_stream, aws_s3_bucket and both aws_lb appear only as refreshed state, never in a diff block, so DNS names did not change. The 4 DynamoDB tables, 6 IAM roles, 6 security groups and 6 log groups change only their module_version tag. The 6 Terraform telemetry events bump moduleVersion.
• The 6 launch templates get a new version with two changes: the new user-data from service-ec2 0.3.5, and a new AMI (ami-07fbe1656fa7f5275 to ami-0878e4d659d91d45e), because service-ec2 selects the most recent Amazon Linux 2023 image at plan time and a newer one appeared between our first upgrade plan on the 21st and the re-plan we applied on the 23rd (the first went stale when AWS credentials expired). The 6 ASGs point at the new version. Any customer upgrade will pick up whatever AMI is current, independent of this PR.
• After apply, all 6 ASGs ran a rolling instance refresh and reported Successful within 26 minutes (AWS EC2 Auto Scaling, via the AWS CLI). Application logs in AWS CloudWatch show launch-before-terminate: the new Iglu instance was serving 8 minutes before the old one shut down, and the new collector was up 2.5 minutes after the apply.
• Collector availability during the upgrade: 1546 GET probes through the ALB from 14:01:30 to 14:31:29 UTC, about one per second and never more than 2 s apart, covering the pipeline apply and all six refreshes. All 1546 returned HTTP 200. Collector downtime: 0 s at 2-second resolution.
• Event loss: 0. All 1546 probe events are in Snowflake, matched on event_id (1546 sent, 1546 landed, 0 missing).
• After the upgrade: 71 more events (20 via snowplow-cli, 51 via the curl loop), 71 accepted, 71 in Snowflake. Upgraded collector checked over SSM (container up, cloud-init clean, new user-data); upgraded enrich user-data checked (new retry loop present, Docker installed on attempt 1).

Findings, none blocking

1. About 123 deprecation warnings on every plan and apply. The >= 4 constraint picks AWS provider 6.x, which deprecates data.aws_region.current.name (used in locals.is_aws_global of every snowplow-devops app module: 111 warnings in the pipeline plan, 8 in the Iglu plan) and the inline server_side_encryption_configuration on aws_s3_bucket (4 warnings, s3-bucket module). Plan and apply succeed. Fix belongs in the modules.
2. App telemetry webhook fails in enrich and the Snowflake loader (Telemetry - Webhook telemetry-g.snowplowanalytics.com/... resulted in exception without a response, INFO, at startup and every 15 min). The network path and generated config look correct from the instance, so the failure is inside the app; not diagnosed further. Same line on 25.10 with the same app versions, so pre-existing and not caused by this PR. Collector and Iglu log no telemetry errors.
3. SSM is not provisioned. The quickstart's access path is SSH via ssh_public_key and ssh_ip_allowlist; the IAM roles carry no SSM permission and the instances do not register with Systems Manager. Port 22 was unreachable from our network, so we attached AmazonSSMManagedInstanceCore out of band for the on-instance checks. If SSM is the intended support path, the modules or docs should say so.
4. Stale READMEs: terraform/aws/iglu_server/default/README.md and secure/README.md on this branch still list rds 0.5.1 while main.tf uses 0.6.0.
5. Teardown: terraform destroy would fail on a non-empty S3 bucket (force_destroy = false; we emptied it first), and the RDS module leaves a final snapshot behind (skip_final_snapshot defaults to false). Both worth a line in the quickstart docs.

Also seen: 62 scanner-generated bad rows (adapter_failures, collector_payload_format_violation; sample was a bot probing /mcp/) while the 25.10 stack sat idle for two days on a public ALB. None from our events.

Not covered: secure variants, Redshift and Databricks targets (their modules are bumped but were not instantiated).

🤖 Generated with Claude Code

[Oguzhan Unlu (oguzhanunlu)]

GCP deployment test: no blockers in the upgrade plan, one significant pre-existing finding (9-minute collector outage on upgrade)

Fresh 26.10 deploy works end to end. Upgrade from 25.10 to 26.10 works end to end: no stateful destroy or replace, the load balancer is untouched, and 0 of the 713 events the collector accepted during the upgrade were lost. But the collector was down for about 9 minutes during the upgrade (467 requests got HTTP 503). The cause is the instance group update policy in snowplow-devops/service-ce/google 0.2.0, which this PR does not change.

snowplow-devops/lb/google 0.3.0 to 0.4.2: the upgrade plan touched the load balancer: no.

Tested this branch at 5dbb62e on GCP (europe-west1), terraform/gcp/iglu_server/default and terraform/gcp/pipeline/default, BigQuery as the warehouse target. Terraform 1.5.7, Google provider 8.8.1. Full plan output is attached to our internal ticket PDP-2903.

Fresh deploy of 26.10

• Iglu 20 to add, pipeline 51 to add, no changes or destroys, applies clean.
• 0 "Deprecated value used" warnings, and 0 warnings of any kind, in terraform validate and in every plan and deploy or upgrade apply.
• 69 test events sent (20 via snowplow-cli, 49 via a curl loop), 69 accepted, 69 in BigQuery. Nothing on either failed-events (bad rows) topic; we attached a pull subscription to each for the test, since the quickstart creates no subscriber for them.
• Collector 3.7.0, enrich 6.2.1 and BigQuery loader 2.0.1 up, startup script exit status 0 (checked over SSH).

Upgrade from 25.10 (main at f48b7cb)

• terraform init -upgrade + plan on the 25.10 state: Iglu 1 to add, 2 to change, 1 to destroy; pipeline 3 to add, 8 to change, 3 to destroy.
• The adds and destroys are only the 4 Compute Engine instance templates, replaced create-before-destroy (templates are immutable). The replacement is forced by the module_version label and the startup script. The decoded collector config differs only in telemetry moduleVersion 0.7.0 to 0.7.1; the other three scripts are marked sensitive in the plan. Ubuntu image unchanged (ubuntu-2404-noble-amd64-v20261024). The 4 managed instance groups point at the new templates.
• Load balancer untouched: google_compute_url_map, google_compute_target_http_proxy, google_compute_global_forwarding_rule, google_compute_backend_service and google_compute_global_address have no planned action in either plan (only "Refreshing state"). Collector and Iglu IPs unchanged.
• Stateful resources untouched: no Pub/Sub topic, Cloud SQL instance or BigQuery dataset has a planned action. The enrich and loader input subscriptions change only their module_version label, in place.
• Collector availability: 1180 GET probes through the load balancer from 12:40:08 to 13:03:30 UTC, never more than 3 s apart, across both applies and the rollout. 713 got HTTP 200, 467 got HTTP 503. 503s from 12:50:44 to 12:59:55 UTC (two isolated 200s inside that window), steady 200 from 12:59:56: about 9 minutes down.
• Accepted events lost: 0. All 713 accepted probes are in BigQuery, matched on event_id. None of the 467 rejected requests landed, so a client that does not retry would lose them.
• After the upgrade: 70 more events, 70 in BigQuery, bad-row topics still empty, all 4 new instances checked over SSH.

Findings, none of them a stateful destroy or replace

1. Collector outage of about 9 minutes on upgrade. From the GCP audit log, the collector instance group:

   • removed its only instance from the group at 12:50:28, which took 6 minutes including the backend service's 300 s connection draining;
   • deleted it;
   • created the replacement only at 12:57:40.

   service-ce 0.2.0 sets max_unavailable_fixed = 3 with no surge (maxSurge 0), and each group has target size 1. The module and its update policy are the same on main, so this PR did not cause it, but from the policy we expect it on every GCP quickstart upgrade that changes an instance template. Suggest allowing surge in service-ce (max_surge_fixed of at least the zone count, max_unavailable_fixed = 0). Iglu, enrich and the loader roll the same way; the Iglu group update took as long as the collector's, so a similar Iglu outage is likely (not measured).

2. Teardown: terraform destroy fails on the BigQuery dataset (still in use), because the events table is created by the loader, not Terraform, and the dataset has no delete_contents_on_destroy. Pre-existing.

3. Teardown: Iglu destroy fails on the Cloud SQL user (role "iglu" cannot be dropped because some objects depend on it): the cloud-sql module deletes the database and the user in parallel, with no dependency between them. Re-running destroy after the database is gone succeeds. cloud-sql 0.4.1 to 0.4.2 changes only CI, the CHANGELOG and the README, so pre-existing; the fix belongs in the module.

4. Enrich logs the same Telemetry - Webhook ... resulted in exception without a response line we saw on AWS. Pre-existing.

Not covered: the secure variants. We did not measure Iglu availability during its own rollout.

🤖 Generated with Claude Code

[Oguzhan Unlu (oguzhanunlu)]

Heads-up from Azure testing (not a result yet): all 18 Azure modules in this release now pin azurerm ~> 4.27.0, while 25.10's modules allow >= 3.58.0. An existing 25.10 stack will therefore have a newer azurerm locked: 4.51.0 was current when 25.10 shipped, and a fresh 25.10 init today resolves 5.11.0. terraform init -upgrade on 26.10 will move those stacks back to 4.27.x. I'm testing whether the plan accepts state written by the newer provider ("Resource instance managed by newer provider version" is the failure I'm watching for) and will report the result here.

[Oguzhan Unlu (oguzhanunlu)]

Azure deployment test: no blockers found; upgrade tested up to plan

Fresh 26.10 deploy works end to end. With 25.10 lock files and state, terraform init -upgrade and plan succeed in all three Azure directories with no destroy or replace. I did not apply the upgrade.

init -upgrade with the new azuread constraint, using the 25.10 lock files: succeeded (azuread 2.43.0 to 2.53.1, azurerm 4.81.0 to 4.27.0).

Tested this branch at 5dbb62e on Azure (westeurope): terraform/azure/base, iglu_server, pipeline with the lake loader. Terraform 1.5.7. Full plan output is attached to our internal ticket PDP-2904.

Correction to my heads-up above

• The pin is in the 12 Azure modules 26.10 uses (18 module calls), not "18 modules".
• 25.10 cannot be deployed on azurerm 5.x at all: a fresh 25.10 init today resolves azurerm 5.11.0, and the plan fails in all three directories (arguments removed or replaced in azurerm 5.0, such as namespace_name on azurerm_eventhub and enable_http2 on the Application Gateway). So the 5.11.0 example in my heads-up is not a real case, and this PR's ~> 4.27.0 pin fixes fresh deploys.
• Stacks first deployed from 25.10 have azurerm 4.51 to 4.81, and upgrading moves them down to 4.27.0. I tested the largest step, from 4.81.0. Stacks carried over from older releases may have older versions locked; not tested.

Fresh deploy of 26.10

• base 7, iglu_server 16, pipeline 47 to add; applies clean.
• terraform validate: 8 "Deprecated attribute" warnings in pipeline (6× namespace_name in the event-hub module, resource_manager_id and azuread application_id in the lake-loader module); the plan also warns about 7 arguments that are removed in v5.0. Harmless under the 4.27 pin.
• 71 test events sent (20 via snowplow-cli, 51 via a curl loop, all accepted), all 71 in the lake as a Delta table, the loop events matched on event_id. Read with DuckDB; I had no access to query it from Synapse and did not try Databricks.
• No bad rows from our events (3 from internet probes before the test started).
• Collector 3.7.0, enrich 6.2.1, lake loader 0.7.0, Iglu 0.14.1 up, cloud-init clean (checked over SSH).

Upgrade from 25.10 (main at f48b7cb, azurerm 4.81.0 and azuread 2.43.0 locked)

• init -upgrade succeeded in all three directories.
• plan (not applied): base no changes; iglu_server 0 add, 3 change, 0 destroy; pipeline 0 add, 9 change, 0 destroy. The changes are module_version tags on NSGs and VMSS, VMSS user_data (only the telemetry module-version label differs; app versions are unchanged), and telemetry events.
• Stateful resources untouched: Postgres flexible server, Event Hubs namespace and hubs, storage account and lake container have no planned action; nor do the Application Gateways, public IPs or the lake loader's azuread app and service principal. No "managed by newer provider version" error: all resources in the 25.10 state plan cleanly on azurerm 4.27.0.

Findings, none blocking in what was tested

1. Plain terraform init fails on 26.10 when a stack is upgraded in place with its 25.10 lock files (the upgrade guide's flow): locked provider ... azurerm 4.81.0 does not match configured version constraint >= 3.58.0, ~> 4.27.0; must use terraform init -upgrade, and the same for azuread in pipeline. The upgrade guide only shows terraform init. Azure users need init -upgrade; worth stating in the release notes or the guide.
2. The VMSS use upgrade_mode = "Manual", so an apply updates the scale set model, not running instances. Harmless for this release (only a telemetry label changes), but worth documenting for releases that change app versions.
3. From azurerm 4.x, ARM_SUBSCRIPTION_ID must be set or plan fails with "subscription_id is a required provider property". The quick start does not mention it. Pre-existing.
4. One Application Gateway (on the 25.10 baseline) took 1h5m to create, the others 11 to 20 min, with no error from Azure. Azure-side, but looks like a hung apply.
5. Enrich and the lake loader log the same telemetry webhook exception as on AWS and GCP. Pre-existing.

Not covered: applying the upgrade, collector downtime during it, events and instance checks after it; querying from Synapse or Databricks; stacks with azurerm older than 4.51 locked.

🤖 Generated with Claude Code

[Oguzhan Unlu (oguzhanunlu)]

Approving. I deployed this branch at 5dbb62e on AWS, GCP and Azure, fresh and as an upgrade from 25.10; results are in my three test comments above. No upgrade plan destroys or replaces a stateful resource on any cloud, and AWS and GCP upgrades lost no events. On Azure this release also fixes fresh deploys, which fail on 25.10 today because its modules resolve azurerm 5.x.

Two small things, neither blocking:

1. Stale READMEs: terraform/aws/iglu_server/default/README.md and terraform/aws/iglu_server/secure/README.md still list snowplow-devops/rds/aws 0.5.1, while main.tf uses 0.6.0. Worth regenerating before merge.
2. Azure release note: with existing 25.10 lock files, plain terraform init fails on this release (locked provider ... azurerm 4.81.0 does not match configured version constraint >= 3.58.0, ~> 4.27.0; must use terraform init -upgrade, and the same for azuread in pipeline). Azure users upgrading need terraform init -upgrade, and from azurerm 4.x they also need ARM_SUBSCRIPTION_ID set. The upgrade guide only shows terraform init, so a line in the release notes would save every Azure user a failed first step.