Re-apply "SCS-0128: Remove admin pass instance creation test"

[toothstone]

Recreation of #1245 after it was erroneously merged and had to be reverted in #1250

Only applicable once #1250 is merged.

[toothstone]

There was an agreement with @mbuechse that SCS-0128 will be amended to only require not failing any tests, so skipping tests is allowed.
This means I can just set the tempest feature flag controlling this as enable_instance_password = false and always skip the problematic test case, so if the revert goes through it's not a problem.

[toothstone]

There was an agreement with @mbuechse that SCS-0128 will be amended to only require not failing any tests, so skipping tests is allowed. This means I can just set the tempest feature flag controlling this as enable_instance_password = false and always skip the problematic test case, so if the revert goes through it's not a problem.

This was reverted in #1285, so I'll reopen this PR.

There are valid security reasons to disallow instance creation with admin passwords, so SCS should not mandate this feature.

[mbuechse]

You are right: on 2026-07-23 we decided to

make SCS-0128-v1 more precise by specifying that the overall test is passed when Tempest has 0 failures (regardless of skips)

And the latest change contradicts that, for the standard now says:

[tempest] MUST NOT report any skipped test cases, except for well-founded cases.

However, there is a priori no reason why the test case in question cannot be (made) one of those well-founded exceptions.

Then again, it has to be discussed whether it should. The above-linked meeting minutes also state

instance creation with admin (read: root) password is not a mandatory feature, compliance checks can't know what the CSP configured

I don't know what is meant here by 'is not a mandatory feature'. Mandatory -- mandated by whom? Maybe it was mandatory in OpenStack-powered Compute, maybe it wasn't. Regardless, I think we, the SCS community, should reconsider this matter and come to a (loose) consensus what we want:

1. keep it mandatory (no skip allowed),
2. make it dependent on the advertised feature set (skip allowed if feature disabled),
3. remove the testcase altogether.

So I'm adding this to the SIG agenda.

[toothstone]

I don't know what is meant here by 'is not a mandatory feature'.

I'm pretty sure this was an early version of the "correctness vs completeness" debate, before we had those terms for it.

Just to avoid further confusion: #1250 was never merged, as of right now, instance creation with admin password is NOT in the list of mandatory test cases/features, so "keep it mandatory" might be a bit misleading in its wording. It would probably be good to clarify that before the SIG discussion.

[toothstone]

Also, versioning the list of test cases would probably be helpful going forward, to avoid such ambiguity.

[mbuechse]

Also, versioning the list of test cases would probably be helpful going forward, to avoid such ambiguity.

We keep a list of exceptions in scs-0128. This list can be amended, and we (edit: CAN) do minor versions and keep a changelog. The admin-password thing was never on this list, and so it should be tested!

[toothstone]

Also, versioning the list of test cases would probably be helpful going forward, to avoid such ambiguity.

We keep a list of exceptions in scs-0128. This list can be amended, and we (edit: CAN) do minor versions and keep a changelog.

Having the human-readable changelog is helpful to humans, but for the technical implementation of automated testing not so much. I'm currently doing "curl | tempest" with the test list, and that would be much nicer and more stable with a canonical URL per SCS-0128 version.

[mbuechse]

I don't see the contradiction. We (can) have a stable URL per major version. That was the idea.