define KaaS in the certification standards

[depressiveRobot]

This PR implements #1135.

[depressiveRobot]

Do we need a new version of SCS-0004 for that change?

[mbuechse]

SCS-compatible Kubernetes clusters

I don't think this is defined anywhere. I suppose you could just drop "SCS-compatible" to make it work.

[janiskemper]

the definition would be that they fulfill the test suite that is defined in this repo and which uses sonobuoy.

I think that is a relevant part for "SCS compatible KaaS". Otherwise we have a definition of "KaaS" (which might be also okay).

What do you suggest @mbuechse ?

[mbuechse]

I would argue that the test suite does test KaaS. The script scs-test-runner.py has commands for "provision" and "unprovision" (as far as I recall). Yes, the script scs-compliance-check.py (which is used by the former) only checks a single cluster. But that is a technical detail IMO.

In this perspective, we already test and certify KaaS (even though there is no explicit testcase for creating a cluster). If everyone knows what KaaS is, we don't need to define it. But the questions seems to have come up time and again.

[mbuechse]

Addendum: What I've written about the testsuite (what it does and how) is just the status quo. I wouldn't say the testsuite is so mature that it couldn't or shouldn't change. In fact, I think it should be refactored in multiple ways. So the testsuite is not the yardstick IMO. It makes sense to me, however, that we regard the testsuite as something that tests KaaS, and if there is a gap currently, then that's just a technical detail that we fix further down the road.

[janiskemper]

I do very much think that we need a definition of KaaS. For example, if we need a fully automated flow to create and access the clusters. This is not straightforward, because some providers allow only humans to access a newly created cluster with OIDC and not machines.

I'm totally fine though with defining KaaS and leaving out SCS-compliant KaaS.

[mbuechse]

I don't think that's correct. It does not define that the clusters must adhere to the standards and therefore fulfill the tests.

I haven't understood the sentence as a full definition, but rather as an additional requirement. You could just add "among other things" to the sentence.

[mbuechse]

BTW here I also mean 3. (as in the comment below)

[depressiveRobot]

The following must be clear in my opinion as a requirement to both, the certification body and the applicants:

• Applicants must provide an environment capable of automatically providing Kubernetes clusters in a reproducible way.
• Each of these clusters must meet the standards (and pass the tests) which means being SCS-compatible/compliant.

That is what should be expressed here to make the certification (process) clear. IMO your third option doesn't make that clear.

Maybe I'm being too strict about this? But then I wonder why we even test if the tests (and thus the standards) don't have to be met?

[mbuechse]

A third time: the tests and standards are not for clusters, but for KaaS!!!

[depressiveRobot]

After bilateral communication, we were able to reconcile our two perspectives. See commit 72ed105.

[mbuechse]

SCS-compliant Kubernetes clusters

(same as above)

[depressiveRobot]

1. Please note, KaaS (Kubernetes as a Service) means that the environment MUST provide SCS-compliant Kubernetes clusters in a fully automated and reproducible way.

or

2. Please note, SCS-compliant KaaS (Kubernetes as a Service) means that the environment MUST provide Kubernetes clusters in a fully automated and reproducible way.

Same here, I think the first option is the correct definition.

[mbuechse]

Actually

3. Please note, KaaS (Kubernetes as a Service) means that the environment MUST provide Kubernetes clusters in a fully automated and reproducible way.

[depressiveRobot]

This conversation is resolved, see comment.

[garloff]

/lgtm