Write ADR for load-balancers

[garloff]

As a KaaS operator, I want to understand exactly what standards my load-balancer must fulfill by reading the relevant ADR.

TODO: Write ADR (according to SCS-0001 standard) to document the requirements tested for in #212.

Definition of Ready:

• User Story is small enough to be finished within one sprint
• User Story is clear and understood by the whole team
• Acceptance criteria are defined
• Acceptance criteria are clear and understood by the whole team

Definition of Done:

• All acceptance criteria are met
• Changes have been reviewed
• CI tests have run successfully
• Documentation has been updated
• Release Notes have been updated

[garloff]

nginx-ingress expects that the single nginx instance on a multi-node cluster works despite externalTrafficPolicy: Local (which means kube-proxy is NOT forwarding connections) on a multi-node cluster. Apparently, nginx-ingress expects the LoadBalancer to detect which nodes are up and load-balance only to the ones where nginx is listening.
This is the most efficient way (by avoiding unneeded forwards), but one could question whether loadbalancers should always detect nodes being down. With the default setting of the OpenStack Cloud Controller Manager, loadbalancers do NOT fulfill this: They don't automatically get a health monitor. This is not unreasonable either -- health monitors can always be requested, though unfortunately, this is not standardized but can only be achieved by an OpenStack specific annotation. Which makes things non-portable :-(
So we will have to default to health-monitors be enabled ...

[garloff]

TODO:

• Investigate why the default to NOT have health monitors was chosen.
• Investigate whether opt-out (by specific annotation) is still possible
• Compare to other CCMs and their defaults

[garloff]

@garloff and @joshmue to look at OCCM in first week of Jan.