getReferencedClusterConnection throws a NullPointerException for a ClusterConnection without status and ignores its generation #98

Current behavior

BaseReconciler.getReferencedClusterConnection reads the phase of the referenced ClusterConnection without two checks (BaseReconciler.java:49-87):

1. No null check for the status. Before the first reconcile of a ClusterConnection, getStatus() is null. Line 73, clusterConnection.getStatus().getPhase(), then throws a NullPointerException.
2. No generation check. A READY from an older generation also counts. After a change of the ClusterConnection spec, for example a new host, a dependent resource uses the new spec before the ClusterConnection reconciler has probed it.

All ten callers (reconcile and cleanup of Database, Schema, Role, Grant and DefaultPrivilege) call the method before their try block. The NullPointerException thus skips handleError: no status is written, and JOSDK retries the reconcile with its own backoff. This is the same type of bug as #94.

This happens when a ClusterConnection and its dependent resources are applied together, for example in one Helm release, and a dependent resource is reconciled first.

Proposal

Do the same as the Garage Operator (BaseReconciler.java, commits c25e3bb and 66ff039):

1. Return Optional.empty() with a log line when the status is null.
2. Return Optional.empty() with a log line when status.observedGeneration is below metadata.generation.
3. Add a Javadoc: an empty result means "not ready yet", not "broken". The callers then set PENDING and try again after 60 seconds.

Add tests for a ClusterConnection without a status, and for a READY ClusterConnection whose observedGeneration is below metadata.generation (aboutbits/garage-operator#3).

Found in the review of the Garage Operator (aboutbits/garage-operator#2).