Improve direct controller update efficiency

[yuwenma]

Checklist

• I did not find a related open enhancement request.
• I understand that enhancement requests filed in the GitHub repository are by default low priority.
• If this request is time-sensitive, I have submitted a corresponding issue with GCP support.

Describe the feature or resource

Overview

In general, Direct controller periodically reconciles and updates the GCP service when it detects a config change. However, each direct resource implements its own "is config changed" logic, due to the differentiation of the GCP services it talks to. This ends up as an increasing developing cost and repeated GCP issues.

Here, we provide some general ideas about the ShouldUpdate logic to improve the efficiency and consistency of the Update method.

Solutions

ShouldUpdate by Proto

We can compare the desired config and actual config in their ProtoBuff format. This is good for those protos that use field_behavior to mark if a field is optional, required, (in)mutable or output-only.

Caveats

• Potential issues normally come from the the proto itself, that some field_behaivor could be mistakenly marked.
• The proto object normally does not have the "defaulting" value, making it hard to tell if user wants to "unmanage/unset" a field, or want to use the default value. This requires case by case analysis.

ShouldUpdate by Snapshot

We can snapshot the previous applied, current desired and actual configurations via etag RFC 7232). To compare the etag diff, Config Connector can decide if to update the GCP resource.

Caveats

• Some fields are not suitable to snapshot, like revision, timestamp. Thus Config Connector needs to keep track of a allowlist.

Handle sensitive fields

Some Config Connector resource (TF-based) use mutable-but-unreadable annotation to track the value of sensitive fields. The downside is mostly for the security concerns, and it lacks the ability of keeping the sensitive data (in k8s Secret or other places) in sync.

In the direct approach, this annotation is disabled to not record the encoded sensitive value in kubernetes annotation.

Long-running sensitive data

• If the GCP service requires the sensitive data itself, the controller could reply on the snapshot to determine whether to update the GCP instance or not. There's security risk because the sensitive data could be transferred as plain text.
• If the GCP service support CMEK, SecretManager, or other encryption services, Config Connector will reference the resource, correspondingly.

short-lived sensitive data

Config Connector periodically reconciles to sync-up with GCP services. For short-lived sensitive data, Config Connector could always re-apply the real time value (if stored in Kubernetes k8s).

The downside is the one additional GCP call every 20-40 minutes. For services that have big write/read throughput but high security requirements, this could be a valid option.

Additional information

No response

Importance

No response