socat gets more resources

[evantahler]

We currently only allocate 0.2 CPU shares to the socat processes, but we've observed that we are hitting that limit:

Rather than remove socat and enforce the same pod affinity for all the pods in the sync, lets start by simply throwing money at the problem!

Future work would be to adjust this dynamically as needed at runtime, or for only those syncs with "fast enough" sources

[evantahler]

@davinchia can you confirm that this file is what is used by Cloud, and we don't overwrite these values?

[evantahler]

@supertopher are there any worries about exhausting our cluster resources?

[evantahler]

I think this is the pattern to add ENV configs? I also think we shouldn't add this to the OSS helm charts for now, sort of like how CONTROL_PLANE_AUTH_ENDPOINT isn't there either.

@davinchia once this is merged, I have no idea how to try our a change in the cloud repo...

[davinchia]

@evantahler yes pattern looks right and yes we shouldn't add to OSS Helm charts for now as we don't really want to expose this to OSS users yet.

After merge we have to:

1. Follow https://internal-docs.airbyte.io/Things-To-Know/Upgrading-OSS-Versions to update Cloud.
2. Update the new values here: https://github.com/airbytehq/airbyte-cloud/blob/master/tools/helm-values-builder/templates/gcp_helm_values.yaml.j2#L223. See https://internal-docs.airbyte.io/How-Tos/Building-Helm-values-files#helm-values-development to understand what is happening.
3. Follow https://internal-docs.airbyte.io/Deploying/Deploying-to-Cloud-using-CICD/One-Click-Deploy-Github-Actions#the-deployment-process to deploy the branch to a dev env for testing. On this step, you might have to manually increase the node pool size of the environment you are testing on.
4. Merge to Cloud master with Auto CD to prod.

I'm PTO after today till Dec 18th so please reach out to @pmossman or @git-phu for help!

[davinchia]

Evan, I'm going to approve. Feel free to merge after addressing the comments.

[evantahler]

Merged with a red test - that's a connector base format error, will be solved by #20163