Allow `spec.replicas` to flow through blue/green deployments.

[im-aIex]

Summary

Currently, when making changes using blue/green, new changes are expected to be made to the green instance before promoting. When changing spec.replicas, the changes are made to both the green and blue instances, resulting in active traffic being impacted prior to promotion.

While there is the spec.strategy.blueGreen.previewReplicaCount, this does not allow for a way to change the replicas of what will eventually be the blue instance, without impacting the existing blue instance.

It would be nice if there were an option to allow the spec.replicas changes to only impact the green instance, waiting for promotion until the blue instance is changed (when pointing to the green ReplicaSet).

Use Cases

Changing spec.replicas and doing load testing against the green instance to ensure rightsizing prior to promoting.

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritize the issues with the most 👍.

[kostis-codefresh]

While there is the spec.strategy.blueGreen.previewReplicaCount, this does not allow for a way to change the replicas of what will eventually be the blue instance, without impacting the existing blue instance

Not sure I follow this. If you have blue/old version with 10 pods and set green/new version with previewReplicaCount to 3 pods, then

• Blue is 10 pods (not impacted by anything)
• Green is 3 pods (because of previewReplicaCount )

It would be nice if there were an option to allow the spec.replicas changes to only impact the green instance

Isn't this exactly previewReplicaCount? It only affects preview/green/new version.

Changing spec.replicas and doing load testing against the green instance to ensure rightsizing prior to promoting.

Isn't this the job of HPA?

[im-aIex]

The issue we run into is that it then requires multiple commits to properly test and deploy a single change.

• Update previewReplicaCount
• Validate changes
• Revert previewReplicaCount and update replicas
• Promote (if any other changes)

Whereas with any other type of change (including e.g. resources) it would just be:

• Update resources (only impacting new/green)
• Validate changes
• Promote

Right now, it seems replicas is the only thing that goes directly to the active/blue instance, essentially bypassing blue/green.

[kostis-codefresh]

Can you explain a bit your use case and what you are trying to do?

Revert previewReplicaCount and update replicas

Why do you need to revert previewReplicaCount? Why not have it as permanent setting on the rollout?

[im-aIex]

The goal is just to not have anything impact the blue/active deployment until it has gone through green/preview.

We wouldn't necessarily need to revert the previewReplicaCount, but it would still be a two-commit operation of updating previewReplicaCount (+ e.g. resources, commit # 1), then to promote that I need to both promote the blue/green (no-commit action) as well update replicas (commit # 2).

Everything else I just commit the change, it goes to green/preview, then I promote it to effect blue/active.

[kostis-codefresh]

The goal is just to not have anything impact the blue/active deployment until it has gone through green/preview.

Sorry, I still don't understand why this requirement is not covered by the current/default behavior of Argo Rollouts.

Let's take a full example. You have an application that needs 10 pods in production, but during internal tests it only needs 3.
You already have a spec in Git that has replicas:10 and previewReplicaCount:3

1. The application is running in production as v1.1 . There is only one version right now with 10 pods for all live users. Everything runs fine.
2. You perform a single commit in the manifest for v1.2 - Argo Rollouts notices the commit and launches 3 new/green pods for this new version. All live users are still on the 10 pods of the blue/stable version
3. You can do as many tests as you want in your leisure on the 3 pods of the new/green version that is at v1.2. There is no impact for all active users as they still use the 10 pods for the blue/stable version still at v1.1
4. Once you are ready to promote (either with CLI or automatically), Argo Rollouts will scale up on its own the new/green pods to 10 and make them the new stable/blue version. Existing users will instantly change from using 10 pods of v1.1 to 10 pods of v1.2
5. After a while the old pods of 1.1 are destroyed. Again there is now a single version of v1.2 with 10 pods (as the replicas field says).

Key points

• There was a single commit to update the manifest from v1.1 to v1.2
• At no point you need to enable/disable previewReplicaCount on your own. It just works for the new/green color
• spec.replicas always affects the current/blue/active version (what the live users see). previewReplicaCount only affects the preview/green version (what you have for your internal tests)
• There was no impact at all for users while you are testing v1.2

The documentation at https://argo-rollouts.readthedocs.io/en/stable/features/bluegreen/#previewreplicacount says that even when using previewReplicaCount Argo Rollouts will always scale up the new/green pods when it becomes the stable version. So in the example above, Argo Rollouts will actually pass 10 pods to live users for v1.2 and NOT 3 pods. It will automatically scale on its own from 3 to 10 pods once v1.2 becomes the new stable/blue color. You don't need to perform any commits to enable/disable previewReplicaCount on your own.

Could you explain how the above scenario does not cover your requirements and what you prefer Argo Rollouts do instead?

[im-aIex]

The end goal with my use-case is that the blue/active will be updated to the new replica count, but only after being validated via green/preview. I want the green/preview to be 1:1 for what will be promoted to blue/active, not specific differences for validation.

So, for a concrete example, let's say I'm rightsizing the overall resource footprint of a service. I want to change replicas from 10 to 3, as well as increase the resources.limits.cpu from 1 to 2.

1. The application is running in production as v1.1. There is only one version right now with 10 pods for all live users. Everything runs fine.
2. I perform the first commit for v1.2, requiring previewReplicaCount to be used to control the green/preview replica count, as well as the desired resources.limits.cpu change.
3. Testing is done against v1.2 green/preview.
4. Once I'm ready to promote, I now need to both:
   1. Promote v1.2 for resources.limits.cpu change - resulting in 10 pods with the higher resource limits
   2. Update replicas to take effect with the blue/active - separate commit, matching previewReplicaCount
      (alternatively swapping the order and having the replicas change in blue/active prior to the resources.limits.cpu)

The issue with previewReplicaCount is that there's no way to do 4.i and 4.ii simultaneously (within the same commit/promote).

what you prefer Argo Rollouts do instead?

The preference would be to have an option to allow changes to replicas to act the same as everything else (e.g. resources.limits.cpu), first only effecting green/preview and then keeping the change when promoting.

Right now, it seems replicas is the only thing that goes directly to the active/blue instance, essentially bypassing blue/green.

Everything else I just commit the change, it goes to green/preview, then I promote it to effect blue/active.