Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zone Aware Routing: Ability to set minimum number of hosts per-zone #38561

Open
jukie opened this issue Feb 25, 2025 · 7 comments · May be fixed by #39058
Open

Zone Aware Routing: Ability to set minimum number of hosts per-zone #38561

jukie opened this issue Feb 25, 2025 · 7 comments · May be fixed by #39058
Labels
area/load balancing enhancement Feature requests. Not bugs or questions. no stalebot Disables stalebot from closing an issue

Comments

@jukie
Copy link
Contributor

jukie commented Feb 25, 2025

Title: Ability to set minimum number of hosts per-zone when using Zone Aware Routing

Description:
In addition to setting minClusterSize, I'd like the ability to set the minimum number of upstream hosts in the local zone in order for zonal routing to be performed.

Relevant Links:

@jukie jukie added enhancement Feature requests. Not bugs or questions. triage Issue requires triage labels Feb 25, 2025
@KBaichoo KBaichoo added area/load balancing and removed triage Issue requires triage labels Feb 26, 2025
@KBaichoo
Copy link
Contributor

Sounds like a reasonable feature.

@jukie
Copy link
Contributor Author

jukie commented Mar 8, 2025

/assign @jukie

Copy link

jukie is not allowed to assign users.

🐱

Caused by: a #38561 (comment) was created by @jukie.

see: more, trace.

@jukie
Copy link
Contributor Author

jukie commented Mar 8, 2025

I'll work on this

Copy link

github-actions bot commented Apr 7, 2025

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale stalebot believes this issue/PR has not been touched recently label Apr 7, 2025
@jukie
Copy link
Contributor Author

jukie commented Apr 7, 2025

/remove-lifecycle stale

@github-actions github-actions bot removed the stale stalebot believes this issue/PR has not been touched recently label Apr 8, 2025
@KBaichoo KBaichoo added the no stalebot Disables stalebot from closing an issue label Apr 8, 2025
@jukie
Copy link
Contributor Author

jukie commented Apr 9, 2025

This might only be applicable when force_locality_direct_routing (#38756) is enabled as a protection mechanism to avoid overloading the local zone.

Stepping through the code I think Envoy would already self-protect small zones. For example consider the below structure:

  // Zone A: 1 local (33.33% local_percentage), 2 upstream hosts (20% upstream_percentage)
  // Zone B: 1 local (33.33% local_percentage), 4 upstream hosts (40% upstream_percentage)
  // Zone C: 1 local (33.33% local_percentage), 4 upstream hosts (40% upstream_percentage)

For zone A local > upstream which means it'd fallback to residual correct?

// If we have lower percent of hosts in the local cluster in the same locality,
// we can push all of the requests directly to upstream cluster in the same locality.
if ((locality_percentages[0].upstream_percentage > 0 &&
locality_percentages[0].upstream_percentage >= locality_percentages[0].local_percentage) ||

So then it steps down and determines what percentage can be routed locally which would be ~60% in this case (20/33.33).

// If we cannot route all requests to the same locality, calculate what percentage can be routed.
// For example, if local percentage is 20% and upstream is 10%
// we can route only 50% of requests directly.
// Local percent can be 0% if there are no upstream hosts in the local locality.
state.local_percent_to_route_ =
upstreamHostsPerLocality.hasLocalLocality() && locality_percentages[0].local_percentage > 0
? locality_percentages[0].upstream_percentage * 10000 /
locality_percentages[0].local_percentage
: 0;

Assuming each zone has 100 request inbound the result should be:

Zone A - 60 requests from local, 0 cross-zone
Zone B - 100 requests from local, 20 cross-zone
Zone C - 100 requests from local, 20 cross-zone

Which already achieves equal distribution and provides protections to smaller zones. I'll plan on just adding this as an extra gate to force_locality_direct_routing but I wanted to first ask if there's additional use cases for min_zone_size to consider. @KBaichoo does the above sound correct?

@jukie jukie linked a pull request Apr 10, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/load balancing enhancement Feature requests. Not bugs or questions. no stalebot Disables stalebot from closing an issue
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants