Skip to content

chore: add arg k_top_api_nodes for dynamic routing to best k nodes#74

Merged
nikolay-komarevskiy merged 2 commits intomainfrom
komarevskiy/use-k-top-nodes-arg
Mar 20, 2025
Merged

chore: add arg k_top_api_nodes for dynamic routing to best k nodes#74
nikolay-komarevskiy merged 2 commits intomainfrom
komarevskiy/use-k-top-nodes-arg

Conversation

@nikolay-komarevskiy
Copy link
Contributor

@nikolay-komarevskiy nikolay-komarevskiy commented Mar 20, 2025

For example setting ic_use_k_top_api_nodes=4 would cause routing to only 4 API nodes using weighted random sampling. NOTE: this is different from the n_ordered_routes().

In /metrics this would manifest in:

# HELP ic_gateway_api_boundary_nodes Number of API boundary nodes with status.
# TYPE ic_gateway_api_boundary_nodes gauge
ic_gateway_api_boundary_nodes{status="healthy"} 4
ic_gateway_api_boundary_nodes{status="total"} 20

We might need another status in_routing as healthy is a bit misleading here.

@nikolay-komarevskiy nikolay-komarevskiy requested a review from a team as a code owner March 20, 2025 07:42
@nikolay-komarevskiy nikolay-komarevskiy merged commit 55f2c02 into main Mar 20, 2025
5 checks passed
@nikolay-komarevskiy nikolay-komarevskiy deleted the komarevskiy/use-k-top-nodes-arg branch March 20, 2025 14:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants