feat(backend): init horizontal clustering#1201
Open
lusergit wants to merge 5 commits intoedgehog-device-manager:mainfrom
Open
feat(backend): init horizontal clustering#1201lusergit wants to merge 5 commits intoedgehog-device-manager:mainfrom
lusergit wants to merge 5 commits intoedgehog-device-manager:mainfrom
Conversation
lusergit
force-pushed
the
push-nrvnswvyxzqv
branch
3 times, most recently
from
6b06d59 to
13cf0b7
Compare
Pull Request Test Coverage Report for Build 0f8a90b75d89dc74710f86829858fc0614fd290a-PR-1201Details
💛 - Coveralls |
lusergit
force-pushed
the
push-nrvnswvyxzqv
branch
12 times, most recently
from
7c1c7c6 to
bc88347
Compare
lusergit
force-pushed
the
push-nrvnswvyxzqv
branch
6 times, most recently
from
ab5cb83 to
5d89de1
Compare
lusergit
force-pushed
the
push-nrvnswvyxzqv
branch
6 times, most recently
from
9a51c35 to
ab965df
Compare
lusergit
force-pushed
the
push-nrvnswvyxzqv
branch
2 times, most recently
from
0f57caf to
4948a33
Compare
lusergit
force-pushed
the
push-nrvnswvyxzqv
branch
4 times, most recently
from
2977a5e to
a475ed8
Compare
lusergit
force-pushed
the
push-nrvnswvyxzqv
branch
4 times, most recently
from
518c0a1 to
159c707
Compare
Signed-off-by: Luca Zaninotto <luca.zaninotto@secomind.com>
Adds libcluster configs to scale edgehog horizontally. This allows edgehog to manage the workload among different replicas for - campaign execution - notifications - reconciliation tasks New environment vairables have been introduced, to allow different edgehog nodes to see eachother based on the deployment strategy. - `EDGEHOG_CLUSTERING_STRATEGY`: one of `none`, `docker-compose` or `kubernetes`. This chooses the strategy edgehog will use to lookup other nodes in the cluster. - `EDGEHOG_CLUSTERING_KUBERNETES_SELECTOR`: The endpoint label to get other edgehog instances. This defaults to `app=edgehog`. - `EDGEHOG_CLUSTERING_KUBERNETES_NAMESPACE` the kubernetes namespace to find other edgehog instances. This defaults to `edgehog`. Signed-off-by: Luca Zaninotto <luca.zaninotto@secomind.com>
When deploying on kubernetes, it is possible to deploy multiple replicas of the backend service. To do so, a couple fo environment variables have been added to instruct edgehog on how to find and connect to other nodes. Signed-off-by: Luca Zaninotto <luca.zaninotto@secomind.com>
Signed-off-by: Luca Zaninotto <luca.zaninotto@secomind.com>
Moves relevant registries and supervisors in the application tree to allow edgehog to scale horizontally. This is done only with registries and supervsors where it makes sens to chare the load: - `Containers.Reconciler`, where the process spawns per-tenant and talks with the DB. - `Tenant.Reconciler`, where tasks and processes are again spawned per-tenant. - `Devices.Reconciler` is moved to a single process managed trough a horde process to make it single per-cluster. Signed-off-by: Luca Zaninotto <luca.zaninotto@secomind.com>
lusergit
force-pushed
the
push-nrvnswvyxzqv
branch
from
159c707 to
66497cd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Horizontally clustering edgheog
Based on #1199
These changes aim at automatically scale edgheog when multiple nodes are able to see each other.
libclusterlibclusteris being added to allow different nodes to see each other. Depending on the deployment environment, different strategies can be selected to query services and discover other edgheog instances in the same cluster.hordeHordehas been added to allow registries with different processes to share the load across multiple replicas. This would allow a better management of active processes and automatically handles netsplits.Checklist
Further Comments
Communication between different services of edgehog happen trough the pubsub module (
Edgehog.PubSub), which internally uses thePhoenix.PubSub. This mechanism already shares messages between different replicas, hence messages are free to pass between one replica and the other(e.g., a campaign is started on one node, hence its process is active on one node, but a deployment is updated in another node. In this case the pubsub mechanism correctly messages all listening services, even the one on different nodes, and the services should be able to work as usual).
Optional: testing
Testing this new feature can be done by adding
In the
docker-compose.yamlfile