Skip to content

docker examples - add depends on and restart policy#1267

Merged
afsalthaj merged 2 commits intogolemcloud:mainfrom
justcoon:docker_compose_fixes
Jan 27, 2025
Merged

docker examples - add depends on and restart policy#1267
afsalthaj merged 2 commits intogolemcloud:mainfrom
justcoon:docker_compose_fixes

Conversation

@justcoon
Copy link
Contributor

@justcoon justcoon commented Jan 25, 2025 •
edited
Loading

added dependency between services and restart policy to avoid issues like

golem-worker-executor-1                | 2025-01-24T10:02:40.658396Z  WARN golem_common::client: gRPC call failed: Status { code: Unavailable, message: "tcp connect error: Connection refused (os error 111)", source: Some(tonic::transport::Error(Transport, ConnectError(ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })))) }, no more retries
golem-worker-executor-1                | 2025-01-24T10:02:40.658556Z  WARN retry{target="shard_manager" op="register" op_id="None" attempt=4}: golem_common::retries: op failure - retrying delay_ms=833 error="Unknown error: Registering with shard manager failed with status: Unavailable, message: \"tcp connect error: Connection refused (os error 111)\", details: [], metadata: MetadataMap { headers: {} }"
golem-worker-executor-1                | 2025-01-24T10:02:41.283919Z  WARN golem_worker_executor_base::services::scheduler: Skipping schedule, shard service is not ready
golem-worker-executor-1                | 2025-01-24T10:02:43.174435Z  WARN golem_common::client: gRPC call failed: Status { code: Unavailable, message: "tcp connect error: Connection refused (os error 111)", source: Some(tonic::transport::Error(Transport, ConnectError(ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })))) }, no more retries
golem-worker-executor-1                | 2025-01-24T10:02:43.174598Z ERROR retry{target="shard_manager" op="register" op_id="None" attempt=5}: golem_common::retries: op failure - no more retries error="Unknown error: Registering with shard manager failed with status: Unavailable, message: \"tcp connect error: Connection refused (os error 111)\", details: [], metadata: MetadataMap { headers: {} }"
golem-shard-manager-1                  | 2025-01-24T10:02:43.175622Z  WARN retry{target="worker_executor_grpc" op="healtcheck" op_id="f16a7ce8f787:9000 (172.21.0.4)" attempt=2}: golem_common::retries: op failure - retrying delay_ms=208 error="gRPC: error status: status: Unknown, message: \"transport error\", details: [], metadata: MetadataMap { headers: {} }"
golem-worker-executor-1                | Error: Unknown error: Registering with shard manager failed with status: Unavailable, message: "tcp connect error: Connection refused (os error 111)", details: [], metadata: MetadataMap { headers: {} }
golem-worker-executor-1                |
golem-worker-executor-1                | Stack backtrace:
golem-worker-executor-1                |    0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
golem-worker-executor-1                |    1: golem_worker_executor_base::Bootstrap::run_server::{{closure}}
golem-worker-executor-1                |    2: golem_worker_executor_base::Bootstrap::run::{{closure}}
golem-worker-executor-1                |    3: worker_executor::async_main::{{closure}}
golem-worker-executor-1                |    4: tokio::runtime::runtime::Runtime::block_on
golem-worker-executor-1                |    5: worker_executor::main
golem-worker-executor-1                |    6: std::sys::backtrace::__rust_begin_short_backtrace
golem-worker-executor-1                |    7: std::rt::lang_start::{{closure}}
golem-worker-executor-1                |    8: std::rt::lang_start_internal
golem-worker-executor-1                |    9: main
golem-worker-executor-1                |   10: <unknown>
golem-worker-executor-1                |   11: __libc_start_main
golem-worker-executor-1                |   12: _start
golem-worker-executor-1 exited with code 1
golem-shard-manager-1                  | 2025-01-24T10:02:45.388727Z  WARN retry{target="worker_executor_grpc" op="healtcheck" op_id="f16a7ce8f787:9000 (172.21.0.4)" attempt=3}: golem_common::retries: op failure - retrying delay_ms=421 error="gRPC: connect timeout"
golem-shard-manager-1                  | 2025-01-24T10:02:47.817623Z  WARN retry{target="worker_executor_grpc" op="healtcheck" op_id="f16a7ce8f787:9000 (172.21.0.4)" attempt=4}: golem_common::retries: op failure - retrying delay_ms=882 error="gRPC: connect timeout"
golem-shard-manager-1                  | 2025-01-24T10:02:50.704612Z ERROR retry{target="worker_executor_grpc" op="healtcheck" op_id="f16a7ce8f787:9000 (172.21.0.4)" attempt=5}: golem_common::retries: op failure - no more retries error="gRPC: connect timeout"
golem-shard-manager-1                  | 2025-01-24T10:02:50.704708Z  INFO golem_shard_manager::shard_management: Initial healthcheck finished

afsalthaj
afsalthaj previously approved these changes Jan 26, 2025
View reviewed changes
afsalthaj
afsalthaj requested changes Jan 26, 2025
View reviewed changes
Copy link
Contributor

@afsalthaj afsalthaj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given it's still not reliable, we will hold this one

@afsalthaj afsalthaj dismissed their stale review January 26, 2025 10:34

Not reliable yet

@justcoon
Copy link
Contributor Author

justcoon commented Jan 26, 2025 •
edited
Loading

added restart policy

golem-shard-manager-1                  | 2025-01-26T10:15:25.191477Z  INFO golem_common::tracing: Tracing initialized tracing_config="{\"stdout\":{\"enabled\":true,\"json\":false,\"json_flatten\":true,\"json_flatten_span\":true,\"ansi\":true,\"compact\":false,\"pretty\":false,\"without_time\":false,\"span_events_active\":false,\"span_events_full\":false},\"file\":{\"enabled\":false,\"json\":true,\"json_flatten\":true,\"json_flatten_span\":true,\"ansi\":false,\"compact\":false,\"pretty\":false,\"without_time\":false,\"span_events_active\":false,\"span_events_full\":false},\"file_dir\":null,\"file_name\":\"shard-manager.log\",\"file_truncate\":true,\"console\":false,\"dtor_friendly\":false}"
golem-shard-manager-1                  | 2025-01-26T10:15:25.201997Z  INFO golem_shard_manager: Golem Shard Manager starting up...
golem-shard-manager-1                  | 2025-01-26T10:15:25.202213Z  INFO golem_service_base::observability: Http server started on 0.0.0.0:8081
golem-shard-manager-1                  | 2025-01-26T10:15:25.202261Z  INFO golem_shard_manager: Using Redis at redis://redis:6380/0
golem-shard-manager-1                  | 2025-01-26T10:15:25.210180Z  INFO golem_shard_manager::shard_management: Initial healthcheck started
golem-shard-manager-1                  | 2025-01-26T10:15:27.212819Z  WARN retry{target="worker_executor_grpc" op="healtcheck" op_id="cb6b52291bc8:9000 (172.21.0.7)" attempt=1}: golem_common::retries: op failure - retrying delay_ms=102 error="gRPC: connect timeout"
golem-shard-manager-1                  | 2025-01-26T10:15:28.321953Z  WARN retry{target="worker_executor_grpc" op="healtcheck" op_id="cb6b52291bc8:9000 (172.21.0.7)" attempt=2}: golem_common::retries: op failure - retrying delay_ms=211 error="gRPC: transport error: transport error"
golem-shard-manager-1                  | 2025-01-26T10:15:30.538986Z  WARN retry{target="worker_executor_grpc" op="healtcheck" op_id="cb6b52291bc8:9000 (172.21.0.7)" attempt=3}: golem_common::retries: op failure - retrying delay_ms=414 error="gRPC: connect timeout"
golem-component-service-1              | 2025-01-26T10:15:30.683464Z  INFO golem_common::tracing: Tracing initialized tracing_config="{\"stdout\":{\"enabled\":true,\"json\":false,\"json_flatten\":true,\"json_flatten_span\":true,\"ansi\":true,\"compact\":false,\"pretty\":false,\"without_time\":false,\"span_events_active\":false,\"span_events_full\":false},\"file\":{\"enabled\":false,\"json\":true,\"json_flatten\":true,\"json_flatten_span\":true,\"ansi\":false,\"compact\":false,\"pretty\":false,\"without_time\":false,\"span_events_active\":false,\"span_events_full\":false},\"file_dir\":null,\"file_name\":\"component-service.log\",\"file_truncate\":true,\"console\":false,\"dtor_friendly\":false}"
golem-component-service-1              | 2025-01-26T10:15:30.683687Z  INFO opentelemetry_sdk:  name="MeterProvider.Built"
golem-component-service-1              | 2025-01-26T10:15:30.683731Z  INFO opentelemetry:  name="MeterProvider.GlobalSet" Global meter provider is set. Meters can now be created using global::meter() or global::meter_with_scope().
golem-component-service-1              | 2025-01-26T10:15:30.687747Z  INFO golem_component_service: Starting cloud server on ports: http: 8083, grpc: 9090
golem-component-service-1              | 2025-01-26T10:15:30.687905Z  INFO golem_service_base::db: DB migration: postgresql://postgres:5432/golem_db?currentSchema=golem_component
golem-component-service-1              | 2025-01-26T10:15:30.701291Z  INFO sqlx::postgres::notice: schema "golem_component" already exists, skipping
golem-component-service-1              | 2025-01-26T10:15:30.708563Z  INFO sqlx::postgres::notice: relation "_sqlx_migrations" already exists, skipping
golem-component-service-1              | 2025-01-26T10:15:30.711522Z  INFO golem_service_base::db: DB Pool: postgresql://postgres:5432/golem_db?currentSchema=golem_component
golem-component-service-1              | 2025-01-26T10:15:30.721477Z  INFO golem_component_service_base::service::component_object_store: FS Component Object Store root: /component_store, prefix:
golem-component-service-1              | 2025-01-26T10:15:30.732855Z  INFO poem::server: listening addr=socket://0.0.0.0:8083
golem-component-service-1              | 2025-01-26T10:15:30.732913Z  INFO poem::server: server started
golem-component-compilation-service-1  | 2025-01-26T10:15:30.932538Z  INFO golem_common::tracing: Tracing initialized tracing_config="{\"stdout\":{\"enabled\":true,\"json\":false,\"json_flatten\":true,\"json_flatten_span\":true,\"ansi\":true,\"compact\":false,\"pretty\":false,\"without_time\":false,\"span_events_active\":false,\"span_events_full\":false},\"file\":{\"enabled\":false,\"json\":true,\"json_flatten\":true,\"json_flatten_span\":true,\"ansi\":false,\"compact\":false,\"pretty\":false,\"without_time\":false,\"span_events_active\":false,\"span_events_full\":false},\"file_dir\":null,\"file_name\":\"component-compilation-service.log\",\"file_truncate\":true,\"console\":false,\"dtor_friendly\":false}"
golem-worker-executor-1                | 2025-01-26T10:15:30.932828Z  INFO golem_common::tracing: Tracing initialized tracing_config="{\"stdout\":{\"enabled\":true,\"json\":false,\"json_flatten\":true,\"json_flatten_span\":true,\"ansi\":true,\"compact\":false,\"pretty\":false,\"without_time\":false,\"span_events_active\":false,\"span_events_full\":false},\"file\":{\"enabled\":false,\"json\":true,\"json_flatten\":true,\"json_flatten_span\":true,\"ansi\":false,\"compact\":false,\"pretty\":false,\"without_time\":false,\"span_events_active\":false,\"span_events_full\":false},\"file_dir\":null,\"file_name\":\"worker-executor.9000.log\",\"file_truncate\":true,\"console\":false,\"dtor_friendly\":false}"
golem-component-compilation-service-1  | 2025-01-26T10:15:30.937863Z  INFO golem_component_compilation_service: Using local file system for blob storage at "/worker_executor_store"
golem-worker-executor-1                | 2025-01-26T10:15:30.939909Z  INFO golem_worker_executor: Golem Worker Executor starting up...
golem-worker-executor-1                | 2025-01-26T10:15:30.940023Z  INFO golem_worker_executor_base: Golem Worker Executor starting up...
golem-component-compilation-service-1  | 2025-01-26T10:15:30.940359Z  INFO golem_service_base::observability: Http server started on 0.0.0.0:8084
golem-worker-executor-1                | 2025-01-26T10:15:30.940368Z  INFO golem_worker_executor_base: Total system memory: 7.75 GiB, Available system memory: 5.91 GiB, Total memory available for workers: 6.20 GiB
golem-component-compilation-service-1  | 2025-01-26T10:15:30.940786Z  INFO golem_component_compilation_service: Server started on port 9091
golem-worker-executor-1                | 2025-01-26T10:15:30.940525Z  INFO golem_worker_executor_base: Using Redis for key-value storage at redis://redis:6380/0
golem-worker-executor-1                | 2025-01-26T10:15:30.940820Z  INFO golem_worker_executor_base: Using the same Redis for indexed-storage
golem-worker-executor-1                | 2025-01-26T10:15:30.940877Z  INFO golem_worker_executor_base: Using local file system for blob storage at "/worker_executor_store"
golem-worker-executor-1                | 2025-01-26T10:15:30.943024Z  INFO golem_worker_executor_base::services::component: Using component API at http://golem-component-service:9090/
golem-shard-manager-1                  | 2025-01-26T10:15:30.955980Z  WARN retry{target="worker_executor_grpc" op="healtcheck" op_id="cb6b52291bc8:9000 (172.21.0.7)" attempt=4}: golem_common::retries: op failure - retrying delay_ms=842 error="gRPC: transport error: transport error"
golem-worker-executor-1                | 2025-01-26T10:15:30.958091Z  INFO golem_worker_executor_base::grpc: Registering worker executor host="cb6b52291bc8" port=9000
golem-worker-service-1                 | 2025-01-26T10:15:31.650036Z  INFO golem_common::tracing: Tracing initialized tracing_config="{\"stdout\":{\"enabled\":true,\"json\":false,\"json_flatten\":true,\"json_flatten_span\":true,\"ansi\":true,\"compact\":false,\"pretty\":false,\"without_time\":false,\"span_events_active\":false,\"span_events_full\":false},\"file\":{\"enabled\":false,\"json\":true,\"json_flatten\":true,\"json_flatten_span\":true,\"ansi\":false,\"compact\":false,\"pretty\":false,\"without_time\":false,\"span_events_active\":false,\"span_events_full\":false},\"file_dir\":null,\"file_name\":\"worker-service.log\",\"file_truncate\":true,\"console\":false,\"dtor_friendly\":false}"
golem-worker-service-1                 | 2025-01-26T10:15:31.650280Z  INFO opentelemetry_sdk:  name="MeterProvider.Built"
golem-worker-service-1                 | 2025-01-26T10:15:31.650326Z  INFO opentelemetry:  name="MeterProvider.GlobalSet" Global meter provider is set. Meters can now be created using global::meter() or global::meter_with_scope().
golem-worker-service-1                 | 2025-01-26T10:15:31.656351Z  INFO golem_service_base::db: DB migration: postgresql://postgres:5432/golem_db?currentSchema=golem_worker
golem-worker-service-1                 | 2025-01-26T10:15:31.668754Z  INFO sqlx::postgres::notice: schema "golem_worker" already exists, skipping
golem-worker-service-1                 | 2025-01-26T10:15:31.673997Z  INFO sqlx::postgres::notice: relation "_sqlx_migrations" already exists, skipping
golem-worker-service-1                 | 2025-01-26T10:15:31.677039Z  INFO golem_service_base::db: DB Pool: postgresql://postgres:5432/golem_db?currentSchema=golem_worker
golem-worker-service-1                 | 2025-01-26T10:15:31.701623Z  INFO poem::server: listening addr=socket://0.0.0.0:9005
golem-worker-service-1                 | 2025-01-26T10:15:31.701690Z  INFO poem::server: server started
golem-worker-service-1                 | 2025-01-26T10:15:31.701714Z  INFO poem::server: listening addr=socket://0.0.0.0:9006
golem-worker-service-1                 | 2025-01-26T10:15:31.701723Z  INFO poem::server: server started
golem-worker-executor-1                | 2025-01-26T10:15:32.591092Z  WARN golem_common::client: gRPC call failed: Status { code: Unavailable, message: "tcp connect error: Connection refused (os error 111)", source: Some(tonic::transport::Error(Transport, ConnectError(ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })))) }, no more retries
golem-worker-executor-1                | 2025-01-26T10:15:32.591250Z  WARN retry{target="shard_manager" op="register" op_id="None" attempt=1}: golem_common::retries: op failure - retrying delay_ms=105 error="Unknown error: Registering with shard manager failed with status: Unavailable, message: \"tcp connect error: Connection refused (os error 111)\", details: [], metadata: MetadataMap { headers: {} }"
golem-worker-executor-1                | 2025-01-26T10:15:32.954460Z  WARN golem_worker_executor_base::services::scheduler: Skipping schedule, shard service is not ready
golem-worker-executor-1                | 2025-01-26T10:15:34.340157Z  WARN golem_common::client: gRPC call failed: Status { code: Unavailable, message: "tcp connect error: Connection refused (os error 111)", source: Some(tonic::transport::Error(Transport, ConnectError(ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })))) }, no more retries
golem-worker-executor-1                | 2025-01-26T10:15:34.340304Z  WARN retry{target="shard_manager" op="register" op_id="None" attempt=2}: golem_common::retries: op failure - retrying delay_ms=202 error="Unknown error: Registering with shard manager failed with status: Unavailable, message: \"tcp connect error: Connection refused (os error 111)\", details: [], metadata: MetadataMap { headers: {} }"
golem-worker-executor-1                | 2025-01-26T10:15:34.956756Z  WARN golem_worker_executor_base::services::scheduler: Skipping schedule, shard service is not ready
golem-worker-executor-1                | 2025-01-26T10:15:36.146171Z  WARN golem_common::client: gRPC call failed: Status { code: Unavailable, message: "tcp connect error: Connection refused (os error 111)", source: Some(tonic::transport::Error(Transport, ConnectError(ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })))) }, no more retries
golem-worker-executor-1                | 2025-01-26T10:15:36.146291Z  WARN retry{target="shard_manager" op="register" op_id="None" attempt=3}: golem_common::retries: op failure - retrying delay_ms=458 error="Unknown error: Registering with shard manager failed with status: Unavailable, message: \"tcp connect error: Connection refused (os error 111)\", details: [], metadata: MetadataMap { headers: {} }"
golem-worker-executor-1                | 2025-01-26T10:15:36.958712Z  WARN golem_worker_executor_base::services::scheduler: Skipping schedule, shard service is not ready
golem-worker-executor-1                | 2025-01-26T10:15:38.240763Z  WARN golem_common::client: gRPC call failed: Status { code: Unavailable, message: "tcp connect error: Connection refused (os error 111)", source: Some(tonic::transport::Error(Transport, ConnectError(ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })))) }, no more retries
golem-worker-executor-1                | 2025-01-26T10:15:38.240869Z  WARN retry{target="shard_manager" op="register" op_id="None" attempt=4}: golem_common::retries: op failure - retrying delay_ms=834 error="Unknown error: Registering with shard manager failed with status: Unavailable, message: \"tcp connect error: Connection refused (os error 111)\", details: [], metadata: MetadataMap { headers: {} }"
golem-worker-executor-1                | 2025-01-26T10:15:38.960608Z  WARN golem_worker_executor_base::services::scheduler: Skipping schedule, shard service is not ready
golem-worker-executor-1                | 2025-01-26T10:15:40.665739Z  WARN golem_common::client: gRPC call failed: Status { code: Unavailable, message: "tcp connect error: Connection refused (os error 111)", source: Some(tonic::transport::Error(Transport, ConnectError(ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })))) }, no more retries
golem-worker-executor-1                | 2025-01-26T10:15:40.665844Z ERROR retry{target="shard_manager" op="register" op_id="None" attempt=5}: golem_common::retries: op failure - no more retries error="Unknown error: Registering with shard manager failed with status: Unavailable, message: \"tcp connect error: Connection refused (os error 111)\", details: [], metadata: MetadataMap { headers: {} }"
golem-shard-manager-1                  | 2025-01-26T10:15:40.666508Z ERROR retry{target="worker_executor_grpc" op="healtcheck" op_id="cb6b52291bc8:9000 (172.21.0.7)" attempt=5}: golem_common::retries: op failure - no more retries error="gRPC: error status: status: Unknown, message: \"transport error\", details: [], metadata: MetadataMap { headers: {} }"
golem-shard-manager-1                  | 2025-01-26T10:15:40.666625Z  INFO golem_shard_manager::shard_management: Initial healthcheck finished
golem-shard-manager-1                  | 2025-01-26T10:15:40.666644Z  INFO golem_shard_manager: Starting health check process...
golem-shard-manager-1                  | 2025-01-26T10:15:40.666651Z  INFO golem_shard_manager: Shard Manager is fully operational.
golem-shard-manager-1                  | 2025-01-26T10:15:40.666657Z  INFO golem_shard_manager: The port read from env is 9002
golem-shard-manager-1                  | 2025-01-26T10:15:40.666691Z  INFO golem_shard_manager: Server started on port 9002
golem-shard-manager-1                  | 2025-01-26T10:15:40.666754Z  INFO golem_shard_manager::shard_management: Pod removed pod=cb6b52291bc8:9000 (172.21.0.7)
golem-shard-manager-1                  | 2025-01-26T10:15:40.667286Z  INFO golem_shard_manager::shard_management: Shard manager beginning rebalance...
golem-shard-manager-1                  | 2025-01-26T10:15:40.667497Z  INFO golem_shard_manager::shard_management: Executing shard unassignments unassignments=[]
golem-shard-manager-1                  | 2025-01-26T10:15:40.667514Z  INFO golem_shard_manager::shard_management: Executing shard assignments assignments=[]
golem-worker-executor-1                | Error: Unknown error: Registering with shard manager failed with status: Unavailable, message: "tcp connect error: Connection refused (os error 111)", details: [], metadata: MetadataMap { headers: {} }
golem-worker-executor-1                |
golem-worker-executor-1                | Stack backtrace:
golem-worker-executor-1                |    0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
golem-worker-executor-1                |    1: golem_worker_executor_base::Bootstrap::run_server::{{closure}}
golem-worker-executor-1                |    2: golem_worker_executor_base::Bootstrap::run::{{closure}}
golem-worker-executor-1                |    3: worker_executor::async_main::{{closure}}
golem-worker-executor-1                |    4: tokio::runtime::runtime::Runtime::block_on
golem-worker-executor-1                |    5: worker_executor::main
golem-worker-executor-1                |    6: std::sys::backtrace::__rust_begin_short_backtrace
golem-worker-executor-1                |    7: std::rt::lang_start::{{closure}}
golem-worker-executor-1                |    8: std::rt::lang_start_internal
golem-worker-executor-1                |    9: main
golem-worker-executor-1                |   10: <unknown>
golem-worker-executor-1                |   11: __libc_start_main
golem-worker-executor-1                |   12: _start
golem-worker-executor-1 exited with code 1
golem-worker-executor-1                | 2025-01-26T10:15:41.067918Z  INFO golem_worker_executor_base::grpc: Registering worker executor host="cb6b52291bc8" port=9000
golem-shard-manager-1                  | 2025-01-26T10:15:41.069769Z  INFO api_request{api="register" api_type="grpc" source_ip="172.21.0.7:60426" host="cb6b52291bc8" port="9000"}: golem_shard_manager: Shard Manager received request to register pod: cb6b52291bc8:9000 (172.21.0.7)
golem-shard-manager-1                  | 2025-01-26T10:15:41.069808Z  INFO api_request{api="register" api_type="grpc" source_ip="172.21.0.7:60426" host="cb6b52291bc8" port="9000"}: golem_common::metrics::api: API request succeeded elapsed_ms=0
golem-shard-manager-1                  | 2025-01-26T10:15:41.069938Z  INFO golem_shard_manager::shard_management: Pod added pod=cb6b52291bc8:9000 (172.21.0.7)
golem-shard-manager-1                  | 2025-01-26T10:15:41.070489Z  INFO golem_shard_manager::shard_management: Shard manager beginning rebalance...
golem-worker-executor-1                | 2025-01-26T10:15:41.070441Z  INFO retry{target="shard_manager" op="register" op_id="None" attempt=1}: golem_common::retries: op success duration_ms=2
golem-shard-manager-1                  | 2025-01-26T10:15:41.070529Z  INFO golem_shard_manager::shard_management: Executing shard unassignments unassignments=[]
golem-worker-executor-1                | 2025-01-26T10:15:41.070525Z  INFO golem_worker_executor_base::grpc: Registered worker executor, waiting for shard assignment...
golem-shard-manager-1                  | 2025-01-26T10:15:41.070542Z  INFO golem_shard_manager::shard_management: Executing shard assignments assignments=[cb6b52291bc8:9000 (172.21.0.7): [<0>..<1023>]]
golem-worker-executor-1                | 2025-01-26T10:15:41.070551Z  INFO golem_worker_executor_base::durable_host: Recovering workers
golem-shard-manager-1                  | 2025-01-26T10:15:41.070576Z  INFO golem_shard_manager::worker_executor: Assigning shards assigned_shards="cb6b52291bc8:9000 (172.21.0.7): [<0>..<1023>]"
golem-worker-executor-1                | 2025-01-26T10:15:41.070558Z  INFO golem_worker_executor_base::durable_host: Finished recovering workers
golem-worker-executor-1                | 2025-01-26T10:15:41.070564Z  INFO golem_worker_executor_base: Starting gRPC server on port 9000
golem-worker-executor-1                | 2025-01-26T10:15:41.070721Z  INFO golem_service_base::observability: Http server started on 0.0.0.0:8082
golem-worker-executor-1                | 2025-01-26T10:15:41.072219Z  INFO api_request{api="assign_shards" api_type="grpc"}: golem_worker_executor_base::durable_host: Recovering workers
golem-worker-executor-1                | 2025-01-26T10:15:41.264554Z  INFO api_request{api="assign_shards" api_type="grpc"}: golem_worker_executor_base::durable_host: Finished recovering workers
golem-worker-executor-1                | 2025-01-26T10:15:41.264611Z  INFO api_request{api="assign_shards" api_type="grpc"}: golem_common::metrics::api: API request succeeded elapsed_ms=192
golem-shard-manager-1                  | 2025-01-26T10:15:41.265045Z  INFO retry{target="worker_executor" op="assign_shards" op_id="cb6b52291bc8:9000 (172.21.0.7)" attempt=1}: golem_common::retries: op success duration_ms=194
golem-shard-manager-1                  | 2025-01-26T10:15:50.670822Z  INFO retry{target="worker_executor_grpc" op="healtcheck" op_id="cb6b52291bc8:9000 (172.21.0.7)" attempt=1}: golem_common::retries: op success duration_ms=1

cc: @afsalthaj

@justcoon justcoon changed the title docker examples - add depends on docker examples - add depends on and restart policy Jan 26, 2025
@justcoon justcoon requested a review from afsalthaj January 26, 2025 11:24
@afsalthaj afsalthaj merged commit 0b27c9d into golemcloud:main Jan 27, 2025
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vigoo vigoo vigoo approved these changes

@afsalthaj afsalthaj Awaiting requested review from afsalthaj

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@justcoon @vigoo @afsalthaj