How to start the load generator when deploying in a Kubernetes cluster?

[riksanyal]

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [x] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

minikube start --cpus=4 --memory 4096 --disk-size 32g -p azure-demo
kubectl apply -f https://raw.githubusercontent.com/Azure-Samples/aks-store-demo/main/aks-store-all-in-one.yaml
cd src/store-front
docker compose up
do above for product-service and order-service

Any log messages given by the failure

There is no failure

Expected/desired behavior

Want the load to be generated. Currently I'm getting logs but the load is not there at all.

OS and Version?

Ubuntu 22.04.4 LTS

Versions

N/A

@pauldotyu

[pauldotyu]

@riksanyal are you looking to run the app using docker compose or run on a local kubernetes cluster? I see in the steps you listed above, you start minikube then do a docker compose up command.

Regardless of how you get the app running (on minikube or docker compose) you should start to see orders being submitted to the order-service via the virtual-worker pod that is deployed to the cluster. It is configured to submit 100 orders per hour and you should see the orders in rabbitmq.

Is that the "load generator" you were looking for or were you looking for something else?

[riksanyal]

@pauldotyu , thanks for getting back.

It seems like the the virtual-worker pod is unable to complete orders. Here is a sample:

No orders to process
No orders to process
No orders to process
Processing orders
Processing 1 orders
Order 30394 processed at 86.48s with status of 1. {"orderId":"30394","customerId":"1193056644","items":[{"productId":9,"quantity":3,"price":12.624281},{"productId":9,"quantity":2,"price":23.119621}],"status":1}
Failed to fetch orders: error sending request for url (http://makeline-service:3001/order/fetch)
Failed to fetch orders: error sending request for url (http://makeline-service:3001/order/fetch)
Processing orders
Processing 2 orders
Order 73582 processed at 257.16s with status of 1. {"orderId":"73582","customerId":"1072043992","items":[{"productId":4,"quantity":1,"price":41.91189},{"productId":3,"quantity":2,"price":49.651657},{"productId":6,"quantity":3,"price":88.467735}],"status":1}
Error completing the order: error sending request for url (http://makeline-service:3001/order)
Failed to fetch orders: error sending request for url (http://makeline-service:3001/order/fetch)
Processing orders
Processing 2 orders
Order 68728 processed at 444.36s with status of 1. {"orderId":"68728","customerId":"1947812545","items":[{"productId":2,"quantity":4,"price":7.727181},{"productId":2,"quantity":2,"price":5.9068127},{"productId":4,"quantity":1,"price":60.958},{"productId":4,"quantity":4,"price":10.1621}],"status":1}
Error completing the order: error sending request for url (http://makeline-service:3001/order)
Failed to fetch orders: error sending request for url (http://makeline-service:3001/order/fetch)
Failed to fetch orders: error sending request for url (http://makeline-service:3001/order/fetch)
Failed to fetch orders: error sending request for url (http://makeline-service:3001/order/fetch)
Processing orders
Processing 2 orders
Order 89625 processed at 765.71s with status of 1. {"orderId":"89625","customerId":"1915184211","items":[{"productId":9,"quantity":4,"price":46.730682},{"productId":8,"quantity":4,"price":13.804795},{"productId":5,"quantity":2,"price":96.82823},{"productId":7,"quantity":1,"price":3.4624524}],"status":1}
Error completing the order: error sending request for url (http://makeline-service:3001/order)
Failed to fetch orders: error sending request for url (http://makeline-service:3001/order/fetch)
Failed to fetch orders: error sending request for url (http://makeline-service:3001/order/fetch)
Failed to fetch orders: error sending request for url (http://makeline-service:3001/order/fetch)
Failed to fetch orders: error sending request for url (http://makeline-service:3001/order/fetch)

This is happening even though the makeline-service pod is healthy, but sometimes failing to insert/save/update orders. Here is the log:

[GIN] 2024/08/05 - 19:39:54 | 200 |      39.511µs |      10.244.0.1 | GET      "/health"
[GIN] 2024/08/05 - 19:39:54 | 200 |      29.121µs |      10.244.0.1 | GET      "/health"
[GIN] 2024/08/05 - 19:39:57 | 200 |      39.991µs |      10.244.0.1 | GET      "/health"
[GIN] 2024/08/05 - 19:39:59 | 200 |      37.362µs |      10.244.0.1 | GET      "/health"
2024/08/05 19:39:59 getting orders
2024/08/05 19:39:59 message received: {"customerId":"2027575324","items":[{"productId":8,"quantity":3,"price":12.219009},{"productId":6,"quantity":3,"price":21.816723},{"productId":6,"quantity":3,"price":84.62012}]}
2024/08/05 19:39:59 getting orders
[GIN] 2024/08/05 - 19:40:00 | 200 |      36.831µs |      10.244.0.1 | GET      "/health"
2024/08/05 19:40:00 no more orders for you: context deadline exceeded
[GIN] 2024/08/05 - 19:40:03 | 200 |      38.831µs |      10.244.0.1 | GET      "/health"
[GIN] 2024/08/05 - 19:40:04 | 200 |      38.351µs |      10.244.0.1 | GET      "/health"
[GIN] 2024/08/05 - 19:40:06 | 200 |      40.391µs |      10.244.0.1 | GET      "/health"
[GIN] 2024/08/05 - 19:40:09 | 200 |      39.901µs |      10.244.0.1 | GET      "/health"
[GIN] 2024/08/05 - 19:40:09 | 200 |       16.35µs |      10.244.0.1 | GET      "/health"
[GIN] 2024/08/05 - 19:40:12 | 200 |      36.791µs |      10.244.0.1 | GET      "/health"
[GIN] 2024/08/05 - 19:40:14 | 200 |      40.332µs |      10.244.0.1 | GET      "/health"
[GIN] 2024/08/05 - 19:40:15 | 200 |       32.44µs |      10.244.0.1 | GET      "/health"
[GIN] 2024/08/05 - 19:40:18 | 200 |      55.332µs |      10.244.0.1 | GET      "/health"
[GIN] 2024/08/05 - 19:40:19 | 200 |      33.941µs |      10.244.0.1 | GET      "/health"
[GIN] 2024/08/05 - 19:40:21 | 200 |      41.691µs |      10.244.0.1 | GET      "/health"
[GIN] 2024/08/05 - 19:40:24 | 200 |       31.89µs |      10.244.0.1 | GET      "/health"
[GIN] 2024/08/05 - 19:40:24 | 200 |      45.382µs |      10.244.0.1 | GET      "/health"
[GIN] 2024/08/05 - 19:40:27 | 200 |      38.511µs |      10.244.0.1 | GET      "/health"
[GIN] 2024/08/05 - 19:40:29 | 200 |      39.731µs |      10.244.0.1 | GET      "/health"
[GIN] 2024/08/05 - 19:40:30 | 200 |      37.211µs |      10.244.0.1 | GET      "/health"
2024/08/05 19:40:31 Failed to insert order: server selection error: server selection timeout, current topology: { Type: Single, Servers: [{ Addr: mongodb:27017, Type: Unknown, Last error: dial tcp 10.99.97.162:27017: connect: connection refused }, ] }
2024/08/05 19:40:31 Failed to save orders to database: server selection error: server selection timeout, current topology: { Type: Single, Servers: [{ Addr: mongodb:27017, Type: Unknown, Last error: dial tcp 10.99.97.162:27017: connect: connection refused }, ] }

Do you know how to resolve this connection refused problem?

[pauldotyu]

Ok, that's interesting... I haven't run into that in a while. In your original issue comment, you mentioned starting up minikube then applying the "aks-store-all-in-one.yaml" manifest and also running a docker compose up This will effectively run two instances of the solution, one within your docker environment and one in a local minikube cluster. Do you know which environment you were running into trouble? I can try to reproduce the error from there.