[query/joins] Datagen jobs never completes #71
Description
Steps to reproduce:
- Just start a new docker setup
- run the entire notebook
03-query/11-joins.ipynb - We will be struck at the triggering the users job.
Even though, clickstream job has finished and we move on to generating the user data, curl localhost:9999/jobs shows that clicks job still has two active sessions and below is the output .
[
{
"name": "clicks",
"config_file": "clickstream/clickstream.json",
"target": {
"type": "file",
"path": "/files/clicks.json"
},
"active_sessions": 2,
"total_records": 1725122,
"start_time": "2024-02-13 05:15:01",
"run_time": 172881.522483,
"status": "COMPLETE",
"status_msg": "Running, Sim Clock: 2024-02-15 05:16:20.150717"
},
{
"name": "users",
"config_file": "clickstream/users_init.json",
"target": {
"type": "file",
"path": "/files/users.json"
},
"active_sessions": 1967,
"total_records": 1967,
"start_time": "2024-02-15 05:16:19",
"run_time": 20.169898,
"status": "RUNNING",
"status_msg": "Starting generator job."
}
]
I tried playing around the concurrency, # of events and hours of data but the issue still exists.