feat: add job monitoring endpoint /backends/jobs #8095
Conversation
✅ Deploy Preview for localai ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
| var once sync.Once | ||
|
|
||
| //Return singleton instance of the JobStore | ||
| func GetStore() *JobStore{ |
There was a problem hiding this comment.
This looks basically superseded by https://github.com/mudler/LocalAI/blob/master/core/http/middleware/trace.go which is used to trace instead any requests (useful for dataset extraction).
@pmarini-nc, wouldn't the current tracing feature already satisfy your issue?
There was a problem hiding this comment.
@mudler I have analyzed core/http/middleware/trace.go.
While trace.go is excellent for historical analysis and debugging, it captures the APIExchange after the request handler returns (Lines 100-125).
Why this PR is distinct and necessary:
The goal of issue #7906 is to have a "top-like" monitoring view. The key requirement is to see jobs while they are executing (e.g., long-running inference or downloads).
Trace: Shows what happened (Past tense).
Job Monitor: Shows what is happening (Present tense).
If we use trace.go, the API user won't see the job until it is already finished, which defeats the purpose of monitoring live resource usage.
This PR inserts the job into the store before next(c) is called, allowing real-time visibility into running processes. I believe keeping this lightweight "Active Job Store" separate from the heavy "Historical Trace Buffer" is the cleanest approach.
@mudler , how can I test it? |
3 participants
Description
This PR addresses issue #7906 by implementing a job tracking system. It allows users to monitor running AI jobs (like chat completions, transcriptions, etc.) via a new API endpoint.
Changes
core/http/jobs: Implemented a thread-safe in-memory store to track job status, start time, and metadata.JobTrackerMiddlewarewhich intercepts/v1/and/api/requests to automatically register them in the job store.GET /backends/jobswhich returns the list of active and recently finished jobs.Related Issue
Fixes #7906
Testing
curlby sending a POST request to/v1/chat/completions./backends/jobsreturns the job details with statusexecuting(during processing) andfinishedorerror(after completion).Sample Output:
[ { "job_id": "4cb9b834-d190-47dc-87b3-b8cfb70f6ab0", "type": "/v1/chat/completions", "model": "gpt-4", "start_time": "2026-01-18T17:09:18.72755107Z", "status": "executing", "client_ip": "4.240.39.193" } ]