1182 ml classification queue #1219
Conversation
| except requests.exceptions.RequestException as e: | ||
| return {"status": JobStatusEnum.FAILED, "message": f"API request failed: {str(e)}"} | ||
|
|
||
| def load_model(self, model_identifier: str) -> bool: |
There was a problem hiding this comment.
rename to request_load_model
| self.model_identifier = model_identifier | ||
|
|
||
| def _unload_all_other_models(self) -> bool: | ||
| """Unload all models except the one managed by this instance""" |
There was a problem hiding this comment.
Need to implement
_request_model_unload: sends the unload request
unload_model: calls request, then send status check request until confirmation of unload, with error handling if max retries met
unload_all_models:run _unload for all models
then we can implement a similar pattern on the loading side as well.
Lastly, lets merge model manager into apiclient, since there is no logical separation.
inference/models/inference.py
Outdated
| """Process this external job and update status/results""" | ||
| try: | ||
| api_client = InferenceAPIClient() | ||
| model_version = ModelVersion.objects.get(classification_type=self.inference_job.classification_type) |
There was a problem hiding this comment.
why do we need model version?
There was a problem hiding this comment.
Check the API documentation to see if a model identifier is needed in conjunction with the job_id
inference/models/inference.py
Outdated
| if new_status == ExternalJobStatus.COMPLETED: | ||
| self.store_results(response.get("results")) | ||
| elif new_status in [ExternalJobStatus.FAILED, ExternalJobStatus.CANCELLED]: | ||
| self.set_error(response.get("message", "")) |
There was a problem hiding this comment.
add note about failed or cancelled if no message is present.
inference/utils/batch.py
Outdated
| """Prepare single URL data for API""" | ||
| return { | ||
| "url_id": url.id, | ||
| "text": url.scraped_text[: self.config.max_text_length], |
There was a problem hiding this comment.
Need to refactor this. It should take in a max text length only, not a url count. And then it should grab as many urls as will fit into the max text length. If a single url exceeds the max text length, then that url can be sent by itself in a batch of 1.
…MPLETED jobs sent for dump to delta migration
Integrate classification queue
…A-IMPACT/COSMOS into 1182-ml-classification-queue
No description provided.