Fix message access in infer_requests#9073
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request modifies the distill_sampler.py file to use attribute-style access for messages instead of dictionary-style access. However, a critical issue was identified as infer_request is a dictionary, and this change will lead to a runtime AttributeError. It is recommended to revert this change.
| completion = self.client.chat.completions.create( | ||
| model=self.model, | ||
| messages=infer_request['messages'], | ||
| messages=infer_request.messages, |
There was a problem hiding this comment.
This change from dictionary-style access (infer_request['messages']) to attribute-style access (infer_request.messages) will cause a runtime AttributeError. The infer_requests list passed to this infer method is populated with dictionaries in DistillSampler.generate, not InferRequest objects. As infer_request is a dictionary, attribute access is not possible.
Please revert this change to use infer_request['messages'] to avoid breaking the code.
| messages=infer_request.messages, | |
| messages=infer_request['messages'], |
1 participant
PR type
PR information
Due to the definition of swift.infer_engine.protocol.InferRequest, the swift.pipelines.sampling.distill_sampler.OpenAIEngine may have to use a different way to access the infer_requests.