Skip to content

added encoder to whisper function in LLMWhisperClient #11

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 30, 2024
Merged

added encoder to whisper function in LLMWhisperClient #11

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
5846fc5
added encoder to whisper function in LLMWhisperClient
ShoaibMajidDar Aug 14, 2024
c6e1b50
changed encoder to encoding and added encoding to whisper_status
ShoaibMajidDar Oct 17, 2024
f681656
changed encoder to encoding and added encoding to whisper_status
ShoaibMajidDar Oct 17, 2024
be02721
added encoding to whisper_retrieve and removed it from whisper_status
ShoaibMajidDar Oct 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions src/unstract/llmwhisperer/client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,7 @@ def whisper(
ocr_provider: str = "advanced",
line_splitter_tolerance: float = 0.4,
horizontal_stretch_factor: float = 1.0,
encoder = "ISO-8859-1"
) -> dict:
"""
Sends a request to the LLMWhisperer API to process a document.
Expand All @@ -190,6 +191,7 @@ def whisper(
ocr_provider (str, optional): The OCR provider. Can be "advanced" or "basic". Defaults to "advanced".
line_splitter_tolerance (float, optional): The line splitter tolerance. Defaults to 0.4.
horizontal_stretch_factor (float, optional): The horizontal stretch factor. Defaults to 1.0.
encoder (str): The character encoding to use for processing the text. Defaults to "ISO-8859-1".

Returns:
dict: The response from the API as a dictionary.
Expand Down Expand Up @@ -268,6 +270,7 @@ def generate():
prepared = req.prepare()
s = requests.Session()
response = s.send(prepared, timeout=self.api_timeout, stream=should_stream)
response.encoding = encoder
if response.status_code != 200 and response.status_code != 202:
message = json.loads(response.text)
message["status_code"] = response.status_code
Expand Down