Skip to content

Commit 99a40a2

Browse files
authored
feat: Add progress bar for async LLM generation (#522)
* feat: Add progress bar for async LLM generation - Add tqdm_asyncio.gather() to show real-time progress during LLM calls - Minimal change: 1 import + 1 line modification - Shows block name, completed/total requests, elapsed time, and rate * style: Fix import sorting for ruff linter
1 parent d633ce3 commit 99a40a2

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

src/sdg_hub/core/blocks/llm/llm_chat_block.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77

88
from litellm import acompletion, completion
99
from pydantic import ConfigDict, Field, SecretStr, field_validator
10+
from tqdm.asyncio import tqdm_asyncio
1011
import litellm
1112

1213
# Third Party
@@ -502,7 +503,9 @@ async def _generate_async(
502503
for messages in messages_list
503504
]
504505

505-
responses = await asyncio.gather(*tasks)
506+
responses = await tqdm_asyncio.gather(
507+
*tasks, desc=self.block_name, unit="req"
508+
)
506509
return responses
507510

508511
except Exception as e:

0 commit comments

Comments
 (0)