Skip to content

Commit 0afc9c0

Browse files
authored
Merge pull request #35 from genlm/clean
Small changes -- remove warning, tidy docs
2 parents 4e308ab + a494dbc commit 0afc9c0

File tree

2 files changed

+2
-5
lines changed

2 files changed

+2
-5
lines changed

docs/index.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
[![codecov](https://codecov.io/github/genlm/genlm-backend/graph/badge.svg?token=PwmHwMJC2y)](https://codecov.io/github/genlm/genlm-backend)
77
[![PyPI](https://img.shields.io/pypi/v/genlm-backend?label=pypi)](https://pypi.org/project/genlm-backend/)
88

9-
GenLM Backend is a high-performance inference backend for language model probabilistic programs, built for seamless integration with the GenLM ecosystem. It provides an **asynchronous**, **autobatched** interface to LLMs served by `vllm` or `transformers`, enabling scalable and efficient inference.
9+
GenLM Backend is a high-performance backend for language model probabilistic programs, built for the GenLM ecosystem. It provides an **asynchronous** and **autobatched** interface to `vllm` and `transformers` language models, enabling scalable and efficient inference.
1010

1111
## Key Features
1212
- Automatic batching of concurrent log-probability requests—enabling efficient large-scale inference without having to write batching logic yourself
@@ -24,7 +24,7 @@ pip install genlm-backend
2424

2525
## Example: Autobatched Sequential Importance Sampling with LLMs
2626

27-
This example demonstrates how `genlm-backend` enables concise, scalable probabilistic inference with language models. It implements a Sequential Importance Sampling (SIS) algorithm in which inference is performed via asynchronous, automatically batched calls to a language model.
27+
This example demonstrates how `genlm-backend` enables concise, scalable probabilistic inference with language models. It implements a Sequential Importance Sampling (SIS) algorithm that makes asynchronous log-probabality requests which get automatically batched by the language model.
2828

2929

3030
```python

genlm/backend/llm/vllm.py

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21,9 +21,6 @@
2121
HAS_VLLM = True
2222
except ImportError: # pragma: no cover
2323
HAS_VLLM = False # pragma: no cover
24-
warnings.warn( # pragma: no cover
25-
"vLLM not installed. Run 'pip install vllm' to use the vLLM-based AsyncLM model."
26-
)
2724

2825
if not HAS_VLLM:
2926

0 commit comments

Comments
 (0)