You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
-[2023/09] Medusa won the [Chai Prize Grant](https://twitter.com/tianle_cai/status/1703891335147897341)🎉 The prize will be used as a development bounty for those who help us achieve milestones in our [roadmap](https://github.com/FasterDecoding/Medusa/issues/3)!
11
+
-[2023/09] Medusa v0.1 is released!
11
12
12
13
---
13
14
## Introduction
@@ -78,7 +79,7 @@ In this initial release, our primary focus is on optimizing Medusa for a batch s
We currently support single-GPU inference with a batch size of 1, which is the most common setup for local model hosting. We are actively working to extend Medusa's capabilities by integrating it into other inference frameworks; please don't hesitate to reach out if you are interested in contributing to this effort.
97
98
98
-
You can use the following command for launching a CLI interface:
99
+
You can use the following command to launch a CLI interface:
99
100
```bash
100
101
CUDA_VISIBLE_DEVICES=0 python -m medusa.inference.cli --model [path of medusa model]
101
102
```
@@ -111,7 +112,7 @@ We take a public version of the ShareGPT dataset, which is a subset of the Vicun
Remark: If you haven't installed `git-lfs`, please install it before clone:
115
+
Remark: If you haven't installed `git-lfs`, please install it before cloning:
115
116
```bash
116
117
git lfs install
117
118
```
@@ -158,7 +159,7 @@ python -m medusa.hf_utils --folder [path of the model folder] --repo [name of th
158
159
```
159
160
160
161
## Codebase Guide
161
-
`medusa/model/medusa_model.py` is the key file for Medusa. It contains the `MedusaModel` class, which is a wrapper of the original model and the new heads. This class also has implementation of a streaming generation method. If you want to dive into the details of Medusa, this is the place to start.
162
+
`medusa/model/medusa_model.py` is the key file for Medusa. It contains the `MedusaModel` class, which is a wrapper of the original model and the new heads. This class also has an implementation of a streaming generation method. If you want to dive into the details of Medusa, this is the place to start.
162
163
163
164
We also provide some illustrative notebooks in `notebooks/` to help you understand the codebase.
0 commit comments