Skip to content

Commit 6d7fa83

Browse files
committed
Update README.md
1 parent 565d802 commit 6d7fa83

File tree

1 file changed

+7
-3
lines changed

1 file changed

+7
-3
lines changed

README.md

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -25,14 +25,18 @@ The app supports three types of suggestion services:
2525

2626
- Models with chat completions API
2727
- Models with completions API
28+
- Models with FIM API
2829
- [Tabby](https://tabby.tabbyml.com)
2930

3031
If you are new to running a model locally, you can try [Ollama](https://ollama.com) and [LM Studio](https://lmstudio.ai).
3132

3233
### Recommended Settings
3334

3435
- Use Tabby since they have extensive experience in code completion.
35-
- Use models with completions API with Fill-in-the-Middle support (for example, codellama:7b-code), and use the "Codellama Fill-in-the-Middle" strategy.
36+
- Use models with completions API with Fill-in-the-Middle support (for example, codellama:7b-code), and use the "Fill-in-the-Middle" strategy.
37+
- Use models with FIM API.
38+
39+
When using custom models to generate suggestions, it is recommended to setup a lower suggestion limit for faster generation.
3640

3741
### Others
3842

@@ -53,8 +57,8 @@ The template format differs in different tools.
5357
- Default: This strategy meticulously explains the context to the model, prompting it to generate a suggestion.
5458
- Naive: This strategy rearranges the code in a naive way to trick the model into believing it's appending code at the end of a file.
5559
- Continue: This strategy employs the "Please Continue" technique to persuade the model that it has started a suggestion and must continue to complete it. (Only effective with the chat completion API).
56-
- CodeLlama Fill-in-the-Middle: It uses special tokens to guide the models to generate suggestions. The models need to support FIM to use it (codellama:xb-code, startcoder, etc.). This strategy uses the special tokens documented by CodeLlama.
57-
- CodeLlama Fill-in-the-Middle with System Prompt: The previous one doesn't have a system prompt telling it what to do. You can try to use it in models that don't support FIM.
60+
- Fill-in-the-Middle: It uses special tokens to guide the models to generate suggestions. The models need to support FIM to use it (codellama:xb-code, startcoder, etc.). You need to setup a prompt format to allow it to work properly. The default prompt format is for codellama.
61+
- Fill-in-the-Middle with System Prompt: The previous one doesn't have a system prompt telling it what to do. You can try to use it in models that don't support FIM.
5862

5963
## Contribution
6064

0 commit comments

Comments
 (0)