Skip to content

Commit 026710d

Browse files
Merge pull request #59 from LLMSQL/55-update-contributingmd-and-readmemd
readme and contributing updated
2 parents eb0b47f + e85f22a commit 026710d

File tree

4 files changed

+69
-21
lines changed

4 files changed

+69
-21
lines changed

CONTRIBUTING.md

Lines changed: 20 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,25 @@ There are several places information about LLMSQL package is located:
88

99
- Our README.md files and comments. We tried to put separate README.md in all important parts of our package. We hope they will help you to understand the package better.
1010
- We created [examples files](https://github.com/LLMSQL/llmsql-benchmark/tree/main/examples), feel free to explore them
11-
- Our [documentation pages](https://llmsql.github.io/llmsql-benchmark/) is currently under development but we will try to serve documentation soon.
11+
- Our [documentation pages](https://llmsql.github.io/llmsql-benchmark/).
12+
13+
- Feel free to reach us on any questions at our discord [LLMSQL discord](https://discord.gg/n9ygeKu6Ba)
14+
15+
## How Can I Get Involved?
16+
17+
To quickly get started, we maintain a list of good first issues, which can be found by [filtering GH Issues](https://github.com/LLMSQL/llmsql-benchmark/issues?q=is%3Aopen+label%3A%22good+first+issue%22+label%3A%22help+wanted%22). These are typically smaller code changes or self-contained features which can be added without extensive familiarity with library internals, and we recommend new contributors consider taking a stab at one of these first if they are feeling uncertain where to begin.
18+
19+
There are a number of distinct ways to contribute to LLMSQL, and all are extremely helpful! A sampling of ways to contribute include:
20+
21+
- **Improving documentation** - Improvements to the documentation, or noting pain points / gaps in documentation, are helpful in order for us to improve the user experience of the package and clarity + coverage of documentation.
22+
- **Testing and devops** - We are very grateful for any assistance in adding tests for the library that can be run for new PRs, and other devops workflows.
23+
- **Proposing or Contributing New Features** - We want LLMSQL to be the best way to interact (evaluate) models on our benchmark. If you have a feature that is not currently supported but desired, feel free to open an issue describing the feature and, if applicable, how you intend to implement it. We would be happy to give feedback on the cleanest way to implement new functionalities and are happy to coordinate with interested contributors.
24+
25+
If you want to resolve an issue , please follow the guidelines in [Code style](#code-style) section.
26+
27+
28+
We hope that this has been helpful, and appreciate your interest in contributing!
29+
1230

1331
## Code Style
1432

@@ -64,6 +82,7 @@ We recommend a few best practices to make your contributions or reported errors
6482

6583
**For Pull Requests:**
6684

85+
- If you contribute a change into [documentation](./docs/), please address your PR merge into `page` branch and not `main` branch. After merging to `page` your contribution will be merged to main.
6786
- PRs should be titled descriptively, and be opened with a brief description of the scope and intent of the new contribution.
6887
- New features should have appropriate documentation added alongside them.
6988
- Aim for code maintainability, and minimize code copying.
@@ -78,18 +97,3 @@ We recommend a few best practices to make your contributions or reported errors
7897
- Provide a *reproducible example* - what is the command you run with our package that results in this error? Have you tried any other steps to resolve it?
7998
- Provide a *full error traceback* of the error that occurs, if applicable. A one-line error message or small screenshot snippet is unhelpful without the surrounding context.
8099
- Note what version of the codebase you are using, and any specifics of your environment and setup that may be relevant.
81-
82-
83-
84-
85-
## How Can I Get Involved?
86-
87-
To quickly get started, we maintain a list of good first issues, which can be found by [filtering GH Issues](https://github.com/LLMSQL/llmsql-benchmark/issues?q=is%3Aopen+label%3A%22good+first+issue%22+label%3A%22help+wanted%22). These are typically smaller code changes or self-contained features which can be added without extensive familiarity with library internals, and we recommend new contributors consider taking a stab at one of these first if they are feeling uncertain where to begin.
88-
89-
There are a number of distinct ways to contribute to LLMSQL, and all are extremely helpful! A sampling of ways to contribute include:
90-
91-
- **Improving documentation** - Improvements to the documentation, or noting pain points / gaps in documentation, are helpful in order for us to improve the user experience of the package and clarity + coverage of documentation.
92-
- **Testing and devops** - We are very grateful for any assistance in adding tests for the library that can be run for new PRs, and other devops workflows.
93-
- **Proposing or Contributing New Features** - We want LLMSQL to be the best way to interact (evaluate) models on our benchmark. If you have a feature that is not currently supported but desired, feel free to open an issue describing the feature and, if applicable, how you intend to implement it. We would be happy to give feedback on the cleanest way to implement new functionalities and are happy to coordinate with interested contributors.
94-
95-
We hope that this has been helpful, and appreciate your interest in contributing!

README.md

Lines changed: 25 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,10 @@
1+
<picture align="center">
2+
<source media="(prefers-color-scheme: dark)" srcset="./assets/logo_black.svg">
3+
<img alt="Pandas Logo" src="./assets/logo_white.svg">
4+
</picture>
5+
6+
7+
18
![Downloads](https://img.shields.io/pypi/dm/llmsql)
29
[![codecov](https://codecov.io/gh/LLMSQL/llmsql-benchmark/branch/main/graph/badge.svg)](https://codecov.io/gh/LLMSQL/llmsql-benchmark)
310
![PyPI Version](https://img.shields.io/pypi/v/llmsql)
@@ -11,7 +18,6 @@ Patched and improved version of the original large crowd-sourced dataset for dev
1118

1219

1320
Our datasets are available for different scenarios on our [HuggingFace page](https://huggingface.co/llmsql-bench).
14-
---
1521

1622
## Overview
1723

@@ -32,7 +38,21 @@ The package doesn't have the dataset, it is stored on our [HuggingFace page](htt
3238
- Support for Hugging Face models out-of-the-box.
3339
- Structured for reproducibility and benchmarking.
3440

35-
---
41+
42+
43+
## Latest News 📣
44+
45+
<!-- * [2025/12] Evaluation class converted to function see [new `evaluate(...)` function](./llmsql/evaluation/evaluate.py#evaluate) -->
46+
47+
* New page version added to [`https://llmsql.github.io/llmsql-benchmark/`](https://llmsql.github.io/llmsql-benchmark/)
48+
49+
* Vllm inference method now supports chat templates, see [`inference_vllm(...)`](./llmsql/inference/inference_vllm.py#inference_vllm).
50+
* Transformers inference now supports custom chat tempalates with `chat_template` argument, see [`inference_transformers(...)`](./llmsql/inference/inference_transformers.py#inference_transformers)
51+
52+
* More stable and deterministic inference with [`inference_vllm(...)`](./llmsql/inference/inference_vllm.py#inference_vllm) function added by setting [some envars](./llmsql/inference/inference_vllm.py)
53+
54+
* `padding_side` argument added to [`inference_transformers(...)`](./llmsql/inference/inference_transformers.py#inference_transformers) function with default `left` option.
55+
3656

3757
## Usage Recommendations
3858

@@ -53,7 +73,7 @@ We therefore recommend that most users:
5373
5474
> [!Tip]
5575
> vllm based inference require vllm optional dependency group installed: `pip install llmsql[vllm]`
56-
---
76+
5777

5878
## Repository Structure
5979

@@ -151,11 +171,11 @@ print(report)
151171

152172
## Contributing
153173

154-
Check out our [open issues](https://github.com/LLMSQL/llmsql-benchmark/issues) and feel free to submit pull requests!
174+
Check out our [open issues](https://github.com/LLMSQL/llmsql-benchmark/issues), fork this repo and feel free to submit pull requests!
155175

156176
We also encourage you to submit new issues!
157177

158-
To get started with development, first fork the repository and install the dev dependencies.
178+
To get started with development, first fork the repository and install basic dependencies with dev dependencies.
159179

160180
For more information on the contributing: check [CONTRIBUTING.md](./CONTRIBUTING.md) and our [documentation page](https://llmsql.github.io/llmsql-benchmark/).
161181

assets/logo_black.svg

Lines changed: 12 additions & 0 deletions
Loading

assets/logo_white.svg

Lines changed: 12 additions & 0 deletions
Loading

0 commit comments

Comments
 (0)