|
| 1 | +# Contributing to LLMSQL |
| 2 | + |
| 3 | +Welcome and thank you for your interest in the LLMSQL package! We welcome contributions, feedback, new issues, pull requests and appreciate your time spent with our package, and hope you find it useful! |
| 4 | + |
| 5 | +## Important Resources |
| 6 | + |
| 7 | +There are several places information about LLMSQL package is located: |
| 8 | + |
| 9 | +- Our README.md files and comments. We tried to put separate README.md in all important parts of our package. We hope they will help you to understand the package better. |
| 10 | +- We created [examples files](https://github.com/LLMSQL/llmsql-benchmark/tree/main/examples), feel free to explore them |
| 11 | +- Our [documentation pages](https://llmsql.github.io/llmsql-benchmark/) is currently under development but we will try to serve documentation soon. |
| 12 | + |
| 13 | +## Code Style |
| 14 | + |
| 15 | +LLMSQL uses [ruff](https://github.com/astral-sh/ruff) for linting, [mypy](https://mypy.readthedocs.io/en/stable/) for type checking and [pdm](https://pdm-project.org/en/latest/) for dependency management. |
| 16 | + |
| 17 | +1. Fork this repository. |
| 18 | + |
| 19 | +2. Install [pdm](https://pdm-project.org/) (Python dependency manager): |
| 20 | + |
| 21 | +```bash |
| 22 | +pip3 install --user pdm |
| 23 | +```` |
| 24 | + |
| 25 | +3. Clone your fork and install the development dependencies: |
| 26 | + |
| 27 | +```bash |
| 28 | +git clone https://github.com/<YOUR_USERNAME>/llmsql-benchmark.git |
| 29 | +cd llmsql-benchmark |
| 30 | +pdm install --without default --with dev |
| 31 | +``` |
| 32 | + |
| 33 | +> **Note**: If you see `Command 'pdm' not found`, it usually means `~/.local/bin` is not on your `PATH`. |
| 34 | +> |
| 35 | +> * On **Linux**, you can run: |
| 36 | +> |
| 37 | +> ```bash |
| 38 | +> export PATH="$HOME/.local/bin:$PATH" |
| 39 | +> ``` |
| 40 | +> |
| 41 | +> (add this line to your shell config, e.g. `~/.bashrc`, `~/.zshrc`, or for fish: `set -U fish_user_paths $HOME/.local/bin $fish_user_paths`) |
| 42 | + |
| 43 | + |
| 44 | + |
| 45 | +Great🎉! You are all set for developing! |
| 46 | + |
| 47 | + |
| 48 | +## Testing |
| 49 | + |
| 50 | +We use [pytest](https://docs.pytest.org/en/latest/) for running unit tests. All library unit tests can be run via: |
| 51 | + |
| 52 | +```bash |
| 53 | +PYTHONPATH=. pdm run pytest --cov=llmsql --cov-report=xml --maxfail=1 --disable-warnings -v |
| 54 | +``` |
| 55 | + |
| 56 | +Please run this command before any changes, just to make sure all code you forked works well by the time the development starts. |
| 57 | + |
| 58 | + |
| 59 | +## Contribution Best Practices |
| 60 | + |
| 61 | +We recommend a few best practices to make your contributions or reported errors easier to assist with. |
| 62 | + |
| 63 | +**For Pull Requests:** |
| 64 | + |
| 65 | +- PRs should be titled descriptively, and be opened with a brief description of the scope and intent of the new contribution. |
| 66 | +- New features should have appropriate documentation added alongside them. |
| 67 | +- Aim for code maintainability, and minimize code copying. |
| 68 | + |
| 69 | +**For Feature Requests:** |
| 70 | + |
| 71 | +- Provide a short paragraph's worth of description. What is the feature you are requesting? What is its motivation, and an example use case of it? How does this differ from what is currently supported? |
| 72 | +
|
| 73 | +**For Bug Reports**: |
| 74 | +
|
| 75 | +- Provide a short description of the bug. |
| 76 | +- Provide a *reproducible example* - what is the command you run with our package that results in this error? Have you tried any other steps to resolve it? |
| 77 | +- Provide a *full error traceback* of the error that occurs, if applicable. A one-line error message or small screenshot snippet is unhelpful without the surrounding context. |
| 78 | +- Note what version of the codebase you are using, and any specifics of your environment and setup that may be relevant. |
| 79 | +
|
| 80 | +
|
| 81 | +
|
| 82 | +
|
| 83 | +## How Can I Get Involved? |
| 84 | +
|
| 85 | +To quickly get started, we maintain a list of good first issues, which can be found by [filtering GH Issues](https://github.com/LLMSQL/llmsql-benchmark/issues?q=is%3Aopen+label%3A%22good+first+issue%22+label%3A%22help+wanted%22). These are typically smaller code changes or self-contained features which can be added without extensive familiarity with library internals, and we recommend new contributors consider taking a stab at one of these first if they are feeling uncertain where to begin. |
| 86 | +
|
| 87 | +There are a number of distinct ways to contribute to LLMSQL, and all are extremely helpful! A sampling of ways to contribute include: |
| 88 | +
|
| 89 | +- **Improving documentation** - Improvements to the documentation, or noting pain points / gaps in documentation, are helpful in order for us to improve the user experience of the package and clarity + coverage of documentation. |
| 90 | +- **Testing and devops** - We are very grateful for any assistance in adding tests for the library that can be run for new PRs, and other devops workflows. |
| 91 | +- **Proposing or Contributing New Features** - We want LLMSQL to be the best way to interact (evaluate) models on our benchmark. If you have a feature that is not currently supported but desired, feel free to open an issue describing the feature and, if applicable, how you intend to implement it. We would be happy to give feedback on the cleanest way to implement new functionalities and are happy to coordinate with interested contributors. |
| 92 | +
|
| 93 | +We hope that this has been helpful, and appreciate your interest in contributing! |
0 commit comments