Merge pull request #233 from Kiln-AI/main

Update Docs for v0.12.0

Author: scosman

.cursorrules

@@ -1,3 +1,4 @@
  - Always assume pydantic 2 (not pydantic 1)
  - Always use pytest for tests
+ - The project supports Python 3.10 and above
 

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,45 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: "[Bug] "
+labels: bug
+assignees: ""
+---
+
+**Describe the bug**
+Insert a clear and concise description of what the bug is.
+
+**Checks**
+
+- [ ] I've read the [troubleshooting guide](https://docs.getkiln.ai/docs/troubleshooting-and-logs)
+- [ ] I've tried to reproduce the problem using another model, and confirmed it's not an issue specific to the model I've chosen.
+- [ ] I've searched [the docs](https://docs.getkiln.ai) for a solution
+- [ ] I've searched for existing Github issues/discussions
+
+**To Reproduce**
+Steps to reproduce the behavior:
+
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Error Logs**
+Please include the logs if the issue shows an error. State that no error is shown if there is no error.
+
+https://app.gitbook.com/o/63UPvNXRQ4QyV9E5eioW/s/EJ4b8A4QiEQlOGbYkXDX/docs/troubleshooting-and-logs
+
+**System Information:**
+
+- OS: [e.g. Windows 11, MacOS]
+- Browser [e.g. chrome, safari]
+- Kiln app Version [e.g. v0.11.1]
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,25 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: "[Feature Request]"
+labels: enhancement
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Checks**
+
+- [ ] I've searched [the docs](https://docs.getkiln.ai) for a solution
+- [ ] I've searched for existing Github issues
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/workflows/build_desktop.yml

@@ -12,9 +12,10 @@ jobs:
         # macos-latest is for arm64 build, macos-13 is for x86_64
         include:
           - os: windows-latest
-          - os: macos-latest # arm64
-          - os: macos-13 # x86_64
-          - os: ubuntu-latest # x86_64 (no arm64 support on GH Actions yet)
+          - os: macos-latest # arm64 macOS
+          - os: macos-13 # x86_64 macOS
+          - os: ubuntu-22.04 # x86_64 linux, oldest available GH runner. Older libc for maximal compatibility
+          - os: ubuntu-22.04-arm # arm linux build
 
     steps:
       - uses: actions/checkout@v4
@@ -38,7 +39,7 @@ jobs:
 
       # Compress MacOS app param ignored on other platforms
       - name: Build Desktop App
-        run: uv run bash ./app/desktop/build_desktop_app.sh --compress-mac-app
+        run: uv run bash ./app/desktop/build_desktop_app.sh
 
       - name: Build Windows Installer
         if: matrix.os == 'windows-latest'
@@ -50,6 +51,31 @@ jobs:
         if: matrix.os == 'windows-latest'
         run: cp ./app/desktop/Output/kilnsetup.exe ./app/desktop/build/dist/Kiln.Windows.Installer.exe
 
+      # Print checksums, so anyone can verify the build
+
+      - name: Checksums (macOS)
+        if: runner.os == 'macOS'
+        run: find ./app/desktop/build/dist/Kiln.app -type f -not -path "*/Contents/Resources/botocore/data/*" -exec md5 {} \;
+
+      - name: Checksums (Linux)
+        if: runner.os == 'Linux'
+        run: find ./app/desktop/build/dist -type f -exec sha256sum {} \;
+
+      - name: Checksums (Windows)
+        if: runner.os == 'Windows'
+        run: |
+          certutil -hashfile ./app/desktop/build/dist/Kiln.Windows.Installer.exe SHA256
+          certutil -hashfile ./app/desktop/build/dist/Kiln/Kiln.exe SHA256
+
+      # MacOS apps have symlinks, and GitHub artifact upload zip will break them. Tar instead.
+      - name: Compress MacOS App
+        if: runner.os == 'macOS'
+        working-directory: ./app/desktop/build/dist
+        run: |
+          echo "Compressing MacOS app"
+          tar czpvf Kiln.app.tgz Kiln.app
+          rm -r Kiln.app
+
       - name: Upload Build
         uses: actions/upload-artifact@v4
         with:

LICENSE.txt

@@ -0,0 +1,18 @@
+Kiln AI - License Information
+Copyright (c) 2024 Chesterfield Laboratories Inc.
+
+This repository contains components under different licenses:
+
+1. Python Libraries (/libs/core and /libs/server directories)
+   Licensed under the MIT License
+   See libs/core/LICENSE.txt and libs/server/LICENSE.txt for details
+
+2. Desktop Application (/app directory)
+   Free to download and use under the Kiln AI Desktop EULA
+   See app/EULA.md and app/LICENSE.txt for details
+
+3. Trademarks
+   The Kiln names and logos are trademarks of Chesterfield Laboratories Inc.
+   All rights reserved.
+
+For the full text of each license, please see the respective LICENSE.txt files in each directory.

README.md

@@ -19,8 +19,8 @@
 | CI      | [![Build and Test](https://github.com/Kiln-AI/kiln/actions/workflows/build_and_test.yml/badge.svg)](https://github.com/Kiln-AI/kiln/actions/workflows/build_and_test.yml) [![Format and Lint](https://github.com/Kiln-AI/kiln/actions/workflows/format_and_lint.yml/badge.svg)](https://github.com/Kiln-AI/kiln/actions/workflows/format_and_lint.yml) [![Desktop Apps Build](https://github.com/Kiln-AI/kiln/actions/workflows/build_desktop.yml/badge.svg)](https://github.com/Kiln-AI/kiln/actions/workflows/build_desktop.yml) [![Web UI Build](https://github.com/Kiln-AI/kiln/actions/workflows/web_format_lint_build.yml/badge.svg)](https://github.com/Kiln-AI/kiln/actions/workflows/web_format_lint_build.yml) [![Test Count Badge](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/scosman/57742c1b1b60d597a6aba5d5148d728e/raw/test_count_kiln.json)](https://github.com/Kiln-AI/kiln/actions/workflows/test_count.yml) [![Test Coverage Badge](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/scosman/57742c1b1b60d597a6aba5d5148d728e/raw/library_coverage_kiln.json)](https://github.com/Kiln-AI/kiln/actions/workflows/test_count.yml) [![Docs](https://github.com/Kiln-AI/Kiln/actions/workflows/build_docs.yml/badge.svg)](https://github.com/Kiln-AI/Kiln/actions/workflows/build_docs.yml) |
 | Package | [![PyPI - Version](https://img.shields.io/pypi/v/kiln-ai.svg?logo=pypi&label=PyPI&logoColor=gold)](https://pypi.org/project/kiln-ai/) [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/kiln-ai.svg?logo=python&label=Python&logoColor=gold)](https://pypi.org/project/kiln-ai/)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
 | Meta    | [![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json)](https://github.com/astral-sh/uv) [![linting - Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff) [![Hatch project](https://img.shields.io/badge/%F0%9F%A5%9A-Hatch-4051b5.svg)](https://github.com/pypa/hatch) [![types - Pyright](https://img.shields.io/badge/types-pyright-blue.svg)](https://github.com/microsoft/pyright) [![Docs](https://img.shields.io/badge/docs-pdoc-blue)](https://kiln-ai.github.io/Kiln/kiln_core_docs/index.html)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
-| Apps    | [![MacOS](https://img.shields.io/badge/MacOS-black?logo=apple)](https://github.com/Kiln-AI/Kiln/releases/latest) [![Windows](https://img.shields.io/badge/Windows-0067b8.svg?logo=data:image/svg%2bxml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPHN2ZyBmaWxsPSIjZmZmIiB2aWV3Qm94PSIwIDAgMzIgMzIiIHZlcnNpb249IjEuMSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KPHBhdGggZD0iTTE2Ljc0MiAxNi43NDJ2MTQuMjUzaDE0LjI1M3YtMTQuMjUzek0xLjAwNCAxNi43NDJ2MTQuMjUzaDE0LjI1NnYtMTQuMjUzek0xNi43NDIgMS4wMDR2MTQuMjU2aDE0LjI1M3YtMTQuMjU2ek0xLjAwNCAxLjAwNHYxNC4yNTZoMTQuMjU2di0xNC4yNTZ6Ij48L3BhdGg+Cjwvc3ZnPg==)](https://github.com/Kiln-AI/Kiln/releases/latest) [![Linux](https://img.shields.io/badge/Linux-444444?logo=linux&logoColor=ffffff)](https://github.com/Kiln-AI/Kiln/releases/latest)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
-| Chat    | [![Discord](https://img.shields.io/badge/Discord-Kiln_AI-blue?logo=Discord&logoColor=white)](https://discord.gg/sVJEzDGu)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
+| Apps    | [![MacOS](https://img.shields.io/badge/MacOS-black?logo=apple)](https://github.com/Kiln-AI/Kiln/releases/latest) [![Windows](https://img.shields.io/badge/Windows-0067b8.svg?logo=data:image/svg%2bxml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPHN2ZyBmaWxsPSIjZmZmIiB2aWV3Qm94PSIwIDAgMzIgMzIiIHZlcnNpb249IjEuMSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KPHBhdGggZD0iTTE2Ljc0MiAxNi43NDJ2MTQuMjUzaDE0LjI1M3YtMTQuMjUzek0xLjAwNCAxNi43NDJ2MTQuMjUzaDE0LjI1NnYtMTQuMjUzek0xNi43NDIgMS4wMDR2MTQuMjU2aDE0LjI1M3YtMTQuMjU2ek0xLjAwNCAxLjAwNHYxNC4yNTZoMTQuMjU2di0xNC4yNTZ6Ij48L3BhdGg+Cjwvc3ZnPg==)](https://github.com/Kiln-AI/Kiln/releases/latest) [![Linux](https://img.shields.io/badge/Linux-444444?logo=linux&logoColor=ffffff)](https://github.com/Kiln-AI/Kiln/releases/latest) ![Github Downsloads](https://img.shields.io/github/downloads/kiln-ai/kiln/total)                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
+| Connect | [![Discord](https://img.shields.io/badge/Discord-Kiln_AI-blue?logo=Discord&logoColor=white)](https://discord.gg/sVJEzDGu) [![Newsletter](https://img.shields.io/badge/Newsletter-kilnai-blue?logo=Substack&logoColor=white)](https://kilnai.substack.com)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
 
 [<img width="220" alt="Download button" src="https://github.com/user-attachments/assets/a5d51b8b-b30a-4a16-a902-ab6ef1d58dc0">](https://github.com/Kiln-AI/Kiln/releases/latest) [<img width="220" alt="Quick start button" src="https://github.com/user-attachments/assets/aff1b35f-72c0-4286-9b28-40a415558359">](https://docs.getkiln.ai/getting-started/quickstart)
 
@@ -29,6 +29,7 @@
 - 🚀 **Intuitive Desktop Apps**: One-click apps for Windows, MacOS, and Linux. Truly intuitive design.
 - 🎛️ **Fine Tuning**: Zero-code fine-tuning for Llama, GPT4o, and Mixtral. Automatic serverless deployment of models.
 - 🤖 **Synthetic Data Generation**: Generate training data with our interactive visual tooling.
+- 🧠 **Reasoning Models**: Train or distill your own custom reasoning models.
 - 🤝 **Team Collaboration**: Git-based version control for your AI datasets. Intuitive UI makes it easy to collaborate with QA, PM, and subject matter experts on structured data (examples, prompts, ratings, feedback, issues, etc.).
 - 📝 **Prompt Generation**: Automatically generate prompts from your data, including chain-of-thought, few-shot, and multi-shot, and more.
 - 🌐 **Wide Model and Provider Support**: Use any model via Ollama, OpenAI, OpenRouter, Fireworks, Groq, AWS, or any OpenAI compatible API.
@@ -60,6 +61,7 @@ Kiln is quite intuitive, so we suggest launching the desktop app and diving in.
 - [Fine Tuning LLM Models](https://docs.getkiln.ai/docs/fine-tuning-guide)
 - [Guide: Train a Reasoning Model](https://docs.getkiln.ai/docs/guide-train-a-reasoning-model)
 - [Reasoning & Chain of Thought](https://docs.getkiln.ai/docs/reasoning-and-chain-of-thought)
+- [Evaluators](https://docs.getkiln.ai/docs/evaluators)
 - [Synthetic Data Generation](https://docs.getkiln.ai/docs/synthetic-data-generation)
 - [Collaborating with Kiln](https://docs.getkiln.ai/docs/collaboration)
 - [Rating and Labeling Data](https://docs.getkiln.ai/docs/reviewing-and-rating)

app/desktop/build_desktop_app.sh

@@ -68,12 +68,3 @@ pyinstaller $(printf %s "$PLATFORM_OPTS")  \
   --noconfirm --distpath=./desktop/build/dist --workpath=./desktop/build/work \
   -n Kiln --specpath=./desktop/build \
   --paths=. ./desktop/desktop.py
-
-# MacOS apps have symlinks, and GitHub artifact upload zip will break them. Tar instead.
-if [[ $* == *--compress-mac-app* && "$(uname)" == "Darwin" ]]; then
-  echo "Compressing MacOS app"
-  cd ./desktop/build/dist
-  tar czpvf Kiln.app.tgz Kiln.app
-  rm -r Kiln.app
-  cd ../../..
-fi

app/desktop/custom_tray.py

@@ -1,8 +1,11 @@
 import io
+import logging
 import sys
 
 import pystray
 
+logger = logging.getLogger(__name__)
+
 
 class KilnTray(pystray.Icon):
     # Special handling for Mac to support dark/light mode and retina icons
@@ -33,7 +36,7 @@ def _assert_image(self):
             self._icon_image.setTemplate_(True)
             # set the logical size of the image, which will be scaled for retina
             self._icon_image.setSize_(logical_size)
-        except Exception as e:
+        except Exception:
             # Continue, this shouldn't be fatal
-            print("Mac Tray Error", e)
+            logger.error("Mac Tray Error", exc_info=True)
         self._status_item.button().setImage_(self._icon_image)  # type: ignore

app/desktop/desktop_server.py

@@ -3,13 +3,14 @@
 import time
 from contextlib import asynccontextmanager
 
+import kiln_ai.datamodel.strict_mode as datamodel_strict_mode
 import kiln_server.server as kiln_server
 import uvicorn
 from fastapi import FastAPI
-from kiln_ai.datamodel import set_strict_mode as set_strict_mode_datamodel
-from kiln_ai.datamodel import strict_mode as strict_mode_datamodel
 
+from app.desktop.log_config import log_config
 from app.desktop.studio_server.data_gen_api import connect_data_gen_api
+from app.desktop.studio_server.eval_api import connect_evals_api
 from app.desktop.studio_server.finetune_api import connect_fine_tune_api
 from app.desktop.studio_server.prompt_api import connect_prompt_api
 from app.desktop.studio_server.provider_api import connect_provider_api
@@ -20,12 +21,12 @@
 
 @asynccontextmanager
 async def lifespan(app: FastAPI):
-    # Set strict mode on startup
-    original_strict_mode = strict_mode_datamodel()
-    set_strict_mode_datamodel(True)
+    # Set datamodel strict mode on startup
+    original_strict_mode = datamodel_strict_mode.strict_mode()
+    datamodel_strict_mode.set_strict_mode(True)
     yield
-    # Reset strict mode on shutdown
-    set_strict_mode_datamodel(original_strict_mode)
+    # Reset datamodel strict mode on shutdown
+    datamodel_strict_mode.set_strict_mode(original_strict_mode)
 
 
 def make_app():
@@ -36,6 +37,7 @@ def make_app():
     connect_settings(app)
     connect_data_gen_api(app)
     connect_fine_tune_api(app)
+    connect_evals_api(app)
 
     # Important: webhost must be last, it handles all other URLs
     connect_webhost(app)
@@ -47,8 +49,8 @@ def server_config(port=8757):
         make_app(),
         host="127.0.0.1",
         port=port,
-        log_level="warning",
         use_colors=False,
+        log_config=log_config(),
     )