Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
a286628
Initial commit
bashandbone Jul 2, 2025
4e93c62
bashandbone Jul 6, 2025
0ac3e37
Integrated ast-grep types in core; vendored ast-grep-language as thre…
bashandbone Jul 7, 2025
6b3e8f6
finished initial langs implementation; added reuse spec headers
bashandbone Jul 7, 2025
d9fe396
added some documentation to thread core to cover ast-grep types, impl…
bashandbone Jul 7, 2025
25ace4f
added more ast-grep foundation, began implementing core services
bashandbone Jul 9, 2025
aa03115
more scaffolding work
bashandbone Jul 9, 2025
43fe783
linting changes
bashandbone Jul 10, 2025
9c98beb
resolving import/export issues after pulling in ast-grep modules
bashandbone Jul 10, 2025
acc0c9c
bashandbone Jul 10, 2025
50f017c
add service layer
bashandbone Jul 10, 2025
cb6cb48
decided to fork ast-grep to get optimal outcomes; that work continues
bashandbone Jul 11, 2025
65e18fe
continued work on forked ast-grep
bashandbone Jul 11, 2025
3b966ed
completed first restructuring of Ast-Grep into a service
bashandbone Jul 11, 2025
45467f3
rematrixing the dependencies
bashandbone Jul 11, 2025
e667ae9
saving state before bringing in the llms...
bashandbone Jul 12, 2025
0dcb5a3
third time's the charm on refactoring, it seems
bashandbone Jul 13, 2025
bf25a28
removed fastmap implementation in favor of a simpler faster hash pattern
bashandbone Jul 13, 2025
0f758a4
about to try separating serialization logic
bashandbone Jul 17, 2025
6579e15
first analysis of serialization separation; prospect poor. Going to t…
bashandbone Jul 18, 2025
6341018
added benchmarks to rule_engine, debug/clone consistency
bashandbone Jul 19, 2025
189ed43
linting and formatting
bashandbone Jul 19, 2025
34cbd08
linting and formatting
bashandbone Jul 19, 2025
81d7759
updates to license updater, improved cla handling
bashandbone Jul 19, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .claude/settings.local.json.license
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
SPDX-FileCopyrightText: 2025 Knitli Inc. <[email protected]>
SPDX-FileContributor: Adam Poulemanos <[email protected]>

SPDX-License-Identifier: MIT OR Apache-2.0
68 changes: 68 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# SPDX-FileCopyrightText: 2025 Knitli Inc. <[email protected]>
# SPDX-FileContributor: Adam Poulemanos <[email protected]>
# SPDX-License-Identifier: MIT OR Apache-2.0
#
# Set default behavior to automatically normalize line endings
* text=auto

# Force most text files to always use LF
*.astro text eol=lf
*.css text eol=lf
*.csv text eol=lf
*.cts text eol=lf
*.go text eol=lf
*.html text eol=lf
*.ini text eol=lf
*.js text eol=lf
*.json text eol=lf
*.jsx text eol=lf
*.md text eol=lf
*.mdx text eol=lf
*.mts text eol=lf
*.pkl text eol=lf
*.py text eol=lf
*.rs text eol=lf
*.sh text eol=lf
*.svelte text eol=lf
*.svg text eol=lf
*.toml text eol=lf
*.ts text eol=lf
*.tsx text eol=lf
*.txt text eol=lf
*.xml text eol=lf
*.yaml text eol=lf
*.yml text eol=lf
.git* text eol=LF
LICENSE text eol=lf
LICENSE* text eol=lf

# Force batch files to use CRLF (Windows scripts)
*.bat text eol=crlf
*.ps* text eol=crlf

# Treat these as binary (no line ending conversion)
*.aac binary
*.avif binary
*.docx binary
*.eot binary
*.exe binary
*.flac binary
*.ico binary
*.jpg binary
*.m4a binary
*.mp3 binary
*.mp4 binary
*.ogg binary
*.otf binary
*.pdf binary
*.png binary
*.pptx binary
*.tar* binary
*.ttf binary
*.wav binary
*.webm binary
*.webp binary
*.woff binary
*.woff2 binary
*.xlsx binary
*.zip binary
12 changes: 12 additions & 0 deletions .github/actionlint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# SPDX-FileCopyrightText: 2025 Knitli Inc. <[email protected]>
# SPDX-FileContributor: Adam Poulemanos <[email protected]>
#
# SPDX-License-Identifier: MIT OR Apache-2.0
# ! Actionlint configuration file to ignore specific warnings
paths:
.github/workflows/cla.yml:
ignore:
- 'property "is_member" is not defined in object type {}'
- >
"github.event.pull_request.title" is potentially untrusted. avoid using it directly in inline scripts. instead, pass it through an environment variable. see https://docs.github.com/en/actions/security-for-github-actions/security-guides/security-hardening-for-github-actions for more details

10 changes: 10 additions & 0 deletions .github/chatmodes/analyze.chatmode.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
# SPDX-FileCopyrightText: 2025 Knitli Inc. <[email protected]>
# SPDX-FileContributor: Adam Poulemanos <[email protected]>
# SPDX-License-Identifier: MIT OR Apache-2.0
description: 'Code Analysis'
tools: ["codebase", "githubRepo", "context7", "sequential-thinking", ]
---
# Expert Code Analyst

You're an experienced code analyst who specializes in identifying and resolving issues in codebases. Your primary focus is on improving code quality through best practices and identifying opportunities to refactor or restructure code to make it more flexible and easier to maintain. The user will ask you to research specific code, modules, or packages within the codebase. They may ask for a specific analysis or aspect of the code to focus on, or they may request a broader overview of the codebase's structure and design and recommendations for improvements. If you identify an opportunity for improving the code quality, you should provide actionable suggestions and code examples to help the user implement the improvements. Unless the user requests a different result, you should produce a report summarizing your findings with specific recommendations and references to specific code snippets by line number and filename.
55 changes: 47 additions & 8 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,47 @@
version: 2
updates:
- package-ecosystem: cargo
directory: "/"
schedule:
interval: daily
time: "08:00"
open-pull-requests-limit: 10
# SPDX-FileCopyrightText: 2025 Knitli Inc. <[email protected]>
# SPDX-FileContributor: Adam Poulemanos <[email protected]>
#
# SPDX-License-Identifier: MIT OR Apache-2.0
version: 2
updates:
- package-ecosystem: cargo
dependency-type: all
update-types:
- version-update:semver-minor
- version-update:semver-patch
versioning-strategy: increase
directory: "/"
schedule:
interval: daily
time: "23:00"
open-pull-requests-limit: 10
commit-message:
prefix: "deps"
assignees:
- codegen-sh[bot]
reviewers:
- codegen-sh[bot]
labels:
- "dependencies"
rebase-strategy: auto
- package-ecosystem: github-actions
dependency-type: all
update-types:
- version-update:semver-major
- version-update:semver-minor
- version-update:semver-patch
versioning-strategy: increase
directory: "/.github/workflows"
schedule:
interval: daily
time: "23:00"
open-pull-requests-limit: 10
commit-message:
prefix: "deps"
assignees:
- codegen-sh[bot]
reviewers:
- codegen-sh[bot]
labels:
- "dependencies"
rebase-strategy: auto
148 changes: 148 additions & 0 deletions .github/dontusefornow.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
<!--
SPDX-FileCopyrightText: 2025 Knitli Inc. <[email protected]>
SPDX-FileContributor: Adam Poulemanos <[email protected]>
SPDX-License-Identifier: MIT OR Apache-2.0
-->

# Copilot Instructions for Thread

## Project Overview

Thread is a Rust code analysis engine that provides intelligent context for AI assistants. The project is transitioning from vendored ast-grep CLI code to a multi-environment service architecture supporting CLI, cloud, WASM, and CI/CD deployments.

## Architecture Guidelines

### Service Layer Pattern
The codebase follows a service abstraction pattern to support multiple environments:

```rust
// Pure service functions (environment-agnostic)
pub async fn scan_with_services(
file_discovery: Arc<dyn FileDiscoveryService>,
config_service: Arc<dyn ConfigurationService>,
options: ScanOptions,
) -> Result<ScanResults>
```

### Crate Organization

- **ag-thread/**: Vendored ast-grep modules being refactored for service layers
- **thread-core/**: Core traits, types, and errors (pure abstractions)
- **thread-engine/**: Main analysis implementation using petgraph
- **thread-parse/**: AST-grep integration and language detection
- **Service-ready crates**: `ag-core`, `ag-search`, `ag-fix`, `ag-types`, `ag-label`
- **Needs refactoring**: `ag-scan`, `ag-utils`, `ag-check-rule` (heavy CLI dependencies)

## Development Commands

Essential commands for this workspace:

```bash
# Build all crates (except WASM)
mise run build
mise run b

# WASM builds
mise run build-wasm # Development (single-threaded)
mise run build-wasm-release # Production optimized

# Testing and quality
mise run test # Tests with cargo nextest
mise run lint # Full linting via hk run check
mise run ci # All CI checks
```

## Key Patterns to Follow

### 1. Service Trait Definitions
When creating new services, follow the pattern from `ag-types`:

```rust
#[async_trait]
pub trait YourService: Send + Sync {
async fn your_method(&self, input: &str) -> Result<Output>;
}
```

### 2. Environment-Agnostic Core Functions
Avoid CLI dependencies in core logic:

```rust
// ✅ Good: Pure function with injected services
pub async fn analyze_with_services(
content: String,
services: &ServiceRegistry
) -> Result<AnalysisResult>

// ❌ Avoid: Direct filesystem or terminal access
pub fn analyze_files(paths: Vec<PathBuf>) -> Result<()>
```

### 3. Multi-Environment Support
Structure implementations for different environments:

```rust
// CLI implementation
impl YourService for CliYourService { /* uses std::fs */ }

// Cloud implementation
impl YourService for CloudYourService { /* uses S3/HTTP */ }

// WASM implementation
impl YourService for WasmYourService { /* uses fetch API */ }
```

## CLI Dependencies Analysis Status

Refer to individual `CLI_DEPENDENCIES.md` files in each ag-thread crate:

- **Immediate attention needed**: `ag-scan/`, `ag-utils/` (heavy CLI dependencies)
- **Service-ready**: `ag-core/`, `ag-search/`, `ag-fix/`, `ag-types/`, `ag-label/`
- **Minor refactoring**: `ag-rule/`, `ag-check-rule/`

## Critical Abstractions

### File Operations
Replace direct filesystem access with service traits:
```rust
// Instead of std::fs::read_to_string
let content = file_service.read_file(path).await?;
```

### Terminal I/O
Replace direct terminal access with service traits:
```rust
// Instead of println! or crossterm
output_service.write(&format!("Result: {}", result)).await?;
```

### Configuration Loading
Replace direct file config loading:
```rust
// Instead of reading YAML files directly
let rules = config_service.load_rules(ConfigSource::Path(path)).await?;
```

## Testing Strategy

- Use `cargo nextest -j 1` for parallel tests with race condition prevention
- Mock service implementations for unit tests
- Environment-specific integration tests for each service implementation
- `RUST_BACKTRACE=1` enabled for debugging

## WASM Considerations

- Default builds are single-threaded for Cloudflare Workers compatibility
- Core logic separated from filesystem operations for WASM portability
- Multi-threaded builds available for browser environments (`--multi-threading`)

When working with WASM targets, ensure no direct filesystem or process dependencies in core libraries.

## Current Development Focus

**Week 1 Sprint**: Establishing service layer foundations
- Refactoring `ag-scan` to use service abstractions
- Creating `ag-services` crate with core trait definitions
- Implementing CLI service adapters to maintain current functionality

The goal is to enable Thread to analyze code and provide AI-friendly context across all deployment environments while maintaining the performance and functionality of the original ast-grep implementation.
88 changes: 88 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# SPDX-FileCopyrightText: 2025 Knitli Inc. <[email protected]>
# SPDX-FileContributor: Adam Poulemanos <[email protected]>
#
# SPDX-License-Identifier: MIT OR Apache-2.0
# ! GitHub Action to run the CI pipeline for Rust projects
# ! This action is triggered on pushes and pull requests to the main and staging branches.
name: CI
on:
push:
branches: [main, staging]
pull_request:
branches: [main, staging]
env:
CARGO_TERM_COLOR: always
jobs:
test:
name: Test Suite
runs-on: ubuntu-latest
strategy:
matrix:
rust:
- stable
- beta
- nightly
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- name: Install Rust
uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ matrix.rust }}
components: rustfmt, clippy
- name: Cache cargo registry
uses: actions/cache@v4
with:
path: ~/.cargo/registry
key: ${{ runner.os }}-cargo-registry-${{ hashFiles('**/Cargo.lock') }}
- name: Cache cargo index
uses: actions/cache@v4
with:
path: ~/.cargo/git
key: ${{ runner.os }}-cargo-index-${{ hashFiles('**/Cargo.lock') }}
- name: Cache cargo build
uses: actions/cache@v4
with:
path: target
key: ${{ runner.os }}-cargo-build-target-${{ hashFiles('**/Cargo.lock') }}
- name: Set up mise
run: |
chmod -R +x ./scripts
./install-mise.sh
MISE="$HOME/.local/bin/mise"
echo \"eval "$($MISE activate bash)"\" >> "$HOME/.bashrc"
source "$HOME/.bashrc"
$MISE run install
- name: Run hk ci workflow
run: >
"$HOME/.local/bin/mise" run ci

security_audit:
name: Security Audit
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: rustsec/[email protected]
with:
token: ${{ secrets.GITHUB_TOKEN }}
coverage:
name: Code Coverage
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
with:
components: llvm-tools-preview
- name: Install cargo-llvm-cov
uses: taiki-e/install-action@cargo-llvm-cov
- name: Generate code coverage
run: cargo llvm-cov --all-features --workspace --lcov --output-path lcov.info
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v4
with:
files: lcov.info
fail_ci_if_error: true
Loading
Loading