Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -70,3 +70,4 @@ packages/benchmark/

# Benchmark studies and session logs
studies/
.test-vectors/
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ Command-line interface for repository indexing and MCP setup.
- `dev mcp install [--cursor]` - Install MCP integration
- `dev mcp uninstall [--cursor]` - Remove MCP integration
- `dev mcp list [--cursor]` - List MCP servers
- `dev gh index` - Index GitHub issues/PRs
- `dev github index` - Index GitHub issues/PRs

### @lytics/dev-agent-subagents

Expand Down Expand Up @@ -391,7 +391,7 @@ pnpm dev # Watch mode
pnpm lint && pnpm typecheck # Quality checks
dev index . # Index repository
dev mcp install --cursor # Install for Cursor
dev gh index # Index GitHub
dev github index # Index GitHub

# Debugging
dev mcp start --verbose # Verbose MCP server
Expand Down
4 changes: 2 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- Memory leaks from unbounded array growth
- Zombie MCP server processes when Cursor closes
- GitHub index not reloading after `dev gh index`
- GitHub index not reloading after `dev github index`
- STDIO transport not handling stdin closure properly

### Security
Expand Down Expand Up @@ -104,7 +104,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
```bash
cd /path/to/repo
dev index .
dev gh index # If using GitHub integration
dev github index # If using GitHub integration
```

3. Reinstall MCP integration:
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -244,8 +244,8 @@ dev git search "authentication fix" # Semantic search over commits
dev git stats # Show indexed commit count

# GitHub integration
dev gh index # Index issues and PRs (also done by dev index)
dev gh search "authentication bug" # Semantic search
dev github index # Index issues and PRs (also done by dev index)
dev github search "authentication bug" # Semantic search

# View statistics
dev stats
Expand Down
24 changes: 12 additions & 12 deletions TROUBLESHOOTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -350,7 +350,7 @@ dev index .

## GitHub Integration

### `dev gh index` fails
### `dev github index` fails

**Common causes:**

Expand Down Expand Up @@ -386,7 +386,7 @@ Failed to fetch issues: spawnSync /bin/sh ENOBUFS
dev index --gh-limit 100

# For dedicated GitHub indexing
dev gh index --limit 100
dev github index --limit 100
```

2. **Adjust limit based on repository size:**
Expand All @@ -397,10 +397,10 @@ Failed to fetch issues: spawnSync /bin/sh ENOBUFS
3. **Index in batches:**
```bash
# Index open items only (usually smaller)
dev gh index --state open --limit 500
dev github index --state open --limit 500

# Then index closed items with lower limit
dev gh index --state closed --limit 100
dev github index --state closed --limit 100
```

**Technical details:**
Expand All @@ -424,10 +424,10 @@ Failed to fetch issues: spawnSync /bin/sh ENOBUFS

3. **Re-index:**
```bash
dev gh index
dev github index
```

**Note:** The `dev_gh` tool automatically reloads when you run `dev gh index` - no restart needed!
**Note:** The `dev_gh` tool automatically reloads when you run `dev github index` - no restart needed!

### GitHub index is stale

Expand All @@ -438,13 +438,13 @@ Use dev_health tool - warns if GitHub index >24h old

**Solution:**
```bash
dev gh index
dev github index
```

**Automation (optional):**
```bash
# Add to crontab for daily updates
0 9 * * * cd /path/to/repo && dev gh index
0 9 * * * cd /path/to/repo && dev github index
```

---
Expand Down Expand Up @@ -534,7 +534,7 @@ Use dev_health tool to check component status
```bash
# Re-index everything
dev index .
dev gh index
dev github index

# Restart MCP server
```
Expand Down Expand Up @@ -689,7 +689,7 @@ rm -rf ~/.dev-agent/indexes/*
# Re-index your repositories
cd /path/to/your/repo
dev index .
dev gh index
dev github index

# Reinstall MCP
dev mcp install --cursor # or without --cursor for Claude Code
Expand Down Expand Up @@ -755,7 +755,7 @@ npm update -g dev-agent
# Re-index repositories (recommended)
cd /path/to/your/repo
dev index .
dev gh index
dev github index

# Restart AI tool
```
Expand Down Expand Up @@ -944,7 +944,7 @@ dev index .

**GitHub Index Stale:**
```bash
dev gh index
dev github index
```

**Repository Not Accessible:**
Expand Down
2 changes: 1 addition & 1 deletion WORKFLOW.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ git checkout main
git pull origin main

# Use GitHub Context to find what to work on next
dev gh search "state:open label:\"Epic: MCP Integration\"" --type issue
dev github search "state:open label:\"Epic: MCP Integration\"" --type issue

# Or use gh CLI directly
gh issue list --milestone "Epic #3: MCP Integration" --state open
Expand Down
4 changes: 2 additions & 2 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,7 @@ dev_gh:

**First, index GitHub:**
```bash
dev gh index
dev github index
```

---
Expand Down Expand Up @@ -339,7 +339,7 @@ dev_health:
dev index .

# After new issues/PRs
dev gh index
dev github index

# Check health
dev_health
Expand Down
2 changes: 1 addition & 1 deletion packages/cli/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@

**Indexer Logging**

- Add `--verbose` flag to `dev index`, `dev git index`, `dev gh index`
- Add `--verbose` flag to `dev index`, `dev git index`, `dev github index`
- Progress spinner shows actual counts: `Embedding 4480/49151 documents (9%)`
- Structured logging with kero logger

Expand Down
13 changes: 3 additions & 10 deletions packages/cli/src/cli.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -58,16 +58,9 @@ describe('CLI Structure', () => {
expect(jsonOption).toBeDefined();
});

it('stats command should have show subcommand with json option', () => {
const subcommands = statsCommand.commands;
const showCommand = subcommands.find((cmd) => cmd.name() === 'show');

expect(showCommand).toBeDefined();

if (showCommand) {
const jsonOption = showCommand.options.find((opt) => opt.long === '--json');
expect(jsonOption).toBeDefined();
}
it('stats command should have json option', () => {
const jsonOption = statsCommand.options.find((opt) => opt.long === '--json');
expect(jsonOption).toBeDefined();
});

it('clean command should have force option', () => {
Expand Down
4 changes: 2 additions & 2 deletions packages/cli/src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ import { cleanCommand } from './commands/clean.js';
import { compactCommand } from './commands/compact.js';
import { dashboardCommand } from './commands/dashboard.js';
import { exploreCommand } from './commands/explore.js';
import { ghCommand } from './commands/gh.js';
import { gitCommand } from './commands/git.js';
import { githubCommand } from './commands/github.js';
import { indexCommand } from './commands/index.js';
import { initCommand } from './commands/init.js';
import { mcpCommand } from './commands/mcp.js';
Expand Down Expand Up @@ -35,7 +35,7 @@ program.addCommand(indexCommand);
program.addCommand(searchCommand);
program.addCommand(exploreCommand);
program.addCommand(planCommand);
program.addCommand(ghCommand);
program.addCommand(githubCommand);
program.addCommand(gitCommand);
program.addCommand(updateCommand);
program.addCommand(statsCommand);
Expand Down
50 changes: 31 additions & 19 deletions packages/cli/src/commands/clean.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,12 @@ import {
getStorageFilePaths,
getStoragePath,
} from '@lytics/dev-agent-core';
import chalk from 'chalk';
import { Command } from 'commander';
import ora from 'ora';
import { loadConfig } from '../utils/config.js';
import { getDirectorySize } from '../utils/file.js';
import { logger } from '../utils/logger.js';
import { output, printCleanSuccess, printCleanSummary } from '../utils/output.js';

export const cleanCommand = new Command('clean')
.description('Clean indexed data and cache')
Expand All @@ -33,21 +34,35 @@ export const cleanCommand = new Command('clean')
await ensureStorageDirectory(storagePath);
const filePaths = getStorageFilePaths(storagePath);

// Calculate sizes of files to be deleted
const files = await Promise.all(
[
{ name: 'Vector store', path: filePaths.vectors },
{ name: 'Indexer state', path: filePaths.indexerState },
{ name: 'GitHub state', path: filePaths.githubState },
{ name: 'Metadata', path: filePaths.metadata },
].map(async (file) => {
try {
const stat = await fs.stat(file.path);
const size = stat.isDirectory() ? await getDirectorySize(file.path) : stat.size;
return { ...file, size };
} catch {
return { ...file, size: null };
}
})
);

const totalSize = files.reduce((sum, file) => sum + (file.size || 0), 0);

// Show what will be deleted
logger.log('');
logger.log(chalk.bold('The following will be deleted:'));
logger.log(` ${chalk.cyan('Storage directory:')} ${storagePath}`);
logger.log(` ${chalk.cyan('Vector store:')} ${filePaths.vectors}`);
logger.log(` ${chalk.cyan('State file:')} ${filePaths.indexerState}`);
logger.log(` ${chalk.cyan('GitHub state:')} ${filePaths.githubState}`);
logger.log(` ${chalk.cyan('Metadata:')} ${filePaths.metadata}`);
logger.log('');
printCleanSummary({
files,
totalSize,
force: options.force,
});

// Confirm unless --force
if (!options.force) {
logger.warn('This action cannot be undone!');
logger.log(`Run with ${chalk.yellow('--force')} to skip this prompt.`);
logger.log('');
process.exit(0);
}

Expand All @@ -56,17 +71,14 @@ export const cleanCommand = new Command('clean')
// Delete storage directory (contains all index files)
try {
await fs.rm(storagePath, { recursive: true, force: true });
spinner.succeed(chalk.green('Cleaned successfully!'));
spinner.succeed('Cleaned successfully');

printCleanSuccess({ totalSize });
} catch (error) {
spinner.fail('Failed to clean');
logger.error(`Error: ${error instanceof Error ? error.message : String(error)}`);
output.error(`Error: ${error instanceof Error ? error.message : String(error)}`);
process.exit(1);
}

logger.log('');
logger.log('All indexed data has been removed.');
logger.log(`Run ${chalk.yellow('dev index')} to re-index your repository.`);
logger.log('');
} catch (error) {
logger.error(`Failed to clean: ${error instanceof Error ? error.message : String(error)}`);
process.exit(1);
Expand Down
39 changes: 14 additions & 25 deletions packages/cli/src/commands/compact.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@ import {
getStoragePath,
RepositoryIndexer,
} from '@lytics/dev-agent-core';
import chalk from 'chalk';
import { Command } from 'commander';
import ora from 'ora';
import { loadConfig } from '../utils/config.js';
import { logger } from '../utils/logger.js';
import { printCompactResults } from '../utils/output.js';

export const compactCommand = new Command('compact')
.description('🗜️ Optimize and compact the vector store')
Expand Down Expand Up @@ -65,36 +65,25 @@ export const compactCommand = new Command('compact')
// @ts-expect-error - accessing private property for optimization
await indexer.vectorStorage.optimize();

const duration = ((Date.now() - startTime) / 1000).toFixed(2);
const duration = (Date.now() - startTime) / 1000;

// Get stats after optimization
const statsAfter = await indexer.getStats();

await indexer.close();

spinner.succeed(chalk.green('Vector store optimized successfully!'));

// Show results
logger.log('');
logger.log(chalk.bold('Optimization Results:'));
logger.log(` ${chalk.cyan('Duration:')} ${duration}s`);
logger.log(` ${chalk.cyan('Total documents:')} ${statsAfter?.vectorsStored || 0}`);

if (options.verbose) {
logger.log('');
logger.log(chalk.bold('Before Optimization:'));
logger.log(` ${chalk.cyan('Storage size:')} ${statsBefore.vectorsStored} vectors`);
logger.log('');
logger.log(chalk.bold('After Optimization:'));
logger.log(` ${chalk.cyan('Storage size:')} ${statsAfter?.vectorsStored || 0} vectors`);
}

logger.log('');
logger.log(
chalk.gray(
'Optimization merges small data fragments, updates indices, and improves query performance.'
)
);
spinner.succeed('Vector store optimized');

// Show results using new formatter
printCompactResults({
duration,
before: {
vectors: statsBefore.vectorsStored,
},
after: {
vectors: statsAfter?.vectorsStored || 0,
},
});
} catch (error) {
spinner.fail('Failed to optimize vector store');
logger.error(error instanceof Error ? error.message : String(error));
Expand Down
Loading