diff --git a/.gitignore b/.gitignore index f51d990ce..a4fa1397f 100644 --- a/.gitignore +++ b/.gitignore @@ -64,4 +64,4 @@ graph-sitter-types/out/** graph-sitter-types/typings/** coverage.json tests/integration/verified_codemods/codemod_data/repo_commits.json - +.codegen/* diff --git a/docs/blog/act-via-code.mdx b/docs/blog/act-via-code.mdx index c9400344b..ff00405d8 100644 --- a/docs/blog/act-via-code.mdx +++ b/docs/blog/act-via-code.mdx @@ -5,32 +5,32 @@ iconType: "solid" description: "The path to advanced code manipulation agents" --- - + -# Act via Code +Two and a half years since the launch of the GPT-3 API, code assistants have emerged as potentially the premier use case of LLMs. The rapid adoption of AI-powered IDEs and prototype builders isn't surprising — code is structured, deterministic, and rich with patterns, making it an ideal domain for machine learning. Experienced developers working with tools like Cursor (myself included) can tell that the field of software engineering is about to go through rapid change. -Two and a half years since the launch of the GPT-3 API, code assistants have emerged as the most powerful and practically useful applications of LLMs. The rapid adoption of AI-powered IDEs and prototype builders isn't surprising — code is structured, deterministic, and rich with patterns, making it an ideal domain for machine learning. As model capabilities continue to scale, we're seeing compounding improvements in code understanding and generation. - -Yet there's a striking gap between what AI agents can understand and what they can actually do. While they can reason about complex architectural changes, debug intricate issues, and propose sophisticated refactors, they often can't execute these ideas. The ceiling isn't intelligence or context—it's the ability to manipulate code at scale. Large-scale modifications remain unreliable or impossible, not because agents don't understand what to do, but because they lack the right interfaces to do it. +Yet there's a striking gap between understanding and action. Today's AI agents can analyze enterprise codebases and propose sophisticated improvements—eliminating tech debt, untangling dependencies, improving modularity. But ask them to actually implement these changes across millions of lines of code, and they hit a wall. Their ceiling isn't intelligence—it's the ability to safely and reliably execute large-scale modifications on real, enterprise codebases. The bottleneck isn't intelligence — it's tooling. By giving AI models the ability to write and execute code that modifies code, we're about to unlock an entire class of tasks that agents already understand but can't yet perform. Code execution environments represent the most expressive tool we could offer an agent—enabling composition, abstraction, and systematic manipulation of complex systems. When paired with ever-improving language models, this will unlock another step function improvement in AI capabilities. ## Beating Minecraft with Code Execution -In mid-2023, a research project called [Voyager](https://voyager.minedojo.org) made waves: it effectively solved Minecraft, performing several multiples better than the prior SOTA on many important dimensions. This was a massive breakthrough — previous reinforcement learning systems had struggled for years with even basic Minecraft tasks. +In mid-2023, a research project called [Voyager](https://voyager.minedojo.org) made waves: it effectively solved Minecraft, performing several multiples better than the prior SOTA. This was a massive breakthrough as previous reinforcement learning systems had struggled for years with even basic Minecraft tasks. -While the AI community was focused on scaling intelligence, Voyager demonstrated something more fundamental: the right tools can unlock entirely new tiers of capability. The same GPT-4 model that struggled with Minecraft using traditional frameworks achieved remarkable results when allowed to write and execute code. This wasn't about raw intelligence—it was about giving the agent a more expressive way to act. +While the AI community was focused on scaling intelligence, Voyager demonstrated something more fundamental: the right tools can unlock entirely new tiers of capability. The same GPT-4 model that struggled with Minecraft using standard agentic frameworks (like [ReAct](https://klu.ai/glossary/react-agent-model)) achieved remarkable results when allowed to write and execute code. This wasn't about raw intelligence—it was about giving the agent a more expressive way to act. -The breakthrough came from a simple yet powerful insight: let the AI write code. Instead of limiting the agent to primitive "tools," Voyager allowed GPT-4 to write and execute [JS programs](https://github.com/MineDojo/Voyager/tree/main/skill_library/trial2/skill/code) that controlled Minecraft actions through a clean API: +The breakthrough came from a simple yet powerful insight: let the AI write code. Instead of limiting the agent to primitive "tools," Voyager allowed GPT-4 to write and execute [JS programs](https://github.com/MineDojo/Voyager/tree/main/skill_library/trial2/skill/code) that controlled Minecraft actions through a clean API. ```javascript +// Example "action program" from Voyager, 2023 +// written by gpt-4 async function chopSpruceLogs(bot) { const spruceLogCount = bot.inventory.count(mcData.itemsByName.spruce_log.id); const logsToMine = 3 - spruceLogCount; @@ -44,7 +44,7 @@ async function chopSpruceLogs(bot) { } ``` -This approach transformed the agent's capabilities. Rather than being constrained to atomic actions like `equipItem(...)`, it could create higher-level operations like [`craftShieldWithFurnace()`](https://github.com/MineDojo/Voyager/blob/main/skill_library/trial2/skill/code/craftShieldWithFurnace.js) through composing JS APIs. The system also implemented a memory mechanism, storing successful programs for reuse in similar situations—effectively building its own library of proven solutions it could later refer to and adapt to similar circumstances. +This approach transformed the agent's capabilities. Rather than being constrained to atomic actions like `equipItem(...)`, it could create higher-level operations like [`craftShieldWithFurnace()`](https://github.com/MineDojo/Voyager/blob/main/skill_library/trial2/skill/code/craftShieldWithFurnace.js) through composing JS APIs. Furthermore, Wang et al. implemented a memory mechanism, in which successful "action programs" could later be recalled, copied, and built upon, effectively enabling the agent to accumulate experience. @@ -56,23 +56,21 @@ As the Voyager authors noted: ## Code is an Ideal Action Space -The implications of code as an action space extend far beyond gaming. Code provides a uniquely powerful interface between AI and real-world systems. When an agent writes code, it gains several critical advantages over traditional atomic tools. +The implications of code as an action space extend far beyond gaming. This architectural insight — letting AI act through code rather than atomic commands — will lead to a step change in the capabilities of AI systems. Nowhere is this more apparent than in software engineering, where agents already understand complex transformations but lack the tools to execute them effectively. + +When an agent writes code, it gains several critical advantages over traditional atomic tools: -### Code is Composable -Code is the ultimate composable medium. Agents can build their own tools by combining simpler operations, wrapping any function as a building block for more complex behaviors. This aligns well with what is perhaps LLMs' premier capability: understanding and interpolating between examples to create new solutions. +- **Composability**: Agents can build their own tools by combining simpler operations. This aligns perfectly with LLMs' demonstrated ability to compose and interpolate between examples to create novel solutions. -### Code Constrains the Action Space -APIs can enforce guardrails that keep agents on track. By designing interfaces that make invalid operations impossible to express, we can prevent entire classes of errors before they happen. The type system becomes a powerful tool for shaping agent behavior. +- **Constrained Action Space**: Well-designed APIs act as guardrails, making invalid operations impossible to express. The type system becomes a powerful tool for preventing entire classes of errors before they happen. -### Code Provides Objective Feedback -Code execution gives immediate, unambiguous feedback. When something goes wrong, you get stack traces and error messages—not just a confidence score. This concrete error signal is invaluable for agents learning to navigate complex systems. +- **Objective Feedback**: Code execution provides immediate, unambiguous feedback through stack traces and error messages—not just confidence scores. This concrete error signal is invaluable for learning. -### Code is a Natural Medium for Collaboration -Programs are a shared language between humans and agents. Code explicitly encodes reasoning in a reviewable format, making agent actions transparent and debuggable. There's no magic—just deterministic execution that can be understood, modified, and improved by both humans and AI. +- **Natural Collaboration**: Programs are a shared language between humans and agents. Code explicitly encodes reasoning in a reviewable format, making actions transparent, debuggable, and easily re-runnable. ## For Software Engineering -This brings us to software engineering, where we see a massive gap between AI's theoretical capabilities and practical achievements. Many code modification tasks are fundamentally programmatic—dependency analysis, refactors, control flow analysis—yet we lack the tools to express them properly. +Software engineering tasks are inherently programmatic and graph-based — dependency analysis, refactors, control flow analysis, etc. Yet today's AI agents interface with code primarily through string manipulation, missing the rich structure that developers and their tools rely on. By giving agents APIs that operate on the codebase's underlying graph structure rather than raw text, we can unlock a new tier of capabilities. Imagine agents that can rapidly traverse dependency trees, analyze control flow, and perform complex refactors while maintaining perfect awareness of the codebase's structure. Consider how a developer thinks about refactoring: it's rarely about direct text manipulation. Instead, we think in terms of high-level operations: "move this function," "rename this variable everywhere," "split this module." These operations can be encoded into a powerful Python API: @@ -84,17 +82,3 @@ for component in codebase.jsx_components: # powerful edit APIs that handle edge cases component.rename(component.name + 'Page') ``` - -This isn't just another code manipulation library—it's a scriptable language server that builds on proven foundations like LSP and codemods, but designed specifically for programmatic analysis and refactoring. - -## What does this look like? - -At Codegen, we've built exactly this system. Our approach centers on four key principles: - -The foundation must be Python, enabling easy composition with existing tools and workflows. Operations must be in-memory for performance, handling large-scale changes efficiently. The system must be open source, allowing developers and AI researchers to extend and enhance it. And perhaps most importantly, it must be thoroughly documented—not just for humans, but for the next generation of AI agents that will build upon it. - -## What does this enable? - -We've already used this approach to merge hundreds of thousands of lines of code in enterprise codebases. Our tools have automated complex tasks like feature flag deletion, test suite reorganization, import cycle elimination, and dead code removal. But more importantly, we've proven that code-as-action-space isn't just theoretical—it's a practical approach to scaling software engineering. - -This is just the beginning. With Codegen, we're providing the foundation for the next generation of code manipulation tools—built for both human developers and AI agents. We believe this approach will fundamentally change how we think about and implement large-scale code changes, making previously impossible tasks not just possible, but routine. \ No newline at end of file diff --git a/docs/blog/codemod-frameworks.mdx b/docs/blog/codemod-frameworks.mdx index e6abd2171..f9c42ad16 100644 --- a/docs/blog/codemod-frameworks.mdx +++ b/docs/blog/codemod-frameworks.mdx @@ -5,6 +5,11 @@ icon: "code-compare" iconType: "solid" --- +# Others to add +- [Abracadabra](https://github.com/nicoespeon/abracadabra) +- [Rope](https://rope.readthedocs.io/en/latest/overview.html#rope-overview) +- [Grit](https://github.com/getgrit/gritql) + Code transformation tools have evolved significantly over the years, each offering unique approaches to programmatic code manipulation. Let's explore the strengths and limitations of major frameworks in this space. ## Python's AST Module diff --git a/docs/blog/posts.mdx b/docs/blog/posts.mdx index f681c1b06..ed1c72110 100644 --- a/docs/blog/posts.mdx +++ b/docs/blog/posts.mdx @@ -4,39 +4,16 @@ icon: "clock" iconType: "solid" --- - -## Static Analysis in the Age of AI Coding Assistants - -Why traditional language servers aren't enough for the future of AI-powered code manipulation - - - - -## Codemod Frameworks - -Comparing popular tools for programmatic code transformation - - - ## Act via Code -Programs are the natural convergence of LLMs and traditional computation. +Why code as an action space will lead to a step function improvement in agent capabilities. diff --git a/docs/building-with-codegen/at-a-glance.mdx b/docs/building-with-codegen/at-a-glance.mdx index 934716d57..125c39738 100644 --- a/docs/building-with-codegen/at-a-glance.mdx +++ b/docs/building-with-codegen/at-a-glance.mdx @@ -60,11 +60,11 @@ Learn how to use Codegen's core APIs to analyze and transform code. Understand function call patterns and manipulate call sites. - Work with module imports, exports, and manage dependencies. + Work with module imports and manage dependencies. +ExportStatement inherits from [Statement](/building-with-codegen/statements-and-code-blocks), providing operations like `remove()` and `insert_before()`. This is particularly useful when you want to manipulate the entire export declaration. + + +## Export Types + +Codegen supports several types of exports: + +```typescript +// Direct exports +export const value = 42; // Value export +export function myFunction() {} // Function export +export class MyClass {} // Class export +export type MyType = string; // Type export +export interface MyInterface {} // Interface export +export enum MyEnum {} // Enum export + +// Re-exports +export { foo, bar } from './other-file'; // Named re-exports +export type { Type } from './other-file'; // Type re-exports +export * from './other-file'; // Wildcard re-exports +export * as utils from './other-file'; // Namespace re-exports + +// Aliased exports +export { foo as foop }; // Basic alias +export { foo as default }; // Default export alias +export { bar as baz } from './other-file'; // Re-export with alias +``` + +## Working with Exports + +The Export API provides methods to identify and filter exports: + +```python +# Check export types +for exp in file.exports: + if exp.is_type_export(): + print(f"Type export: {exp.name}") + elif exp.is_default_export(): + print(f"Default export: {exp.name}") + elif exp.is_wildcard_export(): + print(f"Wildcard export from: {exp.from_file.filepath}") + +# Work with re-exports +for exp in file.exports: + if exp.is_reexport(): + if exp.is_external_export: + print(f"External re-export: {exp.name} from {exp.from_file.filepath}") + else: + print(f"Internal re-export: {exp.name}") +``` + +## Export Resolution + +You can trace exports to their original symbols: + +```python +for exp in file.exports: + if exp.is_reexport(): + # Get original and current symbols + current = exp.exported_symbol + original = exp.resolved_symbol + + print(f"Re-exporting {original.name} from {exp.from_file.filepath}") + print(f"Through: {' -> '.join(e.file.filepath for e in exp.export_chain)}") +``` + +## Common Operations + +Here are common operations for working with exports: + +```python +# Add new export +file.add_export("MyComponent") + +# Add export with alias +file.add_export("MyComponent", alias="default") + +# Convert to type export +export = file.get_export("MyType") +export.make_type_export() + +# Remove export +export.remove() # Removes export but keeps symbol + +# Update export properties +export.update( + name="NewName", + is_type=True, + is_default=False +) +``` + +## Managing Re-exports + +Common patterns for working with re-exports: + +```python +# Create public API +index_file = codebase.get_file("index.ts") + +# Re-export from internal files +for internal_file in codebase.files: + if internal_file.name != "index": + for symbol in internal_file.symbols: + if symbol.is_public: + index_file.add_export( + symbol, + from_file=internal_file + ) + +# Convert default to named exports +for exp in file.exports: + if exp.is_default_export(): + exp.make_named_export() + +# Consolidate re-exports +from collections import defaultdict + +file_exports = defaultdict(list) +for exp in file.exports: + if exp.is_reexport(): + file_exports[exp.from_file].append(exp) + +for from_file, exports in file_exports.items(): + if len(exports) > 1: + # Create consolidated re-export + names = [exp.name for exp in exports] + file.add_export_from_source( + f"export {{ {', '.join(names)} }} from '{from_file.filepath}'" + ) + # Remove individual exports + for exp in exports: + exp.remove() +``` + + +When managing exports, consider the impact on your module's public API. Not all symbols that can be exported should be exported. + \ No newline at end of file diff --git a/docs/building-with-codegen/imports-and-exports.mdx b/docs/building-with-codegen/imports-and-exports.mdx index 4f58d1c05..184812221 100644 --- a/docs/building-with-codegen/imports-and-exports.mdx +++ b/docs/building-with-codegen/imports-and-exports.mdx @@ -102,6 +102,117 @@ for exp in file.exports: # Iterates over all Export symbols empty. +## Working with Imports + +### Working with External Modules + +When working with imports, you often need to distinguish between imports from your project and external packages (like `react` or `lodash`). The [ExternalModule](/api-reference/core/ExternalModule) class helps with this: + +```python +# Check if an import is from an external module +for imp in file.imports: + if isinstance(imp.resolved_symbol, ExternalModule): + print(f"External import: {imp.name} from {imp.module}") + else: + print(f"Local import: {imp.name}") + +# Skip processing external imports +for function in codebase.functions: + for call in function.call_sites: + if isinstance(call.function_definition, ExternalModule): + # Skip external function calls (like React.useState) + continue + # Process local function calls... +``` + +### Common Import Operations + +Here are common operations you can perform on imports: + +```python +# Change the module an import comes from +import_stmt = file.get_import("MyComponent") +import_stmt.set_module("./new/path") # Updates import path + +# Add/update import alias +import_stmt.set_alias("MyAlias") # import X as MyAlias + +# Convert between import styles (TypeScript) +import_stmt.make_type_import() # Converts to 'import type' +import_stmt.make_value_import() # Removes 'type' modifier + +# Update multiple properties +import_stmt.update( + module="./new/path", + alias="NewAlias", + is_type=True +) +``` + +### Bulk Import Management + +Here's how to perform operations on multiple imports: + +```python +# Update all imports from a specific module +old_path = "./old/path" +new_path = "./new/path" + +for imp in file.imports: + if imp.module == old_path: + imp.set_module(new_path) + +# Remove all unused imports +for imp in file.imports: + if not imp.usages and not isinstance(imp.resolved_symbol, ExternalModule): + print(f"Removing unused import: {imp.name}") + imp.remove() + +# Consolidate imports from the same module +from collections import defaultdict + +# Group imports by module +module_imports = defaultdict(list) +for imp in file.imports: + module_imports[imp.module].append(imp) + +# Consolidate each group +for module, imports in module_imports.items(): + if len(imports) > 1: + # Create new combined import + symbols = [imp.name for imp in imports] + file.add_import_from_import_string( + f"import {{ {', '.join(symbols)} }} from '{module}'" + ) + # Remove old imports + for imp in imports: + imp.remove() +``` + +### Import Resolution + +You can trace import chains and resolve symbols: + +```python +# Follow import chain to original symbol +import_stmt = file.get_import("MyComponent") +original_symbol = import_stmt.resolved_symbol + +if original_symbol: + print(f"Imported from: {original_symbol.file.filepath}") + print(f"Original name: {original_symbol.name}") + +# Check if import is re-exported +for imp in file.imports: + if imp.is_reexported: + print(f"Re-exported symbol: {imp.name}") + print(f"Through files: {' -> '.join(exp.file.filepath for exp in imp.export_chain)}") +``` + + +When working with imports, always check if they resolve to external modules before modification. This prevents accidentally modifying imports from third-party packages. + + ## Identifying Export Types Codegen provides several methods to identify and filter different types of exports in your TypeScript code. Here's how to work with various export patterns: diff --git a/docs/building-with-codegen/imports.mdx b/docs/building-with-codegen/imports.mdx new file mode 100644 index 000000000..c66e7736d --- /dev/null +++ b/docs/building-with-codegen/imports.mdx @@ -0,0 +1,170 @@ +--- +title: "The Import API" +sidebarTitle: "Imports" +icon: "file-import" +iconType: "solid" +--- + +The [Import](/api-reference/core/Import) API provides tools for working with imports and managing dependencies between files. + +## Accessing Imports + +You can access these through [File.imports](/api-reference/core/File#imports) and [File.import_statements](/api-reference/core/File#import-statements): + +```python +# Direct access to imports via file +for imp in file.imports: + ... + +# Grab by name of symbol being imported +imp = file.get_import('math') + +# Grab and filter from a codebase +from codegen.sdk import ExternalModule + +external_imports = [i for i in codebase.imports if isinstance(i, ExternalModule)] +``` + +## Common Operations + +The Import API provides several methods for modifying imports: + +```python +# Get a specific import +import_stmt = file.get_import("MyComponent") + +# Change import source +import_stmt.set_module("./new/path") + +# Add/update alias +import_stmt.set_alias("MyAlias") # import X as MyAlias + +# TypeScript-specific operations +import_stmt.make_type_import() # Convert to 'import type' +import_stmt.make_value_import() # Remove 'type' modifier + +# Update multiple properties +import_stmt.update( + module="./new/path", + alias="NewAlias", + is_type=True +) +``` + +## Import Resolution + +Imports can be traced to their original symbols: + +```python +# Follow import chain to source +import_stmt = file.get_import("MyComponent") +original = import_stmt.resolved_symbol + +if original: + print(f"Defined in: {original.file.filepath}") + print(f"Original name: {original.name}") + +# Get file relationships +print(f"From file: {import_stmt.from_file.filepath}") +print(f"To file: {import_stmt.to_file.filepath}") +``` + +## Working with External Modules + +You can determine if an import references an [ExternalModule](/api-reference/core/ExternalModule) by checking the type of [Import.imported_symbol](/api-reference/core/Import#imported-symbol), like so: + +```python +# Check if import is from external package +for imp in file.imports: + if isinstance(imp.imported_symbol, ExternalModule): + print(f"External import: {imp.name} from {imp.module}") + else: + print(f"Local import: {imp.name}") +``` + +Learn more about [external modules here](/building-with-codegen/external-modules) + + +## Bulk Operations + +Here are patterns for working with multiple imports: + +```python +# Update imports from a specific module +old_path = "./old/path" +new_path = "./new/path" + +for imp in file.imports: + if imp.module == old_path: + imp.set_module(new_path) + +# Remove unused imports (excluding external) +for imp in file.imports: + if not imp.usages and not isinstance(imp.resolved_symbol, ExternalModule): + print(f"Removing: {imp.name}") + imp.remove() + +# Consolidate duplicate imports +from collections import defaultdict + +module_imports = defaultdict(list) +for imp in file.imports: + module_imports[imp.module].append(imp) + +for module, imports in module_imports.items(): + if len(imports) > 1: + # Create combined import + symbols = [imp.name for imp in imports] + file.add_import_from_import_string( + f"import {{ {', '.join(symbols)} }} from '{module}'" + ) + # Remove old imports + for imp in imports: + imp.remove() +``` + + +Always check if imports resolve to external modules before modification to avoid breaking third-party package imports. + + +## Import Statements vs Imports + +Codegen provides two levels of abstraction for working with imports: + +- [ImportStatement](/api-reference/core/ImportStatement) - Represents a complete import statement +- [Import](/api-reference/core/Import) - Represents individual imported symbols + + +```python Python +# One ImportStatement containing multiple Import objects +from math import sin, cos as cosine +# Creates: +# - Import for 'sin' +# - Import for 'cos' with alias 'cosine' +``` + +```typescript Typescript +// One ImportStatement containing multiple Import objects +import { sin, cos as cosine } from 'math'; +// Creates: +// - Import for 'sin' +// - Import for 'cos' with alias 'cosine' +``` + + +You can access these through [File.imports](/api-reference/core/File#imports) and [File.import_statements](/api-reference/core/File#import-statements): + +```python +# Direct access to imports +for imp in file.imports: + ... + +# Access to imports via statements +for stmt in file.import_statements: + for imp in stmt.imports: + ... +``` + + +ImportStatement inherits from [Statement](/building-with-codegen/statements-and-code-blocks), providing operations like `remove()` and `insert_before()`. + \ No newline at end of file diff --git a/docs/building-with-codegen/statements-and-code-blocks.mdx b/docs/building-with-codegen/statements-and-code-blocks.mdx index cac0b67ac..b768ba4b1 100644 --- a/docs/building-with-codegen/statements-and-code-blocks.mdx +++ b/docs/building-with-codegen/statements-and-code-blocks.mdx @@ -124,7 +124,7 @@ Codegen supports various statement types, each with specific APIs: ### [Import Statements](../api-reference/core/ImportStatement) / [Export Statements](../api-reference/core/ExportStatement) - See [Import and Exports](../building-with-codegen/imports-and-exports) for + See [imports](/building-with-codegen/imports) and [exports](../building-with-codegen/exports) for more details. diff --git a/docs/images/nether-portal.png b/docs/images/nether-portal.png index 4b8cf6dd5..21312c270 100644 Binary files a/docs/images/nether-portal.png and b/docs/images/nether-portal.png differ diff --git a/docs/introduction/getting-started.mdx b/docs/introduction/getting-started.mdx index 96733d86f..b00f5c344 100644 --- a/docs/introduction/getting-started.mdx +++ b/docs/introduction/getting-started.mdx @@ -9,19 +9,18 @@ Follow our step-by-step tutorial to get up and running with Codegen. ## Installation -We recommend using [`pipx`](https://pypa.github.io/pipx/) to install Codegen globally. `pipx` is a tool to help you install and run Python applications in isolated environments. This isolation ensures that Codegen's dependencies won't conflict with your other Python projects. +We recommend using [uv](https://github.com/astral-sh/uv) to install Codegen's CLI globally. -First, install `pipx` if you haven't already: +First, install `uv` if you haven't already: ```bash -brew install pipx -pipx ensurepath # Ensure pipx binaries are on your PATH +curl -LsSf https://astral.sh/uv/install.sh | sh ``` -Then install Codegen globally: +Then install Codegen: ```bash -pipx install codegen +uv tool install codegen ``` @@ -31,7 +30,8 @@ pipx install codegen ## Quick Start with Jupyter -The fastest way to explore a codebase is using Jupyter. Codegen provides a built-in command to set this up: +The [codgen notebook](/cli/notebook) command creates a virtual environment and opens a jupyter notebook for quick prototyping. This is often the fastest way to get up and running. + ```bash # Navigate to your repository @@ -57,7 +57,7 @@ This will: ## Initializing a Codebase -Instantiating a [`Codebase`](/api-reference/core/Codebase) will automatically parse a codebase and make it available for manipulation. +Instantiating a [Codebase](/api-reference/core/Codebase) will automatically parse a codebase and make it available for manipulation. ```python from codegen import Codebase @@ -83,10 +83,10 @@ Let's explore the codebase we just initialized. Here are some common patterns for code navigation in Codegen: -- Iterate over all [`Functions`](/api-reference/core/Function) with [`Codebase.functions`](/api-reference/core/Codebase#functions) -- View class inheritance with [`Class.superclasses`](/api-reference/core/Class#superclasses) -- View function call-sites with [`Function.call_sites`](/api-reference/core/Function#call-sites) -- View function usages with [`Function.usages`](/api-reference/core/Function#usages) +- Iterate over all [Functions](/api-reference/core/Function) with [Codebase.functions](/api-reference/core/Codebase#functions) +- View class inheritance with [Class.superclasses](/api-reference/core/Class#superclasses) +- View function call-sites with [Function.call_sites](/api-reference/core/Function#call-sites) +- View function usages with [Function.usages](/api-reference/core/Function#usages) ```python # Print overall stats @@ -175,8 +175,8 @@ codebase.commit() In order to commit changes to your filesystem, you must call - [`Codebase.commit()`](/api-reference/core/Codebase#commit). Learn more about - [Commit and Reset](/building-with-codegen/commit-and-reset). + [codebase.commit()](/api-reference/core/Codebase#commit). Learn more about + [commit() and reset()](/building-with-codegen/commit-and-reset). ### Finding Specific Content @@ -281,8 +281,7 @@ if base_class: Learn more about [dependencies and - references](/building-with-codegen/dependencies-and-usages) or [imports - and exports](/building-with-codegen/imports-and-exports). + references](/building-with-codegen/dependencies-and-usages) or [imports](/building-with-codegen/imports) and [exports](/building-with-codegen/exports). ## What's Next? diff --git a/docs/introduction/how-it-works.mdx b/docs/introduction/how-it-works.mdx index 9ce1f8445..ac908a79b 100644 --- a/docs/introduction/how-it-works.mdx +++ b/docs/introduction/how-it-works.mdx @@ -40,7 +40,7 @@ Codegen's graph construction happens in two stages: 1. **AST Parsing**: We use [Tree-sitter](https://tree-sitter.github.io/tree-sitter/) as our foundation for parsing code into Abstract Syntax Trees. Tree-sitter provides fast, reliable parsing across multiple languages. -2. **Multi-file Graph Construction**: Custom parsing logic, implemented in [rustworkx](https://github.com/Qiskit/rustworkx) and Python, analyzes these ASTs to construct a more sophisticated graph structure. This graph captures relationships between [symbols](/api-reference/core/Symbol), [files](/api-reference/core/SourceFile), [imports](/api-reference/core/Import), and more. +2. **Multi-file Graph Construction**: Custom parsing logic, implemented in [rustworkx](https://github.com/Qiskit/rustworkx) and Python, analyzes these ASTs to construct a more sophisticated graph structure. This graph captures relationships between [symbols](/building-with-codegen/symbol-api), [files](/building-with-codegen/files-and-directories), [imports](/building-with-codegen/imports), and more. ### Performance Through Pre-computation diff --git a/docs/introduction/overview.mdx b/docs/introduction/overview.mdx index da55ea2b4..0a21491d1 100644 --- a/docs/introduction/overview.mdx +++ b/docs/introduction/overview.mdx @@ -45,7 +45,7 @@ function.set_docstring('new docstring') # set docstring ## Installation ```bash -pip install codegen +uv tool install codegen ``` ## Get Started diff --git a/docs/mint.json b/docs/mint.json index ae0610cf0..f4f2784ff 100644 --- a/docs/mint.json +++ b/docs/mint.json @@ -43,13 +43,13 @@ "name": "CLI", "url": "/cli" }, - { - "name": "Changelog", - "url": "/changelog" - }, { "name": "Blog", "url": "/blog" + }, + { + "name": "Changelog", + "url": "/changelog" } ], "navigation": [ @@ -98,9 +98,10 @@ "building-with-codegen/the-editable-api", "building-with-codegen/symbol-api", "building-with-codegen/class-api", + "building-with-codegen/imports", + "building-with-codegen/exports", "building-with-codegen/inheritable-behaviors", "building-with-codegen/statements-and-code-blocks", - "building-with-codegen/imports-and-exports", "building-with-codegen/dependencies-and-usages", "building-with-codegen/function-calls-and-callsites", "building-with-codegen/variable-assignments", @@ -133,7 +134,7 @@ }, { "group": "Blog", - "pages": ["blog/posts", "blog/act-via-code", "blog/codemod-frameworks"] + "pages": ["blog/posts", "blog/act-via-code"] }, { "group": "API Reference", diff --git a/docs/prism.css b/docs/prism.css index bdf60227e..73ec879af 100644 --- a/docs/prism.css +++ b/docs/prism.css @@ -99,9 +99,26 @@ pre[class*="language-"] { .dark .prose p code { color: #ffca85 !important; } +.dark .prose p code .comment { + color: #6d6d6d !important; +} .dark .prose ul code { color: #ffca85 !important; } +.dark .prose ul code .comment { + color: #6d6d6d !important; +} + +/* Comments in code blocks should be gray */ +.token.comment { + color: #6d6d6d !important; +} +pre .token.comment { + color: #6d6d6d !important; +} +code .token.comment { + color: #6d6d6d !important; +} /* links are purple */ .prose a { @@ -150,9 +167,6 @@ pre[class*="language-"] { .token.property { color: #ffca85 !important; } -.token.comment { - color: #6d6d6d !important; -} .token.class-name { color: #82e2ff !important; } diff --git a/docs/snippets/Attribute.mdx b/docs/snippets/Attribute.mdx index 13f3118e2..99f8af2a3 100644 --- a/docs/snippets/Attribute.mdx +++ b/docs/snippets/Attribute.mdx @@ -1,5 +1,5 @@ export const Attribute = ({ type, description }) => ( -
+
{type}
diff --git a/docs/snippets/HorizontalDivider.mdx b/docs/snippets/HorizontalDivider.mdx index a8b829fc2..f835261ff 100644 --- a/docs/snippets/HorizontalDivider.mdx +++ b/docs/snippets/HorizontalDivider.mdx @@ -1,6 +1,6 @@ export const HorizontalDivider = ({light=false}) => (
-
-
+
+
) \ No newline at end of file diff --git a/docs/snippets/Parameter.mdx b/docs/snippets/Parameter.mdx index 888701304..cea11e7ef 100644 --- a/docs/snippets/Parameter.mdx +++ b/docs/snippets/Parameter.mdx @@ -1,5 +1,5 @@ export const Parameter = ({name, type, description, defaultValue}) => ( -
+
@@ -13,7 +13,7 @@ export const Parameter = ({name, type, description, defaultValue}) => (
{defaultValue ? (
- default: + default: {defaultValue}
) : ( diff --git a/docs/snippets/ParameterWrapper.mdx b/docs/snippets/ParameterWrapper.mdx index 9866604e5..ce3bcde6d 100644 --- a/docs/snippets/ParameterWrapper.mdx +++ b/docs/snippets/ParameterWrapper.mdx @@ -1,7 +1,7 @@ export const ParameterWrapper = ({ children }) => (

Parameters

-
+
{children}
diff --git a/docs/snippets/Return.mdx b/docs/snippets/Return.mdx index 9404f569e..2d42e9bda 100644 --- a/docs/snippets/Return.mdx +++ b/docs/snippets/Return.mdx @@ -1,7 +1,7 @@ export const Return = ({return_type, description}) => (

Returns

-
+
diff --git a/docs/tutorials/converting-default-exports.mdx b/docs/tutorials/converting-default-exports.mdx index 91c8bd579..dd3064c8b 100644 --- a/docs/tutorials/converting-default-exports.mdx +++ b/docs/tutorials/converting-default-exports.mdx @@ -6,7 +6,7 @@ icon: "arrow-right-arrow-left" iconType: "solid" --- -Codegen provides tools to help you migrate away from default exports to named exports in your TypeScript codebase. This tutorial builds on the concepts covered in [Imports and exports](/building-with-codegen/imports-and-exports) to show you how to automate this conversion process. +Codegen provides tools to help you migrate away from default exports to named exports in your TypeScript codebase. This tutorial builds on the concepts covered in [exports](/building-with-codegen/exports) to show you how to automate this conversion process. ## Overview @@ -134,7 +134,7 @@ Remember to test thoroughly after converting default exports, as this change aff ### Related tutorials - [Managing typescript exports](/tutorials/managing-typescript-exports) -- [Imports and exports](/building-with-codegen/imports-and-exports) +- [Exports](/building-with-codegen/exports) - [Dependencies and usages](/building-with-codegen/dependencies-and-usages) ## Complete Codemod diff --git a/docs/tutorials/flask-to-fastapi.mdx b/docs/tutorials/flask-to-fastapi.mdx index 8cce27e0d..c093e28ee 100644 --- a/docs/tutorials/flask-to-fastapi.mdx +++ b/docs/tutorials/flask-to-fastapi.mdx @@ -25,7 +25,7 @@ Let's walk through each step using Codegen. First, we need to update Flask imports to their FastAPI equivalents and modify the app initialization: - Learn more about [imports here](/building-with-codegen/imports-and-exports). + Learn more about [imports here](/building-with-codegen/imports). ```python diff --git a/docs/tutorials/managing-typescript-exports.mdx b/docs/tutorials/managing-typescript-exports.mdx index 067545402..aa28633f3 100644 --- a/docs/tutorials/managing-typescript-exports.mdx +++ b/docs/tutorials/managing-typescript-exports.mdx @@ -6,7 +6,7 @@ icon: "ship" iconType: "solid" --- -Codegen provides powerful tools for managing and reorganizing exports in TypeScript codebases. This tutorial builds on the concepts covered in [Imports and exports](/building-with-codegen/imports-and-exports) to show you how to automate common export management tasks and ensure your module boundaries stay clean and maintainable. +Codegen provides powerful tools for managing and reorganizing exports in TypeScript codebases. This tutorial builds on the concepts covered in [exports](/building-with-codegen/exports) to show you how to automate common export management tasks and ensure your module boundaries stay clean and maintainable. ## Common Export Management Tasks @@ -194,7 +194,7 @@ Remember that managing exports is an iterative process. You may need to run the ### Related tutorials - [Moving symbols](/building-with-codegen/moving-symbols) -- [Imports and exports](/building-with-codegen/imports-and-exports) +- [Exports](/building-with-codegen/exports) - [Dependencies and usages](/building-with-codegen/dependencies-and-usages) ## Complete Codemod diff --git a/docs/tutorials/organize-your-codebase.mdx b/docs/tutorials/organize-your-codebase.mdx index ff8a188d3..81767871d 100644 --- a/docs/tutorials/organize-your-codebase.mdx +++ b/docs/tutorials/organize-your-codebase.mdx @@ -140,7 +140,7 @@ print("\n✅ Import cycles resolved!") Most operations in Codegen will automatically handle updaging [dependencies](/building-with-codegen/dependencies-and-usages) and - [imports](/building-with-codegen/imports-and-exports). See [Moving + [imports](/building-with-codegen/imports). See [Moving Symbols](/building-with-codegen/moving-symbols) to learn more.