= fetchData();
+```
+
+## Tracking Usages and Dependencies
+
+Like other symbols, Assignments support [usages](/api-reference/core/Assignment#usages) and [dependencies](/api-reference/core/Assignment#dependencies).
+
+```python
+assignment = file.code_block.get_local_var_assignment("userId")
+
+# Get all usages of the assignment
+usages = assignment.usages
+
+# Get all dependencies of the assignment
+dependencies = assignment.dependencies
+```
+
+
+ See [Dependencies and Usages](/building-with-codegen/dependencies-and-usages)
+ for more details.
+
+
+
+---
+title: "Local Variables"
+sidebarTitle: "Local Variables"
+icon: "cube"
+iconType: "solid"
+---
+
+This document explains how to work with local variables in Codegen.
+
+## Overview
+
+Through the [CodeBlock](../api-reference/core/CodeBlock) class, Codegen exposes APIs for analyzing and manipulating local variables within code blocks.
+
+- [local_var_assignments](../api-reference/core/CodeBlock#local-var-assignments): find all [Assignments](../api-reference/core/Assignment) in this scope
+- [get_local_var_assignment(...)](../api-reference/core/CodeBlock#get-local-var-assignment): get specific [Assignments](../api-reference/core/Assignment) by name
+- [rename_local_variable(...)](../api-reference/core/CodeBlock#rename-local-variable): rename variables safely across the current scope
+
+## Basic Usage
+
+Every code block (function body, loop body, etc.) provides access to its local variables:
+
+```python
+# Get all local variables in a function
+function = codebase.get_function("process_data")
+local_vars = function.code_block.local_var_assignments
+for var in local_vars:
+ print(var.name)
+
+# Find a specific variable
+config_var = function.code_block.get_local_var_assignment("config")
+config_var.rename("settings") # Updates all references safely
+
+# Rename a variable used in this scope (but not necessarily declared here)
+function.rename_local_variable("foo", "bar")
+```
+
+## Fuzzy Matching
+
+Codegen supports fuzzy matching when searching for local variables. This allows you to find variables whose names contain a substring, rather than requiring exact matches:
+
+```python
+# Get all local variables containing "config"
+function = codebase.get_function("process_data")
+
+# Exact match - only finds variables named exactly "config"
+exact_matches = function.code_block.get_local_var_assignments("config")
+# Returns: config = {...}
+
+# Fuzzy match - finds any variable containing "config"
+fuzzy_matches = function.code_block.get_local_var_assignments("config", fuzzy_match=True)
+# Returns: config = {...}, app_config = {...}, config_settings = {...}
+
+# Fuzzy matching also works for variable usages
+usages = function.code_block.get_variable_usages("config", fuzzy_match=True)
+
+# And for renaming variables
+function.code_block.rename_variable_usages("config", "settings", fuzzy_match=True)
+# Renames: config -> settings, app_config -> app_settings, config_settings -> settings_settings
+```
+
+
+ Be careful with fuzzy matching when renaming variables, as it will replace the
+ matched substring in all variable names. This might lead to unintended renames
+ like `config_settings` becoming `settings_settings`.
+
+
+
+---
+title: "Comments and Docstrings"
+sidebarTitle: "Comments & Docstrings"
+icon: "comment"
+iconType: "solid"
+---
+
+Codegen enables reading, modifying, and manipulating comments and docstrings while preserving proper formatting.
+
+This guide describes proper usage of the following classes:
+
+- [Comment](/api-reference/core/Comment) - Represents a single comment.
+- [CommentGroup](/api-reference/core/CommentGroup) - Represents a group of comments.
+
+## Accessing with Comments
+
+Comments can be accessed through any symbol or directly from code blocks. Each comment is represented by a `Comment` object that provides access to both the raw source and parsed text:
+
+```python
+# Find all comments in a file
+file = codebase.get_file("my_file.py")
+for comment in file.code_block.comments:
+ print(comment.text)
+
+# Access comments associated with a symbol
+symbol = file.get_symbol("my_function")
+if symbol.comment:
+ print(symbol.comment.text) # Comment text without delimiters
+ print(symbol.comment.source) # Full comment including delimiters
+
+# Access inline comments
+if symbol.inline_comment:
+ print(symbol.inline_comment.text)
+
+# Accessing all comments in a function
+for comment in symbol.code_block.comments:
+ print(comment.text)
+```
+
+### Editing Comments
+
+Comments can be modified using the `edit_text()` method, which handles formatting and delimiters automatically:
+
+```python
+# Edit a regular comment
+symbol.comment.edit_text("Updated comment text")
+
+# Edit an inline comment
+symbol.set_inline_comment("New inline comment")
+```
+
+### Comment Groups
+
+Multiple consecutive comments are automatically grouped into a `CommentGroup`, which can be edited as a single unit:
+
+```python
+# Original comments:
+# First line
+# Second line
+# Third line
+
+comment_group = symbol.comment
+print(comment_group.text) # "First line\nSecond line\nThird line"
+
+# Edit the entire group at once
+comment_group.edit_text("New first line\nNew second line")
+```
+
+## Working with Docstrings
+
+Docstrings are special comments that document functions, classes, and modules. Codegen provides similar APIs for working with docstrings:
+
+```python
+function = file.get_symbol("my_function")
+if function.docstring:
+ print(function.docstring.text) # Docstring content
+ print(function.docstring.source) # Full docstring with delimiters
+```
+
+### Adding Docstrings
+
+You can add docstrings to any symbol that supports them:
+
+```python
+# Add a single-line docstring
+function.set_docstring("A brief description")
+
+# Add a multi-line docstring
+function.set_docstring("""
+ A longer description that
+ spans multiple lines.
+
+ Args:
+ param1: Description of first parameter
+""")
+```
+
+### Language-Specific Formatting
+
+Codegen automatically handles language-specific docstring formatting:
+
+```python
+# Python: Uses triple quotes
+def my_function():
+ """Docstring is formatted with triple quotes."""
+ pass
+```
+
+```typescript
+// TypeScript: Uses JSDoc style
+function myFunction() {
+ /** Docstring is formatted as JSDoc */
+}
+```
+
+### Editing Docstrings
+
+Like comments, docstrings can be modified while preserving formatting:
+
+```python
+# Edit a docstring
+function.docstring.edit_text("Updated documentation")
+
+# Edit a multi-line docstring
+function.docstring.edit_text("""
+ Updated multi-line documentation
+ that preserves indentation and formatting.
+""")
+```
+
+## Comment Operations
+
+Codegen provides utilities for working with comments at scale. For example, you can update or remove specific types of comments across your codebase:
+
+```python
+# Example: Remove eslint disable comments for a specific rule
+for file in codebase.files:
+ for comment in file.code_block.comments:
+ if "eslint-disable" in comment.source:
+ # Check if comment disables specific rule
+ if "@typescript-eslint/no-explicit-any" in comment.text:
+ comment.remove()
+```
+
+
+ When editing multi-line comments or docstrings, Codegen automatically handles
+ indentation and maintains the existing comment style.
+
+
+## Special APIs and AI Integration
+
+### Google Style Docstrings
+
+Codegen supports Google-style docstrings and can handle their specific formatting, using the [CommentGroup.to_google_docstring(...)](/api-reference/core/CommentGroup#to-google-docstring) method.
+
+```python
+# Edit while preserving Google style
+symbol_a = file.get_symbol("SymbolA")
+func_b = symbol_a.get_method("funcB")
+func_b.docstring.to_google_docstring(func_b)
+```
+
+### Using AI for Documentation
+
+Codegen integrates with LLMs to help generate and improve documentation. You can use the [Codebase.ai(...)](/api-reference/core/Codebase#ai) method to:
+
+- Generate comprehensive docstrings
+- Update existing documentation
+- Convert between documentation styles
+- Add parameter descriptions
+
+```python
+# Generate a docstring using AI
+function = codebase.get_function("my_function")
+
+new_docstring = codebase.ai(
+ "Generate a comprehensive docstring in Google style",
+ target=function
+ context={
+ # provide additional context to the LLM
+ 'usages': function.usages,
+ 'dependencies': function.dependencies
+ }
+)
+function.set_docstring(new_docstring)
+```
+
+
+ Learn more about AI documentation capabilities in our [Documentation
+ Guide](/tutorials/creating-documentation) and [LLM Integration
+ Guide](/building-with-codegen/calling-out-to-llms).
+
+
+### Documentation Coverage
+
+You can analyze and improve documentation coverage across your codebase:
+
+```python
+# Count documented vs undocumented functions
+total = 0
+documented = 0
+for function in codebase.functions:
+ total += 1
+ if function.docstring:
+ documented += 1
+
+coverage = (documented / total * 100) if total > 0 else 0
+print(f"Documentation coverage: {coverage:.1f}%")
+```
+
+
+ Check out the [Documentation Guide](/tutorials/creating-documentation) for
+ more advanced coverage analysis and bulk documentation generation.
+
+
+
+---
+title: "External Modules"
+sidebarTitle: "External Modules"
+icon: "box-archive"
+iconType: "solid"
+---
+
+Codegen provides a way to handle imports from external packages and modules through the [ExternalModule](/api-reference/core/ExternalModule) class.
+
+```python
+# Python examples
+import datetime
+from requests import get
+
+# TypeScript/JavaScript examples
+import React from 'react'
+import { useState, useEffect } from 'react'
+import type { ReactNode } from 'react'
+import axios from 'axios'
+```
+
+## What are External Modules?
+
+When writing code, you often import from packages that aren't part of your project - like `datetime` and `requests` in Python, or `react` and `axios` in TypeScript. In Codegen, these are represented as [ExternalModule](/api-reference/core/ExternalModule) instances.
+
+```python
+for imp in codebase.imports:
+ if isinstance(imp.symbol, ExternalModule):
+ print(f"Importing from external package: {imp.resolved_symbol.source}")
+```
+
+
+ External modules are read-only - you can analyze them but can't modify their
+ implementation. This makes sense since they live in your project's
+ dependencies!
+
+
+## Working with External Modules
+
+The most common use case is handling external modules differently from your project's code:
+
+### Identifying Function Calls as External Modules
+
+For [FunctionCall](/api-reference/core/FunctionCall) instances, you can check if the function definition is an [ExternalModule](/api-reference/core/ExternalModule) via the [FunctionCall.function_definition](/api-reference/core/FunctionCall#function-definition) property:
+
+```python
+for fcall in file.function_calls:
+ definition = fcall.function_definition
+ if isinstance(definition, ExternalModule):
+ # Skip external functions
+ print(f'External function: {definition.name}')
+ else:
+ # Process local functions...
+ print(f'Local function: {definition.name}')
+```
+
+### Import Resolution
+
+Similarly, when working with imports, you can determine if they resolve to external modules by checking the [Import.resolved_symbol](/api-reference/core/Import#resolved-symbol) property:
+
+```python
+for imp in file.imports:
+ resolved = imp.resolved_symbol
+ if isinstance(resolved, ExternalModule):
+ print(f"Import from external package: from {imp.module} import {imp.name}")
+```
+
+
+ Use `isinstance(symbol, ExternalModule)` to reliably identify external
+ modules. This works better than checking names or paths since it handles all
+ edge cases.
+
+
+## Properties and Methods
+
+External modules provide several useful properties:
+
+```python
+# Get the module name
+module_name = external_module.name # e.g. "datetime" or "useState"
+
+# Check if it's from node_modules (TypeScript/JavaScript)
+if external_module.filepath == "":
+ print("This is an external package from node_modules")
+```
+
+## Common Patterns
+
+Here are some typical ways you might work with external modules:
+
+### Skip External Processing:
+
+When modifying function calls or imports, skip external modules since they can't be changed:
+
+```python
+# Example from a codemod that adds type hints
+def add_type_hints(function):
+ if isinstance(function.definition, ExternalModule):
+ return # Can't add type hints to external modules like React.FC
+ # Add type hints to local functions...
+```
+
+### Analyze Dependencies
+
+Track which external packages your code uses:
+
+```python
+# Find all external package dependencies
+external_deps = set()
+for imp in codebase.imports:
+ if isinstance(imp.resolved_symbol, ExternalModule):
+ external_deps.add(imp.resolved_symbol.source)
+ # Will find things like 'react', 'lodash', 'datetime', etc.
+```
+
+
+ When working with imports, always handle external modules as a special case.
+ This ensures your codemods work correctly with both local and external code.
+
+
+
+---
+title: "Working with Type Annotations"
+sidebarTitle: "Type Annotations"
+icon: "code"
+iconType: "solid"
+---
+
+This guide covers the core APIs and patterns for working with type annotations in Codegen.
+
+## Type Resolution
+
+Codegen builds a complete dependency graph of your codebase, connecting functions, classes, imports, and their relationships. This enables powerful type resolution capabilities:
+
+```python
+from codegen import Codebase
+
+# Initialize codebase with dependency graph
+codebase = Codebase("./")
+
+# Get a function with a type annotation
+function = codebase.get_file("path/to/file.py").get_function("my_func")
+
+# Resolve its return type to actual symbols
+return_type = function.return_type
+resolved_symbols = return_type.resolved_types # Returns the actual Symbol objects
+
+# For generic types, you can resolve parameters
+if hasattr(return_type, "parameters"):
+ for param in return_type.parameters:
+ resolved_param = param.resolved_types # Get the actual type parameter symbols
+
+# For assignments, resolve their type
+assignment = codebase.get_file("path/to/file.py").get_assignment("my_var")
+resolved_type = assignment.type.resolved_types
+```
+
+
+ Type resolution follows imports and handles complex cases like type aliases, forward references, and generic type parameters.
+
+
+## Core Interfaces
+
+Type annotations in Codegen are built on two key interfaces:
+
+- [Typeable](/api-reference/core/Typeable) - The base interface for any node that can have a type annotation (parameters, variables, functions, etc). Provides `.type` and `.is_typed`.
+- [Type](/api-reference/core/Type) - The base class for all type annotations. Provides type resolution and dependency tracking.
+
+Any node that inherits from `Typeable` will have a `.type` property that returns a `Type` object, which can be used to inspect and modify type annotations.
+
+Learn more about [inheritable behaviors](/building-with-codegen/inheritable-behaviors) like Typeable here
+
+## Core Type APIs
+
+Type annotations can be accessed and modified through several key APIs:
+
+### Function Types
+
+The main APIs for function types are [Function.return_type](/api-reference/python/PyFunction#return-type) and [Function.set_return_type](/api-reference/python/PyFunction#set-return-type):
+
+```python
+# Get return type
+return_type = function.return_type # -> TypeAnnotation
+print(return_type.source) # "List[str]"
+print(return_type.is_typed) # True/False
+
+# Set return type
+function.set_return_type("List[str]")
+function.set_return_type(None) # Removes type annotation
+```
+
+### Parameter Types
+
+Parameters use [Parameter.type](/api-reference/core/Parameter#type) and [Parameter.set_type_annotation](/api-reference/core/Parameter#set-type-annotation):
+
+```python
+for param in function.parameters:
+ # Get parameter type
+ param_type = param.type # -> TypeAnnotation
+ print(param_type.source) # "int"
+ print(param_type.is_typed) # True/False
+
+ # Set parameter type
+ param.set_type("int")
+ param.set_type(None) # Removes type annotation
+```
+
+### Variable Types
+
+Variables and attributes use [Assignment.type](/api-reference/core/Assignment#type) and [Assignment.set_type_annotation](/api-reference/core/Assignment#set-type-annotation). This applies to:
+- Global variables
+- Local variables
+- Class attributes (via [Class.attributes](/api-reference/core/Class#attributes))
+
+```python
+# For global/local assignments
+assignment = file.get_assignment("my_var")
+var_type = assignment.type # -> TypeAnnotation
+print(var_type.source) # "str"
+
+# Set variable type
+assignment.set_type("str")
+assignment.set_type(None) # Removes type annotation
+
+# For class attributes
+class_def = file.get_class("MyClass")
+for attr in class_def.attributes:
+ # Each attribute has an assignment property
+ attr_type = attr.assignment.type # -> TypeAnnotation
+ print(f"{attr.name}: {attr_type.source}") # e.g. "x: int"
+
+ # Set attribute type
+ attr.assignment.set_type("int")
+
+# You can also access attributes directly by index
+first_attr = class_def.attributes[0]
+first_attr.assignment.set_type("str")
+```
+
+## Working with Complex Types
+
+### Union Types
+
+Union types ([UnionType](/api-reference/core/UnionType)) can be manipulated as collections:
+
+```python
+# Get union type
+union_type = function.return_type # -> A | B
+print(union_type.symbols) # ["A", "B"]
+
+# Add/remove options
+union_type.append("float")
+union_type.remove("None")
+
+# Check contents
+if "str" in union_type.options:
+ print("String is a possible type")
+```
+Learn more about [working with collections here](/building-with-codegen/collections)
+
+### Generic Types
+
+Generic types ([GenericType](/api-reference/core/GenericType)) expose their parameters as collection of [Parameters](/api-reference/core/Parameter):
+
+```python
+# Get generic type
+generic_type = function.return_type # -> GenericType
+print(generic_type.base) # "List"
+print(generic_type.parameters) # ["str"]
+
+# Modify parameters
+generic_type.parameters.append("int")
+generic_type.parameters[0] = "float"
+
+# Create new generic
+function.set_return_type("List[str]")
+```
+Learn more about [working with collections here](/building-with-codegen/collections)
+
+### Type Resolution
+
+Type resolution uses [`Type.resolved_value`](/api-reference/core/Type#resolved-value) to get the actual symbols that a type refers to:
+
+```python
+# Get the actual symbols for a type
+type_annotation = function.return_type # -> Type
+resolved_types = type_annotation.resolved_value # Returns an Expression, likely a Symbol or collection of Symbols
+
+# For generic types, resolve each parameter
+if hasattr(type_annotation, "parameters"):
+ for param in type_annotation.parameters:
+ param_types = param.resolved_value # Get symbols for each parameter
+
+# For union types, resolve each option
+if hasattr(type_annotation, "options"):
+ for option in type_annotation.options:
+ option_types = option.resolved_value # Get symbols for each union option
+```
+
+
+---
+title: "Moving Symbols"
+sidebarTitle: "Moving Symbols"
+icon: "arrows-up-down-left-right"
+iconType: "solid"
+---
+
+Codegen provides fast, configurable and safe APIs for moving symbols (functions, classes, variables) between files while automatically handling imports and dependencies.
+
+The key API is [`Symbol.move_to_file(...)`](/api-reference/core/Symbol#move-to-file).
+
+## Basic Symbol Movement
+
+Simply call [`Symbol.move_to_file(...)`](/api-reference/core/Symbol#move-to-file) to move a symbol to a new file.
+
+```python
+# Manipulation code:
+file1 = codebase.get_file("file1.py")
+file2 = codebase.get_file("file2.py")
+
+helper_func = file1.get_symbol("helper")
+
+# Ensure the destination file exists
+if not file2.exists():
+ file2 = codebase.create_file('file2.py')
+
+# Move the symbol
+helper_func.move_to_file(file2)
+```
+
+
+ By default, this will move any dependencies, including imports, to the new
+ file.
+
+
+## Moving Strategies
+
+The [`Symbol.move_to_file(...)`](/api-reference/core/Symbol#move-to-file) method accepts a `strategy` parameter, which can be used to control how imports are updated.
+
+Your options are:
+
+- `"update_all_imports"`: Updates all import statements across the codebase (default)
+- `"add_back_edge"`: Adds import and re-export in the original file
+
+`"add_back_edge"` is useful when moving a symbol that is depended on by other symbols in the original file, and will result in smaller diffs.
+
+
+ `"add_back_edge"` will result in circular dependencies if the symbol has
+ non-import dependencies in it's original file.
+
+
+## Moving Symbols in Bulk
+
+Make sure to call [`Codebase.commit(...)`](/api-reference/core/Codebase#commit) _after_ moving symbols in bulk for performant symbol movement.
+
+```python
+# Move all functions with a specific prefix
+for file in codebase.files:
+ for function in file.functions:
+ if function.name.startswith("pylsp_"):
+ function.move_to_file(
+ shared_file,
+ include_dependencies=True,
+ strategy="update_all_imports"
+ )
+
+# Commit the changes once, at the end
+codebase.commit()
+```
+
+
+---
+title: "Collections"
+sidebarTitle: "Collections"
+icon: "layer-group"
+iconType: "solid"
+---
+
+Codegen enables traversing and manipulating collections through the [List](/api-reference/core/List) and [Dict](/api-reference/core/Dict) classes.
+
+These APIs work consistently across Python and TypeScript while preserving formatting and structure.
+
+## Core Concepts
+
+The [List](/api-reference/core/List) and [Dict](/api-reference/core/Dict) classes provide a consistent interface for working with ordered sequences of elements. Key features include:
+
+- Standard sequence operations (indexing, length, iteration)
+- Automatic formatting preservation
+- Safe modification operations
+- Language-agnostic behavior
+- Comment and whitespace preservation
+
+Collections handle:
+
+- Proper indentation
+- Delimiters (commas, newlines)
+- Multi-line formatting
+- Leading/trailing whitespace
+- Nested structures
+
+## List Operations
+
+Lists in both Python and TypeScript can be manipulated using the same APIs:
+
+```python
+# Basic operations
+items_list = file.get_symbol("items").value # Get list value
+first = items_list[0] # Access elements
+length = len(items_list) # Get length
+items_list[0] = "new" # Modify element
+items_list.append("d") # Add to end
+items_list.insert(1, "x") # Insert at position
+del items_list[1] # Remove element
+
+# Iteration
+for item in items_list:
+ print(item.source)
+
+# Bulk operations
+items_list.clear() # Remove all elements
+```
+
+### Single vs Multi-line Lists
+
+Collections automatically preserve formatting:
+
+```python
+# Source code:
+items = [a, b, c]
+config = [
+ "debug",
+ "verbose",
+ "trace",
+]
+
+# Manipulation code:
+items_list = file.get_symbol("items").value
+items_list.append("d") # Adds new element
+
+config_list = file.get_symbol("config").value
+config_list.append("info") # Adds with formatting
+
+# Result:
+items = [a, b, c, d]
+config = [
+ "debug",
+ "verbose",
+ "trace",
+ "info",
+]
+```
+
+## Dictionary Operations
+
+Dictionaries provide a similar consistent interface:
+
+```python
+# Basic operations
+settings = file.get_symbol("settings").value # Get dict value
+value = settings["key"] # Get value
+settings["key"] = "value" # Set value
+del settings["key"] # Remove key
+has_key = "key" in settings # Check existence
+
+# Iteration
+for key in settings:
+ print(f"{key}: {settings[key]}")
+
+# Bulk operations
+settings.clear() # Remove all entries
+```
+
+
+---
+title: "Traversing the Call Graph"
+sidebarTitle: "Call Graph"
+icon: "sitemap"
+iconType: "solid"
+---
+
+Codegen provides powerful capabilities for analyzing and visualizing function call relationships in your codebase. This guide will show you how to traverse the call graph and create visual representations of function call paths.
+
+## Understanding Call Graph Traversal
+
+At the heart of call graph traversal is the [.function_calls](/api-reference/core/Function#function-calls) property, which returns information about all function calls made within a function:
+
+```python
+def example_function():
+ result = helper_function()
+ process_data()
+ return result
+
+# Get all calls made by example_function
+successors = example_function.function_calls
+for successor in successors:
+ print(f"Call: {successor.source}") # The actual function call
+ print(f"Called: {successor.function_definition.name}") # The function being called
+```
+
+## Building a Call Graph
+
+Here's how to build a directed graph of function calls using NetworkX:
+
+```python
+import networkx as nx
+from codegen.sdk.core.interfaces.callable import FunctionCallDefinition
+from codegen.sdk.core.function import Function
+
+def create_call_graph(start_func, end_func, max_depth=5):
+ G = nx.DiGraph()
+
+ def traverse_calls(parent_func, current_depth):
+ if current_depth > max_depth:
+ return
+
+ # Determine source node
+ if isinstance(parent_func, Function):
+ src_call = src_func = parent_func
+ else:
+ src_func = parent_func.function_definition
+ src_call = parent_func
+
+ # Skip external modules
+ if isinstance(src_func, ExternalModule):
+ return
+
+ # Traverse all function calls
+ for call in src_func.function_calls:
+ func = call.function_definition
+
+ # Skip recursive calls
+ if func.name == src_func.name:
+ continue
+
+ # Add nodes and edges
+ G.add_node(call)
+ G.add_edge(src_call, call)
+
+ # Check if we reached the target
+ if func == end_func:
+ G.add_edge(call, end_func)
+ return
+
+ # Continue traversal
+ traverse_calls(call, current_depth + 1)
+
+ # Initialize graph
+ G.add_node(start_func, color="blue") # Start node
+ G.add_node(end_func, color="red") # End node
+
+ # Start traversal
+ traverse_calls(start_func, 1)
+ return G
+
+# Usage example
+start = codebase.get_function("create_skill")
+end = codebase.get_function("auto_define_skill_description")
+graph = create_call_graph(start, end)
+```
+
+## Filtering and Visualization
+
+You can filter the graph to show only relevant paths and visualize the results:
+
+```python
+# Find all paths between start and end
+all_paths = nx.all_simple_paths(graph, source=start, target=end)
+
+# Create subgraph of only the nodes in these paths
+nodes_in_paths = set()
+for path in all_paths:
+ nodes_in_paths.update(path)
+filtered_graph = graph.subgraph(nodes_in_paths)
+
+# Visualize the graph
+codebase.visualize(filtered_graph)
+```
+
+## Advanced Usage
+
+### Example: Finding Dead Code
+
+You can use call graph analysis to find unused functions:
+
+```python
+def find_dead_code(codebase):
+ dead_functions = []
+ for function in codebase.functions:
+ if not any(function.function_calls):
+ # No other functions call this one
+ dead_functions.append(function)
+ return dead_functions
+```
+
+### Example: Analyzing Call Chains
+
+Find the longest call chain in your codebase:
+
+```python
+def get_max_call_chain(function):
+ G = nx.DiGraph()
+
+ def build_graph(func, depth=0):
+ if depth > 10: # Prevent infinite recursion
+ return
+ for call in func.function_calls:
+ called_func = call.function_definition
+ G.add_edge(func, called_func)
+ build_graph(called_func, depth + 1)
+
+ build_graph(function)
+ return nx.dag_longest_path(G)
+```
+
+
+The `.function_calls` property is optimized for performance and uses Codegen's internal graph structure to quickly traverse relationships. It's much faster than parsing the code repeatedly.
+
+
+
+When traversing call graphs, be mindful of:
+- Recursive calls that could create infinite loops
+- External module calls that might not be resolvable
+- Dynamic/runtime function calls that can't be statically analyzed
+
+
+
+---
+title: "React and JSX"
+sidebarTitle: "React and JSX"
+icon: "react"
+iconType: "brands"
+---
+
+GraphSitter exposes several React and JSX-specific APIs for working with modern React codebases.
+
+Key APIs include:
+
+- [Function.is_jsx](/api-reference/typescript/TSFunction#is-jsx) - Check if a function contains JSX elements
+- [Class.jsx_elements](/api-reference/typescript/TSClass#jsx-elements) - Get all JSX elements in a class
+- [Function.jsx_elements](/api-reference/typescript/TSFunction#jsx-elements) - Get all JSX elements in a function
+- [JSXElement](/api-reference/typescript/JSXElement) - Manipulate JSX elements
+- [JSXProp](/api-reference/typescript/JSXProp) - Manipulate JSX props
+
+
+ See [React Modernization](/tutorials/react-modernization) for tutorials and
+ applications of the concepts described here
+
+
+## Detecting React Components with `is_jsx`
+
+Codegen exposes a `is_jsx` property on both classes and functions, which can be used to check if a symbol is a React component.
+
+```python
+# Check if a function is a React component
+function = file.get_function("MyComponent")
+is_component = function.is_jsx # True for React components
+
+# Check if a class is a React component
+class_def = file.get_class("MyClassComponent")
+is_component = class_def.is_jsx # True for React class components
+```
+
+## Working with JSX Elements
+
+Given a React component, you can access its JSX elements using the [jsx_elements](/api-reference/typescript/TSFunction#jsx-elements) property.
+
+You can manipulate these elements by using the [JSXElement](/api-reference/typescript/JSXElement) and [JSXProp](/api-reference/typescript/JSXProp) APIs.
+
+```python
+# Get all JSX elements in a component
+for element in component.jsx_elements:
+ # Access element name
+ if element.name == "Button":
+ # Wrap element in a div
+ element.wrap("", "
")
+
+ # Get specific prop
+ specific_prop = element.get_prop("className")
+
+ # Iterate over all props
+ for prop in element.props:
+ if prop.name == "className":
+ # Set prop value
+ prop.set_value('"my-classname"')
+
+ # Modify element
+ element.set_name("NewComponent")
+ element.add_prop("newProp", "{value}")
+
+ # Get child JSX elements
+ child_elements = element.jsx_elements
+
+ # Wrap element in a JSX expression (preserves whitespace)
+ element.wrap("", "
")
+```
+
+## Common React Operations
+
+See [React Modernization](/tutorials/react-modernization) for more
+
+### Refactoring Components into Separate Files
+
+Split React components into individual files:
+
+```python
+# Find (named) React components
+react_components = [
+ func for func in codebase.functions
+ if func.is_jsx and func.name is not None
+]
+
+# Filter out those that are not the default export
+non_default_components = [
+ comp for comp in react_components
+ if not comp.export or not comp.export.is_default_export()
+]
+
+# Move these non-default components to new files
+for component in react_components:
+ if component != default_component:
+ # Create new file
+ new_file_path = '/'.join(component.filepath.split('/')[:-1]) + f"{component.name}.tsx"
+ new_file = codebase.create_file(new_file_path)
+
+ # Move component and update imports
+ component.move_to_file(new_file, strategy="add_back_edge")
+```
+
+
+ See [Moving Symbols](/building-with-codegen/moving-symbols) for more details
+ on moving symbols between files.
+
+
+### Updating Component Names and Props
+
+Replace components throughout the codebase with prop updates:
+
+```python
+# Find target component
+new_component = codebase.get_symbol("NewComponent")
+
+for function in codebase.functions:
+ if function.is_jsx:
+ # Update JSX elements
+ for element in function.jsx_elements:
+ if element.name == "OldComponent":
+ # Update name
+ element.set_name("NewComponent")
+
+ # Edit props
+ needs_clsx = not file.has_import("clsx")
+ for prop in element.props:
+ if prop.name == "className":
+ prop.set_value('clsx("new-classname")')
+ needs_clsx = True
+ elif prop.name == "onClick":
+ prop.set_name('handleClick')
+
+ # Add import if needed
+ if needs_clsx:
+ file.add_import_from_import_source("import clsx from 'clsx'")
+
+ # Add import if needed
+ if not file.has_import("NewComponent"):
+ file.add_symbol_import(new_component)
+```
+
+
+---
+title: "Codebase Visualization"
+sidebarTitle: "Visualization"
+icon: "share-nodes"
+iconType: "solid"
+---
+
+Codegen provides the ability to create interactive graph visualizations via the [codebase.visualize(...)](/api-reference/core/Codebase#visualize) method.
+
+These visualizations have a number of applications, including:
+
+- Understanding codebase structure
+- Monitoring critical code paths
+- Analyzing dependencies
+- Understanding inheritance hierarchies
+
+This guide provides a basic overview of graph creation and customization. Like the one below which displays the call_graph for the [modal/client.py](https://github.com/modal-labs/modal-client/blob/v0.72.49/modal/client.py) module.
+
+
+
+
+ Codegen visualizations are powered by [NetworkX](https://networkx.org/) and
+ rendered using [d3](https://d3js.org/what-is-d3).
+
+
+## Basic Usage
+
+The [Codebase.visualize](/api-reference/core/Codebase#visualize) method operates on a NetworkX [DiGraph](https://networkx.org/documentation/stable/reference/classes/graph.DiGraph.html).
+
+```python
+import networkx as nx
+
+# Basic visualization
+G = nx.grid_2d_graph(5, 5)
+# Or start with an empty graph
+# G = nx.DiGraph()
+codebase.visualize(G)
+
+```
+
+It is up to the developer to add nodes and edges to the graph.
+
+### Adding Nodes and Edges
+
+When adding nodes to your graph, you can either add the symbol directly or just its name:
+
+```python
+import networkx as nx
+G = nx.DiGraph()
+function = codebase.get_function("my_function")
+
+# Add the function object directly - enables source code preview
+graph.add_node(function) # Will show function's source code on click
+
+# Add just the name - no extra features
+graph.add_node(function.name) # Will only show the name
+```
+
+
+ Adding symbols to the graph directly (as opposed to adding by name) enables
+ automatic type information, code preview on hover, and more.
+
+
+## Common Visualization Types
+
+### Call Graphs
+
+Visualize how functions call each other and trace execution paths:
+
+```python
+def create_call_graph(entry_point: Function):
+ graph = nx.DiGraph()
+
+ def add_calls(func):
+ for call in func.call_sites:
+ called_func = call.resolved_symbol
+ if called_func:
+ # Add function objects for rich previews
+ graph.add_node(func)
+ graph.add_node(called_func)
+ graph.add_edge(func, called_func)
+ add_calls(called_func)
+
+ add_calls(entry_point)
+ return graph
+
+# Visualize API endpoint call graph
+endpoint = codebase.get_function("handle_request")
+call_graph = create_call_graph(endpoint)
+codebase.visualize(call_graph, root=endpoint)
+```
+
+
+ Learn more about [traversing the call graph
+ here](/building-with-codegen/traversing-the-call-graph).
+
+
+### React Component Trees
+
+Visualize the hierarchy of React components:
+
+```python
+def create_component_tree(root_component: Class):
+ graph = nx.DiGraph()
+
+ def add_children(component):
+ for usage in component.usages:
+ if isinstance(usage.parent, Class) and "Component" in usage.parent.bases:
+ graph.add_edge(component.name, usage.parent.name)
+ add_children(usage.parent)
+
+ add_children(root_component)
+ return graph
+
+# Visualize component hierarchy
+app = codebase.get_class("App")
+component_tree = create_component_tree(app)
+codebase.visualize(component_tree, root=app)
+```
+
+### Inheritance Graphs
+
+Visualize class inheritance relationships:
+
+```python
+import networkx as nx
+
+G = nx.DiGraph()
+base = codebase.get_class("BaseModel")
+
+def add_subclasses(cls):
+ for subclass in cls.subclasses:
+ G.add_edge(cls, subclass)
+ add_subclasses(subclass)
+
+add_subclasses(base)
+
+codebase.visualize(G, root=base)
+```
+
+### Module Dependencies
+
+Visualize dependencies between modules:
+
+```python
+def create_module_graph(start_file: File):
+ G = nx.DiGraph()
+
+ def add_imports(file):
+ for imp in file.imports:
+ if imp.resolved_symbol and imp.resolved_symbol.file:
+ graph.add_edge(file, imp.resolved_symbol.file)
+ add_imports(imp.resolved_symbol.file)
+
+ add_imports(start_file)
+ return graph
+
+# Visualize module dependencies
+main = codebase.get_file("main.py")
+module_graph = create_module_graph(main)
+codebase.visualize(module_graph, root=main)
+```
+
+### Function Modularity
+
+Visualize function groupings by modularity:
+
+```python
+def create_modularity_graph(functions: list[Function]):
+ graph = nx.Graph()
+
+ # Group functions by shared dependencies
+ for func in functions:
+ for dep in func.dependencies:
+ if isinstance(dep, Function):
+ weight = len(set(func.dependencies) & set(dep.dependencies))
+ if weight > 0:
+ graph.add_edge(func.name, dep.name, weight=weight)
+
+ return graph
+
+# Visualize function modularity
+funcs = codebase.functions
+modularity_graph = create_modularity_graph(funcs)
+codebase.visualize(modularity_graph)
+```
+
+## Customizing Visualizations
+
+You can customize your visualizations using NetworkX's attributes while still preserving the smart node features:
+
+```python
+def create_custom_graph(codebase):
+ graph = nx.DiGraph()
+
+ # Add nodes with custom attributes while preserving source preview
+ for func in codebase.functions:
+ graph.add_node(func,
+ color='red' if func.is_public else 'blue',
+ shape='box' if func.is_async else 'oval'
+ )
+
+ # Add edges between actual function objects
+ for func in codebase.functions:
+ for call in func.call_sites:
+ if call.resolved_symbol:
+ graph.add_edge(func, call.resolved_symbol,
+ style='dashed' if call.is_conditional else 'solid',
+ weight=call.count
+ )
+
+ return graph
+```
+
+## Best Practices
+
+1. **Use Symbol Objects for Rich Features**
+
+ ```python
+ # Better: Add symbol objects for rich previews
+ # This will include source code previews, syntax highlighting, type information, etc.
+ for func in api_funcs:
+ graph.add_node(func)
+
+ # Basic: Just names, no extra features
+ for func in api_funcs:
+ graph.add_node(func.name)
+ ```
+
+2. **Focus on Relevant Subgraphs**
+
+ ```python
+ # Better: Visualize specific subsystem
+ api_funcs = [f for f in codebase.functions if "api" in f.filepath]
+ api_graph = create_call_graph(api_funcs)
+ codebase.visualize(api_graph)
+
+ # Avoid: Visualizing entire codebase
+ full_graph = create_call_graph(codebase.functions) # Too complex
+ ```
+
+3. **Use Meaningful Layouts**
+
+ ```python
+ # Group related nodes together
+ graph.add_node(controller_class, cluster="api")
+ graph.add_node(service_class, cluster="db")
+ ```
+
+4. **Add Visual Hints**
+ ```python
+ # Color code by type while preserving rich previews
+ for node in codebase.functions:
+ if "Controller" in node.name:
+ graph.add_node(node, color="red")
+ elif "Service" in node.name:
+ graph.add_node(node, color="blue")
+ ```
+
+## Limitations
+
+- Large graphs may become difficult to read
+- Complex relationships might need multiple views
+- Some graph layouts may take time to compute
+- Preview features only work when adding symbol objects directly
+
+
+
+---
+title: "Calling Out to LLMs"
+sidebarTitle: "LLM Integration"
+icon: "brain"
+iconType: "solid"
+---
+
+Codegen natively integrates with LLMs via the [codebase.ai(...)](../api-reference/core/Codebase#ai) method, which lets you use large language models (LLMs) to help generate, modify, and analyze code.
+
+## Configuration
+
+Before using AI capabilities, you need to provide an OpenAI API key via [codebase.set_ai_key(...)](../api-reference/core/Codebase#set-ai-key):
+
+```python
+# Set your OpenAI API key
+codebase.set_ai_key("your-openai-api-key")
+```
+
+## Calling Codebase.ai(...)
+
+The [Codebase.ai(...)](../api-reference/core/Codebase#ai) method takes three key arguments:
+
+```python
+result = codebase.ai(
+ prompt="Your instruction to the AI",
+ target=symbol_to_modify, # Optional: The code being operated on
+ context=additional_info # Optional: Extra context from static analysis
+)
+```
+
+- **prompt**: Clear instruction for what you want the AI to do
+- **target**: The symbol (function, class, etc.) being operated on - its source code will be provided to the AI
+- **context**: Additional information you want to provide to the AI, which you can gather using GraphSitter's analysis tools
+
+
+ Codegen does not automatically provide any context to the LLM by default. It
+ does not "understand" your codebase, only the context you provide.
+
+
+The context parameter can include:
+
+- A single symbol (its source code will be provided)
+- A list of related symbols
+- A dictionary mapping descriptions to symbols/values
+- Nested combinations of the above
+
+### How Context Works
+
+The AI doesn't automatically know about your codebase. Instead, you can provide relevant context by:
+
+1. Using GraphSitter's static analysis to gather information:
+
+```python
+function = codebase.get_function("process_data")
+context = {
+ "call_sites": function.call_sites, # Where the function is called
+ "dependencies": function.dependencies, # What the function depends on
+ "parent": function.parent, # Class/module containing the function
+ "docstring": function.docstring, # Existing documentation
+}
+```
+
+2. Passing this information to the AI:
+
+```python
+result = codebase.ai(
+ "Improve this function's implementation",
+ target=function,
+ context=context # AI will see the gathered information
+)
+```
+
+## Common Use Cases
+
+### Code Generation
+
+Generate new code or refactor existing code:
+
+```python
+# Break up a large function
+function = codebase.get_function("large_function")
+new_code = codebase.ai(
+ "Break this function into smaller, more focused functions",
+ target=function
+)
+function.edit(new_code)
+
+# Generate a test
+my_function = codebase.get_function("my_function")
+test_code = codebase.ai(
+ f"Write a test for the function {my_function.name}",
+ target=my_function
+)
+my_function.insert_after(test_code)
+```
+
+### Documentation
+
+Generate and format documentation:
+
+```python
+# Generate docstrings for a class
+class_def = codebase.get_class("MyClass")
+for method in class_def.methods:
+ docstring = codebase.ai(
+ "Generate a docstring describing this method",
+ target=method,
+ context={
+ "class": class_def,
+ "style": "Google docstring format"
+ }
+ )
+ method.set_docstring(docstring)
+```
+
+### Code Analysis and Improvement
+
+Use AI to analyze and improve code:
+
+```python
+# Improve function names
+for function in codebase.functions:
+ if codebase.ai(
+ "Does this function name clearly describe its purpose? Answer yes/no",
+ target=function
+ ).lower() == "no":
+ new_name = codebase.ai(
+ "Suggest a better name for this function",
+ target=function,
+ context={"call_sites": function.call_sites}
+ )
+ function.rename(new_name)
+```
+
+### Contextual Modifications
+
+Make changes with full context awareness:
+
+```python
+# Refactor a class method
+method = codebase.get_class("MyClass").get_method("target_method")
+new_impl = codebase.ai(
+ "Refactor this method to be more efficient",
+ target=method,
+ context={
+ "parent_class": method.parent,
+ "call_sites": method.call_sites,
+ "dependencies": method.dependencies
+ }
+)
+method.edit(new_impl)
+```
+
+## Best Practices
+
+1. **Provide Relevant Context**
+
+ ```python
+ # Good: Providing specific, relevant context
+ summary = codebase.ai(
+ "Generate a summary of this method's purpose",
+ target=method,
+ context={
+ "class": method.parent, # Class containing the method
+ "usages": list(method.usages), # How the method is used
+ "dependencies": method.dependencies, # What the method depends on
+ "style": "concise"
+ }
+ )
+
+ # Bad: Missing context that could help the AI
+ summary = codebase.ai(
+ "Generate a summary",
+ target=method # AI only sees the method's code
+ )
+ ```
+
+2. **Gather Comprehensive Context**
+
+ ```python
+ # Gather relevant information before AI call
+ def get_method_context(method):
+ return {
+ "class": method.parent,
+ "call_sites": list(method.call_sites),
+ "dependencies": list(method.dependencies),
+ "related_methods": [m for m in method.parent.methods
+ if m.name != method.name]
+ }
+
+ # Use gathered context in AI call
+ new_impl = codebase.ai(
+ "Refactor this method to be more efficient",
+ target=method,
+ context=get_method_context(method)
+ )
+ ```
+
+3. **Handle AI Limits**
+
+ ```python
+ # Set custom AI request limits for large operations
+ codebase.set_session_options(max_ai_requests=200)
+ ```
+
+4. **Review Generated Code**
+ ```python
+ # Generate and review before applying
+ new_code = codebase.ai(
+ "Optimize this function",
+ target=function
+ )
+ print("Review generated code:")
+ print(new_code)
+ if input("Apply changes? (y/n): ").lower() == 'y':
+ function.edit(new_code)
+ ```
+
+## Limitations and Safety
+
+- The AI doesn't automatically know about your codebase - you must provide relevant context
+- AI-generated code should always be reviewed
+- Default limit of 150 AI requests per codemod execution
+ - Use [set_session_options(...)](../api-reference/core/Codebase#set-session-options) to adjust limits:
+ ```python
+ codebase.set_session_options(max_ai_requests=200)
+ ```
+
+ You can also use `codebase.set_session_options` to increase the execution time and the number of operations allowed in a session. This is useful for handling larger tasks or more complex operations that require additional resources. Adjust the `max_seconds` and `max_transactions` parameters to suit your needs:
+ ```python
+ codebase.set_session_options(max_seconds=300, max_transactions=500)
+ ```
+
+
+---
+title: "Reducing Conditions"
+sidebarTitle: "Reducing Conditions"
+icon: "code-branch"
+iconType: "solid"
+---
+
+Codegen provides powerful APIs for reducing conditional logic to constant values. This is particularly useful for removing feature flags, cleaning up dead code paths, and simplifying conditional logic.
+
+## Overview
+
+The `reduce_condition()` method is available on various conditional constructs:
+
+- [If/else statements](/api-reference/core/IfBlockStatement#reduce-condition)
+- [Ternary expressions](/api-reference/core/TernaryExpression#reduce-condition)
+- [Binary expressions](/api-reference/core/BinaryExpression#reduce-condition)
+- [Function calls](/api-reference/core/FunctionCall#reduce-condition)
+
+When you reduce a condition to `True` or `False`, Codegen automatically:
+
+1. Evaluates which code path(s) to keep
+2. Removes unnecessary branches
+3. Preserves proper indentation and formatting
+
+### Motivating Example
+
+For example, consider the following code:
+
+```python
+flag = get_feature_flag('MY_FEATURE')
+if flag:
+ print('MY_FEATURE: ON')
+else:
+ print('MY_FEATURE: OFF')
+```
+
+`.reduce_condition` allows you to deterministically reduce this code to the following:
+
+```python
+print('MY_FEATURE: ON')
+```
+
+This is useful when a feature flag is fully "rolled out".
+
+## Implementations
+
+### [IfBlockStatements](/api-reference/core/IfBlockStatement#reduce-condition)
+
+You can reduce if/else statements to either their "true" or "false" branch.
+
+For example, in the code snippet above:
+
+```python
+# Grab if statement
+if_block = file.code_block.statements[1]
+
+# Reduce to True branch
+if_block.reduce_condition(True)
+```
+
+This will remove the `else` branch and keep the `print` statement, like so:
+
+```python
+flag = get_feature_flag('MY_FEATURE')
+print('MY_FEATURE: ON')
+```
+
+### Handling Elif Chains
+
+Codegen intelligently handles elif chains when reducing conditions:
+
+```python
+# Original code
+if condition_a:
+ print("A")
+elif condition_b:
+ print("B")
+else:
+ print("C")
+
+# Reduce first condition to False
+if_block.reduce_condition(False)
+# Result:
+if condition_b:
+ print("B")
+else:
+ print("C")
+
+# Reduce elif condition to True
+elif_block.reduce_condition(True)
+# Result:
+print("B")
+```
+
+## Ternary Expressions
+
+Ternary expressions (conditional expressions) can also be reduced:
+
+```python
+# Original code
+result = 'valueA' if condition else 'valueB'
+
+# Reduce to True
+ternary_expr.reduce_condition(True)
+# Result:
+result = 'valueA'
+
+# Reduce to False
+ternary_expr.reduce_condition(False)
+# Result:
+result = 'valueB'
+```
+
+### Nested Ternaries
+
+Codegen handles nested ternary expressions correctly:
+
+```python
+# Original code
+result = 'A' if a else 'B' if b else 'C'
+
+# Reduce outer condition to False
+outer_ternary.reduce_condition(False)
+# Result:
+result = 'B' if b else 'C'
+
+# Then reduce inner condition to True
+inner_ternary.reduce_condition(True)
+# Result:
+result = 'B'
+```
+
+## Binary Operations
+
+Binary operations (and/or) can be reduced to simplify logic:
+
+```python
+# Original code
+result = (x or y) and b
+
+# Reduce x to True
+x_assign.reduce_condition(True)
+# Result:
+result = b
+
+# Reduce y to False
+y_assign.reduce_condition(False)
+# Result:
+result = x and b
+```
+
+## Function Calls
+
+[Function calls](/api-reference/core/FunctionCall#reduce-condition) can also be reduced, which is particularly useful when dealing with hooks or utility functions that return booleans:
+
+```typescript
+// Original code
+const isEnabled = useFeatureFlag("my_feature");
+return isEnabled ? : ;
+
+// After reducing useFeatureFlag to True
+return ;
+```
+
+### Feature Flag Hooks
+
+A common use case is reducing feature flag hooks to constants. Consider the following code:
+
+```typescript
+// Original code
+function MyComponent() {
+ const showNewUI = useFeatureFlag("new_ui_enabled");
+
+ if (showNewUI) {
+ return ;
+ }
+ return ;
+}
+```
+
+We can reduce the `useFeatureFlag` hook to a constant value like so, with [FunctionCall.reduce_condition](/api-reference/core/FunctionCall#reduce-condition):
+
+```python
+hook = codebase.get_function("useFeatureFlag")
+for usage in hook.usages():
+ if isinstance(usage.match, FunctionCall):
+ fcall = usage.match
+ if fcall.args[0].value.content == 'new_ui_enabled':
+ # This will automatically reduce any conditions using the flag
+ fcall.reduce_condition(True)
+```
+
+This produces the following code:
+
+```typescript
+function MyComponent() {
+ return ;
+}
+```
+
+### Comprehensive Example
+
+Here's a complete example of removing a feature flag from both configuration and usage:
+
+```python
+feature_flag_name = "new_ui_enabled"
+target_value = True
+
+# 1. Remove from config
+config_file = codebase.get_file("src/featureFlags/config.ts")
+feature_flag_config = config_file.get_symbol("FEATURE_FLAG_CONFIG").value
+feature_flag_config.pop(feature_flag_name)
+
+# 2. Find and reduce all usages
+hook = codebase.get_function("useFeatureFlag")
+for usage in hook.usages():
+ fcall = usage.match
+ if isinstance(fcall, FunctionCall):
+ # Check if this usage is for our target flag
+ first_arg = fcall.args[0].value
+ if isinstance(first_arg, String) and first_arg.content == feature_flag_name:
+ print(f'Reducing in: {fcall.parent_symbol.name}')
+ # This will automatically reduce:
+ # - Ternary expressions using the flag
+ # - If statements checking the flag
+ # - Binary operations with the flag
+ fcall.reduce_condition(target_value)
+
+# Commit changes to disk
+codebase.commit()
+```
+
+This example:
+
+1. Removes the feature flag from configuration
+2. Finds all usages of the feature flag hook
+3. Reduces each usage to a constant value
+4. Automatically handles all conditional constructs using the flag
+
+
+ When reducing a function call, Codegen automatically handles all dependent
+ conditions. This includes: - [If/else
+ statements](/api-reference/core/IfBlockStatement#reduce-condition) - [Ternary
+ expressions](/api-reference/core/TernaryExpression#reduce-condition) - [Binary
+ operations](/api-reference/core/BinaryExpression#reduce-condition)
+
+
+## TypeScript and JSX Support
+
+Condition reduction works with TypeScript and JSX, including conditional rendering:
+
+```typescript
+// Original JSX
+const MyComponent: React.FC = () => {
+ let isVisible = true;
+ return (
+
+ {isVisible && Visible}
+ {!isVisible && Hidden}
+
+ );
+};
+
+// After reducing isVisible to True
+const MyComponent: React.FC = () => {
+ return (
+
+ Visible
+
+ );
+};
+```
+
+
+ Condition reduction is particularly useful for cleaning up feature flags in
+ React components, where conditional rendering is common.
+
+
+
+---
+title: "Learn by Example"
+sidebarTitle: "At a Glance"
+icon: "graduation-cap"
+iconType: "solid"
+---
+
+Explore our tutorials to learn how to use Codegen for various code transformation tasks.
+
+## Featured Tutorials
+
+
+
+ Generate interactive visualizations of your codebase's structure, dependencies, and relationships.
+
+
+ Create high-quality training data for LLM pre-training similar to word2vec or node2vec
+
+
+ Add, remove, and update feature flags across your application.
+
+
+ Remove unused imports, functions, and variables with confidence.
+
+
+
+## API Migrations
+
+
+
+ Update API calls, handle breaking changes, and manage bulk updates across your codebase.
+
+
+ Update SQLAlchemy code to use the new 2.0-style query interface and patterns.
+
+
+ Convert Flask applications to FastAPI, updating routes and dependencies.
+
+
+ Migrate Python 2 code to Python 3, updating syntax and modernizing APIs.
+
+
+
+## Code Organization
+
+
+
+ Restructure files, enforce naming conventions, and improve project layout.
+
+
+ Split large files, extract shared logic, and manage dependencies.
+
+
+ Organize and optimize TypeScript module exports.
+
+
+ Convert between default and named exports in TypeScript/JavaScript.
+
+
+
+## Testing & Types
+
+
+
+ Convert unittest test suites to pytest's modern testing style.
+
+
+ Add TypeScript types, infer types from usage, and improve type safety.
+
+
+
+## Documentation & AI
+
+
+
+ Generate JSDoc comments, README files, and API documentation.
+
+
+ Generate system prompts, create hierarchical documentation, and optimize for AI assistance.
+
+
+
+
+ Each tutorial includes practical examples, code snippets, and best practices.
+ Follow them in order or jump to the ones most relevant to your needs.
+
+
+
+---
+title: "Migrating APIs"
+sidebarTitle: "API Migrations"
+icon: "webhook"
+iconType: "solid"
+---
+
+API migrations are a common task in large codebases. Whether you're updating a deprecated function, changing parameter names, or modifying return types, Codegen makes it easy to update all call sites consistently.
+
+## Common Migration Scenarios
+
+### Renaming Parameters
+
+When updating parameter names across an API, you need to update both the function definition and all call sites:
+
+```python
+# Find the API function to update
+api_function = codebase.get_function("process_data")
+
+# Update the parameter name
+old_param = api_function.get_parameter("input")
+old_param.rename("data")
+
+# All call sites are automatically updated:
+# process_data(input="test") -> process_data(data="test")
+```
+
+See [dependencies and usages](/building-with-codegen/dependencies-and-usages) for more on updating parameter names and types.
+
+### Adding Required Parameters
+
+When adding a new required parameter to an API:
+
+```python
+# Find all call sites before modifying the function
+call_sites = list(api_function.call_sites)
+
+# Add the new parameter
+api_function.add_parameter("timeout: int")
+
+# Update all existing call sites to include the new parameter
+for call in call_sites:
+ call.add_argument("timeout=30") # Add with a default value
+```
+
+See [function calls and callsites](/building-with-codegen/function-calls-and-callsites) for more on handling call sites.
+
+### Changing Parameter Types
+
+When updating parameter types:
+
+```python
+# Update the parameter type
+param = api_function.get_parameter("user_id")
+param.type = "UUID" # Change from string to UUID
+
+# Find all call sites that need type conversion
+for call in api_function.call_sites:
+ arg = call.get_arg_by_parameter_name("user_id")
+ if arg:
+ # Convert string to UUID
+ arg.edit(f"UUID({arg.value})")
+```
+
+See [working with type annotations](/building-with-codegen/type-annotations) for more on changing parameter types.
+
+### Deprecating Functions
+
+When deprecating an old API in favor of a new one:
+
+```python
+old_api = codebase.get_function("old_process_data")
+new_api = codebase.get_function("new_process_data")
+
+# Add deprecation warning
+old_api.add_decorator('@deprecated("Use new_process_data instead")')
+
+# Update all call sites to use the new API
+for call in old_api.call_sites:
+ # Map old arguments to new parameter names
+ args = [
+ f"data={call.get_arg_by_parameter_name('input').value}",
+ f"timeout={call.get_arg_by_parameter_name('wait').value}"
+ ]
+
+ # Replace the old call with the new API
+ call.replace(f"new_process_data({', '.join(args)})")
+```
+
+## Bulk Updates to Method Chains
+
+When updating chained method calls, like database queries or builder patterns:
+
+```python
+# Find all query chains ending with .execute()
+for execute_call in codebase.function_calls:
+ if execute_call.name != "execute":
+ continue
+
+ # Get the full chain
+ chain = execute_call.call_chain
+
+ # Example: Add .timeout() before .execute()
+ if "timeout" not in {call.name for call in chain}:
+ execute_call.insert_before("timeout(30)")
+```
+
+## Handling Breaking Changes
+
+When making breaking changes to an API, it's important to:
+1. Identify all affected call sites
+2. Make changes consistently
+3. Update related documentation
+4. Consider backward compatibility
+
+Here's a comprehensive example:
+
+```python
+def migrate_api_v1_to_v2(codebase):
+ old_api = codebase.get_function("create_user_v1")
+
+ # Document all existing call patterns
+ call_patterns = {}
+ for call in old_api.call_sites:
+ args = [arg.source for arg in call.args]
+ pattern = ", ".join(args)
+ call_patterns[pattern] = call_patterns.get(pattern, 0) + 1
+
+ print("Found call patterns:")
+ for pattern, count in call_patterns.items():
+ print(f" {pattern}: {count} occurrences")
+
+ # Create new API version
+ new_api = old_api.copy()
+ new_api.rename("create_user_v2")
+
+ # Update parameter types
+ new_api.get_parameter("email").type = "EmailStr"
+ new_api.get_parameter("role").type = "UserRole"
+
+ # Add new required parameters
+ new_api.add_parameter("tenant_id: UUID")
+
+ # Update all call sites
+ for call in old_api.call_sites:
+ # Get current arguments
+ email_arg = call.get_arg_by_parameter_name("email")
+ role_arg = call.get_arg_by_parameter_name("role")
+
+ # Build new argument list with type conversions
+ new_args = [
+ f"email=EmailStr({email_arg.value})",
+ f"role=UserRole({role_arg.value})",
+ "tenant_id=get_current_tenant_id()"
+ ]
+
+ # Replace old call with new version
+ call.replace(f"create_user_v2({', '.join(new_args)})")
+
+ # Add deprecation notice to old version
+ old_api.add_decorator('@deprecated("Use create_user_v2 instead")')
+
+# Run the migration
+migrate_api_v1_to_v2(codebase)
+```
+
+## Best Practices
+
+1. **Analyze First**: Before making changes, analyze all call sites to understand usage patterns
+ ```python
+ # Document current usage
+ for call in api.call_sites:
+ print(f"Called from: {call.parent_function.name}")
+ print(f"With args: {[arg.source for arg in call.args]}")
+ ```
+
+2. **Make Atomic Changes**: Update one aspect at a time
+ ```python
+ # First update parameter names
+ param.rename("new_name")
+
+ # Then update types
+ param.type = "new_type"
+
+ # Finally update call sites
+ for call in api.call_sites:
+ # ... update calls
+ ```
+
+3. **Maintain Backwards Compatibility**:
+ ```python
+ # Add new parameter with default
+ api.add_parameter("new_param: str = None")
+
+ # Later make it required
+ api.get_parameter("new_param").remove_default()
+ ```
+
+4. **Document Changes**:
+ ```python
+ # Add clear deprecation messages
+ old_api.add_decorator('''@deprecated(
+ "Use new_api() instead. Migration guide: docs/migrations/v2.md"
+ )''')
+ ```
+
+
+Remember to test thoroughly after making bulk changes to APIs. While Codegen ensures syntactic correctness, you'll want to verify the semantic correctness of the changes.
+
+
+---
+title: "Codebase Visualization"
+sidebarTitle: "Visualization"
+description: "This guide will show you how to create codebase visualizations using [codegen](/introduction/overview)."
+icon: "share-nodes"
+iconType: "solid"
+---
+
+
+
+
+
+## Overview
+
+To demonstrate the visualization capabilities of the codegen we will generate three different visualizations of PostHog's open source [repository](https://github.com/PostHog/posthog).
+ - [Call Trace Visualization](#call-trace-visualization)
+ - [Function Dependency Graph](#function-dependency-graph)
+ - [Blast Radius Visualization](#blast-radius-visualization)
+
+
+## Call Trace Visualization
+
+Visualizing the call trace of a function is a great way to understand the flow of a function and for debugging. In this tutorial we will create a call trace visualization of the `patch` method of the `SharingConfigurationViewSet` class. View the source code [here](https://github.com/PostHog/posthog/blob/c2986d9ac7502aa107a4afbe31b3633848be6582/posthog/api/sharing.py#L163).
+
+
+### Basic Setup
+First, we'll set up our codebase, graph and configure some basic parameters:
+
+```python
+import networkx as nx
+from codegen import Codebase
+
+# Initialize codebase
+codebase = Codebase("path/to/posthog/")
+
+# Create a directed graph for representing call relationships
+G = nx.DiGraph()
+
+# Configuration flags
+IGNORE_EXTERNAL_MODULE_CALLS = True # Skip calls to external modules
+IGNORE_CLASS_CALLS = False # Include class definition calls
+MAX_DEPTH = 10
+
+COLOR_PALETTE = {
+ "StartFunction": "#9cdcfe", # Light blue - Start Function
+ "PyFunction": "#a277ff", # Soft purple/periwinkle - PyFunction
+ "PyClass": "#ffca85", # Warm peach/orange - PyClass
+ "ExternalModule": "#f694ff" # Bright magenta/pink - ExternalModule
+}
+```
+
+### Building the Visualization
+We'll create a function that will recursively traverse the call trace of a function and add nodes and edges to the graph:
+
+```python
+def create_downstream_call_trace(src_func: Function, depth: int = 0):
+ """Creates call graph by recursively traversing function calls
+
+ Args:
+ src_func (Function): Starting function for call graph
+ depth (int): Current recursion depth
+ """
+ # Prevent infinite recursion
+ if MAX_DEPTH <= depth:
+ return
+
+ # External modules are not functions
+ if isinstance(src_func, ExternalModule):
+ return
+
+ # Process each function call
+ for call in src_func.function_calls:
+ # Skip self-recursive calls
+ if call.name == src_func.name:
+ continue
+
+ # Get called function definition
+ func = call.function_definition
+ if not func:
+ continue
+
+ # Apply configured filters
+ if isinstance(func, ExternalModule) and IGNORE_EXTERNAL_MODULE_CALLS:
+ continue
+ if isinstance(func, Class) and IGNORE_CLASS_CALLS:
+ continue
+
+ # Generate display name (include class for methods)
+ if isinstance(func, Class) or isinstance(func, ExternalModule):
+ func_name = func.name
+ elif isinstance(func, Function):
+ func_name = f"{func.parent_class.name}.{func.name}" if func.is_method else func.name
+
+ # Add node and edge with metadata
+ G.add_node(func, name=func_name,
+ color=COLOR_PALETTE.get(func.__class__.__name__))
+ G.add_edge(src_func, func, **generate_edge_meta(call))
+
+ # Recurse for regular functions
+ if isinstance(func, Function):
+ create_downstream_call_trace(func, depth + 1)
+```
+
+### Adding Edge Metadata
+We can enrich our edges with metadata about the function calls:
+
+```python
+def generate_edge_meta(call: FunctionCall) -> dict:
+ """Generate metadata for call graph edges
+
+ Args:
+ call (FunctionCall): Function call information
+
+ Returns:
+ dict: Edge metadata including name and location
+ """
+ return {
+ "name": call.name,
+ "file_path": call.filepath,
+ "start_point": call.start_point,
+ "end_point": call.end_point,
+ "symbol_name": "FunctionCall"
+ }
+```
+### Visualizing the Graph
+Finally, we can visualize our call graph starting from a specific function:
+```python
+# Get target function to analyze
+target_class = codebase.get_class('SharingConfigurationViewSet')
+target_method = target_class.get_method('patch')
+
+# Add root node
+G.add_node(target_method,
+ name=f"{target_class.name}.{target_method.name}",
+ color=COLOR_PALETTE["StartFunction"])
+
+# Build the call graph
+create_downstream_call_trace(target_method)
+
+# Render the visualization
+codebase.visualize(G)
+```
+
+
+### Take a look
+
+
+View on [codegen.sh](https://www.codegen.sh/codemod/6a34b45d-c8ad-422e-95a8-46d4dc3ce2b0/public/diff)
+
+
+### Common Use Cases
+The call graph visualization is particularly useful for:
+ - Understanding complex codebases
+ - Planning refactoring efforts
+ - Identifying tightly coupled components
+ - Analyzing critical paths
+ - Documenting system architecture
+
+## Function Dependency Graph
+
+Understanding symbol dependencies is crucial for maintaining and refactoring code. This tutorial will show you how to create visual dependency graphs using Codegen and NetworkX. We will be creating a dependency graph of the `get_query_runner` function. View the source code [here](https://github.com/PostHog/posthog/blob/c2986d9ac7502aa107a4afbe31b3633848be6582/posthog/hogql_queries/query_runner.py#L152).
+
+### Basic Setup
+
+We'll use the same basic setup as the [Call Trace Visualization](/tutorials/codebase-visualization#call-trace-visualization) tutorial.
+
+
+### Building the Dependency Graph
+The core function for building our dependency graph:
+```python
+def create_dependencies_visualization(symbol: Symbol, depth: int = 0):
+ """Creates visualization of symbol dependencies
+
+ Args:
+ symbol (Symbol): Starting symbol to analyze
+ depth (int): Current recursion depth
+ """
+ # Prevent excessive recursion
+ if depth >= MAX_DEPTH:
+ return
+
+ # Process each dependency
+ for dep in symbol.dependencies:
+ dep_symbol = None
+
+ # Handle different dependency types
+ if isinstance(dep, Symbol):
+ # Direct symbol reference
+ dep_symbol = dep
+ elif isinstance(dep, Import):
+ # Import statement - get resolved symbol
+ dep_symbol = dep.resolved_symbol if dep.resolved_symbol else None
+
+ if dep_symbol:
+ # Add node with appropriate styling
+ G.add_node(dep_symbol,
+ color=COLOR_PALETTE.get(dep_symbol.__class__.__name__,
+ "#f694ff"))
+
+ # Add dependency relationship
+ G.add_edge(symbol, dep_symbol)
+
+ # Recurse unless it's a class (avoid complexity)
+ if not isinstance(dep_symbol, PyClass):
+ create_dependencies_visualization(dep_symbol, depth + 1)
+```
+
+### Visualizing the Graph
+Finally, we can visualize our dependency graph starting from a specific symbol:
+```python
+# Get target symbol
+target_func = codebase.get_function("get_query_runner")
+
+# Add root node
+G.add_node(target_func, color=COLOR_PALETTE["StartFunction"])
+
+# Generate dependency graph
+create_dependencies_visualization(target_func)
+
+# Render visualization
+codebase.visualize(G)
+```
+
+### Take a look
+
+
+View on [codegen.sh](https://www.codegen.sh/codemod/39a36f0c-9d35-4666-9db7-12ae7c28fc17/public/diff)
+
+
+## Blast Radius visualization
+
+Understanding the impact of code changes is crucial for safe refactoring. A blast radius visualization shows how changes to one function might affect other parts of the codebase by tracing usage relationships. In this tutorial we will create a blast radius visualization of the `export_asset` function. View the source code [here](https://github.com/PostHog/posthog/blob/c2986d9ac7502aa107a4afbe31b3633848be6582/posthog/tasks/exporter.py#L57).
+
+### Basic Setup
+
+We'll use the same basic setup as the [Call Trace Visualization](/tutorials/codebase-visualization#call-trace-visualization) tutorial.
+
+
+### Helper Functions
+We'll create some utility functions to help build our visualization:
+```python
+# List of HTTP methods to highlight
+HTTP_METHODS = ["get", "put", "patch", "post", "head", "delete"]
+
+def generate_edge_meta(usage: Usage) -> dict:
+ """Generate metadata for graph edges
+
+ Args:
+ usage (Usage): Usage relationship information
+
+ Returns:
+ dict: Edge metadata including name and location
+ """
+ return {
+ "name": usage.match.source,
+ "file_path": usage.match.filepath,
+ "start_point": usage.match.start_point,
+ "end_point": usage.match.end_point,
+ "symbol_name": usage.match.__class__.__name__
+ }
+
+def is_http_method(symbol: PySymbol) -> bool:
+ """Check if a symbol is an HTTP endpoint method
+
+ Args:
+ symbol (PySymbol): Symbol to check
+
+ Returns:
+ bool: True if symbol is an HTTP method
+ """
+ if isinstance(symbol, PyFunction) and symbol.is_method:
+ return symbol.name in HTTP_METHODS
+ return False
+```
+
+### Building the Blast Radius Visualization
+The main function for creating our blast radius visualization:
+```python
+def create_blast_radius_visualization(symbol: PySymbol, depth: int = 0):
+ """Create visualization of symbol usage relationships
+
+ Args:
+ symbol (PySymbol): Starting symbol to analyze
+ depth (int): Current recursion depth
+ """
+ # Prevent excessive recursion
+ if depth >= MAX_DEPTH:
+ return
+
+ # Process each usage of the symbol
+ for usage in symbol.usages:
+ usage_symbol = usage.usage_symbol
+
+ # Determine node color based on type
+ if is_http_method(usage_symbol):
+ color = COLOR_PALETTE.get("HTTP_METHOD")
+ else:
+ color = COLOR_PALETTE.get(usage_symbol.__class__.__name__, "#f694ff")
+
+ # Add node and edge to graph
+ G.add_node(usage_symbol, color=color)
+ G.add_edge(symbol, usage_symbol, **generate_edge_meta(usage))
+
+ # Recursively process usage symbol
+ create_blast_radius_visualization(usage_symbol, depth + 1)
+```
+
+### Visualizing the Graph
+Finally, we can create our blast radius visualization:
+```python
+# Get target function to analyze
+target_func = codebase.get_function('export_asset')
+
+# Add root node
+G.add_node(target_func, color=COLOR_PALETTE.get("StartFunction"))
+
+# Build the visualization
+create_blast_radius_visualization(target_func)
+
+# Render graph to show impact flow
+# Note: a -> b means changes to a will impact b
+codebase.visualize(G)
+```
+
+### Take a look
+
+
+View on [codegen.sh](https://www.codegen.sh/codemod/d255db6c-9a86-4197-9b78-16c506858a3b/public/diff)
+
+
+## What's Next?
+
+
+
+ Learn how to use Codegen to create modular codebases.
+
+
+ Learn how to use Codegen to delete dead code.
+
+
+ Learn how to use Codegen to increase type coverage.
+
+
+ Explore the complete API documentation for all Codegen classes and methods.
+
+
+
+---
+title: "Mining Training Data for LLMs"
+sidebarTitle: "Mining Data"
+description: "Learn how to generate training data for large language models using Codegen"
+icon: "network-wired"
+iconType: "solid"
+---
+
+This guide demonstrates how to use Codegen to generate high-quality training data for large language models (LLMs) by extracting function implementations along with their dependencies and usages. This approach is similar to [word2vec](https://www.tensorflow.org/text/tutorials/word2vec) or [node2vec](https://snap.stanford.edu/node2vec/) - given the context of a function, learn to predict the function's implementation.
+
+View the full code in our [examples repository](https://github.com/codegen-sh/codegen-examples/tree/7b978091c3153b687c32928fe10f05425e22f6a5/examples/generate_training_data)
+
+This example works with both Python and Typescript repositories without modification
+
+## Overview
+
+The process involves three main steps:
+
+1. Finding all functions in the codebase
+2. Extracting their implementations, dependencies, and usages
+3. Generating structured training data
+
+Let's walk through each step using Codegen.
+
+## Step 1: Finding Functions and Their Context
+
+First, we will do a "graph expansion" for each function - grab the function's source, as well as the full source of all usages of the function and all dependencies.
+
+See [dependencies and usages](/building-with-codegen/dependencies-and-usages) to learn more about navigating the code graph
+
+First, let's import the types we need from Codegen:
+
+```python
+import codegen
+from codegen import Codebase
+from codegen.sdk.core.external_module import ExternalModule
+from codegen.sdk.core.import_resolution import Import
+from codegen.sdk.core.symbol import Symbol
+```
+
+Here's how we get the full context for each function:
+
+```python
+def get_function_context(function) -> dict:
+ """Get the implementation, dependencies, and usages of a function."""
+ context = {
+ "implementation": {"source": function.source, "filepath": function.filepath},
+ "dependencies": [],
+ "usages": [],
+ }
+
+ # Add dependencies
+ for dep in function.dependencies:
+ # Hop through imports to find the root symbol source
+ if isinstance(dep, Import):
+ dep = hop_through_imports(dep)
+
+ context["dependencies"].append({"source": dep.source, "filepath": dep.filepath})
+
+ # Add usages
+ for usage in function.usages:
+ context["usages"].append({
+ "source": usage.usage_symbol.source,
+ "filepath": usage.usage_symbol.filepath,
+ })
+
+ return context
+```
+
+Notice how we use `hop_through_imports` to resolve dependencies. When working with imports, symbols can be re-exported multiple times. For example, a helper function might be imported and re-exported through several files before being used. We need to follow this chain to find the actual implementation:
+
+```python
+def hop_through_imports(imp: Import) -> Symbol | ExternalModule:
+ """Finds the root symbol for an import."""
+ if isinstance(imp.imported_symbol, Import):
+ return hop_through_imports(imp.imported_symbol)
+ return imp.imported_symbol
+```
+
+This creates a structured representation of each function's context:
+
+```json
+{
+ "implementation": {
+ "source": "def process_data(input: str) -> dict: ...",
+ "filepath": "src/data_processor.py"
+ },
+ "dependencies": [
+ {
+ "source": "def validate_input(data: str) -> bool: ...",
+ "filepath": "src/validators.py"
+ }
+ ],
+ "usages": [
+ {
+ "source": "result = process_data(user_input)",
+ "filepath": "src/api.py"
+ }
+ ]
+}
+```
+
+## Step 2: Processing the Codebase
+
+Next, we process all functions in the codebase to generate our training data:
+
+```python
+def run(codebase: Codebase):
+ """Generate training data using a node2vec-like approach for code embeddings."""
+ # Track all function contexts
+ training_data = {
+ "functions": [],
+ "metadata": {
+ "total_functions": len(codebase.functions),
+ "total_processed": 0,
+ "avg_dependencies": 0,
+ "avg_usages": 0,
+ },
+ }
+
+ # Process each function in the codebase
+ for function in codebase.functions:
+ # Skip if function is too small
+ if len(function.source.split("\n")) < 2:
+ continue
+
+ # Get function context
+ context = get_function_context(function)
+
+ # Only keep functions with enough context
+ if len(context["dependencies"]) + len(context["usages"]) > 0:
+ training_data["functions"].append(context)
+
+ # Update metadata
+ training_data["metadata"]["total_processed"] = len(training_data["functions"])
+ if training_data["functions"]:
+ training_data["metadata"]["avg_dependencies"] = sum(
+ len(f["dependencies"]) for f in training_data["functions"]
+ ) / len(training_data["functions"])
+ training_data["metadata"]["avg_usages"] = sum(
+ len(f["usages"]) for f in training_data["functions"]
+ ) / len(training_data["functions"])
+
+ return training_data
+```
+
+## Step 3: Running the Generator
+
+Finally, we can run our training data generator on any codebase.
+
+See [parsing codebases](/building-with-codegen/parsing-codebases) to learn more
+
+```python
+if __name__ == "__main__":
+ print("Initializing codebase...")
+ codebase = Codebase.from_repo("fastapi/fastapi")
+
+ print("Generating training data...")
+ training_data = run(codebase)
+
+ print("Saving training data...")
+ with open("training_data.json", "w") as f:
+ json.dump(training_data, f, indent=2)
+ print("Training data saved to training_data.json")
+```
+
+This will:
+1. Load the target codebase
+2. Process all functions
+3. Save the structured training data to a JSON file
+
+
+ You can use any Git repository as your source codebase by passing the repo URL
+ to [Codebase.from_repo(...)](/api-reference/core/Codebase#from-repo).
+
+
+## Using the Training Data
+
+The generated data can be used to train LLMs in several ways:
+
+1. **Masked Function Prediction**: Hide a function's implementation and predict it from dependencies and usages
+2. **Code Embeddings**: Generate embeddings that capture semantic relationships between functions
+3. **Dependency Prediction**: Learn to predict which functions are likely to be dependencies
+4. **Usage Pattern Learning**: Train models to understand common usage patterns
+
+For example, to create a masked prediction task:
+
+```python
+def create_training_example(function_data):
+ """Create a masked prediction example from function data."""
+ return {
+ "context": {
+ "dependencies": function_data["dependencies"],
+ "usages": function_data["usages"]
+ },
+ "target": function_data["implementation"]
+ }
+
+# Create training examples
+examples = [create_training_example(f) for f in training_data["functions"]]
+```
+
+
+
+---
+title: "Organizing Your Codebase"
+sidebarTitle: "Organization"
+icon: "folder-tree"
+iconType: "solid"
+---
+
+Codegen SDK provides a powerful set of tools for deterministically moving code safely and efficiently. This guide will walk you through the basics of moving code with Codegen SDK.
+
+Common use cases include:
+
+
+
+
+```python
+print(f"๐ Processing file: {filepath}")
+file = codebase.get_file(filepath)
+
+# Get the directory path for creating new files
+dir_path = file.directory.path if file.directory else ""
+
+# Iterate through all functions in the file
+for function in file.functions:
+ # Create new filename based on function name
+ new_filepath = f"{dir_path}/{function.name}.py"
+ print(f"๐ Creating new file: {new_filepath}")
+
+ # Create the new file
+ new_file = codebase.create_file(new_filepath)
+
+ # Move the function to the new file, including dependencies
+ print(f"โก๏ธ Moving function: {function.name}")
+ function.move_to_file(new_file, include_dependencies=True)
+```
+
+
+
+
+
+```python
+# Dictionary to track modules and their functions
+module_map = {
+ "utils": lambda f: f.name.startswith("util_") or f.name.startswith("helper_"),
+ "api": lambda f: f.name.startswith("api_") or f.name.startswith("endpoint_"),
+ "data": lambda f: f.name.startswith("data_") or f.name.startswith("db_"),
+ "core": lambda f: True # Default module for other functions
+}
+
+print("๐ Starting code organization...")
+
+# Create module directories if they don't exist
+for module in module_map.keys():
+ if not codebase.has_directory(module):
+ print(f"๐ Creating module directory: {module}")
+ codebase.create_directory(module, exist_ok=True)
+
+# Process each file in the codebase
+for file in codebase.files:
+ print(f"\n๐ Processing file: {file.filepath}")
+
+ # Skip if file is already in a module directory
+ if any(file.filepath.startswith(module) for module in module_map.keys()):
+ continue
+
+ # Process each function in the file
+ for function in file.functions:
+ # Determine which module this function belongs to
+ target_module = next(
+ (module for module, condition in module_map.items()
+ if condition(function)),
+ "core"
+ )
+
+ # Create the new file path
+ new_filepath = f"{target_module}/{function.name}.py"
+
+ print(f" โก๏ธ Moving {function.name} to {target_module} module")
+
+ # Create new file and move function
+ if not codebase.has_file(new_filepath):
+ new_file = codebase.create_file(new_filepath)
+ function.move_to_file(new_file, include_dependencies=True)
+
+print("\nโ
Code organization complete!")
+```
+
+
+
+
+
+```python
+# Create a graph to detect cycles
+import networkx as nx
+
+# Build dependency graph
+G = nx.DiGraph()
+
+# Add edges for imports between files
+for file in codebase.files:
+ for imp in file.imports:
+ if imp.from_file:
+ G.add_edge(file.filepath, imp.from_file.filepath)
+
+# Find cycles in the graph
+cycles = list(nx.simple_cycles(G))
+
+if not cycles:
+ print("โ
No import cycles found!")
+ exit()
+
+print(f"๐ Found {len(cycles)} import cycles")
+
+# Process each cycle
+for cycle in cycles:
+ print(f"\nโญ Processing cycle: {' -> '.join(cycle)}")
+
+ # Get the first two files in the cycle
+ file1 = codebase.get_file(cycle[0])
+ file2 = codebase.get_file(cycle[1])
+
+ # Find functions in file1 that are used by file2
+ for function in file1.functions:
+ if any(usage.file == file2 for usage in function.usages):
+ # Create new file for the shared function
+ new_filepath = f"shared/{function.name}.py"
+ print(f" โก๏ธ Moving {function.name} to {new_filepath}")
+
+ if not codebase.has_directory("shared"):
+ codebase.create_directory("shared")
+
+ new_file = codebase.create_file(new_filepath)
+ function.move_to_file(new_file, include_dependencies=True)
+
+print("\nโ
Import cycles resolved!")
+```
+
+
+
+
+
+ Most operations in Codegen will automatically handle updaging
+ [dependencies](/building-with-codegen/dependencies-and-usages) and
+ [imports](/building-with-codegen/imports). See [Moving
+ Symbols](/building-with-codegen/moving-symbols) to learn more.
+
+
+## Basic Symbol Movement
+
+To move a symbol from one file to another, you can use the [move_to_file](/api-reference/core/Function#move-to-file) method.
+
+
+```python python
+# Get the symbol
+symbol_to_move = source_file.get_symbol("my_function")
+# Pick a destination file
+dst_file = codebase.get_file("path/to/dst/location.py")
+# Move the symbol, move all of its dependencies with it (remove from old file), and add an import of symbol into old file
+symbol_to_move.move_to_file(dst_file, include_dependencies=True, strategy="add_back_edge")
+```
+
+```python typescript
+# Get the symbol
+symbol_to_move = source_file.get_symbol("myFunction")
+# Pick a destination file
+dst_file = codebase.get_file("path/to/dst/location.ts")
+# Move the symbol, move all of its dependencies with it (remove from old file), and add an import of symbol into old file
+symbol_to_move.move_to_file(dst_file, include_dependencies=True, strategy="add_back_edge")
+```
+
+
+
+This will move `my_function` to `path/to/dst/location.py`, safely updating all references to it in the process.
+
+## Updating Imports
+
+After moving a symbol, you may need to update imports throughout your codebase. GraphSitter offers two strategies for this:
+
+1. **Update All Imports**: This strategy updates all imports across the codebase to reflect the new location of the symbol.
+
+
+```python python
+symbol_to_move = codebase.get_symbol("symbol_to_move")
+dst_file = codebase.create_file("new_file.py")
+symbol_to_move.move_to_file(dst_file, strategy="update_all_imports")
+```
+
+```python typescript
+symbol_to_move = codebase.get_symbol("symbolToMove")
+dst_file = codebase.create_file("new_file.ts")
+symbol_to_move.move_to_file(dst_file, strategy="update_all_imports")
+```
+
+
+
+Updating all imports can result in very large PRs
+
+2. **Add Back Edge**: This strategy adds an import in the original file that re-imports (and exports) the moved symbol, maintaining backwards compatibility. This will result in fewer total modifications, as existing imports will not need to be updated.
+
+
+```python python
+symbol_to_move = codebase.get_symbol("symbol_to_move")
+dst_file = codebase.create_file("new_file.py")
+symbol_to_move.move_to_file(dst_file, strategy="add_back_edge")
+```
+
+```python typescript
+symbol_to_move = codebase.get_symbol("symbolToMove")
+dst_file = codebase.create_file("new_file.ts")
+symbol_to_move.move_to_file(dst_file, strategy="add_back_edge")
+```
+
+
+
+## Handling Dependencies
+
+By default, Codegen will move all of a symbols dependencies along with it. This ensures that your codebase remains consistent and functional.
+
+
+```python python
+my_symbol = codebase.get_symbol("my_symbol")
+dst_file = codebase.create_file("new_file.py")
+my_symbol.move_to_file(dst_file, include_dependencies=True)
+```
+
+```python typescript
+my_symbol = codebase.get_symbol("mySymbol")
+dst_file = codebase.create_file("new_file.ts")
+my_symbol.move_to_file(dst_file, include_dependencies=True)
+```
+
+
+
+If you set `include_dependencies=False`, only the symbol itself will be moved, and any dependencies will remain in the original file.
+
+## Moving Multiple Symbols
+
+If you need to move multiple symbols, you can do so in a loop:
+
+```python
+source_file = codebase.get_file("path/to/source_file.py")
+dest_file = codebase.get_file("path/to/destination_file.py")
+# Create a list of symbols to move
+symbols_to_move = [source_file.get_function("my_function"), source_file.get_class("MyClass")]
+# Move each symbol to the destination file
+for symbol in symbols_to_move:
+ symbol.move_to_file(dest_file, include_dependencies=True, strategy="update_all_imports")
+```
+
+## Best Practices
+
+1. **Commit After Major Changes**: If you're making multiple significant changes, use `codebase.commit()` between them to ensure the codebase graph is up-to-date.
+
+2. **Re-fetch References**: After a commit, re-fetch any file or symbol references you're working with, as they may have become stale.
+
+3. **Handle Errors**: Be prepared to handle cases where symbols or files might not exist, or where moves might fail due to naming conflicts.
+
+By following these guidelines, you can effectively move symbols around your codebase while maintaining its integrity and functionality.
+
+
+---
+title: "Improving Code Modularity"
+sidebarTitle: "Modularity"
+icon: "diagram-project"
+iconType: "solid"
+---
+
+Codegen SDK provides powerful tools for analyzing and improving code modularity. This guide will help you identify and fix common modularity issues like circular dependencies, tight coupling, and poorly organized imports.
+
+Common use cases include:
+- Breaking up circular dependencies
+- Organizing imports and exports
+- Identifying highly coupled modules
+- Extracting shared code into common modules
+- Analyzing module boundaries
+
+## Analyzing Import Relationships
+
+First, let's see how to analyze import relationships in your codebase:
+
+```python
+import networkx as nx
+from collections import defaultdict
+
+# Create a graph of file dependencies
+def create_dependency_graph():
+ G = nx.DiGraph()
+
+ for file in codebase.files:
+ # Add node for this file
+ G.add_node(file.filepath)
+
+ # Add edges for each import
+ for imp in file.imports:
+ if imp.from_file: # Skip external imports
+ G.add_edge(file.filepath, imp.from_file.filepath)
+
+ return G
+
+# Create and analyze the graph
+graph = create_dependency_graph()
+
+# Find circular dependencies
+cycles = list(nx.simple_cycles(graph))
+if cycles:
+ print("๐ Found circular dependencies:")
+ for cycle in cycles:
+ print(f" โข {' -> '.join(cycle)}")
+
+# Calculate modularity metrics
+print("\n๐ Modularity Metrics:")
+print(f" โข Number of files: {len(graph.nodes)}")
+print(f" โข Number of imports: {len(graph.edges)}")
+print(f" โข Average imports per file: {len(graph.edges)/len(graph.nodes):.1f}")
+```
+
+## Breaking Circular Dependencies
+
+When you find circular dependencies, here's how to break them:
+
+```python
+def break_circular_dependency(cycle):
+ # Get the first two files in the cycle
+ file1 = codebase.get_file(cycle[0])
+ file2 = codebase.get_file(cycle[1])
+
+ # Create a shared module for common code
+ shared_dir = "shared"
+ if not codebase.has_directory(shared_dir):
+ codebase.create_directory(shared_dir)
+
+ # Find symbols used by both files
+ shared_symbols = []
+ for symbol in file1.symbols:
+ if any(usage.file == file2 for usage in symbol.usages):
+ shared_symbols.append(symbol)
+
+ # Move shared symbols to a new file
+ if shared_symbols:
+ shared_file = codebase.create_file(f"{shared_dir}/shared_types.py")
+ for symbol in shared_symbols:
+ symbol.move_to_file(shared_file, strategy="update_all_imports")
+
+# Break each cycle found
+for cycle in cycles:
+ break_circular_dependency(cycle)
+```
+
+## Organizing Imports
+
+Clean up and organize imports across your codebase:
+
+```python
+def organize_file_imports(file):
+ # Group imports by type
+ std_lib_imports = []
+ third_party_imports = []
+ local_imports = []
+
+ for imp in file.imports:
+ if imp.is_standard_library:
+ std_lib_imports.append(imp)
+ elif imp.is_third_party:
+ third_party_imports.append(imp)
+ else:
+ local_imports.append(imp)
+
+ # Sort each group
+ for group in [std_lib_imports, third_party_imports, local_imports]:
+ group.sort(key=lambda x: x.module_name)
+
+ # Remove all existing imports
+ for imp in file.imports:
+ imp.remove()
+
+ # Add imports back in organized groups
+ if std_lib_imports:
+ for imp in std_lib_imports:
+ file.add_import_from_import_string(imp.source)
+ file.insert_after_imports("") # Add newline
+
+ if third_party_imports:
+ for imp in third_party_imports:
+ file.add_import_from_import_string(imp.source)
+ file.insert_after_imports("") # Add newline
+
+ if local_imports:
+ for imp in local_imports:
+ file.add_import_from_import_string(imp.source)
+
+# Organize imports in all files
+for file in codebase.files:
+ organize_file_imports(file)
+```
+
+## Identifying Highly Coupled Modules
+
+Find modules that might need to be split up:
+
+```python
+from collections import defaultdict
+
+def analyze_module_coupling():
+ coupling_scores = defaultdict(int)
+
+ for file in codebase.files:
+ # Count unique files imported from
+ imported_files = {imp.from_file for imp in file.imports if imp.from_file}
+ coupling_scores[file.filepath] = len(imported_files)
+
+ # Count files that import this file
+ importing_files = {usage.file for symbol in file.symbols
+ for usage in symbol.usages if usage.file != file}
+ coupling_scores[file.filepath] += len(importing_files)
+
+ # Sort by coupling score
+ sorted_files = sorted(coupling_scores.items(),
+ key=lambda x: x[1],
+ reverse=True)
+
+ print("\n๐ Module Coupling Analysis:")
+ print("\nMost coupled files:")
+ for filepath, score in sorted_files[:5]:
+ print(f" โข {filepath}: {score} connections")
+
+analyze_module_coupling()
+```
+
+## Extracting Shared Code
+
+When you find highly coupled modules, extract shared code:
+
+```python
+def extract_shared_code(file, min_usages=3):
+ # Find symbols used by multiple files
+ for symbol in file.symbols:
+ # Get unique files using this symbol
+ using_files = {usage.file for usage in symbol.usages
+ if usage.file != file}
+
+ if len(using_files) >= min_usages:
+ # Create appropriate shared module
+ module_name = determine_shared_module(symbol)
+ if not codebase.has_file(f"shared/{module_name}.py"):
+ shared_file = codebase.create_file(f"shared/{module_name}.py")
+ else:
+ shared_file = codebase.get_file(f"shared/{module_name}.py")
+
+ # Move symbol to shared module
+ symbol.move_to_file(shared_file, strategy="update_all_imports")
+
+def determine_shared_module(symbol):
+ # Logic to determine appropriate shared module name
+ if symbol.is_type:
+ return "types"
+ elif symbol.is_constant:
+ return "constants"
+ elif symbol.is_utility:
+ return "utils"
+ else:
+ return "common"
+```
+
+---
+title: "Managing Feature Flags"
+sidebarTitle: "Feature Flags"
+icon: "flag"
+iconType: "solid"
+---
+
+Codegen has been used in production for multi-million line codebases to automatically delete "dead" (rolled-out) feature flags. This guide will walk you through analyzing feature flag usage and safely removing rolled out flags.
+
+
+ Every codebase does feature flags differently. This guide shows common techniques and syntax but likely requires adaptation to codebase-specific circumstances.
+
+
+## Analyzing Feature Flag Usage
+
+Before removing a feature flag, it's important to analyze its usage across the codebase. Codegen provides tools to help identify where and how feature flags are used.
+
+### For Python Codebases
+
+For Python codebases using a `FeatureFlags` class pattern like so:
+```python
+class FeatureFlags:
+ FEATURE_1 = False
+ FEATURE_2 = True
+```
+
+You can use [Class.get_attribute(...)](/api-reference/core/Class#get-attribute) and [Attribute.usages](/api-reference/core/Attribute#usages) to analyze the coverage of your flags, like so:
+
+
+
+```python
+feature_flag_usage = {}
+feature_flag_class = codebase.get_class('FeatureFlag')
+
+if feature_flag_class:
+ # Initialize usage count for all attributes
+ for attr in feature_flag_class.attributes:
+ feature_flag_usage[attr.name] = 0
+
+ # Get all usages of the FeatureFlag class
+ for usage in feature_flag_class.usages:
+ usage_source = usage.usage_symbol.source if hasattr(usage, 'usage_symbol') else str(usage)
+ for flag_name in feature_flag_usage.keys():
+ if f"FeatureFlag.{flag_name}" in usage_source:
+ feature_flag_usage[flag_name] += 1
+
+ sorted_flags = sorted(feature_flag_usage.items(), key=lambda x: x[1], reverse=True)
+
+ print("Feature Flag Usage Table:")
+ print("-------------------------")
+ print(f"{'Feature Flag':<30} | {'Usage Count':<12}")
+ print("-" * 45)
+ for flag, count in sorted_flags:
+ print(f"{flag:<30} | {count:<12}")
+
+ print(f"\nTotal feature flags: {len(sorted_flags)}")
+else:
+ print("โ FeatureFlag enum not found in the codebase")
+```
+
+This will output a table showing all feature flags and their usage counts, helping identify which flags are candidates for removal.
+
+
+ Learn more about [Attributes](/building-with-codegen/class-api#class-attributes) and [tracking usages](/building-with-codegen/dependencies-and-usages) here
+
+
+
+## Removing Rolled Out Flags
+
+Once you've identified a flag that's ready to be removed, Codegen can help safely delete it and its associated code paths.
+
+
+ This primarily leverages Codegen's API for [reduction conditions](/building-with-codegen/reducing-conditions)
+
+
+### Python Example
+
+For Python codebases, here's how to remove a feature flag and its usages:
+
+```python
+flag_name = "FEATURE_TO_REMOVE"
+
+# Get the feature flag variable
+feature_flag_file = codebase.get_file("app/utils/feature_flags.py")
+flag_class = feature_flag_file.get_class("FeatureFlag")
+
+# Check if the flag exists
+flag_var = flag_class.get_attribute(flag_name)
+if not flag_var:
+ print(f'No such flag: {flag_name}')
+ return
+
+# Remove all usages of the feature flag
+for usage in flag_var.usages:
+ if isinstance(usage.parent, IfBlockStatement):
+ # For if statements, reduce the condition to True
+ usage.parent.reduce_condition(True)
+ elif isinstance(usage.parent, WithStatement):
+ # For with statements, keep the code block
+ usage.parent.code_block.unwrap()
+ else:
+ # For other cases, remove the usage
+ usage.remove()
+
+# Remove the flag definition
+flag_var.remove()
+
+# Commit changes
+codebase.commit()
+```
+
+### React/TypeScript Example
+
+For React applications using a hooks-based feature flag system:
+
+```python
+feature_flag_name = "NEW_UI_ENABLED"
+target_value = True # The value to reduce the flag to
+
+print(f'Removing feature flag: {feature_flag_name}')
+
+# 1. Remove from configuration
+config_file = codebase.get_file("src/featureFlags/config.ts")
+feature_flag_config = config_file.get_symbol("FEATURE_FLAG_CONFIG").value
+if feature_flag_name in feature_flag_config.keys():
+ feature_flag_config.pop(feature_flag_name)
+ print('โ
Removed from feature flag config')
+
+# 2. Find and reduce all hook usages
+hook = codebase.get_function("useFeatureFlag")
+for usage in hook.usages:
+ fcall = usage.match
+ if isinstance(fcall, FunctionCall):
+ # Check if this usage is for our target flag
+ first_arg = fcall.args[0].value
+ if isinstance(first_arg, String) and first_arg.content == feature_flag_name:
+ print(f'Reducing in: {fcall.parent_symbol.name}')
+ # This automatically handles:
+ # - Ternary expressions: flag ? :
+ # - If statements: if (flag) { ... }
+ # - Conditional rendering: {flag && }
+ fcall.reduce_condition(target_value)
+
+# 3. Commit changes
+codebase.commit()
+```
+
+This will:
+1. Remove the feature flag from the configuration
+2. Find all usages of the `useFeatureFlag` hook for this flag
+3. Automatically reduce any conditional logic using the flag
+4. Handle common React patterns like ternaries and conditional rendering
+
+
+## Related Resources
+- [Reducing Conditions](/building-with-codegen/reducing-conditions) - Details on condition reduction APIs
+- [Dead Code Removal](/tutorials/deleting-dead-code) - Remove unused code after flag deletion
+
+---
+title: "Deleting Dead Code"
+sidebarTitle: "Dead Code"
+icon: "trash"
+iconType: "solid"
+---
+
+Dead code refers to code that is not being used or referenced anywhere in your codebase.
+
+However, it's important to note that some code might appear unused but should not be deleted, including:
+- Test files and test functions
+- Functions with decorators (which may be called indirectly)
+- Public API endpoints
+- Event handlers or callback functions
+- Code used through reflection or dynamic imports
+
+This guide will show you how to safely identify and remove genuinely unused code while preserving important functionality.
+
+## Overview
+
+To simply identify code without any external usages, you can check for the absence of [Symbol.usages](/api-reference/core/Symbol#usages).
+
+See [Dependencies and Usages](/building-with-codegen/dependencies-and-usages) for more information on how to use these properties.
+
+```python
+# Iterate through all functions in the codebase
+for function in codebase.functions:
+ # Remove functions with no usages
+ if not function.usages:
+ function.remove()
+
+# Commit
+codebase.commit()
+```
+
+
+This will remove all code that is not explicitly referenced elsewhere, including tests, endpoints, etc. This is almost certainly not what you want. We recommend further filtering.
+
+
+## Filtering for Special Cases
+
+To filter out special cases that are not explicitly referenced yet are, nonetheless, worth keeping around, you can use the following pattern:
+
+
+```python
+for function in codebase.functions:
+
+ # Skip test files
+ if "test" in function.file.filepath:
+ continue
+
+ # Skip decorated functions
+ if function.decorators:
+ continue
+
+ # Skip public routes, e.g. next.js endpoints
+ # (Typescript only)
+ if 'routes' in function.file.filepath and function.is_jsx:
+ continue
+
+ # ... etc.
+
+ # Check if the function has no usages and no call sites
+ if not function.usages and not function.call_sites:
+ # Print a message indicating the removal of the function
+ print(f"Removing unused function: {function.name}")
+ # Remove the function from the file
+ function.remove()
+
+# Commit
+codebase.commit()
+```
+
+
+## Cleaning Up Unused Variables
+
+To remove unused variables, you can check for their usages within their scope:
+
+```python typescript
+for func in codebase.functions:
+ # Iterate through local variable assignments in the function
+ for var_assignments in func.code_block.local_var_assignments:
+ # Check if the local variable assignment has no usages
+ if not var_assignments.local_usages:
+ # Remove the local variable assignment
+ var_assignments.remove()
+
+# Commit
+codebase.commit()
+```
+
+
+## Cleaning Up After Removal
+
+After removing dead code, you may need to clean up any remaining artifacts:
+
+```python
+for file in codebase.files:
+ # Check if the file is empty
+ if not file.content.strip():
+ # Print a message indicating the removal of the empty file
+ print(f"Removing empty file: {file.filepath}")
+ # Remove the empty file
+ file.remove()
+
+# commit is NECESSARY to remove the files from the codebase
+codebase.commit()
+
+# Remove redundant newlines
+for file in codebase.files:
+ # Replace three or more consecutive newlines with two newlines
+ file.edit(re.sub(r"\n{3,}", "\n\n", file.content))
+```
+
+
+---
+title: "Increasing Type Coverage"
+sidebarTitle: "Type Coverage"
+icon: "shield-check"
+iconType: "solid"
+---
+
+This guide demonstrates how to analyze and manipulate type annotations with Codegen SDK.
+
+Common use cases include:
+
+- Adding a type to a union or generic type
+- Checking if a generic type has a given subtype
+- Resolving a type annotation
+
+
+ Adding type hints can improve developer experience and [significantly speed up](https://github.com/microsoft/Typescript/wiki/Performance#using-type-annotations) programs like the Typescript compiler and `mypy`.
+
+
+See [Type Annotations](/building-with-codegen/type-annotations) for a general overview of the type maninpulation
+
+## APIs for monitoring types
+
+Codegen programs typically access type annotations through the following APIs:
+- [Parameter.type](/api-reference/core/Parameter#type)
+- [Function.return_type](/api-reference/python/PyFunction#return-type)
+- [Assignment.type](/api-reference/core/Assignment#type)
+
+Each of these has an associated setter.
+
+
+## Finding the extent of your type coverage
+
+To get an indication of your progress on type coverage, analyze the percentage of typed elements across your codebase
+
+```python
+# Initialize counters for parameters
+total_parameters = 0
+typed_parameters = 0
+
+# Initialize counters for return types
+total_functions = 0
+typed_returns = 0
+
+# Initialize counters for class attributes
+total_attributes = 0
+typed_attributes = 0
+
+# Count parameter and return type coverage
+for function in codebase.functions:
+ # Count parameters
+ total_parameters += len(function.parameters)
+ typed_parameters += sum(1 for param in function.parameters if param.is_typed)
+
+ # Count return types
+ total_functions += 1
+ if function.return_type and function.return_type.is_typed:
+ typed_returns += 1
+
+# Count class attribute coverage
+for cls in codebase.classes:
+ for attr in cls.attributes:
+ total_attributes += 1
+ if attr.is_typed:
+ typed_attributes += 1
+
+# Calculate percentages
+param_percentage = (typed_parameters / total_parameters * 100) if total_parameters > 0 else 0
+return_percentage = (typed_returns / total_functions * 100) if total_functions > 0 else 0
+attr_percentage = (typed_attributes / total_attributes * 100) if total_attributes > 0 else 0
+
+# Print results
+print("\nType Coverage Analysis")
+print("---------------------")
+print(f"Parameters: {param_percentage:.1f}% ({typed_parameters}/{total_parameters} typed)")
+print(f"Return types: {return_percentage:.1f}% ({typed_returns}/{total_functions} typed)")
+print(f"Class attributes: {attr_percentage:.1f}% ({typed_attributes}/{total_attributes} typed)")
+```
+
+This analysis gives you a breakdown of type coverage across three key areas:
+1. Function parameters - Arguments passed to functions
+2. Return types - Function return type annotations
+3. Class attributes - Type hints on class variables
+
+
+ Focus first on adding types to the most frequently used functions and classes, as these will have the biggest impact on type checking and IDE support.
+
+
+## Adding simple return type annotations
+
+To add a return type, use `function.set_return_type`. The script below will add a `-> None` return type to all functions that contain no return statements:
+
+
+```python For Python
+for file in codebase.files:
+ # Check if 'app' is in the file's filepath
+ if "app" in file.filepath:
+ # Iterate through all functions in the file
+ for function in file.functions:
+ # Check if the function has no return statements
+ if len(function.return_statements) == 0:
+ # Set the return type to None
+ function.set_return_type("None")
+```
+
+```python For Typescript
+for file in codebase.files:
+ # Check if 'app' is in the file's filepath
+ if "app" in file.filepath:
+ # Iterate through all functions in the file
+ for function in file.functions:
+ # Check if the function has no return statements
+ if len(function.return_statements) == 0:
+ # Set the return type to None
+ function.set_return_type("null")
+```
+
+
+
+## Coming Soon: Advanced Type Inference
+
+Codegen is building out an API for direct interface with `tsc` and `mypy` for precise type inference. Interested piloting this API? Let us know!
+
+---
+title: "Managing TypeScript Exports"
+sidebarTitle: "Export Management"
+description: "Safely and systematically manage exports in your TypeScript codebase"
+icon: "ship"
+iconType: "solid"
+---
+
+Codegen provides powerful tools for managing and reorganizing exports in TypeScript codebases. This tutorial builds on the concepts covered in [exports](/building-with-codegen/exports) to show you how to automate common export management tasks and ensure your module boundaries stay clean and maintainable.
+
+## Common Export Management Tasks
+
+### Collecting and Processing Exports
+
+When reorganizing exports, the first step is identifying which exports need to be processed:
+
+```python
+processed_imports = set()
+
+for file in codebase.files:
+ # Only process files under /src/shared
+ if '/src/shared' not in file.filepath:
+ continue
+
+ # Gather all reexports that are not external exports
+ all_reexports = []
+ for export_stmt in file.export_statements:
+ for export in export_stmt.exports:
+ if export.is_reexport() and not export.is_external_export:
+ all_reexports.append(export)
+
+ # Skip if there are none
+ if not all_reexports:
+ continue
+```
+
+### Moving Exports to Public Files
+
+When centralizing exports in public-facing files:
+
+```python
+# Replace "src/" with "src/shared/"
+resolved_public_file = export.resolved_symbol.filepath.replace("src/", "src/shared/")
+
+# Get relative path from the "public" file back to the original file
+relative_path = codebase.get_relative_path(
+ from_file=resolved_public_file,
+ to_file=export.resolved_symbol.filepath
+)
+
+# Ensure the "public" file exists
+if not codebase.has_file(resolved_public_file):
+ target_file = codebase.create_file(resolved_public_file, sync=True)
+else:
+ target_file = codebase.get_file(resolved_public_file)
+
+# If target file already has a wildcard export for this relative path, skip
+if target_file.has_export_statement_for_path(relative_path, "WILDCARD"):
+ has_wildcard = True
+ continue
+```
+
+### Managing Different Export Types
+
+Codegen can handle all types of exports automatically:
+
+
+
+ ```python
+ # A) Wildcard export, e.g. `export * from "..."`
+ if export.is_wildcard_export():
+ target_file.insert_before(f'export * from "{relative_path}"')
+ ```
+
+
+
+ ```python
+ # B) Type export, e.g. `export type { Foo, Bar } from "..."`
+ elif export.is_type_export():
+ # Does this file already have a type export statement for the path?
+ statement = file.get_export_statement_for_path(relative_path, "TYPE")
+ if statement:
+ # Insert into existing statement
+ if export.is_aliased():
+ statement.insert(0, f"{export.resolved_symbol.name} as {export.name}")
+ else:
+ statement.insert(0, f"{export.name}")
+ else:
+ # Insert a new type export statement
+ if export.is_aliased():
+ target_file.insert_before(
+ f'export type {{ {export.resolved_symbol.name} as {export.name} }} '
+ f'from "{relative_path}"'
+ )
+ else:
+ target_file.insert_before(
+ f'export type {{ {export.name} }} from "{relative_path}"'
+ )
+ ```
+
+
+
+ ```python
+ # C) Normal export, e.g. `export { Foo, Bar } from "..."`
+ else:
+ statement = file.get_export_statement_for_path(relative_path, "EXPORT")
+ if statement:
+ # Insert into existing statement
+ if export.is_aliased():
+ statement.insert(0, f"{export.resolved_symbol.name} as {export.name}")
+ else:
+ statement.insert(0, f"{export.name}")
+ else:
+ # Insert a brand-new normal export statement
+ if export.is_aliased():
+ target_file.insert_before(
+ f'export {{ {export.resolved_symbol.name} as {export.name} }} '
+ f'from "{relative_path}"'
+ )
+ else:
+ target_file.insert_before(
+ f'export {{ {export.name} }} from "{relative_path}"'
+ )
+ ```
+
+
+
+## Updating Import References
+
+After moving exports, you need to update all import references:
+
+```python
+# Now update all import usages that refer to this export
+for usage in export.symbol_usages():
+ if isinstance(usage, TSImport) and usage not in processed_imports:
+ processed_imports.add(usage)
+
+ # Translate the resolved_public_file to the usage file's TS config import path
+ new_path = usage.file.ts_config.translate_import_path(resolved_public_file)
+
+ if has_wildcard and export.name != export.resolved_symbol.name:
+ name = f"{export.resolved_symbol.name} as {export.name}"
+ else:
+ name = usage.name
+
+ if usage.is_type_import():
+ new_import = f'import type {{ {name} }} from "{new_path}"'
+ else:
+ new_import = f'import {{ {name} }} from "{new_path}"'
+
+ usage.file.insert_before(new_import)
+ usage.remove()
+
+# Remove the old export from the original file
+export.remove()
+
+# If the file ends up with no exports, remove it entirely
+if not file.export_statements and len(file.symbols) == 0:
+ file.remove()
+```
+
+## Best Practices
+
+1. **Check for Wildcards First**: Always check for existing wildcard exports before adding new ones:
+```python
+if target_file.has_export_statement_for_path(relative_path, "WILDCARD"):
+ has_wildcard = True
+ continue
+```
+
+2. **Handle Path Translations**: Use TypeScript config for path translations:
+```python
+new_path = usage.file.ts_config.translate_import_path(resolved_public_file)
+```
+
+3. **Clean Up Empty Files**: Remove files that no longer contain exports or symbols:
+```python
+if not file.export_statements and len(file.symbols) == 0:
+ file.remove()
+```
+
+## Next Steps
+
+After reorganizing your exports:
+
+1. Run your test suite to verify everything still works
+2. Review the generated import statements
+3. Check for any empty files that should be removed
+4. Verify that all export types (wildcard, type, named) are working as expected
+
+
+Remember that managing exports is an iterative process. You may need to run the codemod multiple times as your codebase evolves.
+
+
+### Related tutorials
+- [Moving symbols](/building-with-codegen/moving-symbols)
+- [Exports](/building-with-codegen/exports)
+- [Dependencies and usages](/building-with-codegen/dependencies-and-usages)
+
+## Complete Codemod
+
+Here's the complete codemod that you can copy and use directly:
+
+```python
+processed_imports = set()
+
+for file in codebase.files:
+ # Only process files under /src/shared
+ if '/src/shared' not in file.filepath:
+ continue
+
+ # Gather all reexports that are not external exports
+ all_reexports = []
+ for export_stmt in file.export_statements:
+ for export in export_stmt.exports:
+ if export.is_reexport() and not export.is_external_export:
+ all_reexports.append(export)
+
+ # Skip if there are none
+ if not all_reexports:
+ continue
+
+ for export in all_reexports:
+ has_wildcard = False
+
+ # Replace "src/" with "src/shared/"
+ resolved_public_file = export.resolved_symbol.filepath.replace("src/", "src/shared/")
+
+ # Get relative path from the "public" file back to the original file
+ relative_path = codebase.get_relative_path(
+ from_file=resolved_public_file,
+ to_file=export.resolved_symbol.filepath
+ )
+
+ # Ensure the "public" file exists
+ if not codebase.has_file(resolved_public_file):
+ target_file = codebase.create_file(resolved_public_file, sync=True)
+ else:
+ target_file = codebase.get_file(resolved_public_file)
+
+ # If target file already has a wildcard export for this relative path, skip
+ if target_file.has_export_statement_for_path(relative_path, "WILDCARD"):
+ has_wildcard = True
+ continue
+
+ # Compare "public" path to the local file's export.filepath
+ if codebase._remove_extension(resolved_public_file) != codebase._remove_extension(export.filepath):
+
+ # A) Wildcard export, e.g. `export * from "..."`
+ if export.is_wildcard_export():
+ target_file.insert_before(f'export * from "{relative_path}"')
+
+ # B) Type export, e.g. `export type { Foo, Bar } from "..."`
+ elif export.is_type_export():
+ # Does this file already have a type export statement for the path?
+ statement = file.get_export_statement_for_path(relative_path, "TYPE")
+ if statement:
+ # Insert into existing statement
+ if export.is_aliased():
+ statement.insert(0, f"{export.resolved_symbol.name} as {export.name}")
+ else:
+ statement.insert(0, f"{export.name}")
+ else:
+ # Insert a new type export statement
+ if export.is_aliased():
+ target_file.insert_before(
+ f'export type {{ {export.resolved_symbol.name} as {export.name} }} '
+ f'from "{relative_path}"'
+ )
+ else:
+ target_file.insert_before(
+ f'export type {{ {export.name} }} from "{relative_path}"'
+ )
+
+ # C) Normal export, e.g. `export { Foo, Bar } from "..."`
+ else:
+ statement = file.get_export_statement_for_path(relative_path, "EXPORT")
+ if statement:
+ # Insert into existing statement
+ if export.is_aliased():
+ statement.insert(0, f"{export.resolved_symbol.name} as {export.name}")
+ else:
+ statement.insert(0, f"{export.name}")
+ else:
+ # Insert a brand-new normal export statement
+ if export.is_aliased():
+ target_file.insert_before(
+ f'export {{ {export.resolved_symbol.name} as {export.name} }} '
+ f'from "{relative_path}"'
+ )
+ else:
+ target_file.insert_before(
+ f'export {{ {export.name} }} from "{relative_path}"'
+ )
+
+ # Now update all import usages that refer to this export
+ for usage in export.symbol_usages():
+ if isinstance(usage, TSImport) and usage not in processed_imports:
+ processed_imports.add(usage)
+
+ # Translate the resolved_public_file to the usage file's TS config import path
+ new_path = usage.file.ts_config.translate_import_path(resolved_public_file)
+
+ if has_wildcard and export.name != export.resolved_symbol.name:
+ name = f"{export.resolved_symbol.name} as {export.name}"
+ else:
+ name = usage.name
+
+ if usage.is_type_import():
+ new_import = f'import type {{ {name} }} from "{new_path}"'
+ else:
+ new_import = f'import {{ {name} }} from "{new_path}"'
+
+ usage.file.insert_before(new_import)
+ usage.remove()
+
+ # Remove the old export from the original file
+ export.remove()
+
+ # If the file ends up with no exports, remove it entirely
+ if not file.export_statements and len(file.symbols) == 0:
+ file.remove()
+```
+
+---
+title: "Converting Default Exports"
+sidebarTitle: "Default Export Conversion"
+description: "Convert default exports to named exports in your TypeScript codebase"
+icon: "arrow-right-arrow-left"
+iconType: "solid"
+---
+
+Codegen provides tools to help you migrate away from default exports to named exports in your TypeScript codebase. This tutorial builds on the concepts covered in [exports](/building-with-codegen/exports) to show you how to automate this conversion process.
+
+## Overview
+
+Default exports can make code harder to maintain and refactor. Converting them to named exports provides several benefits:
+
+- Better IDE support for imports and refactoring
+- More explicit and consistent import statements
+- Easier to track symbol usage across the codebase
+
+## Converting Default Exports
+
+Here's how to convert default exports to named exports:
+
+```python
+for file in codebase.files:
+ target_file = file.filepath
+ if not target_file:
+ print(f"โ ๏ธ Target file not found: {filepath}")
+ continue
+
+ # Get corresponding non-shared file
+ non_shared_path = target_file.filepath.replace('/shared/', '/')
+ if not codebase.has_file(non_shared_path):
+ print(f"โ ๏ธ No matching non-shared file for: {filepath}")
+ continue
+
+ non_shared_file = codebase.get_file(non_shared_path)
+ print(f"๐ Processing {target_file.filepath}")
+
+ # Process individual exports
+ for export in target_file.exports:
+ # Handle default exports
+ if export.is_reexport() and export.is_default_export():
+ print(f" ๐ Converting default export '{export.name}'")
+ default_export = next((e for e in non_shared_file.default_exports), None)
+ if default_export:
+ default_export.make_non_default()
+
+ print(f"โจ Fixed exports in {target_file.filepath}")
+```
+
+## Understanding the Process
+
+Let's break down how this works:
+
+
+
+ ```python
+ # Process individual exports
+ for export in target_file.exports:
+ # Handle default exports
+ if export.is_reexport() and export.is_default_export():
+ print(f" ๐ Converting default export '{export.name}'")
+ ```
+
+ The code identifies default exports by checking:
+ 1. If it's a re-export (`is_reexport()`)
+ 2. If it's a default export (`is_default_export()`)
+
+
+
+ ```python
+ default_export = next((e for e in non_shared_file.default_exports), None)
+ if default_export:
+ default_export.make_non_default()
+ ```
+
+ For each default export:
+ 1. Find the corresponding export in the non-shared file
+ 2. Convert it to a named export using `make_non_default()`
+
+
+
+ ```python
+ # Get corresponding non-shared file
+ non_shared_path = target_file.filepath.replace('/shared/', '/')
+ if not codebase.has_file(non_shared_path):
+ print(f"โ ๏ธ No matching non-shared file for: {filepath}")
+ continue
+
+ non_shared_file = codebase.get_file(non_shared_path)
+ ```
+
+ The code:
+ 1. Maps shared files to their non-shared counterparts
+ 2. Verifies the non-shared file exists
+ 3. Loads the non-shared file for processing
+
+
+
+## Best Practices
+
+1. **Check for Missing Files**: Always verify files exist before processing:
+```python
+if not target_file:
+ print(f"โ ๏ธ Target file not found: {filepath}")
+ continue
+```
+
+2. **Log Progress**: Add logging to track the conversion process:
+```python
+print(f"๐ Processing {target_file.filepath}")
+print(f" ๐ Converting default export '{export.name}'")
+```
+
+3. **Handle Missing Exports**: Check that default exports exist before converting:
+```python
+default_export = next((e for e in non_shared_file.default_exports), None)
+if default_export:
+ default_export.make_non_default()
+```
+
+## Next Steps
+
+After converting default exports:
+
+1. Run your test suite to verify everything still works
+2. Update any import statements that were using default imports
+3. Review the changes to ensure all exports were converted correctly
+4. Consider adding ESLint rules to prevent new default exports
+
+
+Remember to test thoroughly after converting default exports, as this change affects how other files import the converted modules.
+
+
+### Related tutorials
+- [Managing typescript exports](/tutorials/managing-typescript-exports)
+- [Exports](/building-with-codegen/exports)
+- [Dependencies and usages](/building-with-codegen/dependencies-and-usages)
+
+## Complete Codemod
+
+Here's the complete codemod that you can copy and use directly:
+
+```python
+
+for file in codebase.files:
+ target_file = file.filepath
+ if not target_file:
+ print(f"โ ๏ธ Target file not found: {filepath}")
+ continue
+
+ # Get corresponding non-shared file
+ non_shared_path = target_file.filepath.replace('/shared/', '/')
+ if not codebase.has_file(non_shared_path):
+ print(f"โ ๏ธ No matching non-shared file for: {filepath}")
+ continue
+
+ non_shared_file = codebase.get_file(non_shared_path)
+ print(f"๐ Processing {target_file.filepath}")
+
+ # Process individual exports
+ for export in target_file.exports:
+ # Handle default exports
+ if export.is_reexport() and export.is_default_export():
+ print(f" ๐ Converting default export '{export.name}'")
+ default_export = next((e for e in non_shared_file.default_exports), None)
+ if default_export:
+ default_export.make_non_default()
+
+ print(f"โจ Fixed exports in {target_file.filepath}")
+
+```
+
+---
+title: "Creating Documentation"
+sidebarTitle: "Documentation"
+icon: "book"
+iconType: "solid"
+---
+
+This guide demonstrates how to determine docs coverage and create documentation for your codebase.
+
+This primarily leverages two APIs:
+- [`codebase.ai(...)`](/api-reference/core/Codebase#ai) for generating docstrings
+- [`function.set_docstring(...)`](/api-reference/core/HasBlock#set-docstring) for modifying them
+
+## Determining Documentation Coverage
+
+In order to determine the extent of your documentation coverage, you can iterate through all symbols of interest and count the number of docstrings:
+
+To see your current documentation coverage, you can iterate through all symbols of interest and count the number of docstrings:
+
+```python python
+# Initialize counters
+total_functions = 0
+functions_with_docs = 0
+total_classes = 0
+classes_with_docs = 0
+
+# Check functions
+for function in codebase.functions:
+ total_functions += 1
+ if function.docstring:
+ functions_with_docs += 1
+
+# Check classes
+for cls in codebase.classes:
+ total_classes += 1
+ if cls.docstring:
+ classes_with_docs += 1
+
+# Calculate percentages
+func_coverage = (functions_with_docs / total_functions * 100) if total_functions > 0 else 0
+class_coverage = (classes_with_docs / total_classes * 100) if total_classes > 0 else 0
+
+# Print results with emojis
+print("\n๐ Documentation Coverage Report:")
+print(f"\n๐ Functions:")
+print(f" โข Total: {total_functions}")
+print(f" โข Documented: {functions_with_docs}")
+print(f" โข Coverage: {func_coverage:.1f}%")
+
+print(f"\n๐ Classes:")
+print(f" โข Total: {total_classes}")
+print(f" โข Documented: {classes_with_docs}")
+print(f" โข Coverage: {class_coverage:.1f}%")
+
+print(f"\n๐ฏ Overall Coverage: {((functions_with_docs + classes_with_docs) / (total_functions + total_classes) * 100):.1f}%")
+```
+
+Which provides the following output:
+```
+๐ Documentation Coverage Report:
+๐ Functions:
+ โข Total: 1384
+ โข Documented: 331
+ โข Coverage: 23.9%
+๐ Classes:
+ โข Total: 453
+ โข Documented: 91
+ โข Coverage: 20.1%
+๐ฏ Overall Coverage: 23.0%
+```
+
+## Identifying Areas of Low Documentation Coverage
+
+
+To identify areas of low documentation coverage, you can iterate through all directories and count the number of functions with docstrings.
+
+Learn more about [`Directories` here](/building-with-codegen/files-and-directories).
+
+```python python
+# Track directory stats
+dir_stats = {}
+
+# Analyze each directory
+for directory in codebase.directories:
+ # Skip test, sql and alembic directories
+ if any(x in directory.path.lower() for x in ['test', 'sql', 'alembic']):
+ continue
+
+ # Get undecorated functions
+ funcs = [f for f in directory.functions if not f.is_decorated]
+ total = len(funcs)
+
+ # Only analyze dirs with >10 functions
+ if total > 10:
+ documented = sum(1 for f in funcs if f.docstring)
+ coverage = (documented / total * 100)
+ dir_stats[directory.path] = {
+ 'total': total,
+ 'documented': documented,
+ 'coverage': coverage
+ }
+
+# Find lowest coverage directory
+if dir_stats:
+ lowest_dir = min(dir_stats.items(), key=lambda x: x[1]['coverage'])
+ path, stats = lowest_dir
+
+ print(f"๐ Lowest coverage directory: '{path}'")
+ print(f" โข Total functions: {stats['total']}")
+ print(f" โข Documented: {stats['documented']}")
+ print(f" โข Coverage: {stats['coverage']:.1f}%")
+
+ # Print all directory stats for comparison
+ print("\n๐ All directory coverage rates:")
+ for path, stats in sorted(dir_stats.items(), key=lambda x: x[1]['coverage']):
+ print(f" '{path}': {stats['coverage']:.1f}% ({stats['documented']}/{stats['total']} functions)")
+```
+
+Which provides the following output:
+```python
+๐ Lowest coverage directory: 'codegen-backend/app/utils/github_utils/branch'
+ โข Total functions: 12
+ โข Documented: 0
+ โข Coverage: 0.0%
+๐ All directory coverage rates:
+ 'codegen-backend/app/utils/github_utils/branch': 0.0% (0/12 functions)
+ 'codegen-backend/app/utils/slack': 14.3% (2/14 functions)
+ 'codegen-backend/app/modal_app/github': 18.2% (2/11 functions)
+ 'codegen-backend/app/modal_app/slack': 18.2% (2/11 functions)
+ 'codegen-backend/app/utils/github_utils/webhook': 21.4% (6/28 functions)
+ 'codegen-backend/app/modal_app/cron': 23.1% (3/13 functions)
+ 'codegen-backend/app/utils/github_utils': 23.5% (39/166 functions)
+ 'codegen-backend/app/codemod': 25.0% (7/28 functions)
+```
+
+## Leveraging AI for Generating Documentation
+
+For non-trivial codebases, it can be challenging to achieve full documentation coverage.
+
+The most efficient way to edit informative docstrings is to use [codebase.ai](/api-reference/core/Codebase#ai) to generate docstrings, then use the [set_docstring](/api-reference/core/HasBlock#set-docstring) method to update the docstring.
+
+Learn more about using AI in our [guides](/building-with-codegen/calling-out-to-llms).
+
+```python python
+# Import datetime for timestamp
+from datetime import datetime
+
+# Get current timestamp
+timestamp = datetime.now().strftime("%B %d, %Y")
+
+print("๐ Generating and Updating Function Documentation")
+
+# Process all functions in the codebase
+for function in codebase.functions:
+ current_docstring = function.docstring()
+
+ if current_docstring:
+ # Update existing docstring to be more descriptive
+ new_docstring = codebase.ai(
+ f"Update the docstring for {function.name} to be more descriptive and comprehensive.",
+ target=function
+ )
+ new_docstring += f"\n\nUpdated on: {timestamp}"
+ else:
+ # Generate new docstring for function
+ new_docstring = codebase.ai(
+ f"Generate a comprehensive docstring for {function.name} including parameters, return type, and description.",
+ target=function
+ )
+ new_docstring += f"\n\nCreated on: {timestamp}"
+
+ # Set the new or updated docstring
+ function.set_docstring(new_docstring)
+```
+
+
+
+## Adding Explicit Parameter Names and Types
+
+Alternatively, you can also rely on deterministic string formatting to edit docstrings.
+
+To add "Google-style" parameter names and types to a function docstring, you can use the following code snippet:
+
+```python python
+# Iterate through all functions in the codebase
+for function in codebase.functions:
+ # Skip if function already has a docstring
+ if function.docstring:
+ continue
+
+ # Build parameter documentation
+ param_docs = []
+ for param in function.parameters:
+ param_type = param.type.source if param.is_typed else "Any"
+ param_docs.append(f" {param.name} ({param_type}): Description of {param.name}")
+
+ # Get return type if present
+ return_type = function.return_type.source if function.return_type else "None"
+
+ # Create Google-style docstring
+ docstring = f'''"""
+ Description of {function.name}.
+
+ Args:
+{chr(10).join(param_docs)}
+
+ Returns:
+ {return_type}: Description of return value
+ """'''
+
+ # Set the new docstring
+ function.set_docstring(docstring)
+```
+
+
+---
+title: "React Modernization"
+sidebarTitle: "React Modernization"
+icon: "react"
+iconType: "brands"
+description: "Modernize your React codebase with Codegen"
+---
+
+Codegen SDK provides powerful APIs for modernizing React codebases. This guide will walk you through common React modernization patterns.
+
+Common use cases include:
+
+- Upgrading to modern APIs, including React 18+
+- Automatically memoizing components
+- Converting to modern hooks
+- Standardizing prop types
+- Organizing components into individual files
+
+and much more.
+
+## Converting Class Components to Functions
+
+Here's how to convert React class components to functional components:
+
+```python
+# Find all React class components
+for class_def in codebase.classes:
+ # Skip if not a React component
+ if not class_def.is_jsx or "Component" not in [base.name for base in class_def.bases]:
+ continue
+
+ print(f"Converting {class_def.name} to functional component")
+
+ # Extract state from constructor
+ constructor = class_def.get_method("constructor")
+ state_properties = []
+ if constructor:
+ for statement in constructor.code_block.statements:
+ if "this.state" in statement.source:
+ # Extract state properties
+ state_properties = [prop.strip() for prop in
+ statement.source.split("{")[1].split("}")[0].split(",")]
+
+ # Create useState hooks for each state property
+ state_hooks = []
+ for prop in state_properties:
+ hook_name = f"[{prop}, set{prop[0].upper()}{prop[1:]}]"
+ state_hooks.append(f"const {hook_name} = useState(null);")
+
+ # Convert lifecycle methods to effects
+ effects = []
+ if class_def.get_method("componentDidMount"):
+ effects.append("""
+ useEffect(() => {
+ // TODO: Move componentDidMount logic here
+ }, []);
+ """)
+
+ if class_def.get_method("componentDidUpdate"):
+ effects.append("""
+ useEffect(() => {
+ // TODO: Move componentDidUpdate logic here
+ });
+ """)
+
+ # Get the render method
+ render_method = class_def.get_method("render")
+
+ # Create the functional component
+ func_component = f"""
+const {class_def.name} = ({class_def.get_method("render").parameters[0].name}) => {{
+ {chr(10).join(state_hooks)}
+ {chr(10).join(effects)}
+
+ {render_method.code_block.source}
+}}
+"""
+
+ # Replace the class with the functional component
+ class_def.edit(func_component)
+
+ # Add required imports
+ file = class_def.file
+ if not any("useState" in imp.source for imp in file.imports):
+ file.add_import_from_import_string("import { useState, useEffect } from 'react';")
+```
+
+## Migrating to Modern Hooks
+
+Convert legacy patterns to modern React hooks:
+
+```python
+# Find components using legacy patterns
+for function in codebase.functions:
+ if not function.is_jsx:
+ continue
+
+ # Look for common legacy patterns
+ for call in function.function_calls:
+ # Convert withRouter to useNavigate
+ if call.name == "withRouter":
+ # Add useNavigate import
+ function.file.add_import_from_import_string(
+ "import { useNavigate } from 'react-router-dom';"
+ )
+ # Add navigate hook
+ function.insert_before_first_return("const navigate = useNavigate();")
+ # Replace history.push calls
+ for history_call in function.function_calls:
+ if "history.push" in history_call.source:
+ history_call.edit(
+ history_call.source.replace("history.push", "navigate")
+ )
+
+ # Convert lifecycle methods in hooks
+ elif call.name == "componentDidMount":
+ call.parent.edit("""
+useEffect(() => {
+ // Your componentDidMount logic here
+}, []);
+""")
+```
+
+## Standardizing Props
+
+### Inferring Props from Usage
+
+Add proper prop types and TypeScript interfaces based on how props are used:
+
+```python
+# Add TypeScript interfaces for props
+for function in codebase.functions:
+ if not function.is_jsx:
+ continue
+
+ # Get props parameter
+ props_param = function.parameters[0] if function.parameters else None
+ if not props_param:
+ continue
+
+ # Collect used props
+ used_props = set()
+ for prop_access in function.function_calls:
+ if f"{props_param.name}." in prop_access.source:
+ prop_name = prop_access.source.split(".")[1]
+ used_props.add(prop_name)
+
+ # Create interface
+ if used_props:
+ interface_def = f"""
+interface {function.name}Props {{
+ {chr(10).join(f' {prop}: any;' for prop in used_props)}
+}}
+"""
+ function.insert_before(interface_def)
+ # Update function signature
+ function.edit(function.source.replace(
+ f"({props_param.name})",
+ f"({props_param.name}: {function.name}Props)"
+ ))
+```
+
+### Extracting Inline Props
+
+Convert inline prop type definitions to separate type declarations:
+
+```python
+# Iterate over all files in the codebase
+for file in codebase.files:
+ # Iterate over all functions in the file
+ for function in file.functions:
+ # Check if the function is a React functional component
+ if function.is_jsx: # Assuming is_jsx indicates a function component
+ # Check if the function has inline props definition
+ if len(function.parameters) == 1 and isinstance(function.parameters[0].type, Dict):
+ # Extract the inline prop type
+ inline_props: TSObjectType = function.parameters[0].type.source
+ # Create a new type definition for the props
+ props_type_name = f"{function.name}Props"
+ props_type_definition = f"type {props_type_name} = {inline_props};"
+
+ # Set the new type for the parameter
+ function.parameters[0].set_type_annotation(props_type_name)
+ # Add the new type definition to the file
+ function.insert_before('\n' + props_type_definition + '\n')
+```
+
+This will convert components from:
+
+```typescript
+function UserCard({ name, age }: { name: string; age: number }) {
+ return (
+
+ {name} ({age})
+
+ );
+}
+```
+
+To:
+
+```typescript
+type UserCardProps = { name: string; age: number };
+
+function UserCard({ name, age }: UserCardProps) {
+ return (
+
+ {name} ({age})
+
+ );
+}
+```
+
+
+ Extracting prop types makes them reusable and easier to maintain. It also
+ improves code readability by separating type definitions from component logic.
+
+
+## Updating Fragment Syntax
+
+Modernize React Fragment syntax:
+
+```python
+for function in codebase.functions:
+ if not function.is_jsx:
+ continue
+
+ # Replace React.Fragment with <>
+ for element in function.jsx_elements:
+ if element.name == "React.Fragment":
+ element.edit(element.source.replace(
+ "",
+ "<>"
+ ).replace(
+ "",
+ ""
+ ))
+```
+
+## Organizing Components into Individual Files
+
+A common modernization task is splitting files with multiple components into a more maintainable structure where each component has its own file. This is especially useful when modernizing legacy React codebases that might have grown organically.
+
+```python
+# Initialize a dictionary to store files and their corresponding JSX components
+files_with_jsx_components = {}
+
+# Iterate through all files in the codebase
+for file in codebase.files:
+ # Check if the file is in the components directory
+ if 'components' not in file.filepath:
+ continue
+
+ # Count the number of JSX components in the file
+ jsx_count = sum(1 for function in file.functions if function.is_jsx)
+
+ # Only proceed if there are multiple JSX components
+ if jsx_count > 1:
+ # Identify non-default exported components
+ non_default_components = [
+ func for func in file.functions
+ if func.is_jsx and not func.is_exported
+ ]
+ default_components = [
+ func for func in file.functions
+ if func.is_jsx and func.is_exported and func.export.is_default_export()
+ ]
+
+ # Log the file path and its components
+ print(f"๐ {file.filepath}:")
+ for component in default_components:
+ print(f" ๐ข {component.name} (default)")
+ for component in non_default_components:
+ print(f" ๐ต {component.name}")
+
+ # Create a new directory path based on the original file's directory
+ new_dir_path = "/".join(file.filepath.split("/")[:-1]) + "/" + file.name.split(".")[0]
+ codebase.create_directory(new_dir_path, exist_ok=True)
+
+ # Create a new file path for the component
+ new_file_path = f"{new_dir_path}/{component.name}.tsx"
+ new_file = codebase.create_file(new_file_path)
+
+ # Log the movement of the component
+ print(f" ๐ซธ Moved to: {new_file_path}")
+
+ # Move the component to the new file
+ component.move_to_file(new_file, strategy="add_back_edge")
+```
+
+This script will:
+
+1. Find files containing multiple React components
+2. Create a new directory structure based on the original file
+3. Move each non-default exported component to its own file
+4. Preserve imports and dependencies automatically
+5. Keep default exports in their original location
+
+For example, given this structure:
+
+```
+components/
+ Forms.tsx # Contains Button, Input, Form (default)
+```
+
+It will create:
+
+```
+components/
+ Forms.tsx # Contains Form (default)
+ forms/
+ Button.tsx
+ Input.tsx
+```
+
+
+ The `strategy="add_back_edge"` parameter ensures that any components that were
+ previously co-located can still import each other without circular
+ dependencies. Learn more about [moving
+ code](/building-with-codegen/moving-symbols) here.
+
+
+
+
+---
+title: "Migrating from unittest to pytest"
+sidebarTitle: "Unittest to Pytest"
+description: "Learn how to migrate unittest test suites to pytest using Codegen"
+icon: "vial"
+iconType: "solid"
+---
+
+Migrating from [unittest](https://docs.python.org/3/library/unittest.html) to [pytest](https://docs.pytest.org/) involves converting test classes and assertions to pytest's more modern and concise style. This guide will walk you through using Codegen to automate this migration.
+
+
+You can find the complete example code in our [examples repository](https://github.com/codegen-sh/codegen-examples/tree/7b978091c3153b687c32928fe10f05425e22f6a5/examples/unittest_to_pytest).
+
+
+## Overview
+
+The migration process involves four main steps:
+
+1. Converting test class inheritance and setup/teardown methods
+2. Updating assertions to pytest style
+3. Converting test discovery patterns
+4. Modernizing fixture usage
+
+Let's walk through each step using Codegen.
+
+## Step 1: Convert Test Classes and Setup Methods
+
+The first step is to convert unittest's class-based tests to pytest's function-based style. This includes:
+
+- Removing `unittest.TestCase` inheritance
+- Converting `setUp` and `tearDown` methods to fixtures
+- Updating class-level setup methods
+
+```python
+# From:
+class TestUsers(unittest.TestCase):
+ def setUp(self):
+ self.db = setup_test_db()
+
+ def tearDown(self):
+ self.db.cleanup()
+
+ def test_create_user(self):
+ user = self.db.create_user("test")
+ self.assertEqual(user.name, "test")
+
+# To:
+import pytest
+
+@pytest.fixture
+def db():
+ db = setup_test_db()
+ yield db
+ db.cleanup()
+
+def test_create_user(db):
+ user = db.create_user("test")
+ assert user.name == "test"
+```
+
+## Step 2: Update Assertions
+
+Next, we'll convert unittest's assertion methods to pytest's plain assert statements:
+
+```python
+# From:
+def test_user_validation(self):
+ self.assertTrue(is_valid_email("user@example.com"))
+ self.assertFalse(is_valid_email("invalid"))
+ self.assertEqual(get_user_count(), 0)
+ self.assertIn("admin", get_roles())
+ self.assertRaises(ValueError, parse_user_id, "invalid")
+
+# To:
+def test_user_validation():
+ assert is_valid_email("user@example.com")
+ assert not is_valid_email("invalid")
+ assert get_user_count() == 0
+ assert "admin" in get_roles()
+ with pytest.raises(ValueError):
+ parse_user_id("invalid")
+```
+
+## Step 3: Update Test Discovery
+
+pytest uses a different test discovery pattern than unittest. We'll update the test file names and patterns:
+
+```python
+# From:
+if __name__ == '__main__':
+ unittest.main()
+
+# To:
+# Remove the unittest.main() block entirely
+# Rename test files to test_*.py or *_test.py
+```
+
+## Step 4: Modernize Fixture Usage
+
+Finally, we'll update how test dependencies are managed using pytest's powerful fixture system:
+
+```python
+# From:
+class TestDatabase(unittest.TestCase):
+ @classmethod
+ def setUpClass(cls):
+ cls.db_conn = create_test_db()
+
+ def setUp(self):
+ self.transaction = self.db_conn.begin()
+
+ def tearDown(self):
+ self.transaction.rollback()
+
+# To:
+@pytest.fixture(scope="session")
+def db_conn():
+ return create_test_db()
+
+@pytest.fixture
+def transaction(db_conn):
+ transaction = db_conn.begin()
+ yield transaction
+ transaction.rollback()
+```
+
+## Common Patterns
+
+Here are some common patterns you'll encounter when migrating to pytest:
+
+1. **Parameterized Tests**
+
+```python
+# From:
+def test_validation(self):
+ test_cases = [("valid@email.com", True), ("invalid", False)]
+ for email, expected in test_cases:
+ with self.subTest(email=email):
+ self.assertEqual(is_valid_email(email), expected)
+
+# To:
+@pytest.mark.parametrize("email,expected", [
+ ("valid@email.com", True),
+ ("invalid", False)
+])
+def test_validation(email, expected):
+ assert is_valid_email(email) == expected
+```
+
+2. **Exception Testing**
+
+```python
+# From:
+def test_exceptions(self):
+ self.assertRaises(ValueError, process_data, None)
+ with self.assertRaises(TypeError):
+ process_data(123)
+
+# To:
+def test_exceptions():
+ with pytest.raises(ValueError):
+ process_data(None)
+ with pytest.raises(TypeError):
+ process_data(123)
+```
+
+3. **Temporary Resources**
+
+```python
+# From:
+def setUp(self):
+ self.temp_dir = tempfile.mkdtemp()
+
+def tearDown(self):
+ shutil.rmtree(self.temp_dir)
+
+# To:
+@pytest.fixture
+def temp_dir():
+ dir = tempfile.mkdtemp()
+ yield dir
+ shutil.rmtree(dir)
+```
+
+## Tips and Notes
+
+1. pytest fixtures are more flexible than unittest's setup/teardown methods:
+
+ - They can be shared across test files
+ - They support different scopes (function, class, module, session)
+ - They can be parameterized
+
+2. pytest's assertion introspection provides better error messages by default:
+
+ ```python
+ # pytest shows a detailed comparison
+ assert result == expected
+ ```
+
+3. You can gradually migrate to pytest:
+
+ - pytest can run unittest-style tests
+ - Convert one test file at a time
+ - Start with assertion style updates before moving to fixtures
+
+4. Consider using pytest's built-in fixtures:
+ - `tmp_path` for temporary directories
+ - `capsys` for capturing stdout/stderr
+ - `monkeypatch` for modifying objects
+ - `caplog` for capturing log messages
+
+
+---
+title: "Migrating from SQLAlchemy 1.6 to 2.0"
+sidebarTitle: "SQLAlchemy 1.6 to 2.0"
+description: "Learn how to migrate SQLAlchemy 1.6 codebases to 2.0 using Codegen"
+icon: "layer-group"
+iconType: "solid"
+---
+
+Migrating from [SQLAlchemy](https://www.sqlalchemy.org/) 1.6 to 2.0 involves several API changes to support the new 2.0-style query interface. This guide will walk you through using Codegen to automate this migration, handling query syntax, session usage, and ORM patterns.
+
+
+You can find the complete example code in our [examples repository](https://github.com/codegen-sh/codegen-examples/tree/7b978091c3153b687c32928fe10f05425e22f6a5/examples/sqlalchemy_1.6_to_2.0).
+
+
+## Overview
+
+The migration process involves three main steps:
+
+1. Converting legacy Query objects to select() statements
+2. Updating session execution patterns
+3. Modernizing ORM relationship declarations
+
+Let's walk through each step using Codegen.
+
+## Step 1: Convert Query to Select
+
+First, we need to convert legacy Query-style operations to the new select() syntax:
+
+```python
+def convert_query_to_select(file):
+ """Convert Query-style operations to select() statements"""
+ for call in file.function_calls:
+ if call.name == "query":
+ # Convert query(Model) to select(Model)
+ call.set_name("select")
+
+ # Update method chains
+ if call.parent and call.parent.is_method_chain:
+ chain = call.parent
+ if "filter" in chain.source:
+ # Convert .filter() to .where()
+ chain.source = chain.source.replace(".filter(", ".where(")
+ if "filter_by" in chain.source:
+ # Convert .filter_by(name='x') to .where(Model.name == 'x')
+ model = call.args[0].value
+ conditions = chain.source.split("filter_by(")[1].split(")")[0]
+ new_conditions = []
+ for cond in conditions.split(","):
+ if "=" in cond:
+ key, value = cond.split("=")
+ new_conditions.append(f"{model}.{key.strip()} == {value.strip()}")
+ chain.edit(f".where({' & '.join(new_conditions)})")
+```
+
+This transforms code from:
+
+```python
+# Legacy Query style
+session.query(User).filter_by(name='john').filter(User.age >= 18).all()
+```
+
+to:
+
+```python
+# New select() style
+session.execute(
+ select(User).where(User.name == 'john').where(User.age >= 18)
+).scalars().all()
+```
+
+
+ SQLAlchemy 2.0 standardizes on select() statements for all queries, providing
+ better type checking and a more consistent API.
+
+
+## Step 2: Update Session Execution
+
+Next, we update how queries are executed with the Session:
+
+```python
+def update_session_execution(file):
+ """Update session execution patterns for 2.0 style"""
+ for call in file.function_calls:
+ if call.name == "query":
+ # Find the full query chain
+ chain = call
+ while chain.parent and chain.parent.is_method_chain:
+ chain = chain.parent
+
+ # Wrap in session.execute() if needed
+ if not chain.parent or "execute" not in chain.parent.source:
+ chain.edit(f"execute(select{chain.source[5:]})")
+
+ # Add .scalars() for single-entity queries
+ if len(call.args) == 1:
+ chain.edit(f"{chain.source}.scalars()")
+```
+
+This converts patterns like:
+
+```python
+# Old style
+users = session.query(User).all()
+first_user = session.query(User).first()
+```
+
+to:
+
+```python
+# New style
+users = session.execute(select(User)).scalars().all()
+first_user = session.execute(select(User)).scalars().first()
+```
+
+
+ The new execution pattern is more explicit about what's being returned, making
+ it easier to understand and maintain type safety.
+
+
+## Step 3: Update ORM Relationships
+
+Finally, we update relationship declarations to use the new style:
+
+```
+
+```
+
+
+---
+title: "Fixing Import Loops"
+description: "Learn how to identify and fix problematic import loops using Codegen."
+icon: "arrows-rotate"
+iconType: "solid"
+---
+
+
+
+
+
+Import loops occur when two or more Python modules depend on each other, creating a circular dependency. While some import cycles can be harmless, others can lead to runtime errors and make code harder to maintain.
+
+In this tutorial, we'll explore how to identify and fix problematic import cycles using Codegen.
+
+
+You can find the complete example code in our [examples repository](https://github.com/codegen-sh/codegen-examples/tree/main/examples/removing_import_loops_in_pytorch).
+
+
+## Overview
+
+The steps to identify and fix import loops are as follows:
+1. Detect import loops
+2. Visualize them
+3. Identify problematic cycles with mixed static/dynamic imports
+4. Fix these cycles using Codegen
+
+# Step 1: Detect Import Loops
+- Create a graph
+- Loop through imports in the codebase and add edges between the import files
+- Find strongly connected components using Networkx (the import loops)
+```python
+G = nx.MultiDiGraph()
+
+# Add all edges to the graph
+for imp in codebase.imports:
+ if imp.from_file and imp.to_file:
+ edge_color = "red" if imp.is_dynamic else "black"
+ edge_label = "dynamic" if imp.is_dynamic else "static"
+
+ # Store the import statement and its metadata
+ G.add_edge(
+ imp.to_file.filepath,
+ imp.from_file.filepath,
+ color=edge_color,
+ label=edge_label,
+ is_dynamic=imp.is_dynamic,
+ import_statement=imp, # Store the whole import object
+ key=id(imp.import_statement),
+ )
+# Find strongly connected components
+cycles = [scc for scc in nx.strongly_connected_components(G) if len(scc) > 1]
+
+print(f"๐ Found {len(cycles)} import cycles:")
+for i, cycle in enumerate(cycles, 1):
+ print(f"\nCycle #{i}:")
+ print(f"Size: {len(cycle)} files")
+
+ # Create subgraph for this cycle to count edges
+ cycle_subgraph = G.subgraph(cycle)
+
+ # Count total edges
+ total_edges = cycle_subgraph.number_of_edges()
+ print(f"Total number of imports in cycle: {total_edges}")
+
+ # Count dynamic and static imports separately
+ dynamic_imports = sum(1 for u, v, data in cycle_subgraph.edges(data=True) if data.get("color") == "red")
+ static_imports = sum(1 for u, v, data in cycle_subgraph.edges(data=True) if data.get("color") == "black")
+
+ print(f"Number of dynamic imports: {dynamic_imports}")
+ print(f"Number of static imports: {static_imports}")
+```
+
+
+## Understanding Import Cycles
+
+Not all import cycles are problematic! Here's an example of a cycle that one may think would cause an error but it does not because due to using dynamic imports.
+
+```python
+# top level import in in APoT_tensor.py
+from quantizer.py import objectA
+```
+
+```python
+# dynamic import in quantizer.py
+def some_func():
+ # dynamic import (evaluated when some_func() is called)
+ from APoT_tensor.py import objectB
+```
+
+
+
+A dynamic import is an import defined inside of a function, method or any executable body of code which delays the import execution until that function, method or body of code is called.
+
+You can use `imp.is_dynamic` to check if the import is dynamic allowing you to investigate imports that are handled more intentionally.
+
+# Step 2: Visualize Import Loops
+- Create a new subgraph to visualize one cycle
+- color and label the edges based on their type (dynamic/static)
+- visualize the cycle graph using `codebase.visualize(graph)`
+
+```python
+cycle = cycles[0]
+
+def create_single_loop_graph(cycle):
+ cycle_graph = nx.MultiDiGraph() # Changed to MultiDiGraph to support multiple edges
+ cycle = list(cycle)
+ for i in range(len(cycle)):
+ for j in range(len(cycle)):
+ # Get all edges between these nodes from original graph
+ edge_data_dict = G.get_edge_data(cycle[i], cycle[j])
+ if edge_data_dict:
+ # For each edge between these nodes
+ for edge_key, edge_data in edge_data_dict.items():
+ # Add edge with all its attributes to cycle graph
+ cycle_graph.add_edge(cycle[i], cycle[j], **edge_data)
+ return cycle_graph
+
+
+cycle_graph = create_single_loop_graph(cycle)
+codebase.visualize(cycle_graph)
+```
+
+
+
+
+
+
+# Step 3: Identify problematic cycles with mixed static & dynamic imports
+
+The import loops that we are really concerned about are those that have mixed static/dynamic imports.
+
+Here's an example of a problematic cycle that we want to fix:
+
+```python
+# In flex_decoding.py
+from .flex_attention import (
+ compute_forward_block_mn,
+ compute_forward_inner,
+ # ... more static imports
+)
+
+# Also in flex_decoding.py
+def create_flex_decoding_kernel(*args, **kwargs):
+ from .flex_attention import set_head_dim_values # dynamic import
+```
+
+It's clear that there is both a top level and a dynamic import that imports from the *same* module. Thus, this can cause issues if not handled carefully.
+
+
+
+Let's find these problematic cycles:
+
+```python
+def find_problematic_import_loops(G, sccs):
+ """Find cycles where files have both static and dynamic imports between them."""
+ problematic_cycles = []
+
+ for i, scc in enumerate(sccs):
+ if i == 2: # skipping the second import loop as it's incredibly long (it's also invalid)
+ continue
+ mixed_import_files = {} # (from_file, to_file) -> {dynamic: count, static: count}
+
+ # Check all file pairs in the cycle
+ for from_file in scc:
+ for to_file in scc:
+ if G.has_edge(from_file, to_file):
+ # Get all edges between these files
+ edges = G.get_edge_data(from_file, to_file)
+
+ # Count imports by type
+ dynamic_count = sum(1 for e in edges.values() if e["color"] == "red")
+ static_count = sum(1 for e in edges.values() if e["color"] == "black")
+
+ # If we have both types between same files, this is problematic
+ if dynamic_count > 0 and static_count > 0:
+ mixed_import_files[(from_file, to_file)] = {"dynamic": dynamic_count, "static": static_count, "edges": edges}
+
+ if mixed_import_files:
+ problematic_cycles.append({"files": scc, "mixed_imports": mixed_import_files, "index": i})
+
+ # Print findings
+ print(f"Found {len(problematic_cycles)} cycles with mixed imports:")
+ for i, cycle in enumerate(problematic_cycles):
+ print(f"\nโ ๏ธ Problematic Cycle #{i + 1}:")
+ print(f"\nโ ๏ธ Index #{cycle['index']}:")
+ print(f"Size: {len(cycle['files'])} files")
+
+ for (from_file, to_file), data in cycle["mixed_imports"].items():
+ print("\n๐ Mixed imports detected:")
+ print(f" From: {from_file}")
+ print(f" To: {to_file}")
+ print(f" Dynamic imports: {data['dynamic']}")
+ print(f" Static imports: {data['static']}")
+
+ return problematic_cycles
+
+problematic_cycles = find_problematic_import_loops(G, cycles)
+```
+
+# Step 4: Fix the loop by moving the shared symbols to a separate `utils.py` file
+One common fix to this problem to break this cycle is to move all the shared symbols to a separate `utils.py` file. We can do this using the method `symbol.move_to_file`:
+
+```python
+# Create new utils file
+utils_file = codebase.create_file("torch/_inductor/kernel/flex_utils.py")
+
+# Get the two files involved in the import cycle
+decoding_file = codebase.get_file("torch/_inductor/kernel/flex_decoding.py")
+attention_file = codebase.get_file("torch/_inductor/kernel/flex_attention.py")
+attention_file_path = "torch/_inductor/kernel/flex_attention.py"
+decoding_file_path = "torch/_inductor/kernel/flex_decoding.py"
+
+# Track symbols to move
+symbols_to_move = set()
+
+# Find imports from flex_attention in flex_decoding
+for imp in decoding_file.imports:
+ if imp.from_file and imp.from_file.filepath == attention_file_path:
+ # Get the actual symbol from flex_attention
+ if imp.imported_symbol:
+ symbols_to_move.add(imp.imported_symbol)
+
+# Move identified symbols to utils file
+for symbol in symbols_to_move:
+ symbol.move_to_file(utils_file)
+
+print(f"๐ Moved {len(symbols_to_move)} symbols to flex_utils.py")
+for symbol in symbols_to_move:
+ print(symbol.name)
+```
+
+```python
+# run this command to have the changes take effect in the codebase
+codebase.commit()
+```
+
+Next Steps
+Verify all tests pass after the migration and fix other problematic import loops using the suggested strategies:
+ 1. Move the shared symbols to a separate file
+ 2. If a module needs imports only for type hints, consider using `if TYPE_CHECKING` from the `typing` module
+ 3. Use lazy imports using `importlib` to load imports dynamically
+
+---
+title: "Migrating from Python 2 to Python 3"
+sidebarTitle: "Python 2 to 3"
+description: "Learn how to migrate Python 2 codebases to Python 3 using Codegen"
+icon: "snake"
+iconType: "solid"
+---
+
+Migrating from Python 2 to Python 3 involves several syntax and API changes. This guide will walk you through using Codegen to automate this migration, handling print statements, string handling, iterators, and more.
+
+
+You can find the complete example code in our [examples repository](https://github.com/codegen-sh/codegen-examples/tree/7b978091c3153b687c32928fe10f05425e22f6a5/examples/python2_to_python3).
+
+
+## Overview
+
+The migration process involves five main steps:
+
+1. Converting print statements to function calls
+2. Updating Unicode to str
+3. Converting raw_input to input
+4. Updating exception handling syntax
+5. Modernizing iterator methods
+
+Let's walk through each step using Codegen.
+
+## Step 1: Convert Print Statements
+
+First, we need to convert Python 2's print statements to Python 3's print function calls:
+
+```python
+def convert_print_statements(file):
+ """Convert Python 2 print statements to Python 3 function calls"""
+ lines = file.content.split('\n')
+ new_content = []
+
+ for line in lines:
+ stripped = line.strip()
+ if stripped.startswith('print '):
+ indent = line[:len(line) - len(line.lstrip())]
+ args = stripped[6:].strip()
+ new_content.append(f"{indent}print({args})")
+ else:
+ new_content.append(line)
+
+ if new_content != lines:
+ file.edit('\n'.join(new_content))
+```
+
+This transforms code from:
+
+```python
+print "Hello, world!"
+print x, y, z
+```
+
+to:
+
+```python
+print("Hello, world!")
+print(x, y, z)
+```
+
+
+ In Python 3, `print` is a function rather than a statement, requiring
+ parentheses around its arguments.
+
+
+## Step 2: Update Unicode to str
+
+Next, we update Unicode-related code to use Python 3's unified string type:
+
+```python
+def update_unicode_to_str(file):
+ """Convert Unicode-related code to str for Python 3"""
+ # Update imports from 'unicode' to 'str'
+ for imp in file.imports:
+ if imp.name == 'unicode':
+ imp.set_name("str")
+
+ # Update function calls from Unicode to str
+ for func_call in file.function_calls:
+ if func_call.name == "unicode":
+ func_call.set_name("str")
+
+ # Check function arguments for Unicode references
+ for arg in func_call.args:
+ if arg.value == "unicode":
+ arg.set_value("str")
+
+ # Find and update Unicode string literals (u"...")
+ for string_literal in file.find('u"'):
+ if string_literal.source.startswith('u"') or string_literal.source.startswith("u'"):
+ new_string = string_literal.source[1:] # Remove the 'u' prefix
+ string_literal.edit(new_string)
+```
+
+This converts code from:
+
+```python
+from __future__ import unicode_literals
+text = unicode("Hello")
+prefix = u"prefix"
+```
+
+to:
+
+```python
+text = str("Hello")
+prefix = "prefix"
+```
+
+
+ Python 3 unifies string types, making the `unicode` type and `u` prefix
+ unnecessary.
+
+
+## Step 3: Convert raw_input to input
+
+Python 3 renames `raw_input()` to `input()`:
+
+```python
+def convert_raw_input(file):
+ """Convert raw_input() calls to input()"""
+ for call in file.function_calls:
+ if call.name == "raw_input":
+ call.edit(f"input{call.source[len('raw_input'):]}")
+```
+
+This updates code from:
+
+```python
+name = raw_input("Enter your name: ")
+```
+
+to:
+
+```python
+name = input("Enter your name: ")
+```
+
+
+ Python 3's `input()` function always returns a string, like Python 2's
+ `raw_input()`.
+
+
+## Step 4: Update Exception Handling
+
+Python 3 changes the syntax for exception handling:
+
+```python
+def update_exception_syntax(file):
+ """Update Python 2 exception handling to Python 3 syntax"""
+ for editable in file.find("except "):
+ if editable.source.lstrip().startswith("except") and ", " in editable.source and " as " not in editable.source:
+ parts = editable.source.split(",", 1)
+ new_source = f"{parts[0]} as{parts[1]}"
+ editable.edit(new_source)
+```
+
+This converts code from:
+
+```python
+try:
+ process_data()
+except ValueError, e:
+ print(e)
+```
+
+to:
+
+```python
+try:
+ process_data()
+except ValueError as e:
+ print(e)
+```
+
+
+ Python 3 uses `as` instead of a comma to name the exception variable.
+
+
+## Step 5: Update Iterator Methods
+
+Finally, we update iterator methods to use Python 3's naming:
+
+```python
+def update_iterators(file):
+ """Update iterator methods from Python 2 to Python 3"""
+ for cls in file.classes:
+ next_method = cls.get_method("next")
+ if next_method:
+ # Create new __next__ method with same content
+ new_method_source = next_method.source.replace("def next", "def __next__")
+ cls.add_source(new_method_source)
+ next_method.remove()
+```
+
+This transforms iterator classes from:
+
+```python
+class MyIterator:
+ def next(self):
+ return self.value
+```
+
+to:
+
+```python
+class MyIterator:
+ def __next__(self):
+ return self.value
+```
+
+
+ Python 3 renames the `next()` method to `__next__()` for consistency with
+ other special methods.
+
+
+## Running the Migration
+
+You can run the complete migration using our example script:
+
+```bash
+git clone https://github.com/codegen-sh/codegen-examples.git
+cd codegen-examples/python2_to_python3
+python run.py
+```
+
+The script will:
+
+1. Process all Python [files](/api-reference/python/PyFile) in your codebase
+2. Apply the transformations in the correct order
+3. Maintain your code's functionality while updating to Python 3 syntax
+
+## Next Steps
+
+After migration, you might want to:
+
+- Add type hints to your code
+- Use f-strings for string formatting
+- Update dependencies to Python 3 versions
+- Run the test suite to verify functionality
+
+Check out these related tutorials:
+
+- [Increase Type Coverage](/tutorials/increase-type-coverage)
+- [Organizing Your Codebase](/tutorials/organize-your-codebase)
+- [Creating Documentation](/tutorials/creating-documentation)
+
+## Learn More
+
+- [Python 3 Documentation](https://docs.python.org/3/)
+- [What's New in Python 3](https://docs.python.org/3/whatsnew/3.0.html)
+- [Codegen API Reference](/api-reference)
+- [Dependencies and Usages](/building-with-codegen/dependencies-and-usages)
+
+
+---
+title: "Migrating from Flask to FastAPI"
+sidebarTitle: "Flask to FastAPI"
+icon: "bolt"
+iconType: "solid"
+---
+
+Migrating from [Flask](https://flask.palletsprojects.com/) to [FastAPI](https://fastapi.tiangolo.com/) involves several key changes to your codebase. This guide will walk you through using Codegen to automate this migration, handling imports, route decorators, static files, and template rendering.
+
+You can find the complete example code in our [examples repository](https://github.com/codegen-sh/codegen-examples/tree/7b978091c3153b687c32928fe10f05425e22f6a5/examples/flask_to_fastapi_migration)
+
+## Overview
+
+The migration process involves four main steps:
+
+1. Updating imports and initialization
+2. Converting route decorators
+3. Setting up static file handling
+4. Updating template handling
+
+Let's walk through each step using Codegen.
+
+## I: Update Imports and Initialization
+
+First, we need to update Flask imports to their FastAPI equivalents and modify the app initialization:
+
+
+ Learn more about [imports here](/building-with-codegen/imports).
+
+
+```python
+from codegen import Codebase
+
+# Parse the codebase
+codebase = Codebase("./")
+
+# Update imports and initialization
+for file in codebase.files:
+ # Update Flask to FastAPI imports
+ for imp in file.imports:
+ if imp.name == "Flask":
+ imp.set_name("FastAPI")
+ elif imp.module == "flask":
+ imp.set_module("fastapi")
+
+ # Update app initialization
+ for call in file.function_calls:
+ if call.name == "Flask":
+ call.set_name("FastAPI")
+ # Remove __name__ argument (not needed in FastAPI)
+ if len(call.args) > 0 and call.args[0].value == "__name__":
+ call.args[0].remove()
+```
+
+This transforms code from:
+
+```python
+from flask import Flask
+app = Flask(__name__)
+```
+
+to:
+
+```python
+from fastapi import FastAPI
+app = FastAPI()
+```
+
+
+ FastAPI doesn't require the `__name__` argument that Flask uses for template
+ resolution. Codegen automatically removes it during migration.
+
+
+## II: Convert Route Decorators
+
+Next, we update Flask's route decorators to FastAPI's operation decorators:
+
+```python
+for function in file.functions:
+ for decorator in function.decorators:
+ if "@app.route" in decorator.source:
+ route = decorator.source.split('"')[1]
+ method = "get" # Default to GET
+ if "methods=" in decorator.source:
+ methods = decorator.source.split("methods=")[1].split("]")[0]
+ if "post" in methods.lower():
+ method = "post"
+ elif "put" in methods.lower():
+ method = "put"
+ elif "delete" in methods.lower():
+ method = "delete"
+ decorator.edit(f'@app.{method}("{route}")')
+```
+
+This converts decorators from Flask style:
+
+```python
+@app.route("/users", methods=["POST"])
+def create_user():
+ pass
+```
+
+to FastAPI style:
+
+```python
+@app.post("/users")
+def create_user():
+ pass
+```
+
+
+ FastAPI provides specific decorators for each HTTP method, making the API more
+ explicit and enabling better type checking and OpenAPI documentation.
+
+
+## III: Setup Static Files
+
+FastAPI handles static files differently than Flask. We need to add the StaticFiles mounting:
+
+```python
+# Add StaticFiles import
+file.add_import_from_import_string("from fastapi.staticfiles import StaticFiles")
+
+# Mount static directory
+file.add_symbol_from_source(
+ 'app.mount("/static", StaticFiles(directory="static"), name="static")'
+)
+```
+
+This sets up static file serving equivalent to Flask's automatic static file handling.
+
+
+ FastAPI requires explicit mounting of static directories, which provides more
+ flexibility in how you serve static files.
+
+
+## IV: Update Template Handling
+
+Finally, we update the template rendering to use FastAPI's Jinja2Templates:
+
+```python
+for func_call in file.function_calls:
+ if func_call.name == "render_template":
+ # Convert to FastAPI's template response
+ func_call.set_name("Jinja2Templates(directory='templates').TemplateResponse")
+ if len(func_call.args) > 1:
+ # Convert template variables to context dict
+ context_arg = ", ".join(
+ f"{arg.name}={arg.value}" for arg in func_call.args[1:]
+ )
+ func_call.set_kwarg("context", f"{'{'}{context_arg}{'}'}")
+ # Add required request parameter
+ func_call.set_kwarg("request", "request")
+```
+
+This transforms template rendering from Flask style:
+
+```python
+@app.get("/users")
+def list_users():
+ return render_template("users.html", users=users)
+```
+
+to FastAPI style:
+
+```python
+@app.get("/users")
+def list_users(request: Request):
+ return Jinja2Templates(directory="templates").TemplateResponse(
+ "users.html",
+ context={"users": users},
+ request=request
+ )
+```
+
+
+ FastAPI requires the `request` object to be passed to templates. Codegen
+ automatically adds this parameter during migration.
+
+
+## Running the Migration
+
+You can run the complete migration using our example script:
+
+```bash
+git clone https://github.com/codegen-sh/codegen-examples.git
+cd codegen-examples/flask_to_fastapi_migration
+python run.py
+```
+
+The script will:
+
+1. Process all Python [files](/api-reference/python/PyFile) in your codebase
+2. Apply the transformations in the correct order
+3. Maintain your code's functionality while updating to FastAPI patterns
+
+## Next Steps
+
+After migration, you might want to:
+
+- Add type hints to your route parameters
+- Set up dependency injection
+- Add request/response models
+- Configure CORS and middleware
+
+Check out these related tutorials:
+
+- [Increase Type Coverage](/tutorials/increase-type-coverage)
+- [Managing TypeScript Exports](/tutorials/managing-typescript-exports)
+- [Organizing Your Codebase](/tutorials/organize-your-codebase)
+
+## Learn More
+
+- [FastAPI Documentation](https://fastapi.tiangolo.com/)
+- [Codegen API Reference](/api-reference)
+- [Moving Symbols Guide](/building-with-codegen/moving-symbols)
+- [Dependencies and Usages](/building-with-codegen/dependencies-and-usages)
+ 2. Open the Command Palette (Cmd/Ctrl + Shift + P)
+ 3. Type "Python: Select Interpreter"
+ 
+ 4. Choose "Enter interpreter path"
+ 5. Navigate to and select:
+ ```bash
+ .codegen/.venv/bin/python
+ ```
+
+ Alternatively, create a `.vscode/settings.json`:
+ ```json
+ {
+ "python.defaultInterpreterPath": "${workspaceFolder}/.codegen/.venv/bin/python",
+ "python.analysis.extraPaths": [
+ "${workspaceFolder}/.codegen/.venv/lib/python3.12/site-packages"
+ ]
+ }
+ ```
+ 
+ Collaborating with Cursor's assistant and the Codegen system prompt
+
+
+In addition, when you [create](/cli/create) a codemod with "-d", Codegen generates an optimized system prompt in `.codegen/codemods/{name}/{name}-system-prompt.txt`. This prompt contains:
+- Relevant Codegen API documentation
+- Examples of relevant transformations
+- Context about your specific task
+
+