diff --git a/docs/codeflash-concepts/how-codeflash-works.mdx b/docs/codeflash-concepts/how-codeflash-works.mdx index f5608e37f..6a71de952 100644 --- a/docs/codeflash-concepts/how-codeflash-works.mdx +++ b/docs/codeflash-concepts/how-codeflash-works.mdx @@ -57,7 +57,7 @@ Codeflash runs tests for the target function using either pytest or unittest fra #### Performance benchmarking -Codeflash implements [several techniques](/codeflash-concepts/benchmarking.md) to measure code performance accurately. In particular, it runs multiple iterations of the code in a loop to determine the best performance with the minimum runtime. Codeflash compares the performance of the original code against the optimization, requiring at least a 10% speed improvement before considering it to be faster. This approach eliminates most runtime measurement variability, even on noisy CI systems and virtual machines. The final runtime Codeflash reports is the minimum total time it took to run all the test cases. +Codeflash implements [several techniques](/codeflash-concepts/benchmarking) to measure code performance accurately. In particular, it runs multiple iterations of the code in a loop to determine the best performance with the minimum runtime. Codeflash compares the performance of the original code against the optimization, requiring at least a 10% speed improvement before considering it to be faster. This approach eliminates most runtime measurement variability, even on noisy CI systems and virtual machines. The final runtime Codeflash reports is the minimum total time it took to run all the test cases. ## Creating Pull Requests diff --git a/docs/docs.json b/docs/docs.json index b3ccf9027..a869e26d2 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -25,25 +25,24 @@ ] }, { - "group": "πŸš€ Getting Started", + "group": "πŸš€ Quickstart", "pages": [ "getting-started/local-installation" ] }, { - "group": "πŸ”§ Continuous Optimization", + "group": "⚑ Optimizing with Codeflash", "pages": [ - "getting-started/codeflash-github-actions", - "optimizing-with-codeflash/optimize-prs", - "optimizing-with-codeflash/benchmarking" + "optimizing-with-codeflash/one-function", + "optimizing-with-codeflash/trace-and-optimize", + "optimizing-with-codeflash/codeflash-all" ] }, { - "group": "⚑ Optimization Workflows", + "group": "✨ Continuous Optimization", "pages": [ - "optimizing-with-codeflash/one-function", - "optimizing-with-codeflash/trace-and-optimize", - "optimizing-with-codeflash/codeflash-all" + "optimizing-with-codeflash/codeflash-github-actions", + "optimizing-with-codeflash/benchmarking" ] }, { @@ -62,21 +61,7 @@ } ] } - ], - "global": { - "anchors": [ - { - "anchor": "GitHub", - "href": "https://github.com/codeflash-ai", - "icon": "github" - }, - { - "anchor": "Discord", - "href": "https://www.codeflash.ai/discord", - "icon": "discord" - } - ] - } + ] }, "logo": { "light": "/images/codeflash_light.svg", @@ -85,8 +70,14 @@ "navbar": { "links": [ { - "label": "Contact", - "href": "mailto:contact@codeflash.ai" + "label": "Discord", + "href": "https://www.codeflash.ai/discord", + "icon": "discord" + }, + { + "label": "GitHub", + "href": "https://github.com/codeflash-ai/codeflash", + "icon": "github" }, { "label": "Blog", @@ -113,7 +104,7 @@ "footer": { "socials": { "discord": "https://www.codeflash.ai/discord", - "github": "https://github.com/codeflash-ai", + "github": "https://github.com/codeflash-ai/codeflash", "linkedin": "https://www.linkedin.com/company/codeflash-ai" }, "links": [ diff --git a/docs/getting-started/local-installation.mdx b/docs/getting-started/local-installation.mdx index 9547512b9..9474bb014 100644 --- a/docs/getting-started/local-installation.mdx +++ b/docs/getting-started/local-installation.mdx @@ -6,8 +6,6 @@ icon: "download" Codeflash is installed and configured on a per-project basis. -You can install Codeflash locally for a project by running the following command in the project's virtual environment: - ### Prerequisites Before installing Codeflash, ensure you have: @@ -15,7 +13,9 @@ Before installing Codeflash, ensure you have: 1. **Python 3.9 or above** installed 2. **A Python project** with a virtual environment 3. **Project dependencies installed** in your virtual environment -4. **Tests** (optional) for your code (Codeflash uses tests to verify optimizations) + +Good to have (optional): +1. **Unit Tests** that Codeflash uses to ensure correctness of the optimizations **Virtual Environment Required** @@ -23,14 +23,15 @@ Before installing Codeflash, ensure you have: Always install Codeflash in your project's virtual environment, not globally. Make sure your virtual environment is activated before proceeding. ```bash -# Example: Activate your virtual environment source venv/bin/activate # On Linux/Mac # or -#venv\Scripts\activate # On Windows +venv\Scripts\activate # On Windows ``` + +You can install Codeflash locally for a project by running the following command in the project's virtual environment: ```bash pip install codeflash ``` @@ -39,37 +40,32 @@ pip install codeflash **Codeflash is a Development Dependency** We recommend installing Codeflash as a development dependency. -It doesn't need to be installed as part of your package requirements. -Codeflash is intended to be used locally and as part of development workflows such as CI. +Codeflash is intended to be used in development workflows locally and as part of CI. +Try to always use the latest version of Codeflash as it improves quickly. -```toml Poetry -[tool.poetry.dependencies.dev] -codeflash = "^latest" -``` - ```bash uv uv add --dev codeflash ``` -```bash pip -pip install --dev codeflash +```bash poetry +poetry add codeflash@latest --group dev ``` -Codeflash uses cloud-hosted AI models to optimize your code. You'll need an API key to use it. +Codeflash uses cloud-hosted AI models and integrations with GitHub. You'll need an API key to authorize your access. -1. Visit the [Codeflash Web App](https://app.codeflash.ai/) +1. Visit the [Codeflash Web App](https://app.codeflash.ai/) 2. Sign up with your GitHub account (free) 3. Navigate to the [API Key](https://app.codeflash.ai/app/apikeys) page to generate your API key **Free Tier Available** -Codeflash offers a **free tier** with a limited number of optimizations per month. Perfect for trying it out or small projects! +Codeflash offers a **free tier** with a limited number of optimizations. Perfect for trying it out on small projects! @@ -77,10 +73,6 @@ Codeflash offers a **free tier** with a limited number of optimizations per mont Navigate to your project's root directory (where your `pyproject.toml` file is or should be) and run: ```bash -# Make sure you're in your project root -cd /path/to/your/project - -# Run the initialization codeflash init ``` @@ -89,30 +81,31 @@ If you don't have a pyproject.toml file yet, the codeflash init command will ask **What's pyproject.toml?** -`pyproject.toml` is a configuration file that is used to specify build tool settings for Python projects. -pyproject.toml is the modern replacement for setup.py and requirements.txt files. -It's the new standard for Python package metadata. +`pyproject.toml` is a configuration file that is used to specify build and tool settings for Python projects. +`pyproject.toml` is the modern replacement for setup.py and requirements.txt files. When running `codeflash init`, you will see the following prompts: ```text -1. Enter your Codeflash API key: -2. Which Python module do you want me to optimize going forward? (e.g. my_module) -3. Where are your tests located? (e.g. tests/) -4. Which test framework do you use? (pytest/unittest) +1. Enter your Codeflash API key: +2. Install the GitHub app. +3. Which Python module do you want me to optimize going forward? (e.g. my_module) +4. Where are your tests located? (e.g. tests/) +5. Which test framework do you use? (pytest/unittest) +6. Install GitHub actions for Continuous optimization? ``` -After you have answered these questions, Codeflash will be configured for your project. -The configuration will be saved in the `pyproject.toml` file in the root directory of your project. -To understand the configuration options, and set more advanced options, see the [Configuration](/configuration) page. +After you have answered these questions, the Codeflash configuration will be saved in the `pyproject.toml` file. +To understand the configuration options, and set more advanced options, see the [Manual Configuration](/configuration) page. ### Step 4: Install the Codeflash GitHub App {/* TODO: Justify to users Why we need the user to install Github App even in local Installation or local optimization? */} -Finally, if you have not done so already, Codeflash will ask you to install the Github App in your repository. The Codeflash GitHub App allows access to your repository to the codeflash-ai bot to open PRs, review code, and provide optimization suggestions. +Finally, if you have not done so already, Codeflash will ask you to install the GitHub App in your repository. +The Codeflash GitHub App allows access to your repository to the codeflash-ai bot to open PRs, review code, and provide optimization suggestions. Please [install the Codeflash GitHub app](https://github.com/apps/codeflash-ai/installations/select_target) by choosing the repository you want to install @@ -128,30 +121,29 @@ Once configured, you can start optimizing your code immediately: # Optimize a specific function codeflash --file path/to/your/file.py --function function_name -# Or optimize locally without creating a PR -codeflash --file path/to/your/file.py --function function_name --no-pr +# Or optimize all functions in your codebase +codeflash --all ``` - -**Pro tip**: Start with a single function to see how Codeflash works before running it on your entire codebase. - - - - Want to see Codeflash in action? Check out our **optimize-me** repository with real examples. + + + Want to see Codeflash in action and don't know what code to optimize? Check out our **optimize-me** repository with code ready to optimize. **What's included:** - Sample Python code with performance issues - Tests for verification - Pre-configured `pyproject.toml` - - Before/after optimization examples in PRs - + + Fork the [optimize-me](https://github.com/codeflash-ai/optimize-me) repo to your GitHub account by clicking "Fork" on the top of the page. This allows Codeflash to open Pull Requests with the optimizations it found on your forked repo. + + ```bash -git clone https://github.com/codeflash-ai/optimize-me.git +git clone https://github.com/your_github_username/optimize-me.git cd optimize-me ``` @@ -159,7 +151,7 @@ cd optimize-me ```bash python -m venv .venv -source .venv/bin/activate # or venv\Scripts\activate on Windows +source .venv/bin/activate pip install -r requirements.txt pip install codeflash ``` @@ -193,7 +185,9 @@ codeflash --all # optimize the entire repo - Use the `--verbose` flag for detailed output: + Do know that not all functions can be optimized as no optimization opportunities may exist for them. This is fine and expected. + + To investigate further, use the `--verbose` flag for detailed output: ```bash codeflash optimize --verbose ``` @@ -203,13 +197,7 @@ codeflash --all # optimize the entire repo - 🚫 Why certain functions were skipped - ⚠️ Detailed error messages - πŸ“Š Performance analysis results - - - **Common Reasons Functions Are Skipped:** - - Function is too simple (less than 3 lines) - - Function has no clear performance bottleneck - - Function contains unsupported constructs - + @@ -223,7 +211,7 @@ codeflash --all # optimize the entire repo pytest --collect-only # Check your pyproject.toml configuration - cat pyproject.toml | grep -A 5 "\[tool.codeflash\]" + cat pyproject.toml | grep -A 8 "\[tool.codeflash\]" ``` @@ -233,5 +221,5 @@ codeflash --all # optimize the entire repo - Learn about [Codeflash Concepts](/codeflash-concepts/how-codeflash-works) - Explore [Optimization workflows](/optimizing-with-codeflash/one-function) -- Set up [GitHub Actions integration](/getting-started/codeflash-github-actions) +- Set up [Pull Request Optimization](/optimizing-with-codeflash/codeflash-github-actions) - Read [configuration options](/configuration) for advanced setups diff --git a/docs/getting-the-best-out-of-codeflash.mdx b/docs/getting-the-best-out-of-codeflash.mdx index 4a98badc2..9945a1532 100644 --- a/docs/getting-the-best-out-of-codeflash.mdx +++ b/docs/getting-the-best-out-of-codeflash.mdx @@ -6,14 +6,11 @@ sidebarTitle: "Best Practices" keywords: ["best practices", "tips", "github actions", "tracer", "optimization", "workflow"] --- -# Getting the best out of Codeflash +Codeflash is a powerful tool; here are our recommendations based on how the Codeflash team and our customers use Codeflash. -Codeflash is a powerful tool; here are our recommendations based on how the Codeflash team uses Codeflash. +### Install the GitHub App and actions workflow -### Install the Github App and actions workflow - -After you install Codeflash on an actively developed project, [installing the GitHub App](getting-started/codeflash-github-actions) and setting up the -GitHub Actions workflow will automatically optimize your code whenever new pull requests are opened. This ensures you get the best version of any changes you make to your code without any extra effort. We find that PRs are also the best time to review these changes, because the code is fresh in your mind. +After you install Codeflash on an actively developed project, [installing the GitHub Actions](optimizing-with-codeflash/codeflash-github-actions) will automatically optimize your code whenever new pull requests are opened. This ensures you get the best version of any changes you make to your code without any extra effort. We find that PRs are also the best time to review these changes, because the code is fresh in your mind. ### Find and optimize entire scripts with the Codeflash Tracer @@ -21,13 +18,18 @@ Find the best results by running [Codeflash Optimize](optimizing-with-codeflash/ This internally runs a profiler, captures inputs to all the functions your script calls, and uses those inputs to create Replay tests and benchmarks. The optimizations you get with this method, show you how much faster your workflow will get plus guarantee that your workflow won't break if you merge in the optimizations. -### ### Find optimizations on your whole codebase with `codeflash --all` If you have a lot of existing code, run [`codeflash --all`](optimizing-with-codeflash/codeflash-all) to discover and fix any slow code in your project. Codeflash will open new pull requests for any optimizations it finds, and you can review and merge them at your own pace. +It is first recommended to trace your tests to achieve higher quality optimizations with this approach + +```bash +codeflash optimize --trace-only -m pytest tests/ ; codeflash --all +``` + ### Review the PRs Codeflash opens diff --git a/docs/images/codeflash_pr_suggestion_1.png b/docs/images/codeflash_pr_suggestion_1.png new file mode 100644 index 000000000..9a44ecb96 Binary files /dev/null and b/docs/images/codeflash_pr_suggestion_1.png differ diff --git a/docs/images/codeflash_social_card.png b/docs/images/codeflash_social_card.png deleted file mode 100644 index ee6139be2..000000000 Binary files a/docs/images/codeflash_social_card.png and /dev/null differ diff --git a/docs/index.mdx b/docs/index.mdx index 432521a93..24d0d1561 100644 --- a/docs/index.mdx +++ b/docs/index.mdx @@ -1,17 +1,17 @@ --- -title: "What is Codeflash?" -description: "AI-powered Python performance optimizer that automatically speeds up your code while verifying correctness" +title: "Codeflash is an AI performance optimizer for Python code" icon: "rocket" sidebarTitle: "Overview" keywords: ["python", "performance", "optimization", "AI", "code analysis", "benchmarking"] --- -Codeflash speeds up Python code by figuring out the best way to rewrite your code while verifying that the behavior of the code is unchanged. +Codeflash speeds up any Python code by figuring out the best way to rewrite it while verifying that the behavior of the code is unchanged, and verifying real speed +gains through performance benchmarking. The optimizations Codeflash finds are generally better algorithms, opportunities to remove wasteful compute, better logic, utilizing caching and utilization of more efficient library methods. Codeflash -does not modify the system architecture of your code, but it tries to find the most efficient implementation of that architecture. +does not modify the system architecture of your code, but it tries to find the most efficient implementation of your current architecture. -### Features +### How to use Codeflash @@ -21,6 +21,13 @@ does not modify the system architecture of your code, but it tries to find the m ``` + + Automatically find optimizations for Pull Requests with GitHub Actions integration. + ```bash + codeflash init-actions + ``` + + End-to-end optimization of entire Python workflows with execution tracing. ```bash @@ -35,12 +42,7 @@ does not modify the system architecture of your code, but it tries to find the m ``` - - Automatically find optimization code changes in Pull Requests with GitHub Actions integration. - ```bash - codeflash init-actions - ``` - + ### How does Codeflash verify correctness? diff --git a/docs/optimizing-with-codeflash/benchmarking.mdx b/docs/optimizing-with-codeflash/benchmarking.mdx index d0dd464f8..b58cd2239 100644 --- a/docs/optimizing-with-codeflash/benchmarking.mdx +++ b/docs/optimizing-with-codeflash/benchmarking.mdx @@ -1,21 +1,21 @@ --- -title: "Using Benchmarks in CI" +title: "Optimize Performance Benchmarks with every Pull Request" description: "Configure and use pytest-benchmark integration for performance-critical code optimization" icon: "chart-line" -sidebarTitle: "CI Benchmarks" +sidebarTitle: Setup Benchmarks to Optimize keywords: ["benchmarks", "CI", "pytest-benchmark", "performance testing", "github actions", "benchmark mode"] --- -**Performance-critical optimization** - Define benchmarks for your most important functions and let Codeflash measure the real-world impact of every optimization on your performance metrics. +**Performance-critical optimization** - Define benchmarks for your most important code sections and let Codeflash optimize and measure the real-world impact of every optimization on your performance metrics. -Benchmark mode is an easy way for users to define workflows that are performance-critical and need to be optimized. -For example, if a user has an important function that requires minimal latency, the user can define a benchmark for that function. -Codeflash will then calculate the impact (if any) of any optimization on the performance of that function. +Benchmark mode is an easy way to define workflows that are performance-critical and need to be optimized and run fast. +Codeflash will run the benchmark, understand how the current code change in the Pull Request is affecting the benchmark. +It will then try to optimize the new code for the benchmark and calculate the impact of any optimization on the speed of that benchmark. ## Using Codeflash in Benchmark Mode -1. **Create a benchmarks root** +1. **Create a benchmarks root:** Create a directory for benchmarks if it does not already exist. @@ -31,7 +31,7 @@ Codeflash will then calculate the impact (if any) of any optimization on the per formatter-cmds = ["disabled"] ``` -2. **Define your benchmarks** +2. **Define your benchmarks:** Currently, Codeflash only supports benchmarks written as pytest-benchmarks. Check out the [pytest-benchmark](https://pytest-benchmark.readthedocs.io/en/stable/index.html) documentation for more information on syntax. @@ -50,7 +50,7 @@ Codeflash will then calculate the impact (if any) of any optimization on the per The pytest-benchmark format is simply used as an interface. The plugin is actually not used - Codeflash will run these benchmarks with its own pytest plugin -3. **Run Codeflash** +3. **Run and Test Codeflash:** Run Codeflash with the `--benchmark` flag. Note that benchmark mode cannot be used with `--all`. @@ -65,13 +65,15 @@ Codeflash will then calculate the impact (if any) of any optimization on the per ``` -4. **Run Codeflash in CI** +4. **Run Codeflash :** - Benchmark mode is best used together with Codeflash as a Github Action. This way, with every PR, you will know the impact of Codeflash's optimizations on your benchmarks. + Benchmark mode is best used together with Codeflash as a GitHub Action. This way, + Codeflash will trace through your benchmark and optimize the functions modified in your Pull Request to speed up the benchmark. + It will also report the impact of Codeflash's optimizations on your benchmarks. - Use `codeflash init` for an easy way to set up Codeflash as a Github Action (with the option to enable benchmark mode). + Use `codeflash init` for an easy way to set up Codeflash as a GitHub Action. - Otherwise, you can run the following command in your Codeflash GitHub Action: + After that, you can add the `--benchmark` argument to codeflash to enable benchmarks optimization. ```bash codeflash --benchmark @@ -84,7 +86,7 @@ Codeflash will then calculate the impact (if any) of any optimization on the per 1. Codeflash identifies benchmarks in the benchmarks-root directory. -2. The benchmarks are run so that runtime statistics and information can be recorded. +2. The benchmarks are run so that runtime statistics and inputs can be recorded. 3. Replay tests are generated so the performance of optimization candidates on the exact inputs used in the benchmarks can be measured. @@ -97,5 +99,3 @@ Codeflash will then calculate the impact (if any) of any optimization on the per Using Codeflash with benchmarks is a great way to find optimizations that really matter. - -Codeflash is actively working on this feature and will be adding new capabilities in the near future! \ No newline at end of file diff --git a/docs/optimizing-with-codeflash/codeflash-all.mdx b/docs/optimizing-with-codeflash/codeflash-all.mdx index eddaf41a6..92a232e67 100644 --- a/docs/optimizing-with-codeflash/codeflash-all.mdx +++ b/docs/optimizing-with-codeflash/codeflash-all.mdx @@ -1,8 +1,8 @@ --- title: "Optimize Your Entire Codebase" -description: "Automatically optimize all functions in your project with Codeflash's comprehensive analysis" +description: "Automatically optimize all codepaths in your project with Codeflash's comprehensive analysis" icon: "database" -sidebarTitle: "Entire Codebase" +sidebarTitle: "Optimize Entire Codebase" keywords: ["codebase optimization", "all functions", "batch optimization", "github app", "checkpoint", "recovery"] --- @@ -19,13 +19,34 @@ codeflash --all This requires the Codeflash GitHub App to be installed in your repository. -This is a powerful feature that can help you optimize your entire codebase in one go. +This is a powerful feature that can help you optimize your entire codebase in one go. It also discovers and runs any unit tests covering the function under optimization. + Since it runs on all the functions in your codebase, it can take some time to complete, please be patient. As this runs you will see Codeflash opening pull requests for each function it successfully optimizes. +If you only want to optimize a subdirectory you can run: +```bash +codeflash --all path/to/dir +``` + + + If your project has a good number of unit tests, we can trace those to achieve higher quality results. + The following approach is recommended instead: + ```bash + codeflash optimize --trace-only -m pytest tests/ ; codeflash --all + ``` + This will run your test suite, trace all the code covered by your tests, ensuring higher correctness guarantees + and better performance benchmarking, and help create optimizations for code where the LLMs struggle to generate and run tests. + + Even though `codeflash --all` discovers any existing unit tests. It currently can only discover any test that directly calls the + function under optimization. Tracing all the tests helps ensure correctness for code that may be indirectly called by your tests. + + ## Important considerations - **Dedicated Optimization Machine:** Optimizing the entire codebase may require considerable timeβ€”up to one day. It's recommended to allocate a dedicated machine specifically for this long-running optimization task. - **Minimize Background Processes:** To achieve optimal results, avoid running other processes on the optimization machine. Additional processes can introduce noise into Codeflash's runtime measurements, reducing the quality of the optimizations. Although Codeflash tolerates some runtime fluctuations, minimizing noise ensures the highest optimization quality. - **Checkpoint and Recovery:** Codeflash automatically creates checkpoints as it identifies optimizations. If the optimization process is interrupted or encounters issues, you can resume the process by re-running `codeflash --all`. The command will prompt you to continue from the most recent checkpoint. + + diff --git a/docs/getting-started/codeflash-github-actions.mdx b/docs/optimizing-with-codeflash/codeflash-github-actions.mdx similarity index 68% rename from docs/getting-started/codeflash-github-actions.mdx rename to docs/optimizing-with-codeflash/codeflash-github-actions.mdx index 37ec7b147..280cc5987 100644 --- a/docs/getting-started/codeflash-github-actions.mdx +++ b/docs/optimizing-with-codeflash/codeflash-github-actions.mdx @@ -1,21 +1,25 @@ --- -title: "GitHub Actions Integration" -description: "Automatically optimize pull requests with Codeflash GitHub Actions workflow" +title: "Auto Optimize Pull Requests" +description: "Automatically optimize new code in pull requests with Codeflash GitHub Actions workflow" icon: "github" --- -{/* TODO: Add more pictures to guide better */} -Codeflash can automatically optimize your code when new pull requests are opened. +Optimizing new code in Pull Requests is the best way to ensure that all code you and your team ship is performant +in the future. Automating optimization in the Pull Request stage how most teams use Codeflash, to +continuously find optimizations for their new code. -To be able to scan new code for performance optimizations, Codeflash requires a GitHub action workflow to -be installed which runs the Codeflash optimization logic on every new pull request. +To scan new code for performance optimizations, Codeflash uses a GitHub Action workflow which runs +the Codeflash optimization logic on the new code in every pull request. If the action workflow finds an optimization, it communicates with the Codeflash GitHub -App through our secure servers and asks it to suggest new changes to the pull request. +App and asks it to suggest new changes to the pull request. This is the most useful way of using Codeflash, where you set it up once and all your new code gets optimized. So setting this up is highly recommended. +## Pull Request Optimization 30 seconds demo + + ## Prerequisites @@ -38,18 +42,18 @@ So setting this up is highly recommended. codeflash init-actions ``` This command will automatically create the GitHub Actions workflow file and guide you through the setup process. + +Alternatively running `codeflash init` also asks to setup the github actions. - + Open a new pull request to your GitHub project. You'll see: - βœ… A new Codeflash workflow running in GitHub Actions - πŸ€– The codeflash-ai bot commenting with optimization suggestions (if any are found) + +Ensure that your Python environment installation works correctly and codeflash is able to run. - - -**Recommended approach** - This automated setup ensures you get the latest workflow configuration with all best practices included. - @@ -153,4 +157,16 @@ Customize the dependency installation based on your Python package manager: +## How the Pull Request Optimization Suggestion looks + +Codeflash creates a new dependent Pull Request for you to review with the reported speedups, helpful explanation for the optimization +and the proof of correctness. The pull request has the code change for you to review and accept. + +![Codeflash PR Review](/images/codeflash_pr_suggestion_1.png) + + +Sometimes it also makes an inline suggestion with the optimization. + +![Codeflash PR Suggestion](/images/code-suggestion.png) +We hope you enjoy the performance unlock the Pull Request optimization enables. \ No newline at end of file diff --git a/docs/optimizing-with-codeflash/one-function.mdx b/docs/optimizing-with-codeflash/one-function.mdx index 433e2e4f1..7aa84f6ba 100644 --- a/docs/optimizing-with-codeflash/one-function.mdx +++ b/docs/optimizing-with-codeflash/one-function.mdx @@ -2,7 +2,7 @@ title: "Optimize a Single Function" description: "Target and optimize individual Python functions for maximum performance gains" icon: "bullseye" -sidebarTitle: "Single Function" +sidebarTitle: "Optimize Single Function" keywords: ["function optimization", "single function", "class methods", "performance", "targeted optimization"] --- diff --git a/docs/optimizing-with-codeflash/optimize-prs.mdx b/docs/optimizing-with-codeflash/optimize-prs.mdx deleted file mode 100644 index fcd1ee354..000000000 --- a/docs/optimizing-with-codeflash/optimize-prs.mdx +++ /dev/null @@ -1,33 +0,0 @@ ---- -title: "Optimize Pull Requests" -description: "Automatically optimize code changes in pull requests with GitHub Actions integration" -icon: "code-merge" -sidebarTitle: "PR Optimization" -keywords: ["pull requests", "github actions", "code review", "automated optimization", "dependent PR", "suggestions"] ---- - - -**Continuous optimization** - After initial setup, Codeflash will automatically review every new pull request and suggest performance improvements through comments and dependent PRs. - - -## How to optimize a pull request -After following the setup steps in the [Automate Code Optimization with GitHub Actions](/getting-started/codeflash-github-actions) guide, -Codeflash will automatically optimize your pull requests when they are opened. - -If Codeflash finds any successful optimizations, it will comment on the pull request asking you to review the changes. - -![Codeflash PR Comment](/images/review-comment.png) - -Codeflash can ask you to review the changes in two ways: -### Opening a dependent pull request -Codeflash will open a new pull request with the optimized code. -You can review the changes in this pull request, make changes if you want, and merge it if you are satisfied with the optimizations. -The changes will be merged back into the original pull request as a new commit. - -![Codeflash PR Review](/images/dependent-pr.png) -### Reviewing the changes in the original pull request -If the suggested changes are small and only affect the modified lines, Codeflash will suggest the changes in the original pull request itself. -You can choose to accept or reject the changes directly in the original pull request. -The changes can be added to a batch of changes in the original pull request as a new commit. - -![Codeflash PR Review](/images/code-suggestion.png) diff --git a/docs/optimizing-with-codeflash/trace-and-optimize.mdx b/docs/optimizing-with-codeflash/trace-and-optimize.mdx index 9452b0b59..d57ec319c 100644 --- a/docs/optimizing-with-codeflash/trace-and-optimize.mdx +++ b/docs/optimizing-with-codeflash/trace-and-optimize.mdx @@ -1,26 +1,28 @@ --- -title: "Trace & Optimize Workflows" +title: "Trace & Optimize E2E Workflows" description: "End-to-end optimization of entire Python workflows with execution tracing" icon: "route" -sidebarTitle: "Trace & Optimize" +sidebarTitle: "Optimize E2E Workflows" keywords: ["tracing", "workflow optimization", "replay tests", "end-to-end", "script optimization", "context manager"] --- -Codeflash supports optimizing an entire Python script end-to-end by tracing the script's execution and generating Replay Tests. Tracing follows the execution of a script, profiles it and captures inputs to all called functions, allowing them to be replayed during optimization. Codeflash uses these Replay Tests to optimize all functions called in the script, starting from the most important ones. +Codeflash supports optimizing an entire Python script end-to-end by tracing the script's execution and generating Replay Tests. +Tracing follows the execution of a script, profiles it and captures inputs to all functions it called, allowing them to be replayed during optimization. +Codeflash uses these Replay Tests to optimize all functions called in the script, starting from the most important ones. -To optimize a script, `python myscript.py`, replace `python` with `codeflash optimize` and run the following command: +To optimize a script, `python myscript.py`, simply replace `python` with `codeflash optimize` and run the following command: ```bash codeflash optimize myscript.py ``` -To optimize code called by pytest tests that you could normally run like `python -m pytest tests/`, use this command: +You can also optimize code called by pytest tests that you could normally run like `python -m pytest tests/`, this provides for a good workload to optimize. Run this command: ```bash codeflash optimize -m pytest tests/ ``` -This powerful command creates high-quality optimizations, making it ideal when you need to optimize a workflow or script. The initial tracing process can be slow, so try to limit your script's runtime to under 1 minute for best results. If your workflow is longer, consider tracing it into smaller sections by using the Codeflash tracer as a context manager (point 3 below). +The powerful `codeflash optimize` command creates high-quality optimizations, making it ideal when you need to optimize a workflow or script. The initial tracing process can be slow, so try to limit your script's runtime to under 1 minute for best results. If your workflow is longer, consider tracing it into smaller sections by using the Codeflash tracer as a context manager (point 3 below). The generated replay tests and the trace file are for the immediate optimization use, don't add them to git. @@ -32,10 +34,11 @@ The generated replay tests and the trace file are for the immediate optimization ## What is the codeflash optimize command? `codeflash optimize` tries to do everything that an expert engineer would do while optimizing a workflow. It profiles your code, traces the execution of your workflow and generates a set of test cases that are derived from how your code is actually run. -Codeflash Tracer works by recording the inputs of your functions as they are called in your codebase. These inputs are then used to generate test cases that are representative of the real-world usage of your functions. +Codeflash Tracer works by recording the inputs of your functions as they are called in your codebase, and generating +regression tests with those inputs. We call these generated test cases "Replay Tests" because they replay the inputs that were recorded during the tracing phase. +These replay tests are representative of the real-world usage of your functions. -Then, Codeflash Optimizer can use these replay tests to verify correctness and calculate accurate performance gains for the optimized functions. Using Replay Tests, Codeflash can verify that the optimized functions produce the same output as the original function and also measure the performance gains of the optimized function on the real-world inputs. This way you can be *sure* that the optimized function causes no changes of behavior for the traced workflow and also, that it is faster than the original function. To get more confidence on the correctness of the code, we also generate several LLM generated test cases and discover any existing unit cases you may have. @@ -57,15 +60,16 @@ Codeflash script optimizer can be used in three ways: codeflash optimize path/to/your/file.py --your_options ``` - The above command should suffice in most situations. You can add a argument like `codeflash optimize -o trace_file_path.trace` if you want to customize the trace file location. Otherwise, it defaults to `codeflash.trace` in the current working directory. + The above command should suffice in most situations. + To customize the trace file location you can specify it like `codeflash optimize -o trace_file_path.trace`. Otherwise, it defaults to `codeflash.trace` in the current working directory. 2. **Trace and optimize as two separate steps** If you want more control over the tracing and optimization process. You can trace first and then optimize with the replay tests later. Each replay test is associated with a trace file. - To first create just the trace file, run + To create just the trace file first, run - ```python + ```bash codeflash optimize -o trace_file.trace --trace-only path/to/your/file.py --your_options ``` @@ -79,7 +83,7 @@ Codeflash script optimizer can be used in three ways: - `--tracer-timeout`: The maximum time in seconds to trace the entire workflow. Default is indefinite. This is useful while tracing really long workflows. 3. **As a Context Manager -** - To trace only very specific sections of your codeflash, You can also use the Codeflash Tracer as a context manager. + To trace only specific sections of your code, You can also use the Codeflash Tracer as a context manager. You can wrap the code you want to trace in a `with` statement as follows - ```python @@ -89,7 +93,7 @@ Codeflash script optimizer can be used in three ways: model.predict() # Your code here ``` - This is much faster than tracing the whole script. Sometimes, if tracing the whole script fails, then the Context Manager can also be used to trace the code sections. + This is much faster than tracing the whole script. It can also help if tracing the whole script fails. After this finishes, you can optimize using the generated replay tests. @@ -97,7 +101,7 @@ Codeflash script optimizer can be used in three ways: codeflash --replay-test /path/to/test_replay_test_0.py ``` - More Options for the Tracer: + More Options for the Tracer Context Manager: - `disable`: If set to `True`, the tracer will not trace the code. Default is `False`. - `max_function_count`: The maximum number of times to trace a single function. More calls to a function will not be traced. Default is 100.