|
| 1 | +--- |
| 2 | +title: App-Level Patching |
| 3 | +--- |
| 4 | + |
| 5 | +:::warning Experimental Feature |
| 6 | +App-level patching is an experimental feature that requires setting the `COPA_EXPERIMENTAL=1` environment variable to enable. This feature is under active development, and future releases may introduce breaking changes. Feedback is welcome! |
| 7 | +::: |
| 8 | + |
| 9 | +Copa supports patching application-level dependencies, such as Python packages, in addition to operating system packages. This feature allows you to update vulnerable libraries and packages in various programming language ecosystems. |
| 10 | + |
| 11 | +## Overview |
| 12 | + |
| 13 | +App-level patching works by scanning and updating application dependencies found in your container images. Unlike OS-level patching which updates system packages, app-level patching focuses on: |
| 14 | + |
| 15 | +- Python packages (`pip` is the supported package manager) |
| 16 | +- Node.js packages (`npm` is the supported package manager, for both user applications and globally-installed packages) |
| 17 | + |
| 18 | +Please note that app-level patching requires scanner results that identify vulnerabilities in application libraries. |
| 19 | + |
| 20 | +## Package Type Filtering |
| 21 | + |
| 22 | +Copa supports filtering between different types of packages using the `--pkg-types` flag: |
| 23 | + |
| 24 | +```bash |
| 25 | +# Patch only OS packages (default) |
| 26 | +copa patch -i $IMAGE --pkg-types os ... |
| 27 | + |
| 28 | +# Patch only library/app-level packages |
| 29 | +copa patch -i $IMAGE --pkg-types library ... |
| 30 | + |
| 31 | +# Patch both OS and library packages |
| 32 | +copa patch -i $IMAGE --pkg-types os,library ... |
| 33 | +``` |
| 34 | + |
| 35 | +### Package Type Options |
| 36 | + |
| 37 | +- `os`: Operating system packages (APT, YUM, APK, etc.). This is the default behavior if no `--pkg-types` flag is specified. |
| 38 | +- `library`: Application-level packages (Python, Node.js, etc.) |
| 39 | +- `os,library`: Both types |
| 40 | + |
| 41 | +This filtering is particularly useful when you want to: |
| 42 | + |
| 43 | +- Apply different patch policies to OS vs. application dependencies |
| 44 | +- Separate OS security updates from application updates |
| 45 | +- Reduce the scope of changes in production environments |
| 46 | + |
| 47 | +## Patch Level Control |
| 48 | + |
| 49 | +Copa allows you to control how aggressively application-level packages are updated based on their versioning. This is particularly useful for managing compatibility and stability in your applications. The `--library-patch-level` flag determines the maximum version bump allowed for library updates: |
| 50 | + |
| 51 | +```bash |
| 52 | +# Only apply patch-level updates (e.g., 2.6.0 → 2.6.1) |
| 53 | +copa patch -i $IMAGE --pkg-types library --library-patch-level patch ... |
| 54 | + |
| 55 | +# Allow minor version updates (e.g., 2.6.0 → 2.7.0, prefer 2.6.1) |
| 56 | +copa patch -i $IMAGE --pkg-types library --library-patch-level minor ... |
| 57 | + |
| 58 | +# Allow major version updates (e.g., 2.6.0 → 3.0.0, prefer 2.6.1) |
| 59 | +copa patch -i $IMAGE --pkg-types library --library-patch-level major ... |
| 60 | +``` |
| 61 | + |
| 62 | +Please note that the `--library-patch-level` flag requires the `--pkg-types library` option to be set. Default behavior is `patch` level if not specified. |
| 63 | + |
| 64 | +### Patch Level Behavior |
| 65 | + |
| 66 | +The patch level determines the maximum version bump allowed for library updates: |
| 67 | + |
| 68 | +#### `patch` Level (Recommended) |
| 69 | + |
| 70 | +- **Allows**: `2.6.0` → `2.6.1`, `2.6.2`, etc. |
| 71 | +- **Blocks**: `2.6.0` → `2.7.0` or `3.0.0` |
| 72 | +- **Use case**: Conservative updates, minimal risk of breaking changes |
| 73 | + |
| 74 | +#### `minor` Level |
| 75 | + |
| 76 | +- **Allows**: `2.6.0` → `2.6.1` (preferred) or `2.7.0` |
| 77 | +- **Blocks**: `2.6.0` → `3.0.0` |
| 78 | +- **Preference**: If both `2.6.1` and `2.7.0` are available, it will choose `2.6.1` (patch) over `2.7.0` (minor) |
| 79 | +- **Use case**: Moderate updates, some new features acceptable |
| 80 | + |
| 81 | +#### `major` Level |
| 82 | + |
| 83 | +- **Allows**: Any version update |
| 84 | +- **Preference**: When comma-separated versions are available, prefers Patch > Minor > Major for compatibility. When no comma-separated versions exist, picks the highest version to fix all CVEs. |
| 85 | +- **Example**: If both `2.6.1` and `2.7.0` are available, it will choose `2.6.1` for better compatibility |
| 86 | +- **Use case**: Aggressive updates, all fixes applied regardless of compatibility risk |
| 87 | + |
| 88 | +:::warning |
| 89 | +Please note that `copa` does not guarantee compatibility with all versions. The patch level only controls the maximum version bump allowed. Always test your application after patching. |
| 90 | +::: |
| 91 | + |
| 92 | +### Special Package Handling |
| 93 | + |
| 94 | +Some packages have special handling due to their nature: |
| 95 | + |
| 96 | +- **certifi**: Always updated to the latest version regardless of patch level setting |
| 97 | + |
| 98 | +## Usage Examples |
| 99 | + |
| 100 | +To use app-level patching, you need to have a scanner result file that contains vulnerabilities for application-level packages. Below is an example of how to scan a Python image and then apply patches using Copa. |
| 101 | + |
| 102 | +```bash |
| 103 | +export COPA_EXPERIMENTAL=1 |
| 104 | +export IMAGE=python:3.11.0 |
| 105 | + |
| 106 | +# Scan for Python package vulnerabilities |
| 107 | +trivy image --vuln-type os,library --ignore-unfixed -f json -o python-scan.json $IMAGE |
| 108 | +``` |
| 109 | + |
| 110 | +### Basic Usage |
| 111 | + |
| 112 | +```bash |
| 113 | +# Apply patch-level Python package updates only |
| 114 | +copa patch \ |
| 115 | + -i $IMAGE \ |
| 116 | + -r python-scan.json \ |
| 117 | + --pkg-types os,library \ |
| 118 | + --library-patch-level patch |
| 119 | +``` |
| 120 | + |
| 121 | +### Ignoring Errors |
| 122 | + |
| 123 | +Sometimes, certain packages may not be compatible with the patching process or may not have available updates. In such cases, you can use the `--ignore-errors` flag to allow Copa to continue patching other packages even if some fail. This is useful in environments where you want to apply as many updates as possible without failing the entire patching process. |
| 124 | + |
| 125 | +```bash |
| 126 | +# Apply patch-level Python package updates, ignoring errors |
| 127 | +copa patch \ |
| 128 | + -i $IMAGE \ |
| 129 | + -r python-scan.json \ |
| 130 | + --pkg-types os,library \ |
| 131 | + --library-patch-level major \ |
| 132 | + --ignore-errors |
| 133 | +``` |
| 134 | + |
| 135 | +## Limitations |
| 136 | + |
| 137 | +Due to the nature of app-level patching, it may not be as comprehensive as OS-level patching. Some known limitations are: |
| 138 | + |
| 139 | +### Python |
| 140 | + |
| 141 | +#### Dependency Resolution |
| 142 | + |
| 143 | +Copa does not perform dependency resolution for application-level packages. It applies updates based on the scanner results without checking for compatibility with other packages in the environment. This means that while Copa can update vulnerable packages, it may not resolve all dependency conflicts that arise from those updates. |
| 144 | + |
| 145 | +For example, if a package has a strict version requirement that conflicts with the updated version, you may encounter errors like: |
| 146 | + |
| 147 | +```shell |
| 148 | +#8 8.971 ERROR: Cannot install azure-cli and paramiko==3.4.0 because these package versions have conflicting dependencies. |
| 149 | +#8 8.971 |
| 150 | +#8 8.971 The conflict is caused by: |
| 151 | +#8 8.971 The user requested paramiko==3.4.0 |
| 152 | +#8 8.971 azure-cli-core 2.40.0 depends on paramiko<3.0.0 and >=2.0.8 |
| 153 | +#8 8.971 |
| 154 | +#8 8.971 To fix this you could try to: |
| 155 | +#8 8.971 1. loosen the range of package versions you've specified |
| 156 | +#8 8.971 2. remove package versions to allow pip attempt to solve the dependency conflict |
| 157 | +``` |
| 158 | + |
| 159 | +#### Python Version Compatibility |
| 160 | + |
| 161 | +Copa does not check whether the updated Python packages are compatible with the Python version in the image. For example, if you update a package that requires Python 3.12 to a version that is not compatible with Python 3.11, you may encounter runtime or dependency resolution errors. |
| 162 | + |
| 163 | +#### Testing and Validation |
| 164 | + |
| 165 | +Due to the nature of app-level patching, it is _highly recommended_ to thoroughly test your application after applying updates. Copa does not perform any automated testing or validation of the patched application, so you should ensure that your application functions correctly with the updated dependencies. |
| 166 | + |
| 167 | +#### Non Existent Versions |
| 168 | + |
| 169 | +Trivy provides vulnerability data for Python dependencies using [GitHub Security Advisories](https://github.com/advisories) (GHSA). However, it does not check whether the patched version exists in the [Python Package Index](https://pypi.org) (PyPI). For example, [`GHSA-3749-ghw9-m3mg`](https://github.com/advisories/GHSA-3749-ghw9-m3mg) contains a vulnerability for torch package, but the patched version `2.0.2.7.1-rc1` does not exist in PyPI at the time of this writing. |
| 170 | + |
| 171 | +#### Virtual Environment and Package Manager Support |
| 172 | + |
| 173 | +Currently, only Python packages managed by `pip` are supported. We have not evaluated or implemented support for virtual environments, or other Python package managers like `conda` or `poetry` and others. This might break compatibility with applications that use these package managers. |
| 174 | + |
| 175 | +#### Replacing PyPI |
| 176 | + |
| 177 | +Copa does not support replacing the default [Python Package Index](https://pypi.org) (PyPI) with a custom index or mirror at this time. This means that all package updates are fetched from the official PyPI repository, which may not be suitable for all environments, especially those with strict network policies or private package registries. |
| 178 | + |
| 179 | +### Node.js |
| 180 | + |
| 181 | +Copa supports patching Node.js applications and globally-installed npm packages. When scanning an image for vulnerabilities, Copa will: |
| 182 | + |
| 183 | +1. **User Applications**: Detect and patch packages defined in `package.json` files. |
| 184 | +2. **Global Packages**: Detect and patch globally-installed npm packages (e.g., `eslint`, `typescript`, etc.). |
| 185 | + |
| 186 | +#### Usage Example |
| 187 | + |
| 188 | +```bash |
| 189 | +export COPA_EXPERIMENTAL=1 |
| 190 | +export IMAGE=node:18 |
| 191 | + |
| 192 | +# Scan for Node.js package vulnerabilities |
| 193 | +trivy image --vuln-type os,library --ignore-unfixed -f json -o nodejs-scan.json $IMAGE |
| 194 | + |
| 195 | +# Apply patch-level Node.js package updates |
| 196 | +copa patch \ |
| 197 | + -i $IMAGE \ |
| 198 | + -r nodejs-scan.json \ |
| 199 | + --pkg-types os,library \ |
| 200 | + --library-patch-level patch |
| 201 | +``` |
| 202 | + |
| 203 | +#### Node.js Limitations |
| 204 | + |
| 205 | +##### Node.js Dependency Resolution |
| 206 | + |
| 207 | +Like Python, Copa does not perform full dependency resolution. It applies updates based on scanner results without checking for compatibility conflicts. |
| 208 | + |
| 209 | +More importantly, the `npm overrides` strategy is only capable of patching **transitive dependencies** (dependencies of your dependencies). It **cannot** patch a package that is listed as a **direct dependency** in your application's `package.json`. Attempting to do so will result in an `EOVERRIDE` error from `npm`, and the patch for that package will be skipped. |
| 210 | + |
| 211 | +##### Patching Core Tooling (npm, corepack) |
| 212 | + |
| 213 | +Attempting to patch the `npm` package manager itself is an unsupported edge case. The `npm` project has internal dependencies that are not available on the public registry, which causes the `npm install` process to fail. To ensure stability and prevent hangs, Copa's patching logic is configured to automatically **skip patching `npm` and `corepack`** when they are detected as globally-installed packages. |
| 214 | + |
| 215 | +##### Node.js Package Manager Support |
| 216 | + |
| 217 | +Currently, only `npm` is supported. Other Node.js package managers like `yarn` or `pnpm` are not supported at this time. |
| 218 | + |
| 219 | +##### Incompatible Project Setups |
| 220 | + |
| 221 | +The patching process is designed for standard `npm`-based projects. Images built with other package managers or non-standard project structures will likely fail to patch. Known incompatibilities include: |
| 222 | + |
| 223 | +- **Projects using `yarn` or `pnpm`:** These package managers have different dependency resolution mechanisms and file structures (e.g., `yarn.lock`, `.pnp.cjs`). |
| 224 | +- **Projects using `patch:` protocol:** Some projects apply custom patches to their dependencies using a `patch:` directive. The `npm` version in most containers does not support this protocol, causing an `EUNSUPPORTEDPROTOCOL` error. |
| 225 | +- **Non-standard project structures:** Some frameworks, like Meteor, bundle dependencies in a way that doesn't follow the standard single `package.json` at the project root. This can confuse the application detection logic. |
| 226 | + |
| 227 | +##### Node.js Native Modules |
| 228 | + |
| 229 | +Copa uses the `--ignore-scripts` flag when installing Node.js packages to avoid issues with native module compilation (node-gyp). This means: |
| 230 | + |
| 231 | +- Packages with native dependencies may not build their native components. |
| 232 | +- Most security patches work without native rebuilds. |
| 233 | +- In rare cases, functionality relying on native modules might be affected. |
| 234 | + |
| 235 | +##### Node.js Testing and Validation |
| 236 | + |
| 237 | +As with Python packages, it is *highly recommended* to thoroughly test your Node.js application after applying updates. Copa does not perform automated testing or validation of the patched application. |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?v=4&size=40){kind=avatar}
0 commit comments