|
| 1 | +# AGENTS.md |
| 2 | + |
| 3 | +## Repository Structure |
| 4 | + |
| 5 | +Key components: |
| 6 | + |
| 7 | +- `server` - Core of the OpenSearch server. Includes the distributed system framework as well as most of the search and indexing functionality. |
| 8 | +- `plugins/*` - Optional plugins that extend interfaces defined in `server`. Plugins run in their own classloader that is a child of the `server` classloader. |
| 9 | +- `modules/*` - Architecturally the same as plugins, but are included by default and are not uninstallable. |
| 10 | +- `libs/*` - Libraries that can be used by `server` or any plugin or module. |
| 11 | +- `buildSrc` - The build framework, used by this repository and all external plugin repositories. This is published as the `build-tools` artifact. |
| 12 | +- `sandbox` - Contains `libs`, `modules`, and `plugins` that are under development. These are not included in non-snapshot builds. |
| 13 | +- `distribution` - Builds tar, zip, rpm, and deb packages. |
| 14 | +- `qa` - Integration tests requiring multiple modules/plugins, multi-version cluster tests, and tests in unusual configurations. |
| 15 | +- `test` - Test framework and test fixtures used across the project. Published for external plugin testing. |
| 16 | + |
| 17 | +## Build |
| 18 | + |
| 19 | +JDK 21 is the minimum supported. `JAVA_HOME` must be set. |
| 20 | + |
| 21 | +``` |
| 22 | +./gradlew assemble # build all distributions |
| 23 | +./gradlew localDistro # build for local platform only |
| 24 | +./gradlew run # run OpenSearch from source |
| 25 | +./gradlew generateProto # regenerate protobuf code (if compilation errors) |
| 26 | +``` |
| 27 | + |
| 28 | +## Testing |
| 29 | + |
| 30 | +### Unit Testing |
| 31 | +- Defined in `<component>/src/test` directory |
| 32 | +- Test class names must end with "Tests" |
| 33 | +- Base class: `OpenSearchTestCase` |
| 34 | + |
| 35 | +### Internal Cluster Tests |
| 36 | +- Defined in `<component>/src/internalClusterTest` directory |
| 37 | +- Test class names must end with "IT" |
| 38 | +- Base classes: `OpenSearchSingleNodeTestCase` (single node), `OpenSearchIntegTestCase` (multi-node) |
| 39 | +- These tests are extremely prone to race conditions. Ensure that you are using appropriate concurrency primitives or polling when asserting on a condition that completes asynchronously. |
| 40 | + |
| 41 | +### REST Tests |
| 42 | +- YAML-based REST tests: `./gradlew :rest-api-spec:yamlRestTest` |
| 43 | +- Java REST tests: `./gradlew :<module>:javaRestTest` (base class: `OpenSearchRestTestCase`) |
| 44 | + |
| 45 | +### Running Tests |
| 46 | + |
| 47 | +``` |
| 48 | +./gradlew check # all verification tasks (unit, integration, static checks) |
| 49 | +./gradlew precommit # precommit checks only |
| 50 | +./gradlew internalClusterTest # in-memory cluster integration tests only |
| 51 | +./gradlew test # unit tests only |
| 52 | +
|
| 53 | +# run a specific test |
| 54 | +./gradlew server:test --tests "*.ReplicaShardBatchAllocatorTests.testNoAsyncFetchData" |
| 55 | +./gradlew :server:internalClusterTest --tests "org.opensearch.action.admin.ClientTimeoutIT.testNodesInfoTimeout" |
| 56 | +
|
| 57 | +# run with a specific seed for reproducibility |
| 58 | +./gradlew test -Dtests.seed=DEADBEEF |
| 59 | +
|
| 60 | +# repeat a test N times |
| 61 | +./gradlew server:test --tests "*.ReplicaShardBatchAllocatorTests.testNoAsyncFetchData" -Dtests.iters=N |
| 62 | +``` |
| 63 | + |
| 64 | +### Writing Good Tests |
| 65 | + |
| 66 | +- Prefer unit tests over multi-threaded integration tests when unit tests can provide the same test coverage. |
| 67 | +- Use randomization for parameters not expected to affect behavior (e.g., shard count for aggregation tests), not for coverage. If different code paths exist (e.g., 1 shard vs 2+ shards), write separate tests for each. |
| 68 | +- Never use `Thread.sleep` directly. Use `assertBusy` or `waitUntil` for polling, or instrument code with concurrency primitives like `CountDownLatch` for deterministic waiting. |
| 69 | +- Do not depend on specific segment topology. If needed, disable background refreshes and force merge after indexing. |
| 70 | +- Clean up all resources at the end of each test. The base test class checks for open file handles and running threads after tear down. |
| 71 | +- Do not abuse randomization in multi-threaded tests because it can make failures non-reproducible. |
| 72 | +- When testing functionality behind a feature flag, use `FeatureFlags.TestUtils` and the `@LockFeatureFlag` annotation. |
| 73 | +- Use the `-Dtests.iters=N` parameter to repeat new and modified tests with many different random seeds to ensure stability. |
| 74 | + |
| 75 | +## Java Formatting |
| 76 | + |
| 77 | +- Formatted with Eclipse JDT formatter via Spotless Gradle plugin. |
| 78 | +- Run `./gradlew spotlessJavaCheck` to check, `./gradlew spotlessApply` to fix. |
| 79 | +- 4-space indent, 140-character line width. |
| 80 | +- Wildcard imports are forbidden. |
| 81 | +- Prefer `foo == false` over `!foo` for readability. |
| 82 | + |
| 83 | +## Adding Dependencies |
| 84 | + |
| 85 | +When adding or removing a dependency in any `build.gradle` (non-test scope): |
| 86 | +1. Copy the library's `LICENSE.txt` and `NOTICE.txt` to `<component>/licenses/<artifact>-LICENSE.txt` and `<artifact>-NOTICE.txt`. |
| 87 | +2. Run `./gradlew :<component>:updateSHAs` to generate the SHA file. |
| 88 | +3. Verify with `./gradlew :<component>:check`. |
| 89 | + |
| 90 | +## Backwards Compatibility |
| 91 | + |
| 92 | +- Use `Version.onOrAfter` / `Version.before` checks when changing on-disk formats or encodings. |
| 93 | +- Mark public API classes with `@PublicApi` (backwards compatibility guaranteed), `@InternalApi` (no guarantees), `@ExperimentalApi` (may change any time), or `@DeprecatedApi`. |
| 94 | +- User-facing API changes require the `>breaking` PR label, a CHANGELOG entry, and deprecation log messages via `DeprecationLogger`. |
| 95 | +- Run `./gradlew japicmp` to check API compatibility against the latest release. |
| 96 | + |
| 97 | +## Commits |
| 98 | + |
| 99 | +Ensure `./gradlew precommit` passes before creating a commit. Write commit message titles focused on user impact and not implementation details. |
| 100 | + |
| 101 | +- Good: `Allow slash in snapshot file name validation` |
| 102 | +- Bad: `Add Strings#validFileNameExcludingSlash method` |
| 103 | + |
| 104 | +Commit message titles should be limited to 50 characters, followed by a blank line, and the body should be wrapped at 72 characters. Include all relevant context in commit messages but avoid excess verbosity. Users must understand and accept the Developer Certificate of Origin (DCO) and all commits must be signed off accordingly. |
| 105 | + |
| 106 | +## Pull Requests |
| 107 | + |
| 108 | +Always push to the user's fork. Never push to the upstream `opensearch-project/OpenSearch` repo. Never push directly to `main`. If a user fork does not exist, ask the contributor to create one. |
0 commit comments