diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 24b7a27b0..a66169b62 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,60 +1,136 @@ -# Typical development workflow +# Contributing -Fork the repo, work on an issue +This guide explains how to set up your environment, make changes, and submit a PR. -## Updating the generated Kibana client. +## Development Setup -If your work involves the Kibana API, the endpoints may or may not be included in the generated client. -Check [generated/kbapi](./generated/kbapi/) for more details. +* Fork and clone the repo. +* Setup your preferred IDE (IntelliJ, VSCode, etc.) -## Acceptance tests +Requirements: +* [Terraform](https://www.terraform.io/downloads.html) >= 1.0.0 +* [Go](https://golang.org/doc/install) >= 1.25 +* Docker (for acceptance tests) -```bash -make docker-testacc -``` +## Development Workflow -Run a single test with terraform debug enabled: -```bash -env TF_LOG=DEBUG make docker-testacc TESTARGS='-run ^TestAccResourceDataStreamLifecycle$$' -``` +* Create a new branch for your changes. +* Make your changes. See [Useful Commands](#useful-commands) and [Debugging](#running--debugging-the-provider). +* Validate your changes + * Run unit and acceptance tests (See [Running Acceptance Tests](#running-acceptance-tests)). + * Run `make lint` to check linting and formatting. For this check to succeed, all changes must have been committed. + * All checks also run automatically on every PR. +* Submit your PR for review. +* Add a changelog entry in `CHANGELOG.md` under the `Unreleased` section. This will be included in the release notes of the next release. The changelog entry references the PR, so it has to be added after the PR has been opened. -A way to forward debug logs to a file: -```bash -env TF_ACC_LOG_PATH=/tmp/tf.log TF_ACC_LOG=DEBUG TF_LOG=DEBUG make docker-testacc -``` +When creating new resources: +* Use the [Plugin Framework](https://developer.hashicorp.com/terraform/plugin/framework/getting-started/code-walkthrough) for new resources. + * Use an existing resource (e.g. `internal/elasticsearch/security/system_user`) as a template. + * Some resources use the deprecated Terraform SDK, so only resources using the new Terraform Framework should be used as reference. +* Use the generated API clients to interact with the Kibana APIs. (See [Working with Generated API Clients](#working-with-generated-api-clients) +* Add a documentation template and examples for the resource. See [Updating Documentation](#updating-documentation) for more details. +* Write unit and acceptance tests. +### Useful Commands -## Update documentation +* `make build`: Build the provider. +* `make lint`: Lints and formats the code. +* `make test`: Run unit tests. +* `make docs-generate`: Generate documentation. +* [Running & Debugging the Provider](#running--debugging-the-provider) +* [Running Acceptance Tests](#running-acceptance-tests) -Update documentation templates in `./templates` directory and re-generate docs via: -```bash -make docs-generate -``` +### Running & Debugging the Provider -## Update `./CHANGELOG.md` +You can run the currently checked-out code for local testing and use it with Terraform. -List of previous commits is a good example of what should be included in the changelog. +Also see [Terraform docs on debugging](https://developer.hashicorp.com/terraform/plugin/debugging#starting-a-provider-in-debug-mode). +Run the provider in debug mode and reattach the provider in Terraform: +* Launch `main.go` with the `-debug` flag from your IDE. + * Or launch it with `go run main.go -debug` from the command line. +* After launching, the provider will print an env var. Copy the printed `TF_REATTACH_PROVIDERS='{…}'` value. +* Export it in your shell where you run Terraform: `export TF_REATTACH_PROVIDERS='{…}'`. +* Terraform will now talk to your debug instance, and you can set breakpoints. -## Pull request +### Running Acceptance Tests -Format the code before pushing: -```bash -make fmt -``` +Acceptance tests spin up Elasticsearch, Kibana, and Fleet with Docker and run tests in a Go container. -Check if the linting: ```bash -make lint -``` +# Start Elasticsearch, Kibana, and Fleet +make docker-fleet -Create a PR and check acceptance test matrix is green. +# Run all tests +make testacc -## Run provider with local terraform +# Run a specific test +make testacc TESTARGS='-run ^TestAccResourceDataStreamLifecycle$$' -TBD +# Cleanup created docker containers +make docker-clean +``` -## Releasing +### Working with Generated API Clients + +If your work involves the Kibana API, the API client can be generated directly from the Kibana OpenAPI specs: +- For Kibana APIs, use the generated client in `generated/kbapi`. +- To add new endpoints, see [generated/kbapi/README.md](generated/kbapi/README.md). +- Regenerate clients with: + ```sh + make transform generate + ``` + +The codebase includes a number of deprecated clients which should not be used anymore: +- `libs/go-kibana-rest`: Fork of an external library, which is not maintained anymore. +- `generated/alerting`, `generated/connectors`, `generated/slo`: Older generated clients, but based on non-standard specs. If any of these APIs are needed, they should be included in the `kbapi` client. + +### Updating Documentation + +Docs are generated from templates in `templates/` and examples in `examples/`. +* Update or add templates and examples. +* Run `make docs-generate` to produce files under `docs/`. +* Commit the generated files. `make lint` will fail if docs are stale. + +## Project Structure + +A quick overview over what's in each folder: + +* `docs/` - Documentation files + * `data-sources/` - Documentation for Terraform data sources + * `guides/` - User guides and tutorials + * `resources/` - Documentation for Terraform resources +* `examples/` - Example Terraform configurations + * `cloud/` - Examples using the cloud to launch testing stacks + * `data-sources/` - Data source usage examples + * `resources/` - Resource usage examples + * `provider/` - Provider configuration examples +* `generated/` - Auto-generated clients from the `generate-clients` make target + * `kbapi/` - Kibana API client + * `alerting/` - (Deprecated) Kibana alerting API client + * `connectors/` - (Deprecated) Kibana connectors API client + * `slo/` - (Deprecated) SLO (Service Level Objective) API client +* `internal/` - Internal Go packages + * `acctest/` - Acceptance test utilities + * `clients/` - API client implementations + * `elasticsearch/` - Elasticsearch-specific logic + * `fleet/` - Fleet management functionality + * `kibana/` - Kibana-specific logic + * `models/` - Data models and structures + * `schema/` - Connection schema definitions for plugin framework + * `utils/` - Utility functions + * `versionutils/` - Version handling utilities +* `libs/` - External libraries + * `go-kibana-rest/` - (Deprecated) Kibana REST API client library +* `provider/` - Core Terraform provider implementation +* `scripts/` - Utility scripts for development and CI +* `templates/` - Template files for documentation generation + * `data-sources/` - Data source documentation templates + * `resources/` - Resource documentation templates + * `guides/` - Guide documentation templates +* `xpprovider/` - Additional provider functionality needed for Crossplane + +## Releasing (maintainers) Releasing is implemented in CI pipeline. @@ -65,4 +141,4 @@ To release a new provider version: - updates CHANGELOG.md with the list of changes being released. [Example](https://github.com/elastic/terraform-provider-elasticstack/commit/be866ebc918184e843dc1dd2f6e2e1b963da386d). -* Once the PR is merged, the release CI pipeline can be started by pushing a new release tag to the `main` branch. +* Once the PR is merged, the release CI pipeline can be started by pushing a new release tag to the `main` branch. (`git tag v0.11.13 && git push origin v0.11.13`) diff --git a/README.md b/README.md index 9adefc90d..1de54e1c0 100644 --- a/README.md +++ b/README.md @@ -76,64 +76,9 @@ provider "elasticstack" { } ``` - ## Developing the Provider -If you wish to work on the provider, you'll first need [Go](http://www.golang.org) installed on your machine (see [Requirements](#requirements)). - -To compile the provider, run `go install`. This will build the provider and put the provider binary in the `$GOPATH/bin` directory. - -To install the provider locally into the `~/.terraform.d/plugins/...` directory one can use `make install` command. This will allow to refer this provider directly in the Terraform configuration without needing to download it from the registry. - -To generate or update documentation, run `make gen`. All the generated docs will have to be committed to the repository as well. - -In order to run the full suite of Acceptance tests, run `make testacc`. - -If you have [Docker](https://docs.docker.com/get-docker/) installed, you can use following command to start the Elasticsearch container and run Acceptance tests against it: - -```sh -$ make docker-testacc -``` - -To clean up the used containers and to free up the assigned container names, run `make docker-clean`. - -Note: there have been some issues encountered when using `tfenv` for local development. It's recommended you move your version management for terraform to `asdf` instead. - - -### Requirements - -- [Terraform](https://www.terraform.io/downloads.html) >= 1.0.0 -- [Go](https://golang.org/doc/install) >= 1.19 - - -### Building The Provider - -1. Clone the repository -1. Enter the repository directory -1. Build the provider using the `make install` command: -```sh -$ make install -``` - - -### Adding Dependencies - -This provider uses [Go modules](https://github.com/golang/go/wiki/Modules). -Please see the Go documentation for the most up to date information about using Go modules. - -To add a new dependency `github.com/author/dependency` to your Terraform provider: - -``` -go get github.com/author/dependency -go mod tidy -``` - -Then commit the changes to `go.mod` and `go.sum`. - -### Generating Kibana clients - -Kibana clients for some APIs are generated based on Kibana OpenAPI specs. -Please see [Makefile](./Makefile) tasks for more details. +See [CONTRIBUTING.md](CONTRIBUTING.md) ## Support