From 0110facad068896cb4e0cea05b3b953d940733c0 Mon Sep 17 00:00:00 2001 From: Mashhur Date: Wed, 18 Dec 2024 11:44:26 -0800 Subject: [PATCH 1/3] Add plugin development notes in a readme. --- README.md | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 71 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 09f7c12e..016eac75 100644 --- a/README.md +++ b/README.md @@ -7,4 +7,74 @@ It can be used in a Logstash pipeline to perform the transformations that are ap ## Documentation Logstash provides infrastructure to automatically generate documentation for this plugin from its [asciidoc source](docs/index.asciidoc). -All plugin documentation are placed under one [central location](http://www.elastic.co/guide/en/logstash/current/). \ No newline at end of file +All plugin documentation are placed under one [central location](http://www.elastic.co/guide/en/logstash/current/). + +## Development +### Prerequisites +- Clone the [plugin repo](https://github.com/elastic/logstash-filter-elastic_integration) +- Check out the branch you would like to build, such as `8.x` or `9.x/main`, etc... + - Note that, `8.series` branches require JDK17 and `9.series`/`main` requires JDK21. +- Set the Logstash source path in the `gradle.properties` file. Open the file and update the `LOGSTASH_PATH`. +- Make sure Logstash is compiled. +- Point to the Elasticsearch branch/version to build the plugin with. Open the `gradle.properties` file and update `ELASTICSEARCH_TREEISH`. +- Point to the Elasticsearch branch/version to build the plugin with. Open the `gradle.properties` file and update `ELASTICSEARCH_TREEISH`. + +### Build +We use Gradle tool to build this plugin. Gradle seeks Logstash core jars from the `LOGSTASH_PATH` path, downloads the Elasticsearch version defined with `ELASTICSEARCH_TREEISH` and build the plugin. +The build process may fail if the Elasticsearch interfaces the plugin is using have changed. + +The following command builds plugin and generates the jar file, can be locally installed in Logstash core and verified: +```shell +./gradlew clean vendor localGem +``` + +### Plugin test +The recommendation to run tests to align on our common [`plugins/.ci`](https://github.com/logstash-plugins/.ci) tool. +Run the following command under the plugin source dir to download the CI tool: +```shell +mkdir -p .ci-test && curl -sL --retry 5 --retry-delay 5 https://github.com/logstash-plugins/.ci/archive/1.x.tar.gz | tar zxvf - --keep-old-files --strip-components=1 -C .ci-test +``` + +Export environment variables: +```shell +SNAPSHOT=false # set to true if targeting the SNAPSHOT +ELASTIC_STACK_VERSION=8.17.0 # stack version to run the plugin on +INTEGRATION=false # set to true if you want integration tests to run, it also requires SECURE_INTEGRATION=true +``` + +Run tests on the local docker environment, make sure your docker-engine is running. +```shell +./.ci/docker-setup.sh && ./.ci/docker-run.sh +``` + +### Running your unpublished plugin in Logstash + +#### Run in a local Logstash clone +- Edit Logstash `Gemfile` and add the local plugin path, for example: +```ruby +gem "logstash-filter-awesome", :path => "/your/local/logstash-filter-awesome" +``` +- Install plugin +```sh +bin/logstash-plugin install --no-verify +``` +- Run Logstash with your plugin +```sh +bin/logstash -e 'filter {awesome {}}' +``` +At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash. + +#### Run in an installed Logstash +- Edit Logstash `Gemfile` and add the local plugin path, for example: +```ruby +gem "logstash-filter-awesome", :path => "/your/local/logstash-filter-awesome" +``` +- Build your plugin gem +```sh +./gradlew clean vendor localGem +``` +- Install the plugin from the Logstash home +```sh +bin/logstash-plugin install --no-verify +``` +- Start Logstash and proceed to test the plugin \ No newline at end of file From 9d825265d7ed8d1558ad9ee1961fa7a63341c28c Mon Sep 17 00:00:00 2001 From: Mashhur <99575341+mashhurs@users.noreply.github.com> Date: Wed, 18 Dec 2024 11:51:18 -0800 Subject: [PATCH 2/3] Update README.md Co-authored-by: Cas Donoghue --- README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/README.md b/README.md index 016eac75..f91a5903 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,6 @@ All plugin documentation are placed under one [central location](http://www.elas - Set the Logstash source path in the `gradle.properties` file. Open the file and update the `LOGSTASH_PATH`. - Make sure Logstash is compiled. - Point to the Elasticsearch branch/version to build the plugin with. Open the `gradle.properties` file and update `ELASTICSEARCH_TREEISH`. -- Point to the Elasticsearch branch/version to build the plugin with. Open the `gradle.properties` file and update `ELASTICSEARCH_TREEISH`. ### Build We use Gradle tool to build this plugin. Gradle seeks Logstash core jars from the `LOGSTASH_PATH` path, downloads the Elasticsearch version defined with `ELASTICSEARCH_TREEISH` and build the plugin. From 7bdab9876550895b34f90ac2620e2d2a75b6054b Mon Sep 17 00:00:00 2001 From: Mashhur <99575341+mashhurs@users.noreply.github.com> Date: Wed, 18 Dec 2024 12:00:30 -0800 Subject: [PATCH 3/3] Update README.md Co-authored-by: Cas Donoghue --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index f91a5903..3ae5e8b7 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ All plugin documentation are placed under one [central location](http://www.elas - Point to the Elasticsearch branch/version to build the plugin with. Open the `gradle.properties` file and update `ELASTICSEARCH_TREEISH`. ### Build -We use Gradle tool to build this plugin. Gradle seeks Logstash core jars from the `LOGSTASH_PATH` path, downloads the Elasticsearch version defined with `ELASTICSEARCH_TREEISH` and build the plugin. +We use Gradle tool to build this plugin. Gradle is configured to load the Logstash core jars from the `LOGSTASH_PATH` path, download the Elasticsearch version defined with `ELASTICSEARCH_TREEISH` and build the plugin. The build process may fail if the Elasticsearch interfaces the plugin is using have changed. The following command builds plugin and generates the jar file, can be locally installed in Logstash core and verified: