|
| 1 | +{% raw %} |
| 2 | +<html> |
| 3 | + <head></head> |
| 4 | + <body> |
| 5 | + <div id="preamble"> |
| 6 | + <div class="sectionbody"> |
| 7 | + <div class="paragraph _abstract"> |
| 8 | + <p>Kroxylicious' composable filter chains and pluggable API mean that you can write your own filters to apply your own rules to the Kafka protocol, using the Java programming language.</p> |
| 9 | + </div> |
| 10 | + <div class="paragraph"> |
| 11 | + <p>In this quick start you will build a custom filter and use it to modify messages being sent to/consumed from Kafka, learn about filter configuration and running custom filters, and find a starting point for developing your own custom filters with your own rules and logic.</p> |
| 12 | + </div> |
| 13 | + </div> |
| 14 | + </div> |
| 15 | + <div class="sect1"> |
| 16 | + <h2 id="getting_started">1. Getting started</h2> |
| 17 | + <div class="sectionbody"> |
| 18 | + <div class="sect2"> |
| 19 | + <h3 id="prerequisites">1.1. Prerequisites</h3> |
| 20 | + <div class="paragraph"> |
| 21 | + <p>To start developing your own custom filters for Kroxylicious, you will need to install <a href="https://openjdk.org/projects/jdk/21/">JDK 21</a>.</p> |
| 22 | + </div> |
| 23 | + <div class="paragraph"> |
| 24 | + <p>You’ll also need to install the <a href="https://maven.apache.org/index.html">Apache Maven CLI</a> and one of either <a href="https://podman.io/docs/installation">Podman</a> or <a href="https://docs.docker.com/install/">Docker</a>.</p> |
| 25 | + </div> |
| 26 | + <div class="admonitionblock note"> |
| 27 | + <table> |
| 28 | + <tbody> |
| 29 | + <tr> |
| 30 | + <td class="icon"><i class="fa icon-note" title="Note"></i></td> |
| 31 | + <td class="content">If you are using Podman, you may encounter issues with the integration tests. There are instructions <a href="https://github.com/kroxylicious/kroxylicious/blob/main/DEV_GUIDE.md#running-integration-tests-on-podman">here</a> to resolve this.</td> |
| 32 | + </tr> |
| 33 | + </tbody> |
| 34 | + </table> |
| 35 | + </div> |
| 36 | + </div> |
| 37 | + <div class="sect2"> |
| 38 | + <h3 id="generate_a_sample_filter_project">1.2. Generate a Sample Filter Project</h3> |
| 39 | + <div class="paragraph"> |
| 40 | + <p>The easiest way to learn how to build custom filters is with our <code>kroxylicious-filter-archetype</code> maven archetype, which will generate some basic find-and-replace filters for you to experiment with. Begin by generating a sample project:</p> |
| 41 | + </div> |
| 42 | + <div class="listingblock"> |
| 43 | + <div class="content"> |
| 44 | + <pre class="CodeRay highlight"><code data-lang="shell">mvn archetype:generate -DarchetypeGroupId=io.kroxylicious \ |
| 45 | + -DarchetypeArtifactId=kroxylicious-filter-archetype \ |
| 46 | + -DarchetypeVersion=0.17.1 \ |
| 47 | + -DgroupId=org.example \ |
| 48 | + -DartifactId=sample-filter \ |
| 49 | + -Dversion=1.0-SNAPSHOT \ |
| 50 | + -DinteractiveMode=false |
| 51 | +cd sample-filter</code></pre> |
| 52 | + </div> |
| 53 | + </div> |
| 54 | + </div> |
| 55 | + </div> |
| 56 | + </div> |
| 57 | + <div class="sect1"> |
| 58 | + <h2 id="build_the_sample_filter_project">2. Build the Sample Filter project</h2> |
| 59 | + <div class="sectionbody"> |
| 60 | + <div class="paragraph"> |
| 61 | + <p>Building the sample project is easy!</p> |
| 62 | + </div> |
| 63 | + <div class="listingblock"> |
| 64 | + <div class="content"> |
| 65 | + <pre class="CodeRay highlight"><code data-lang="shell">mvn verify</code></pre> |
| 66 | + </div> |
| 67 | + </div> |
| 68 | + <div class="paragraph"> |
| 69 | + <p>Note that the sample project includes automated unit and integration tests. The project’s powerful integration tests run against an in-VM Kafka cluster, enabling you to rapidly iterate on your filter’s business logic.</p> |
| 70 | + </div> |
| 71 | + </div> |
| 72 | + </div> |
| 73 | + <div class="sect1"> |
| 74 | + <h2 id="run_the_built_sample_filter_module_in_a_proxy">3. Run the built Sample Filter module in a Proxy</h2> |
| 75 | + <div class="sectionbody"> |
| 76 | + <div class="paragraph"> |
| 77 | + <p>To build and run your sample filter</p> |
| 78 | + </div> |
| 79 | + <div class="listingblock"> |
| 80 | + <div class="content"> |
| 81 | + <pre class="CodeRay highlight"><code data-lang="shell">chmod +x run.sh |
| 82 | +./run.sh --config sample-proxy-config.yaml</code></pre> |
| 83 | + </div> |
| 84 | + </div> |
| 85 | + <div class="paragraph"> |
| 86 | + <p>Send a Kafka message containing <code>foo</code> through the proxy. You should see the content transformed, with the word <code>foo</code> replaced by <code>baz</code> in the message received by the consumer.</p> |
| 87 | + </div> |
| 88 | + <div class="listingblock"> |
| 89 | + <div class="content"> |
| 90 | + <pre class="CodeRay highlight"><code data-lang="shell">podman run -it --net host \ |
| 91 | + --entrypoint /opt/kafka/bin/kafka-console-producer.sh \ |
| 92 | + apache/kafka:4.1.0 \ |
| 93 | + --bootstrap-server localhost:9192 \ |
| 94 | + --topic hello |
| 95 | +>foo |
| 96 | + |
| 97 | +podman run -it --net host \ |
| 98 | + --entrypoint /opt/kafka/bin/kafka-console-consumer.sh \ |
| 99 | + apache/kafka:4.1.0 \ |
| 100 | + --bootstrap-server localhost:9192 \ |
| 101 | + --topic hello \ |
| 102 | + --from-beginning |
| 103 | +baz</code></pre> |
| 104 | + </div> |
| 105 | + </div> |
| 106 | + </div> |
| 107 | + </div> |
| 108 | + <div class="sect1"> |
| 109 | + <h2 id="configure_the_filter">4. Configure the Filter</h2> |
| 110 | + <div class="sectionbody"> |
| 111 | + <div class="paragraph"> |
| 112 | + <p>Filters can be added and removed by altering the <code>filterDefinitions</code> list in the <code>sample-proxy-config.yaml</code> file. You can also reconfigure the sample filters by changing the configuration values in this file. Note that the proxy must be restarted when you modify the configuration.</p> |
| 113 | + </div> |
| 114 | + <div class="paragraph"> |
| 115 | + <p>The <strong>SampleFetchResponseFilter</strong> and <strong>SampleProduceRequestFilter</strong> each have two configuration values that must be specified for them to work:</p> |
| 116 | + </div> |
| 117 | + <div class="ulist"> |
| 118 | + <ul> |
| 119 | + <li> |
| 120 | + <p><code>findValue</code> - the string the filter will search for in the produce/fetch data</p> |
| 121 | + </li> |
| 122 | + <li> |
| 123 | + <p><code>replacementValue</code> - the string the filter will replace the value above with</p> |
| 124 | + </li> |
| 125 | + </ul> |
| 126 | + </div> |
| 127 | + <div class="sect2"> |
| 128 | + <h3 id="default_configuration">4.1. Default Configuration</h3> |
| 129 | + <div class="paragraph"> |
| 130 | + <p>The default configuration for <strong>SampleProduceRequestFilter</strong> is:</p> |
| 131 | + </div> |
| 132 | + <div class="listingblock"> |
| 133 | + <div class="content"> |
| 134 | + <pre class="CodeRay highlight"><code data-lang="yaml"><span class="key">filterDefinitions</span>: |
| 135 | + - <span class="string"><span class="content">name: produce-request-filter</span></span> |
| 136 | + <span class="key">type</span>: <span class="string"><span class="content">SampleProduceRequestFilterFactory</span></span> |
| 137 | + <span class="key">config</span>: |
| 138 | + <span class="key">findValue</span>: <span class="string"><span class="content">foo</span></span> |
| 139 | + <span class="key">replacementValue</span>: <span class="string"><span class="content">bar</span></span></code></pre> |
| 140 | + </div> |
| 141 | + </div> |
| 142 | + <div class="paragraph"> |
| 143 | + <p>This means that it will search for the string <code>foo</code> in the produce data and replace all occurrences with the string <code>bar</code>. For example, if a Kafka Producer sent a produce request with data <code>{"myValue":"foo"}</code>, the filter would transform this into <code>{"myValue":"bar"}</code> and Kroxylicious would send that to the Kafka Broker instead.</p> |
| 144 | + </div> |
| 145 | + <div class="paragraph"> |
| 146 | + <p>The default configuration for <strong>SampleFetchResponseFilter</strong> is:</p> |
| 147 | + </div> |
| 148 | + <div class="listingblock"> |
| 149 | + <div class="content"> |
| 150 | + <pre class="CodeRay highlight"><code data-lang="yaml"><span class="key">filterDefinitions</span>: |
| 151 | + - <span class="string"><span class="content">name: fetch-response-filter</span></span> |
| 152 | + <span class="key">type</span>: <span class="string"><span class="content">SampleFetchResponseFilterFactory</span></span> |
| 153 | + <span class="key">config</span>: |
| 154 | + <span class="key">findValue</span>: <span class="string"><span class="content">bar</span></span> |
| 155 | + <span class="key">replacementValue</span>: <span class="string"><span class="content">baz</span></span></code></pre> |
| 156 | + </div> |
| 157 | + </div> |
| 158 | + <div class="paragraph"> |
| 159 | + <p>This means that it will search for the string <code>bar</code> in the fetch data and replace all occurrences with the string <code>baz</code>. For example, if a Kafka Broker sent a fetch response with data <code>{"myValue":"bar"}</code>, the filter would transform this into <code>{"myValue":"baz"}</code> and Kroxylicious would send that to the Kafka Consumer instead.</p> |
| 160 | + </div> |
| 161 | + </div> |
| 162 | + <div class="sect2"> |
| 163 | + <h3 id="modify">4.2. Modify</h3> |
| 164 | + <div class="paragraph"> |
| 165 | + <p>Now that you know how the sample filters work, you can start modifying them! Replace the <code>SampleFilterTransformer</code> logic with your own code, change which messages they apply to, or whatever else you like!</p> |
| 166 | + </div> |
| 167 | + </div> |
| 168 | + </div> |
| 169 | + </div> |
| 170 | + </body> |
| 171 | +</html> |
| 172 | +{% endraw %} |
0 commit comments