add JBang UDF example

Author: ferenc-csaky

user-defined-function/README.md

@@ -1,24 +1,27 @@
 
 # Custom User-Defined Functions in Flink
-This guide will lead you through the process of creating, compiling, and deploying a user-defined functions (UDFs) in Apache Flink using DataSQRL. We will specifically focus on a function called MyScalarFunction, which will double the values of input numbers, and then deploy and execute the function in Flink.
+This guide will lead you through the process of creating, compiling, and deploying a user-defined functions (UDFs) in Apache Flink using DataSQRL.
+We will specifically focus on a function called `MyScalarFunction`, which will double the values of input numbers, and then deploy and execute the function in Flink.
 
 ## Introduction
-User-defined functions (UDFs) in Flink are powerful tools that allow for the extension of the system's built-in functionality. UDFs can be used to perform operations on data that are not covered by the built-in functions.
+User-defined functions (UDFs) in Flink are powerful tools that allow for the extension of the system's built-in functionality.
+UDFs can be used to perform operations on data that are not covered by the built-in functions.
 
-## Creating a User-Defined Function
-1. **Project Structure:** The `myjavafunction` folder contains a sample Java project, demonstrating the structure and necessary components of a Flink UDF.
+We support two main ways to ship UDFs for SQRL applications:
+1. [**JBang**](./jbang): For simple, standalone UDFs that has no or only lightweight dependencies, DataSQRL can build and ship UDFs on the fly via [JBang](https://www.jbang.dev/). 
+2. [**Assemble JAR Manually**](./maven-project): For more complex UDF projects with common abstract UDF layers, complex logic, a custom Maven or Gradle project can be crated.
+  Then the manually built JAR(s) can be placed to a project folder, which DataSQRL will recognize and ship.  
 
-2. **Defining the Function:** The main component of this project is the MyScalarFunction class. This is the implementation of a custom flink function. DataSQRL recognizes flink functions that extend UserDefinedFunction.
+To learn more about each option, please see the example project and their specific readme.
 
-3. **ServiceLoader Entry:** The function must be registered with a ServiceLoader entry. This is essential for DataSQRL to recognize and use your UDF.
-- **AutoService Library:** The example includes the AutoService library by Google, simplifying the creation of ServiceLoader META-INF manifest entries.
-
-4. **Jar Compiling:** Compile the sample project and build a jar. This jar is what DataSQRL will use to discover your function. It reads the manifest entries for any UserDefinedFunction classes and load them into DataSQRL for use in queries. It can be placed into any folder relative to the sqrl root folder which will translate to the import path. In the example, we will use the `target` directory that the compilation process creates.
+The next sections contain instructions about how to run and interact with the example projects, and those steps are the same for both variants.
 
 ## SQRL Compilation and Packaging
 1. **SQRL Compilation:** Compile the SQRL using DataSQRL's command interface, which prepares your script for deployment in the Flink environment.
 
 ```shell
+# "cd jbang" OR "cd maven-project"
+
 docker run --rm -v $PWD:/build datasqrl/cmd:latest compile myudf.sqrl
 ```
 
@@ -52,4 +55,4 @@ query {
     myFnc
   }
 }'
-```
+```

user-defined-function/jbang/README.md

@@ -0,0 +1,8 @@
+## Creating JBang UDFs
+
+1. **Structure:** The `usrlib` contains a sample Java UDF file, but any folder name or subfolder can be used inside the DataSQRL project root.
+2. **Defining the Function:** The main component of this project is the `MyScalarFunction` class.
+   This is the implementation of a custom flink function. DataSQRL recognizes flink functions that extend any base Flink UDF class.
+3. **Defining Dependencies:** To fetch dependencies JBang will look for lines that start with `//DEPS` and define a proper artifact. 
+  The bare minimum that any Flink UDF will require is `flink-table-common`, because the custom UDF has to extend a Flink built-in UDF parent class.
+4. **JAR Compiling:** The compile and JAR build is handled by JBang, that DataSQRL will do automatically on the fly.

user-defined-function/jbang/myudf.sqrl

@@ -0,0 +1,8 @@
+IMPORT usrlib.MyScalarFunction;
+
+-- Capture some input data
+CREATE TABLE InputData (
+    val BIGINT NOT NULL
+);
+
+MyTable := SELECT val, MyScalarFunction(val, val) AS myUdf FROM InputData;

user-defined-function/jbang/usrlib/MyScalarFunction.java

@@ -0,0 +1,10 @@
+//DEPS org.apache.flink:flink-table-common:1.19.3
+
+import org.apache.flink.table.functions.ScalarFunction;
+
+public class MyScalarFunction extends ScalarFunction {
+
+  public long eval(long a, long b) {
+    return a + b;
+  }
+}

user-defined-function/maven-project/README.md

@@ -0,0 +1,11 @@
+## Creating a UDF Maven Project
+
+1. **Project Structure:** The `myjavafunction` folder contains a sample Java project, demonstrating the structure and necessary components of a Flink UDF.
+2. **Defining the Function:** The main component of this project is the `MyScalarFunction` class.
+   This is the implementation of a custom flink function. DataSQRL recognizes flink functions that extend any base Flink UDF class.
+3. **ServiceLoader Entry:** The function must be registered with a ServiceLoader entry. This is essential for DataSQRL to recognize and use your UDF.
+4. **AutoService Library:** The example includes the AutoService library by Google, simplifying the creation of ServiceLoader `META-INF` manifest entries.
+5. **JAR Compiling:** Compile the sample project and build a jar. This jar is what DataSQRL will use to discover your function.
+   It reads the manifest entries for any UserDefinedFunction classes and load them into DataSQRL for use in queries.
+   It can be placed into any folder relative to the sqrl root folder which will translate to the import path.
+   In the example, we will use the `target` directory that the compilation process creates.

…-defined-function/myjavafunction/pom.xml …ion/maven-project/myjavafunction/pom.xmluser-defined-function/myjavafunction/pom.xml renamed to user-defined-function/maven-project/myjavafunction/pom.xml user-defined-function/myjavafunction/pom.xml renamed to user-defined-function/maven-project/myjavafunction/pom.xml

@@ -18,11 +18,12 @@
       <artifactId>auto-service</artifactId>
       <version>1.0.1</version>
     </dependency>
+
     <dependency>
       <groupId>org.apache.flink</groupId>
       <artifactId>flink-table-api-java-bridge</artifactId>
-      <version>1.16.1</version>
+      <version>1.19.3</version>
       <scope>provided</scope>
     </dependency>
   </dependencies>
-</project>
+</project>

…ain/java/com/myudf/MyScalarFunction.java …ain/java/com/myudf/MyScalarFunction.javauser-defined-function/myjavafunction/src/main/java/com/myudf/MyScalarFunction.java renamed to user-defined-function/maven-project/myjavafunction/src/main/java/com/myudf/MyScalarFunction.java user-defined-function/myjavafunction/src/main/java/com/myudf/MyScalarFunction.java renamed to user-defined-function/maven-project/myjavafunction/src/main/java/com/myudf/MyScalarFunction.java

@@ -8,5 +8,4 @@ CREATE TABLE InputData (
     val BIGINT NOT NULL
 );
 
-MyTable := SELECT val, MyScalarFunction(val, val) AS myFnc
-           FROM InputData;
+MyTable := SELECT val, MyScalarFunction(val, val) AS myUdf FROM InputData;