Fixes documentation and javadoc

Author: DigitalSmile

CONCEPTS.md

@@ -0,0 +1,41 @@
+# Concepts
+
+The main concept behind the library is to provide easy access to native structures and functions of dynamic libraries written in different languages.
+Java FFM API provides solid core objects and patterns to interact with native code, but is lacking of DevEx when working with real native code.
+That said, the developer needs to write a lot of boilerplate code to make s simple native call and understand the whole native-java engine under the hood.
+
+To solve this issue `native-memory-processor` consists of two major parts:
+1) `annotation` project, which declares core annotations and some helpers to work with native code in runtime.
+    - `@NativeMemory` - main annotation to interface. Provide header files for parsing or leave empty if you need just native function generation.
+    - `@NativeMemoryOptions` - annotation to interface, can define optional advanced options to code generation.
+    - `@Structs` / `@Struct` - annotations to interface, defines the intent to parse structures.
+    - `@Enums` / `@Enum` - annotations to interface, defines the intent to parse enums.
+    - `@Unions` / `@Union` - annotations to interface, defines the intent to parse unions.
+    - `@NativeManualFunction` - annotation to method in interface, defines the native function to be generated.
+    - `@ByAddress` / `@Returns` - annotation to parameter of method in interface, defines parameter option to send as a pointer / value and native function return object.
+
+2) `annotation-processor` project, which provides processing of annotation described above at compile time in few steps:
+    - indicate the main `@NativeMemory` annotation on interface
+    - get provided header files (if any) and generate structures/enums/unions as defined by corresponding annotations
+    - get annotated by `@NativeManualFunction` methods in interface and generate java-to-native code
+
+In general, library tries to encapsulate the hardcore part of native interacting by code generation and provide user-specific classes with variety of options.
+
+## Difference from jextract
+
+The OpenJDK team had already created a similar tool, called `jextract` to keep developer hands clean from native code.
+This is a standalone CLI tool, which works with `libclang` to parse provided header files and generate the Java FFM-ready code.
+While it is great in concept, I found it 'not-so-friendly' for a person, who never worked with native code before.
+Mainly, the interacting and generated code is hard to understand in different ways:
+- structure/function names usually uses `_` and `$` signs or its combinations in generated methods
+- no support of opaque structures
+- it is HUGE! Simple `gpiochip_info` structure from `gpio.h` kernel UAPI of three fields transforms to 284 lines of code (sic!)
+- jextract holds no context of what it is generating. Meaning you can get a header file for defining primitive types by kernel (e.g. `__kernel_sighandler_t` for above structure), which is usually do not needed in your code
+- there is no simple way to understand the connections between generated classes, because of it's size and naming
+- it is standalone binary, which is needed to be connected somehow to your gradle/maven build toolchain
+
+I tried to get all advantages of `jextract` and create more user-friendly version of it.
+Still both `native-memory-processor` and `jextract` uses the same parsing library `libclang` and the quality of parsers are the same, but the process is very different:
+- annotation processing by gradle/maven instead of standalone binary
+- very specific control over the objects you are parsing with context based approach
+- more helpers and code validation

README.md

@@ -4,6 +4,8 @@
 ![Maven Central Version](https://img.shields.io/maven-central/v/io.github.digitalsmile.native/annotation-processor?label=annotation-processor)
 ![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/digitalsmile/native-memory-processor/gradle.yml)
 
+#### Create an interop code with ease.
+
 ## Introduction
 
 With the release of JDK 22 the new Foreign Function & Memory API (FFM API) has been introduced from preview phase.
@@ -17,7 +19,8 @@ This project goal is to combine the power of FFM API and Java, like annotation p
 
 ## Features
 - two separate libraries - `annotation` and `annotation-processor` working with code generation during compile time
-- C/C++ types support in top level: struct, union, enum
+- C/C++ types support in top level: struct, union, enum (including anonymous and opaque)
+- context based generation: combine several native libraries within one code base 
 - can load third party native libraries or use already loaded by classloader
 - flexible native-to-java function declarations
 - pass to native functions primitives and objects, by value or just a pointer
@@ -38,9 +41,9 @@ Windows hosts are not yet supported.
 ```groovy
 dependencies {
     // Annotations to use for code generation
-    implementation 'io.github.digitalsmile.native:annotation:{$version}'
+    implementation 'io.github.digitalsmile.native:annotation:${version}'
     // Process annotations and generate code at compile time
-    annotationProcessor 'io.github.digitalsmile.native:annotation-processor:{$version}'
+    annotationProcessor 'io.github.digitalsmile.native:annotation-processor:${version}'
 }
 ```
 
@@ -50,18 +53,17 @@ dependencies {
 @Structs
 @Enums
 public interface GPIO {
-    @NativeFunction(name = "ioctl", useErrno = true, returnType = int.class)
+    @NativeManualFunction(name = "ioctl", useErrno = true)
     int nativeCall(int fd, long command, int data) throws NativeMemoryException;
 }
 ```
 This code snippet will generate all structures and enums within the header file `gpio.h` (located in `resources` folder), as well as `GPIONative` class with call implementation of `ioctl` native function.
-Find more examples in [documentation](USAGE.md) or in my other project https://github.com/digitalsmile/gpio 
+
+Observe concepts in [doucmentation](CONCEPTS.md) or find more examples in [getting started](USAGE.md). You can also look at [tests](annotation-processor-test/src/test/java/io/github/digitalsmile/gpio) or in my other project https://github.com/digitalsmile/gpio 
 
 3) Enjoy! :)
 
 ## Plans
 
 - more support of native C/C++ types and patterns
-- validation of structures parameters and custom validation support
-- adding more context on different header files, especially connected with each other
-
+- validation of structures parameters and custom validation support

USAGE.md

@@ -1,45 +1,73 @@
 # Getting started
 
-## Concepts
-
-The main concept behind the library is to provide easy access to native structures and functions of dynamic libraries written in different languages.
-Java FFM API provides solid core objects and patterns to interact with native code, but is lacking of DevEx when working with real native code.
-That said, the developer needs to write a lot of boilerplate code to make s simple native call and understand the whole native-java engine under the hood.
-
-To solve this issue `native-memory-processor` consists of two major parts:
-1) `annotation` project, which declares core annotations and some helpers to work with native code in runtime.
-   - `@NativeMemory` - main annotation to interface. Provide header files for parsing or leave empty if you need just native function generation.
-   - `@NativeMemoryOptions` - annotation to interface, can define optional advanced options to code generation.
-   - `@Structs` / `@Struct` - annotations to interface, defines the intent to parse structures.
-   - `@Enums` / `@Enum` - annotations to interface, defines the intent to parse enums.
-   - `@Unions` / `@Union` - annotations to interface, defines the intent to parse unions.
-   - `@NativeFunction` - annotation to method in interface, defines the native function to be generated.
-   - `@ByAddress` / `@Returns` - annotation to parameter of method in interface, defines parameter option to send as a pointer / value and native function return object.
-   
-2) `annotation-processor` project, which provides processing of annotation described above at compile time in few steps:
-   - indicate the main `@NativeMemory` annotation on interface
-   - get provided header files (if any) and generate structures/enums/unions as defined by corresponding annotations
-   - get annotated by `@NativeFunction` methods in interface and generate java-to-native code 
-
-In general, library tries to encapsulate the hardcore part of native interacting by code generation and provide user-specific classes with variety of options. 
-
-## Difference from jextract
-
-The OpenJDK team had already created a similar tool, called `jextract` to keep developer hands clean from native code.
-This is a standalone CLI tool, which works with `libclang` to parse provided header files and generate the Java FFM-ready code.
-While it is great in concept, I found it 'not-so-friendly' for a person, who never worked with native code before.
-Mainly, the interacting and generated code is hard to understand in different ways:
-- structure/function names usually uses `_` and `$` signs or its combinations in generated methods
-- it is HUGE! Simple `gpiochip_info` structure from `gpio.h` kernel UAPI of three fields transforms to 284 lines of code (sic!)
-- jextract holds no context of what it is generating. Meaning you can get a header file for defining primitive types by kernel (e.g. `__kernel_sighandler_t` for above structure), which is usually do not needed in your code
-- there is no simple way to understand the connections between generated classes, because of it's size and naming
-- it is standalone binary, which is needed to be connected somehow to your gradle/maven build toolchain
-
-I tried to get all advantages of `jextract` and create more user-friendly version of it.
-Still both `native-memory-processor` and `jextract` uses the same parsing library `libclang` and the quality of parsers are the same, but the process is very different:
-- annotation processing by gradle/maven instead of standalone binary
-- very specific control over the objects you are parsing with context based approach
-- more helpers and code validation
+## Quick Start
+
+1) Locate your library header files. Usually, it is a single header file, which interconnects all needed structures/classes. E.g.
+```shell
+git clone --depth 1 https://github.com/curl/curl.git
+cd curl/include/curl
+pwd
+ls curl.h
+```
+2) Observe the header file and choose what structures/enums/functions you need.
+3) Create an interface `Libcurl` in your java project and locate your `systemIncludes` (or skip if you have it in your 
+system path). Fill in the header path and selected structures as follows:
+```java
+@NativeMemory(headers = "libcurl/curl/include/curl/curl.h")
+@NativeMemoryOptions(systemIncludes = {
+        "/usr/lib/gcc/x86_64-linux-gnu/12/include/"
+    }, processRootConstants = true)
+@Structs({
+    @Struct(name = "CURL", javaName = "CurlInstance")
+})
+@Enums({
+    @Enum(name = "CURLcode", javaName = "CURLCode"),
+    @Enum(name = "CURLoption", javaName = "CURLOption")
+})
+public interface Libcurl {}
+```
+4) Run build and observe the generated classes in project `build/generated/sources/annotationProcessor/java/main/{you-package-name}`.
+The package name defaults to where the `Libcurl` interface created.
+5) Locate you dynamic library `libcurl.so` (or `libcurl.dylib`). You can use precompiled binary or compile it by yourself.
+6) Set up the native functions mapping (this is a minimum to run hello world example):
+```java
+public interface Libcurl {
+
+    @NativeManualFunction(name = "curl_easy_init", library = "/usr/lib/x86_64-linux-gnu/libcurl.so")
+    CURL easyInit() throws NativeMemoryException;
+
+    @NativeManualFunction(name = "curl_global_init", library = "/usr/lib/x86_64-linux-gnu/libcurl.so")
+    CURLcode globalInit(long flags) throws NativeMemoryException;
+
+    @NativeManualFunction(name = "curl_easy_setopt", library = "/usr/lib/x86_64-linux-gnu/libcurl.so")
+    CURLcode easySetOpt(CURL curl, CURLoption option, String value) throws NativeMemoryException;
+
+    @NativeManualFunction(name = "curl_easy_setopt", library = "/usr/lib/x86_64-linux-gnu/libcurl.so")
+    CURLcode easySetOpt(CURL curl, CURLoption option, @ByAddress long value) throws NativeMemoryException;
+
+    @NativeManualFunction(name = "curl_easy_perform", library = "/usr/lib/x86_64-linux-gnu/libcurl.so")
+    CURLcode easyPerform(CURL curl) throws NativeMemoryException;
+}
+```
+8) Run build and observe `LibcurlNative` class at the same location.
+7) Write your example method
+```java
+    public static void main(String[] args) throws NativeMemoryException {
+        try(var libcurl = new LibcurlNative()) {
+            var code = libcurl.globalInit(CurlConstants.CURL_GLOBAL_DEFAULT);
+            assertEquals(code, CURLcode.CURLE_OK);
+            var curl = libcurl.easyInit();
+            code = libcurl.easySetOpt(curl, CURLoption.CURLOPT_URL, "https://example.com");
+            assertEquals(code, CURLcode.CURLE_OK);
+            code = libcurl.easySetOpt(curl, CURLoption.CURLOPT_FOLLOWLOCATION, 1L);
+            assertEquals(code, CURLcode.CURLE_OK);
+
+            code = libcurl.easyPerform(curl);
+            assertEquals(code, CURLcode.CURLE_OK);
+        }
+    }
+```
+8) Run the main method. You should see the html page of `https://example.com` in stdout.
 
 ## Example usages
 ### Example 1
@@ -50,7 +78,7 @@ Generate all structures, enums and unions from `gpio.h` and generate file `GPION
 @Enums
 @Unions
 public interface GPIO {
-    @NativeFunction(name = "ioctl", useErrno = true, returnType = int.class)
+    @NativeManualFunction(name = "ioctl", useErrno = true)
     int nativeCall(int fd, long command, int data) throws NativeMemoryException;
 }
 ```
@@ -64,7 +92,7 @@ Method declaration will forward the output of `ioctl` to user code, since `retur
         @Struct(name = "gpio_v2_line_info", javaName = "LineInfo")
 })
 public interface GPIO {
-    @NativeFunction(name = "ioctl", returnType = int.class)
+    @NativeManualFunction(name = "ioctl")
     int nativeCall(int fd, long command, int data) throws NativeMemoryException;
 }
 ```
@@ -83,10 +111,10 @@ You can declare a freshly generated structure to use within the native functions
         @Struct(name = "gpio_v2_line_info", javaName = "LineInfo")
 })
 public interface GPIO {
-    @NativeFunction(name = "ioctl", returnType = int.class)
+    @NativeManualFunction(name = "ioctl")
     ChipInfo nativeCall(int fd, long command, @Returns ChipInfo data) throws NativeMemoryException;
 
-    @NativeFunction(name = "ioctl", useErrno = true, returnType = int.class)
+    @NativeManualFunction(name = "ioctl", useErrno = true)
     long nativeCall(int fd, long command, @Returns @ByAddress long data) throws NativeMemoryException;
 }
 ```
@@ -106,7 +134,7 @@ You can use system properties `-Dversion=6.2.0-39` to pass variables into header
 )
 @Structs
 public interface GPIO {
-    @NativeFunction(name = "ioctl", returnType = int.class)
+    @NativeManualFunction(name = "ioctl")
     GpiochipInfo nativeCall(int fd, long command, @Returns GpiochipInfo data) throws NativeMemoryException;
 }
 ```
@@ -119,10 +147,10 @@ Generate `some_struct` structure from `library2_header.h` and implement two nati
 @NativeMemory(header = "library2_header.h")
 @Structs
 public interface GPIO {
-    @NativeFunction(name = "some_func1", library = "/some/path/to/library1.so")
+    @NativeManualFunction(name = "some_func1", library = "/some/path/to/library1.so")
     void call(int data) throws NativeMemoryException;
 
-    @NativeFunction(name = "some_func2", library = "library2")
+    @NativeManualFunction(name = "some_func2", library = "library2")
     SomeStruct nativeCall(int param, @Returns SomeStruct data) throws NativeMemoryException;
 }
 ```

annotation-processor-test/src/test/java/io/github/digitalsmile/gpio/libcurl/Libcurl.java

@@ -40,5 +40,4 @@ public interface Libcurl {
 
     @NativeManualFunction(name = "curl_easy_perform", library = "/usr/lib/x86_64-linux-gnu/libcurl.so")
     CURLcode easyPerform(CURL curl) throws NativeMemoryException;
-
 }

annotation-processor-test/src/test/java/io/github/digitalsmile/gpio/libcurl/LibcurlTest.java

@@ -23,4 +23,19 @@ public void libcurl() throws NativeMemoryException {
             assertEquals(code, CURLcode.CURLE_OK);
         }
     }
+
+    public static void main(String[] args) throws NativeMemoryException {
+        try(var libcurl = new LibcurlNative()) {
+            var code = libcurl.globalInit(CurlConstants.CURL_GLOBAL_DEFAULT);
+            assertEquals(code, CURLcode.CURLE_OK);
+            var curl = libcurl.easyInit();
+            code = libcurl.easySetOpt(curl, CURLoption.CURLOPT_URL, "https://example.com");
+            assertEquals(code, CURLcode.CURLE_OK);
+            code = libcurl.easySetOpt(curl, CURLoption.CURLOPT_FOLLOWLOCATION, 1L);
+            assertEquals(code, CURLcode.CURLE_OK);
+
+            code = libcurl.easyPerform(curl);
+            assertEquals(code, CURLcode.CURLE_OK);
+        }
+    }
 }

annotation/src/main/java/io/github/digitalsmile/annotation/ArenaType.java

@@ -1,10 +1,31 @@
 package io.github.digitalsmile.annotation;
 
-import java.lang.foreign.Arena;
-
+/**
+ * Class represents <code>Arena</code> type in string representation for code generation.
+ */
 public enum ArenaType {
-    GLOBAL, AUTO, CONFINED, SHARED;
+    /**
+     * Arena.global()
+     */
+    GLOBAL,
+    /**
+     * Arena.ofAuto()
+     */
+    AUTO,
+    /**
+     * Arena.ofConfined()
+     */
+    CONFINED,
+    /**
+     * Arena.ofShared()
+     */
+    SHARED;
 
+    /**
+     * Gets the string representation of arena type.
+     *
+     * @return string representation of arena type
+     */
     public String arena() {
         return switch (this) {
             case AUTO -> "ofAuto()";

annotation/src/main/java/io/github/digitalsmile/annotation/NativeMemoryOptions.java

@@ -20,13 +20,44 @@
      */
     boolean processRootConstants() default false;
 
+    /**
+     * Defines <code>Arena</code> that should be used within the generated code.
+     * Defaults to <code>Arena.ofAuto()</code> if not specified.
+     *
+     * @return arena type to be used
+     */
     ArenaType arena() default ArenaType.AUTO;
 
+    /**
+     * Defines the <code>packageName</code> to be used for storing generated code.
+     * Leave empty to have the interface package name as default.
+     *
+     * @return package name to be used
+     */
     String packageName() default "";
 
+    /**
+     * Defines the native code include search path to be passed to <code>libclang</code>.
+     * Use if you have library with complex search path.
+     *
+     * @return include array to be passed to <code>libclang</code>
+     */
     String[] includes() default {};
+
+    /**
+     * Defines the system includes search path to be passed to <code>libclang</code>.
+     * If you have issues with standard C/C++ header files, provide the path to search with this option.
+     *
+     * @return system include array to be passed to <code>libclang</code>
+     */
     String[] systemIncludes() default {};
 
+    /**
+     * Defines if header files are system and should be parsed.
+     * By default, system header processing is off to simplify the generated output and skip unused kernel/system structures.
+     *
+     * @return true if system headers should be parsed
+     */
     boolean systemHeader() default false;
 
     boolean debugMode() default false;