Skip to content

Commit 5f7e0e4

Browse files
joewizclaude
joewiz
and
claude
committed
[feature] Native EXPath HTTP Client Module for eXist-db
Standalone XAR implementing the EXPath HTTP Client 1.0 spec (http://expath.org/ns/http-client) using Java's built-in java.net.http.HttpClient — zero external dependencies. Key fix: JSON and text responses return xs:string (not xs:base64Binary), matching BaseX behavior and the EXPath spec. Features: - http:send-request() with 1, 2, and 3 argument arities - All HTTP methods: GET, POST, PUT, DELETE, HEAD, OPTIONS, PATCH - Content-Type classification: XML → document-node(), text/JSON → xs:string - HTTP Basic auth, redirects, timeouts, gzip, multipart - EXPath error codes HC001–HC006 Includes GitHub Actions CI with Docker-based smoke tests and 113 tests (43 unit + 70 integration). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 5a7d386 commit 5f7e0e4

File tree

14 files changed

+2816
-10
lines changed

14 files changed

+2816
-10
lines changed

‎.github/workflows/exist.yml‎

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
# This workflow builds a xar archive, deploys it into exist and runs a smoke test.
2+
3+
name: exist-db CI
4+
5+
on: [push, pull_request]
6+
7+
jobs:
8+
build:
9+
runs-on: ubuntu-latest
10+
strategy:
11+
fail-fast: false
12+
matrix:
13+
exist-version: [latest]
14+
15+
steps:
16+
- uses: actions/checkout@v6
17+
18+
# Build with Maven
19+
- name: Set up JDK 21
20+
uses: actions/setup-java@v5
21+
with:
22+
java-version: '21'
23+
distribution: 'temurin'
24+
cache: maven
25+
26+
# Extract exist-core JAR from Docker image for compilation,
27+
# since the published 7.0.0-SNAPSHOT in the Maven repo is stale.
28+
- name: Pull eXist-db image
29+
run: docker pull existdb/existdb:${{ matrix.exist-version }}
30+
31+
- name: Install exist-core from Docker image into local Maven repo
32+
run: |
33+
id=$(docker create existdb/existdb:${{ matrix.exist-version }})
34+
docker cp "$id:/exist/lib/exist.uber.jar" /tmp/exist.uber.jar
35+
docker rm "$id"
36+
mvn install:install-file -Dfile=/tmp/exist.uber.jar \
37+
-DgroupId=org.exist-db -DartifactId=exist-core \
38+
-Dversion=7.0.0-SNAPSHOT -Dpackaging=jar -q
39+
40+
- name: Build with Maven
41+
run: mvn clean package -DskipTests -q
42+
43+
# Deploy XAR in Container
44+
- name: Start eXist-db container
45+
run: |
46+
docker run -dit -p 8080:8080 \
47+
-v ${{ github.workspace }}/target:/exist/autodeploy \
48+
--name exist --rm --health-interval=1s --health-start-period=1s \
49+
existdb/existdb:${{ matrix.exist-version }}
50+
51+
- name: Wait for eXist-db to start and deploy packages
52+
timeout-minutes: 5
53+
run: |
54+
while ! docker logs exist | grep -q "Server has started"; \
55+
do sleep 6s; \
56+
done
57+
sleep 5
58+
59+
# Smoke tests: verify module loads and key functionality works
60+
# Note: The XAR module cannot override the built-in http-client in
61+
# the Docker image, so these tests verify the built-in module works.
62+
# The key JSON-as-string fix is validated by the Maven integration tests.
63+
- name: Test http:send-request basic GET
64+
run: |
65+
result=$(curl -sf -u admin: -H "Content-Type: application/xml" --data '
66+
<query xmlns="http://exist.sourceforge.net/NS/exist">
67+
<text><![CDATA[
68+
import module namespace http = "http://expath.org/ns/http-client";
69+
let $r := http:send-request(
70+
<http:request method="GET" href="https://httpbin.org/get"/>
71+
)
72+
return string($r[1]/@status)
73+
]]></text>
74+
</query>' "http://localhost:8080/exist/rest/db")
75+
echo "$result"
76+
echo "$result" | grep -q "200" || (echo "FAIL: expected status 200" && exit 1)
77+
78+
- name: Test http:send-request POST with body
79+
run: |
80+
result=$(curl -sf -u admin: -H "Content-Type: application/xml" --data '
81+
<query xmlns="http://exist.sourceforge.net/NS/exist">
82+
<text><![CDATA[
83+
import module namespace http = "http://expath.org/ns/http-client";
84+
let $r := http:send-request(
85+
<http:request method="POST" href="https://httpbin.org/post">
86+
<http:body media-type="text/plain">hello</http:body>
87+
</http:request>
88+
)
89+
return string($r[1]/@status)
90+
]]></text>
91+
</query>' "http://localhost:8080/exist/rest/db")
92+
echo "$result"
93+
echo "$result" | grep -q "200" || (echo "FAIL: expected status 200" && exit 1)
94+
95+
- name: Test http:send-request returns response headers
96+
run: |
97+
result=$(curl -sf -u admin: -H "Content-Type: application/xml" --data '
98+
<query xmlns="http://exist.sourceforge.net/NS/exist">
99+
<text><![CDATA[
100+
import module namespace http = "http://expath.org/ns/http-client";
101+
let $r := http:send-request(
102+
<http:request method="GET" href="https://httpbin.org/get"/>
103+
)
104+
return count($r[1]/http:header) > 0
105+
]]></text>
106+
</query>' "http://localhost:8080/exist/rest/db")
107+
echo "$result"
108+
echo "$result" | grep -q "true" || (echo "FAIL: expected headers" && exit 1)
109+
110+
- name: Test http:send-request XML response is parsed
111+
run: |
112+
result=$(curl -sf -u admin: -H "Content-Type: application/xml" --data '
113+
<query xmlns="http://exist.sourceforge.net/NS/exist">
114+
<text><![CDATA[
115+
import module namespace http = "http://expath.org/ns/http-client";
116+
let $r := http:send-request(
117+
<http:request method="GET" href="https://httpbin.org/xml"/>
118+
)
119+
return count($r[2]//slide) > 0
120+
]]></text>
121+
</query>' "http://localhost:8080/exist/rest/db")
122+
echo "$result"
123+
echo "$result" | grep -q "true" || (echo "FAIL: expected parsed XML" && exit 1)
124+
125+
- name: Upload XAR artifact
126+
uses: actions/upload-artifact@v4
127+
with:
128+
name: exist-http-client-xar
129+
path: target/exist-http-client-*.xar

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
.env
2+
.mvn/
23
target/
34
*.class
45
*.jar

‎LICENSE‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
GNU LESSER GENERAL PUBLIC LICENSE
2+
Version 2.1, February 1999
3+
4+
Copyright (C) 1991, 1999 Free Software Foundation, Inc.
5+
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
6+
Everyone is permitted to copy and distribute verbatim copies
7+
of this license document, but changing it is not allowed.
8+
9+
[This is the first released version of the Lesser GPL. It also counts
10+
as the successor of the GNU Library Public License, version 2, hence
11+
the version number 2.1.]

‎README.md‎

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
# EXPath HTTP Client Module for eXist-db
2+
3+
A **drop-in replacement** for eXist-db's bundled EXPath HTTP Client. Same namespace (`http://expath.org/ns/http-client`), same `http:send-request()` function, same `http:` prefix — but built on Java's `java.net.http.HttpClient` with zero external dependencies.
4+
5+
**One intentional behavior change:** text and JSON responses now return `xs:string` instead of `xs:base64Binary`. This is a bug fix per the [EXPath spec](http://expath.org/spec/http-client). Code using `util:binary-to-string()` still works (the function accepts strings), but the workaround is no longer needed.
6+
7+
## Compatibility
8+
9+
- **eXist-db 6.x:** The bundled module's `conf.xml` registration takes precedence over XAR-installed modules. The XAR works once the bundled registration is removed from `conf.xml` (planned for 7.0).
10+
- **eXist-db 7.0+:** The XAR takes over after the bundled module is removed from `conf.xml`.
11+
12+
## Install
13+
14+
Download the `.xar` from CI build artifacts and install with the eXist-db Package Manager or the `xst` CLI:
15+
16+
```bash
17+
xst package install exist-http-client-0.9.0-SNAPSHOT.xar
18+
```
19+
20+
## Function
21+
22+
The module provides a single function with three arities:
23+
24+
```xquery
25+
http:send-request($request as element(http:request)?) as item()+
26+
27+
http:send-request($request as element(http:request)?,
28+
$href as xs:string?) as item()+
29+
30+
http:send-request($request as element(http:request)?,
31+
$href as xs:string?,
32+
$bodies as item()*) as item()+
33+
```
34+
35+
**Module namespace:** `http://expath.org/ns/http-client`
36+
37+
### Basic GET
38+
39+
```xquery
40+
import module namespace http = "http://expath.org/ns/http-client";
41+
42+
let $response := http:send-request(
43+
<http:request method="GET" href="https://httpbin.org/get"/>
44+
)
45+
return string($response[1]/@status)
46+
```
47+
48+
### JSON API
49+
50+
With the new module, JSON responses are `xs:string` — `parse-json()` works directly:
51+
52+
```xquery
53+
import module namespace http = "http://expath.org/ns/http-client";
54+
55+
let $response := http:send-request(
56+
<http:request method="GET" href="https://httpbin.org/json"/>
57+
)
58+
return parse-json($response[2])?slideshow?title
59+
```
60+
61+
> **Note:** On eXist 6.x with the bundled module still active, JSON responses are `xs:base64Binary`. Use `parse-json(util:binary-to-string($response[2]))` for code that must work with both modules.
62+
63+
### POST with Body
64+
65+
```xquery
66+
import module namespace http = "http://expath.org/ns/http-client";
67+
68+
http:send-request(
69+
<http:request method="POST" href="https://httpbin.org/post">
70+
<http:header name="Accept" value="application/json"/>
71+
<http:body media-type="text/plain">Hello from XQuery</http:body>
72+
</http:request>
73+
)
74+
```
75+
76+
### POST with XML Body
77+
78+
```xquery
79+
import module namespace http = "http://expath.org/ns/http-client";
80+
81+
http:send-request(
82+
<http:request method="POST" href="https://httpbin.org/post">
83+
<http:body media-type="application/xml">
84+
<order><item id="1">Widget</item></order>
85+
</http:body>
86+
</http:request>
87+
)
88+
```
89+
90+
## Features
91+
92+
- **HTTP methods:** GET, POST, PUT, DELETE, HEAD, OPTIONS, PATCH
93+
- **Content-Type classification:** XML → `document-node()`, JSON/text → `xs:string`, binary → `xs:base64Binary`
94+
- **Request headers** and **body content** (text, JSON, XML, form data)
95+
- **HTTP Basic authentication** (`username`, `password`, `auth-method` attributes)
96+
- **Redirect following** (configurable via `follow-redirect`)
97+
- **Timeouts** with EXPath HC006 error
98+
- **gzip** response decompression
99+
- **Multipart** response handling
100+
- **`status-only`** and **`override-media-type`** attributes
101+
- **EXPath error codes** HC001–HC006
102+
103+
## Content-Type Handling
104+
105+
| Content-Type | Return Type |
106+
|---|---|
107+
| `text/xml`, `application/xml`, `*+xml` | `document-node()` (parsed XML) |
108+
| `text/html`, `application/xhtml+xml` | `document-node()` (parsed HTML) |
109+
| `text/*` (plain, css, csv) | `xs:string` |
110+
| `application/json`, `*+json` | `xs:string` |
111+
| `application/javascript` | `xs:string` |
112+
| Everything else | `xs:base64Binary` |
113+
114+
## What changed from the bundled module
115+
116+
This is a drop-in replacement. The namespace, prefix, and function signature are identical.
117+
118+
| | Bundled (`extensions/expath`) | This package (`exist-http-client`) |
119+
|---|---|---|
120+
| **Namespace** | `http://expath.org/ns/http-client` | `http://expath.org/ns/http-client` |
121+
| **Prefix** | `http:` | `http:` |
122+
| **Function** | `http:send-request()` (3 arities) | `http:send-request()` (3 arities) |
123+
| **Dependencies** | `http-client-java` + `tools-java` (Apache HttpClient) | Zero — `java.net.http.HttpClient` |
124+
| **JSON responses** | `xs:base64Binary` | `xs:string` (bug fix) |
125+
| **Text responses** | `xs:string` for `text/*`, `xs:base64Binary` for `application/json` | `xs:string` for all text types |
126+
127+
### Migration
128+
129+
Import statements require **no changes**:
130+
131+
```xquery
132+
import module namespace http = "http://expath.org/ns/http-client";
133+
```
134+
135+
All three arities of `http:send-request` are compatible. The only behavioral change is that JSON responses (and other text-like types) are now returned as `xs:string` instead of `xs:base64Binary`. Code using `util:binary-to-string()` still works — the function accepts strings.
136+
137+
## Build
138+
139+
```bash
140+
JAVA_HOME=/path/to/java-21 mvn clean package -DskipTests
141+
```
142+
143+
Run integration tests (requires exist-core 7.0.0-SNAPSHOT in your local Maven repo):
144+
145+
```bash
146+
mvn test -Pintegration-tests
147+
```
148+
149+
## License
150+
151+
[GNU Lesser General Public License v2.1](https://opensource.org/licenses/LGPL-2.1)

‎pom.xml‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313

1414
<groupId>org.exist-db</groupId>
1515
<artifactId>exist-http-client</artifactId>
16-
<version>1.0.0</version>
16+
<version>0.9.0-SNAPSHOT</version>
1717

1818
<name>EXPath HTTP Client Module</name>
1919
<description>Native EXPath HTTP Client Module for eXist-db using java.net.http.HttpClient</description>
@@ -45,8 +45,8 @@
4545
<exist.version>7.0.0-SNAPSHOT</exist.version>
4646

4747
<!-- used in the EXPath Package Descriptor -->
48-
<package-name>http://exist-db.org/apps/http-client</package-name>
49-
<package-abbrev>http-client</package-abbrev>
48+
<package-name>http://exist-db.org/pkg/http-client</package-name>
49+
<package-abbrev>exist-http-client</package-abbrev>
5050

5151
<http-client.module.namespace>http://expath.org/ns/http-client</http-client.module.namespace>
5252
<http-client.module.java.classname>org.exist.xquery.modules.httpclient.HttpClientModule</http-client.module.java.classname>
@@ -67,6 +67,13 @@
6767
<profile>
6868
<id>integration-tests</id>
6969
<dependencies>
70+
<!-- exist-core at test scope so its transitive deps (Saxon etc.) are available -->
71+
<dependency>
72+
<groupId>org.exist-db</groupId>
73+
<artifactId>exist-core</artifactId>
74+
<version>${exist.version}</version>
75+
<scope>test</scope>
76+
</dependency>
7077
<dependency>
7178
<groupId>org.exist-db</groupId>
7279
<artifactId>exist-core</artifactId>

0 commit comments

Comments
 (0)