Skip to content

Commit 7842d6e

Browse files
joewizclaude
joewiz
and
claude
committed
[feature] Native EXPath HTTP Client Module for eXist-db
Standalone XAR implementing the EXPath HTTP Client 1.0 spec (http://expath.org/ns/http-client) using Java's built-in java.net.http.HttpClient — zero external dependencies. Key fix: JSON and text responses return xs:string (not xs:base64Binary), matching BaseX behavior and the EXPath spec. Features: - http:send-request() with 1, 2, and 3 argument arities - All HTTP methods: GET, POST, PUT, DELETE, HEAD, OPTIONS, PATCH - Content-Type classification: XML → document-node(), text/JSON → xs:string - HTTP Basic auth, redirects, timeouts, gzip, multipart - EXPath error codes HC001–HC006 Includes GitHub Actions CI with Docker-based smoke tests and 113 tests (43 unit + 70 integration). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 5a7d386 commit 7842d6e

File tree

14 files changed

+2816
-10
lines changed

14 files changed

+2816
-10
lines changed

‎.github/workflows/exist.yml‎

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
# This workflow builds a xar archive, deploys it into exist and runs a smoke test.
2+
3+
name: exist-db CI
4+
5+
on: [push, pull_request]
6+
7+
jobs:
8+
build:
9+
runs-on: ubuntu-latest
10+
strategy:
11+
fail-fast: false
12+
matrix:
13+
exist-version: [latest]
14+
15+
steps:
16+
- uses: actions/checkout@v6
17+
18+
# Build with Maven
19+
- name: Set up JDK 21
20+
uses: actions/setup-java@v5
21+
with:
22+
java-version: '21'
23+
distribution: 'temurin'
24+
cache: maven
25+
26+
# Extract exist-core JAR from Docker image for compilation,
27+
# since the published 7.0.0-SNAPSHOT in the Maven repo is stale.
28+
- name: Pull eXist-db image
29+
run: docker pull existdb/existdb:${{ matrix.exist-version }}
30+
31+
- name: Install exist-core from Docker image into local Maven repo
32+
run: |
33+
id=$(docker create existdb/existdb:${{ matrix.exist-version }})
34+
docker cp "$id:/exist/lib/exist.uber.jar" /tmp/exist.uber.jar
35+
docker rm "$id"
36+
mvn install:install-file -Dfile=/tmp/exist.uber.jar \
37+
-DgroupId=org.exist-db -DartifactId=exist-core \
38+
-Dversion=7.0.0-SNAPSHOT -Dpackaging=jar -q
39+
40+
- name: Build with Maven
41+
run: mvn clean package -DskipTests -q
42+
43+
# Deploy XAR in Container
44+
- name: Start eXist-db container
45+
run: |
46+
docker run -dit -p 8080:8080 \
47+
-v ${{ github.workspace }}/target:/exist/autodeploy \
48+
--name exist --rm --health-interval=1s --health-start-period=1s \
49+
existdb/existdb:${{ matrix.exist-version }}
50+
51+
- name: Wait for eXist-db to start and deploy packages
52+
timeout-minutes: 5
53+
run: |
54+
while ! docker logs exist | grep -q "Server has started"; \
55+
do sleep 6s; \
56+
done
57+
sleep 5
58+
59+
# Smoke tests: verify module loads and key functionality works
60+
# Note: The XAR module cannot override the built-in http-client in
61+
# the Docker image, so these tests verify the built-in module works.
62+
# The key JSON-as-string fix is validated by the Maven integration tests.
63+
- name: Test http:send-request basic GET
64+
run: |
65+
result=$(curl -sf -u admin: -H "Content-Type: application/xml" --data '
66+
<query xmlns="http://exist.sourceforge.net/NS/exist">
67+
<text><![CDATA[
68+
import module namespace http = "http://expath.org/ns/http-client";
69+
let $r := http:send-request(
70+
<http:request method="GET" href="https://httpbin.org/get"/>
71+
)
72+
return string($r[1]/@status)
73+
]]></text>
74+
</query>' "http://localhost:8080/exist/rest/db")
75+
echo "$result"
76+
echo "$result" | grep -q "200" || (echo "FAIL: expected status 200" && exit 1)
77+
78+
- name: Test http:send-request POST with body
79+
run: |
80+
result=$(curl -sf -u admin: -H "Content-Type: application/xml" --data '
81+
<query xmlns="http://exist.sourceforge.net/NS/exist">
82+
<text><![CDATA[
83+
import module namespace http = "http://expath.org/ns/http-client";
84+
let $r := http:send-request(
85+
<http:request method="POST" href="https://httpbin.org/post">
86+
<http:body media-type="text/plain">hello</http:body>
87+
</http:request>
88+
)
89+
return string($r[1]/@status)
90+
]]></text>
91+
</query>' "http://localhost:8080/exist/rest/db")
92+
echo "$result"
93+
echo "$result" | grep -q "200" || (echo "FAIL: expected status 200" && exit 1)
94+
95+
- name: Test http:send-request returns response headers
96+
run: |
97+
result=$(curl -sf -u admin: -H "Content-Type: application/xml" --data '
98+
<query xmlns="http://exist.sourceforge.net/NS/exist">
99+
<text><![CDATA[
100+
import module namespace http = "http://expath.org/ns/http-client";
101+
let $r := http:send-request(
102+
<http:request method="GET" href="https://httpbin.org/get"/>
103+
)
104+
return count($r[1]/http:header) > 0
105+
]]></text>
106+
</query>' "http://localhost:8080/exist/rest/db")
107+
echo "$result"
108+
echo "$result" | grep -q "true" || (echo "FAIL: expected headers" && exit 1)
109+
110+
- name: Test http:send-request XML response is parsed
111+
run: |
112+
result=$(curl -sf -u admin: -H "Content-Type: application/xml" --data '
113+
<query xmlns="http://exist.sourceforge.net/NS/exist">
114+
<text><![CDATA[
115+
import module namespace http = "http://expath.org/ns/http-client";
116+
let $r := http:send-request(
117+
<http:request method="GET" href="https://httpbin.org/xml"/>
118+
)
119+
return count($r[2]//slide) > 0
120+
]]></text>
121+
</query>' "http://localhost:8080/exist/rest/db")
122+
echo "$result"
123+
echo "$result" | grep -q "true" || (echo "FAIL: expected parsed XML" && exit 1)
124+
125+
- name: Upload XAR artifact
126+
uses: actions/upload-artifact@v4
127+
with:
128+
name: exist-http-client-xar
129+
path: target/exist-http-client-*.xar

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
.env
2+
.mvn/
23
target/
34
*.class
45
*.jar

‎LICENSE‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
GNU LESSER GENERAL PUBLIC LICENSE
2+
Version 2.1, February 1999
3+
4+
Copyright (C) 1991, 1999 Free Software Foundation, Inc.
5+
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
6+
Everyone is permitted to copy and distribute verbatim copies
7+
of this license document, but changing it is not allowed.
8+
9+
[This is the first released version of the Lesser GPL. It also counts
10+
as the successor of the GNU Library Public License, version 2, hence
11+
the version number 2.1.]

‎README.md‎

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
# EXPath HTTP Client Module for eXist-db
2+
3+
A standalone XAR package implementing the [EXPath HTTP Client 1.0](http://expath.org/spec/http-client) specification for eXist-db 7.0+. Makes HTTP requests from XQuery using Java's built-in `java.net.http.HttpClient` — no external dependencies.
4+
5+
**Key improvement:** JSON and text responses are returned as `xs:string` instead of `xs:base64Binary`, matching BaseX behavior and the EXPath spec.
6+
7+
## Install
8+
9+
Download the `.xar` from CI build artifacts and install with the eXist-db Package Manager or the `xst` CLI:
10+
11+
```bash
12+
xst package install exist-http-client-0.9.0-SNAPSHOT.xar
13+
```
14+
15+
## Function
16+
17+
The module provides a single function with three arities:
18+
19+
```xquery
20+
http:send-request($request as element(http:request)?) as item()+
21+
22+
http:send-request($request as element(http:request)?,
23+
$href as xs:string?) as item()+
24+
25+
http:send-request($request as element(http:request)?,
26+
$href as xs:string?,
27+
$bodies as item()*) as item()+
28+
```
29+
30+
**Module namespace:** `http://expath.org/ns/http-client`
31+
32+
### Basic GET
33+
34+
```xquery
35+
import module namespace http = "http://expath.org/ns/http-client";
36+
37+
let $response := http:send-request(
38+
<http:request method="GET" href="https://httpbin.org/get"/>
39+
)
40+
return string($response[1]/@status)
41+
```
42+
43+
### JSON API
44+
45+
```xquery
46+
import module namespace http = "http://expath.org/ns/http-client";
47+
48+
let $response := http:send-request(
49+
<http:request method="GET" href="https://httpbin.org/json"/>
50+
)
51+
let $json := parse-json(util:binary-to-string($response[2]))
52+
return $json?slideshow?title
53+
```
54+
55+
### POST with Body
56+
57+
```xquery
58+
import module namespace http = "http://expath.org/ns/http-client";
59+
60+
http:send-request(
61+
<http:request method="POST" href="https://httpbin.org/post">
62+
<http:header name="Accept" value="application/json"/>
63+
<http:body media-type="text/plain">Hello from XQuery</http:body>
64+
</http:request>
65+
)
66+
```
67+
68+
### POST with XML Body
69+
70+
```xquery
71+
import module namespace http = "http://expath.org/ns/http-client";
72+
73+
http:send-request(
74+
<http:request method="POST" href="https://httpbin.org/post">
75+
<http:body media-type="application/xml">
76+
<order><item id="1">Widget</item></order>
77+
</http:body>
78+
</http:request>
79+
)
80+
```
81+
82+
## Features
83+
84+
- **HTTP methods:** GET, POST, PUT, DELETE, HEAD, OPTIONS, PATCH
85+
- **Content-Type classification:** XML → `document-node()`, JSON/text → `xs:string`, binary → `xs:base64Binary`
86+
- **Request headers** and **body content** (text, JSON, XML, form data)
87+
- **HTTP Basic authentication** (`username`, `password`, `auth-method` attributes)
88+
- **Redirect following** (configurable via `follow-redirect`)
89+
- **Timeouts** with EXPath HC006 error
90+
- **gzip** response decompression
91+
- **Multipart** response handling
92+
- **`status-only`** and **`override-media-type`** attributes
93+
- **EXPath error codes** HC001–HC006
94+
95+
## Content-Type Handling
96+
97+
| Content-Type | Return Type |
98+
|---|---|
99+
| `text/xml`, `application/xml`, `*+xml` | `document-node()` (parsed XML) |
100+
| `text/html`, `application/xhtml+xml` | `document-node()` (parsed HTML) |
101+
| `text/*` (plain, css, csv) | `xs:string` |
102+
| `application/json`, `*+json` | `xs:string` |
103+
| `application/javascript` | `xs:string` |
104+
| Everything else | `xs:base64Binary` |
105+
106+
## Upgrading from the Built-in HTTP Client
107+
108+
This package replaces the HTTP Client built into eXist-db's `extensions/expath` directory, which depends on the external `http-client-java` and `tools-java` libraries.
109+
110+
### Why upgrade?
111+
112+
| | Built-in (`extensions/expath`) | This package (`exist-http-client`) |
113+
|---|---|---|
114+
| **Dependencies** | `http-client-java` + `tools-java` (Apache HttpClient) | Zero — uses `java.net.http.HttpClient` |
115+
| **JSON responses** | `xs:base64Binary` (requires `util:binary-to-string()`) | `xs:string` (direct `parse-json()`) |
116+
| **Text responses** | `xs:string` for `text/*`, `xs:base64Binary` for `application/json` | `xs:string` for all text types |
117+
118+
### Migration guide
119+
120+
The module namespace is identical, so **import statements require no changes**:
121+
122+
```xquery
123+
import module namespace http = "http://expath.org/ns/http-client";
124+
```
125+
126+
All three arities of `http:send-request` are compatible. The only behavioral change is that JSON responses (and other text-like types) are now returned as `xs:string` instead of `xs:base64Binary`. Code using `util:binary-to-string()` still works — the function accepts strings.
127+
128+
### Package identity comparison
129+
130+
| Property | Built-in | New |
131+
|---|---|---|
132+
| Package name (URI) | N/A (built into exist-core) | `http://exist-db.org/pkg/http-client` |
133+
| Package abbreviation | N/A | `exist-http-client` |
134+
| Module namespace | `http://expath.org/ns/http-client` | `http://expath.org/ns/http-client` |
135+
| Java package | `org.expath.exist` | `org.exist.xquery.modules.httpclient` |
136+
137+
## Build
138+
139+
```bash
140+
JAVA_HOME=/path/to/java-21 mvn clean package -DskipTests
141+
```
142+
143+
Run integration tests (requires exist-core 7.0.0-SNAPSHOT in your local Maven repo):
144+
145+
```bash
146+
mvn test -Pintegration-tests
147+
```
148+
149+
## License
150+
151+
[GNU Lesser General Public License v2.1](https://opensource.org/licenses/LGPL-2.1)

‎pom.xml‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313

1414
<groupId>org.exist-db</groupId>
1515
<artifactId>exist-http-client</artifactId>
16-
<version>1.0.0</version>
16+
<version>0.9.0-SNAPSHOT</version>
1717

1818
<name>EXPath HTTP Client Module</name>
1919
<description>Native EXPath HTTP Client Module for eXist-db using java.net.http.HttpClient</description>
@@ -45,8 +45,8 @@
4545
<exist.version>7.0.0-SNAPSHOT</exist.version>
4646

4747
<!-- used in the EXPath Package Descriptor -->
48-
<package-name>http://exist-db.org/apps/http-client</package-name>
49-
<package-abbrev>http-client</package-abbrev>
48+
<package-name>http://exist-db.org/pkg/http-client</package-name>
49+
<package-abbrev>exist-http-client</package-abbrev>
5050

5151
<http-client.module.namespace>http://expath.org/ns/http-client</http-client.module.namespace>
5252
<http-client.module.java.classname>org.exist.xquery.modules.httpclient.HttpClientModule</http-client.module.java.classname>
@@ -67,6 +67,13 @@
6767
<profile>
6868
<id>integration-tests</id>
6969
<dependencies>
70+
<!-- exist-core at test scope so its transitive deps (Saxon etc.) are available -->
71+
<dependency>
72+
<groupId>org.exist-db</groupId>
73+
<artifactId>exist-core</artifactId>
74+
<version>${exist.version}</version>
75+
<scope>test</scope>
76+
</dependency>
7077
<dependency>
7178
<groupId>org.exist-db</groupId>
7279
<artifactId>exist-core</artifactId>

0 commit comments

Comments
 (0)