Skip to content

Commit d63e43b

Browse files
james-benchlopcode
authored andcommitted
Add operationalisation section to README, guidance on jemalloc
1 parent d68fa4f commit d63e43b

File tree

6 files changed

+81
-2
lines changed

6 files changed

+81
-2
lines changed

CONTRIBUTING.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,9 @@ library is already used by some larger businesses (which is cool!), and I care a
1515
Please discuss any bigger changes with me **before** submitting a Pull Request - I can help you refine your idea better
1616
that way, and I don't want to waste anybody's time: [Discussions](https://github.com/lopcode/vips-ffm/discussions).
1717

18+
As part of a pull request I will probably edit commits on the branch, and will squash them down, but will be careful to
19+
retain your contributor metadata so you're named appropriately as a contributor on GitHub.
20+
1821
I haven't currently defined a code of conduct for this project specifically, but please refer to the CoC [in libvips](https://github.com/libvips/libvips/blob/master/CODE_OF_CONDUCT.md)
1922
for guidance on expected behaviour.
2023

README.md

Lines changed: 38 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -193,6 +193,43 @@ like Android where it's hard to set the system library path), you can do so usin
193193
* glib: `vipsffm.libpath.glib.override`
194194
* gobject: `vipsffm.libpath.gobject.override`
195195

196+
## Operationalisation
197+
198+
libvips maintain [a checklist](https://www.libvips.org/API/8.17/developer-checklist.html#linux-memory-allocator) of
199+
things to be aware of when using the library. Of particular note for vips-ffm is memory usage - especially if the library
200+
is used for long-running application (like a server).
201+
202+
#### Operation Cache
203+
204+
At the time of writing, libvips maintains a cache of the 100 most recent operations ([see docs](https://www.libvips.org/API/8.17/how-it-works.html#operation-cache)).
205+
If running an image proxy, or something that processes lots of different images, you won't see any benefit, and can disable it:
206+
207+
```java
208+
Vips.init();
209+
Vips.disableOperationCache();
210+
```
211+
212+
### Memory Allocation
213+
214+
On glibc-based Linux systems (e.g. Debian, Red Hat), the default memory allocator performs poorly for long-running,
215+
multithreaded processes with frequent small allocations. Using an alternative allocator like jemalloc can reduce the
216+
off-heap footprint of the JVM when using libvips.
217+
218+
Note that the jemalloc project is going through [some turbulence](https://jasone.github.io/2025/06/12/jemalloc-postmortem/)
219+
at the moment. Facebook have [forked it](https://github.com/facebook/jemalloc), though its maintenance status is
220+
currently unknown.
221+
222+
An example of using jemalloc on Ubuntu:
223+
1. Install jemalloc
224+
```sh
225+
apt install jemalloc-dev
226+
```
227+
2. Set the `LD_PRELOAD` environment variable before running your application.
228+
```sh
229+
export LD_PRELOAD=/usr/local/lib/libjemalloc.so
230+
java -jar ...
231+
```
232+
196233
## Project goals
197234

198235
Ideas and suggestions are welcome, but please make sure they fit in to these goals, or you have a good argument about
@@ -224,4 +261,4 @@ Thank you for being enthusiastic about the project!
224261
* And only after a GitHub Release is made
225262
* Run `./publish_release_to_maven_central.sh <version matching github release version, including v prefix>`
226263

227-
[1]: https://docs.oracle.com/en/java/javase/23/core/memory-segments-and-arenas.html
264+
[1]: https://docs.oracle.com/en/java/javase/23/core/memory-segments-and-arenas.html

core/src/main/java/app/photofox/vipsffm/Vips.java

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,9 @@ public static void init(boolean allowUntrusted, boolean detectLeaks) {
1717
VipsHelper.leak_set(detectLeaks);
1818
}
1919

20+
/// Provides a scoped arena to provide a memory boundary for running libvips operations
21+
///
22+
/// After the scope has ended, any memory allocated whilst using libvips within it will be freed
2023
public static void run(VipsRunnable runnable) {
2124
try (var arena = Arena.ofConfined()) {
2225
runnable.run(arena);
@@ -26,4 +29,22 @@ public static void run(VipsRunnable runnable) {
2629
public static void shutdown() {
2730
VipsHelper.shutdown();
2831
}
32+
33+
/// Permits untrusted operations, such as loading PDFs
34+
///
35+
/// vips-ffm blocks these by default - see the [libvips docs](https://www.libvips.org/API/8.17/func.block_untrusted_set.html)
36+
/// for guidance
37+
public static void allowUntrustedOperations() {
38+
VipsHelper.block_untrusted_set(false);
39+
}
40+
41+
/// Disables the libvips operations cache
42+
///
43+
/// At the time of writing libvips caches 100 operations by default, which might not be useful in long-running
44+
/// applications (like servers).
45+
///
46+
/// See also: [libvips docs](https://www.libvips.org/API/8.17/how-it-works.html#operation-cache)
47+
public static void disableOperationCache() {
48+
VipsHelper.cache_set_max(0);
49+
}
2950
}
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
FROM ubuntu:24.04
2+
ENV JAVA_HOME=/opt/java/openjdk
3+
COPY --from=eclipse-temurin:23 $JAVA_HOME $JAVA_HOME
4+
ENV PATH="${JAVA_HOME}/bin:${PATH}"
5+
6+
COPY sample /opt/sample
7+
COPY run_samples.sh /opt/run_samples.sh
8+
9+
RUN apt update && apt install libvips-dev jemalloc-dev -y
10+
RUN vips --version
11+
12+
WORKDIR /opt
13+
CMD ./run_samples.sh

run_docker_tests.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ echo "building samples..."
77
echo "running docker tests..."
88
WORKSPACE_DIR="$PWD"
99

10-
docker_tests=("debian-12" "ubuntu-2404")
10+
docker_tests=("debian-12" "ubuntu-2404" "ubuntu-2404-jemalloc")
1111
for docker_test in "${docker_tests[@]}"; do
1212
echo "testing \"$docker_test\""
1313
pushd "docker_tests/$docker_test"

run_samples.sh

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,11 @@ if [[ "$OSTYPE" == "darwin"* ]]; then
1515
export JAVA_PATH_OPTS="-Dvipsffm.libpath.vips.override=/opt/homebrew/lib/libvips.dylib"
1616
fi
1717

18+
if test -f /usr/local/lib/libjemalloc.so; then
19+
echo "found jemalloc - using it"
20+
export LD_PRELOAD=$LD_PRELOAD;/usr/local/lib/libjemalloc.so
21+
fi
22+
1823
echo "running samples..."
1924
java $JAVA_PATH_OPTS -jar sample/build/libs/sample-all.jar 2>&1 | tee sample_output.log
2025

0 commit comments

Comments
 (0)