Skip to content
Merged
Show file tree
Hide file tree
Changes from 81 commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
17f0cf7
Started initial implementation
DavyLandman Nov 20, 2023
bb6c56f
First working test of directory watcher
DavyLandman Dec 25, 2023
860ac75
Improved tests by removing sleeps
DavyLandman Dec 25, 2023
d3a47b4
Implemented initial recursive support
DavyLandman Dec 26, 2023
e6fc78a
Simplified recursive watcher
DavyLandman Dec 27, 2023
4313b3e
Simplified JDKPoller singleton and avoided extra thread
DavyLandman Dec 27, 2023
82615a3
Added recursive tests
DavyLandman Feb 24, 2024
a0ea7db
Implemented virtual events for new nested directories and files
DavyLandman Feb 24, 2024
a12b50e
Got checker framework to properly work
DavyLandman Feb 24, 2024
d58eff4
Added initial support for overflow event
DavyLandman Feb 25, 2024
4f22795
Rewrote tests to start with a fresh directory
DavyLandman Feb 26, 2024
4b3c167
Added full event support to the watcher interface
DavyLandman Feb 26, 2024
2194412
Making sure not to leak watches
DavyLandman Feb 27, 2024
515a56a
Added overflow support for the recursive watcher
DavyLandman Mar 17, 2024
6812d78
Refactoring the functions to make them smaller
DavyLandman Mar 17, 2024
aff2b72
Rewrote recursive watcher to report to relative root
DavyLandman Mar 30, 2024
829e506
Execute syntatic events in the correct order
DavyLandman Mar 31, 2024
c0a7f3f
Added torture tests to make sure everything works, even in very busy IO
DavyLandman Jul 19, 2024
0858ca2
Cleanup of code
DavyLandman Jul 19, 2024
46acd38
Bundle registered watches to avoid duplicate FS watches
DavyLandman Sep 5, 2024
de3ea33
Implemented single file support
DavyLandman Sep 5, 2024
939e57b
Improve how long we wait for failure condition
DavyLandman Sep 7, 2024
03ab088
Made the poller loop a bit more resistant to misbehaving callbacks
DavyLandman Sep 12, 2024
54d4d88
Added test for delete behavior
DavyLandman Sep 13, 2024
8cc2c39
Wrote javadocs
DavyLandman Sep 13, 2024
dd18b9b
Improved handling of deletes
DavyLandman Sep 13, 2024
6a41f57
[ci] starting with ci
DavyLandman Sep 13, 2024
1d4ced3
[ci] also run checker-framework
DavyLandman Sep 13, 2024
0ccca4c
Fixed broken tests
DavyLandman Sep 13, 2024
800d1b0
[ci] also test under different jdks
DavyLandman Sep 14, 2024
9cf1997
[ci] make sure deletes are also torture tested
DavyLandman Sep 14, 2024
e028181
Fixed null errors found by CF
DavyLandman Sep 14, 2024
174f4d6
Slight tweak to the documentation
DavyLandman Sep 14, 2024
fde2d6e
Improving the torture test to wait a bit long before events have stab…
DavyLandman Sep 14, 2024
2bf0c75
Cleanup of test api usage
DavyLandman Sep 14, 2024
66d588f
Trying to make the test run better on ci
DavyLandman Sep 14, 2024
bc777ef
Fixing broken test
DavyLandman Sep 16, 2024
c8d4fd5
Trying to make a better stabilizing torture test detection
DavyLandman Sep 16, 2024
524d690
Change smoke test to wait for appropriate time
DavyLandman Sep 16, 2024
d8170f8
Trying to make give the tests a bit more time
DavyLandman Sep 16, 2024
b26a1a3
Improved the tests by splitting up the 2 torture tests
DavyLandman Sep 16, 2024
6698332
Lets make sure sync events are generated more correctly
DavyLandman Sep 16, 2024
3157d62
Better print for torture test
DavyLandman Sep 16, 2024
44ed90c
Trying to unbreak the test suite
DavyLandman Sep 16, 2024
51a6930
Only start watch after the directory has been processed, to avoid a r…
DavyLandman Sep 16, 2024
c9cee56
Disable delete test, which will mostly fail, as it can be a race with…
DavyLandman Sep 16, 2024
424ad48
Fixed race on new directory watches in recursive watch in a better wa…
DavyLandman Sep 16, 2024
dae3f53
Trying to get the test to be faster
DavyLandman Sep 16, 2024
f282d49
Adding test to see if on linux the race breaks it
DavyLandman Sep 16, 2024
a5a76d6
Checking that we log what is thrown
DavyLandman Sep 16, 2024
6e9ca1b
Trying to give windows a bit more time to work through the tests
DavyLandman Sep 16, 2024
1666cf3
Trying to break the deadlock on linux
DavyLandman Sep 16, 2024
f97d8e0
Rewrote the register path to reduce the time in a limited thread spacve
DavyLandman Sep 17, 2024
7b0bfbc
Added a catchup-loop to make sure we're not missing events in the rec…
DavyLandman Sep 17, 2024
2090e35
Removed extra set that was just a premature optimization
DavyLandman Sep 17, 2024
51cfb9e
Trying to really wait for all events to have stabilized
DavyLandman Sep 18, 2024
5ca0fe3
Longer wait for stabilization
DavyLandman Sep 18, 2024
1fbc69b
Running sync in a background thread to get initial events faster
DavyLandman Sep 18, 2024
49da0a9
Trying my best to stabilize the torture tests
DavyLandman Sep 18, 2024
bfe3d74
Using await to just wait for the condition we care about
DavyLandman Sep 18, 2024
38f9b9d
Increased timeouts on windows
DavyLandman Sep 18, 2024
eb388e8
Increased timeouts on windows
DavyLandman Sep 18, 2024
14c501d
Increased timeouts on windows
DavyLandman Sep 18, 2024
bcc19a6
Moved towards native file watch support for windows and rewrote the A…
DavyLandman Sep 24, 2024
0d5a969
Nullable fix
DavyLandman Sep 24, 2024
6eca5ff
Increase the pressure a bit for the torture test
DavyLandman Sep 24, 2024
3a106ff
Do not print the exception everytime
DavyLandman Sep 24, 2024
e4f87a9
Applied all the comments from @sungshik
DavyLandman Sep 25, 2024
538b045
Added a bit more comment
DavyLandman Sep 25, 2024
39a8b74
Added registration test
DavyLandman Oct 1, 2024
e1c7186
Extra test around registration
DavyLandman Oct 1, 2024
096a6f9
Renamed enums and processes review comments
DavyLandman Oct 14, 2024
de22a03
Working on the fix for the race Sung found
DavyLandman Oct 14, 2024
7006548
Removed races around registering and unregistering by adding a big ol…
DavyLandman Oct 14, 2024
80628ab
Tweaked the test a bit
DavyLandman Oct 14, 2024
d6b1f28
Improved torture tests
DavyLandman Oct 14, 2024
4f8cf59
Fixed bug around fast registration and unregistration in JDKPoller
DavyLandman Oct 14, 2024
85efd25
Fixed test that would be triggered by events spread over different wo…
DavyLandman Oct 14, 2024
5bcf1ce
Remove JDK watch only after a delay, just to avoid hammering the regi…
DavyLandman Oct 14, 2024
ac9c73e
Extended readme with example and description
DavyLandman Oct 14, 2024
69870e2
Slight wording fix
DavyLandman Oct 14, 2024
9095927
Apply suggestions from code review
DavyLandman Oct 15, 2024
fb01d39
Improve documentation of `PATH_ONLY` watch scopes
sungshik Oct 18, 2024
23b239b
Refine test (deleteOfFileInDirectoryShouldBeVisible)
sungshik Oct 18, 2024
88170a2
Tweaked documentation a bit
DavyLandman Nov 11, 2024
87c63e6
Moved JDK classes to its own namespace
DavyLandman Nov 11, 2024
d392f1c
Improved impl directory
DavyLandman Nov 11, 2024
982ce65
Make sure to have a dedicated interface, to allow for future addition…
DavyLandman Nov 11, 2024
d1b6e9e
Missed refactor of test
DavyLandman Nov 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Editor configuration, see http://editorconfig.org
root = true

[*]
charset = utf-8
indent_style = space
indent_size = 2
insert_final_newline = true
trim_trailing_whitespace = true
max_line_length = 80

[*.sh]
end_of_line = lf

[*.java]
indent_size = 4
max_line_length = 120
45 changes: 45 additions & 0 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Build and Test
on:
push:
branches:
- main
tags:
- 'v[0-9]+.*'
pull_request:
branches:
- main

jobs:
test:
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
jdk: [11, 17, 21]
fail-fast: false
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- name: Set up JDK
uses: actions/setup-java@v4
with:
java-version: ${{ matrix.jdk }}
distribution: 'temurin'
cache: 'maven'

- name: test
run: mvn -B clean test
env:
DELAY_FACTOR: 3

checker-framework:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up JDK 11
uses: actions/setup-java@v4
with:
java-version: 11
distribution: 'temurin'
cache: 'maven'

- run: mvn -B -Pchecker-framework clean compile
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,5 @@
# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
hs_err_pid*
replay_pid*

/target
3 changes: 3 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"java.configuration.updateBuildConfiguration": "automatic"
}
53 changes: 52 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,59 @@
# java-watch
a java file watcher that works across platforms and supports recursion and single file watches
a java file watcher that works across platforms and supports recursion, single file watches, and tries to make sure no events are missed.

## Features

Currently working features in java-watch:

- Recursive watches, even if platform doesn't support it natively.
- Recursive watches also work inside directories created after the watch started
- Even in case of overflow you will get notifications of **new** directories (and it's recursive files), modification events will however not be simulated
- Single file watches
- Multiple watches for the same directory are merged to avoid overloading the kernel
- Events are process on a worker pool, which you can customize.

Future features:

- Avoid poll based watcher in macOS/OSX that only detects changes every 2 seconds
- Support file watches natively in linux
- Monitor only specific events (such as only CREATES)

## Usage

Import dependency in pom.xml:

```xml
<dependency>
<groupId>engineering.swat</groupId>
<artifactId>java-watch</artifactId>
<version>${java-watch-version}</version>
</dependency>
```

Start using java-watch:

```java
var directory = Path.of("tmp", "test-dir");
var watcherSetup = Watcher.watch(directory, WatchScope.PATH_AND_CHILDREN)
.withExecutor(Executors.newCachedThreadPool()) // optionally configure a custom thread pool
.onEvent(watchEvent -> {
System.err.println(watchEvent);
});

try(var active = watcherSetup.start()) {
System.out.println("Monitoring files, press any key to stop");
System.in.read();
}
// after active.close(), the watch is stopped and
// no new events will be scheduled on the threadpool
```

## Related work

Before starting this library, we wanted to use existing libraries, but they all lacked proper support for recursive file watches or lacked configurability. This library now has a growing collection of tests and a small API that should allow for future improvements without breaking compatibility.

The following section describes the related work research on the libraries and underlying limitations.

After reading the documentation of the following discussion on file system watches:

- [Paul Millr's nodejs chokidar](https://github.com/paulmillr/chokidar)
Expand Down
192 changes: 192 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,192 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>engineering.swat</groupId>
<artifactId>java-watch</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>

<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<checkerframework.version>3.42.0</checkerframework.version>
<junit.version>5.10.2</junit.version>
<log4j.version>2.23.0</log4j.version>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
</properties>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.10.1</version>
<configuration>
<release>11</release>
<compilerArgument>-parameters</compilerArgument>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<version>3.2.1</version>
<executions>
<execution>
<id>attach-sources</id>
<goals>
<goal>jar</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-release-plugin</artifactId>
<configuration>
<tagNameFormat>v@{project.version}</tagNameFormat>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>3.4.0</version>
<configuration>
<additionalparam>-Xdoclint:none</additionalparam>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.8</version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>report</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

<dependencies>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-params</artifactId>
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.awaitility</groupId>
<artifactId>awaitility</artifactId>
<version>4.2.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.checkerframework</groupId>
<artifactId>checker-qual</artifactId>
<version>${checkerframework.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>${log4j.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>${log4j.version}</version>
</dependency>
</dependencies>

<profiles>
<profile> <!-- run with: mvn clean compile -P checker-framework -->
<id>checker-framework</id>
<build>
<plugins>
<plugin>
<!-- This plugin will set properties values using dependency information -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>properties</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.10.1</version>
<configuration>
<fork>true</fork>
<release>11</release>
<compilerArgs>
<arg>-Xmaxerrs</arg>
<arg>10000</arg>
<arg>-Xmaxwarns</arg>
<arg>10000</arg>
<!-- we have to open up the jdk modules for checker framework to work. note if we get a modules.java this has to change-->
<arg>-J--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED</arg>
<arg>-J--add-exports=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED</arg>
<arg>-J--add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED</arg>
<arg>-J--add-exports=jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED</arg>
<arg>-J--add-exports=jdk.compiler/com.sun.tools.javac.model=ALL-UNNAMED</arg>
<arg>-J--add-exports=jdk.compiler/com.sun.tools.javac.processing=ALL-UNNAMED</arg>
<arg>-J--add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED</arg>
<arg>-J--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED</arg>
<arg>-J--add-opens=jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED</arg>
<arg>-Astubs=src/main/checkerframework</arg>
</compilerArgs>
<annotationProcessorPaths>
<path>
<groupId>org.checkerframework</groupId>
<artifactId>checker</artifactId>
<version>${checkerframework.version}</version>
</path>
</annotationProcessorPaths>
<annotationProcessors>
<!-- Add all the checkers you want to enable here -->
<annotationProcessor>
org.checkerframework.checker.nullness.NullnessChecker
</annotationProcessor>
</annotationProcessors>
</configuration>
</plugin>
</plugins>
</build>

<dependencies>
<dependency>
<groupId>org.checkerframework</groupId>
<artifactId>checker</artifactId>
<version>${checkerframework.version}</version>
<scope>provided</scope>
</dependency>
</dependencies>

</profile>
</profiles>
</project>
Loading