Testing

Netatalk Testing

At present, Netatalk has two test automation suites, both written in pure C:

• afpd component integration tests
• afptest (testsuite) system integration test suite

The former can be run stateless through the Meson test runner.

The latter requires a test environment with a correctly configured and running netatalk instance.

The component integration test code is located in test/afpd, and the system integration tests in test/testsuite.

Testing policy

Any changeset to the Netatalk AFP server components should include updates to the test suites, when needed. Any novel test failures should be fixed, and new AFP functionality should be covered by new tests.

It is strongly recommended to run the tests locally before submitting a PR.

Run the afpd tests

In order to run the tests, build Netatalk with tests enabled, then run the meson test target from within the build directory:

meson setup build -Dwith-tests=true
meson compile -C build
cd build
meson test

Passing results should look something like this:

ninja: Entering directory `/home/atalk/devel/netatalk/build'
ninja: no work to do.
1/2 afpd integration tests - setup        OK              0.42s
2/2 afpd integration tests - run          OK              1.98s

Ok:                 2   
Expected Fail:      0   
Fail:               0   
Unexpected Pass:    0   
Skipped:            0   
Timeout:            0   

Full log written to /home/atalk/devel/netatalk/build/meson-logs/testlog.txt

Note that the suite contains multiple tests. See test.c for the full list of assertions.

Run the spectest test suite

The spectest test suite ensures that there are no breakages in AFP specification conformance. The test runner binary is called afp_spectest.

First, build netatalk with the testsuite enabled. Installation may need root privileges.

meson setup build -Dwith-testsuite=true
meson compile -C build
meson install -C build

Stop netatalk if running, and then configure the environment.

Debian env prep

The following steps were run on Debian Linux, but may work on other Linux flavors. This is a modified script from what's running with autopkgtest in Debian CI. We only set up the requirements for the tier 1 spectests, namely those that can run remotely, without test suite access to the host file system. Root privileges are required to execute these commands.

groupadd -f afpusers
useradd -G afpusers atalk1
useradd -G afpusers atalk2
echo "atalk1:afpafp" | chpasswd
echo "atalk2:afpafp" | chpasswd
mkdir -p /tmp/afptest1
mkdir -p /tmp/afptest2
chown atalk1:afpusers /tmp/afptest1
chown atalk1:afpusers /tmp/afptest2
chmod 2775 /tmp/afptest1
chmod 2775 /tmp/afptest2

FreeBSD env prep

pw groupadd afpusers
pw useradd atalk1 -m -G afpusers
pw useradd atalk2 -m -G afpusers
passwd atalk1
passwd atalk2
mkdir -p /tmp/afptest1
mkdir -p /tmp/afptest2
chown atalk1:afpusers /tmp/afptest1
chown atalk1:afpusers /tmp/afptest2
chmod 2775 /tmp/afptest1
chmod 2775 /tmp/afptest2

OpenBSD env prep

The following steps were run on OpenBSD, and may work on other BSD flavors.

groupadd afpusers
useradd -m -G afpusers atalk1
useradd -m -G afpusers atalk2
passwd atalk1
passwd atalk2
mkdir -p /tmp/afptest1
mkdir -p /tmp/afptest2
chown atalk1:afpusers /tmp/afptest1
chown atalk1:afpusers /tmp/afptest2
chmod 2775 /tmp/afptest1
chmod 2775 /tmp/afptest2

Run tests

Modify the netatalk config files with test users, volumes, and UAMs.

cat <<AFP > /usr/local/etc/afp.conf
[Global]
uam list = uams_clrtxt.so uams_guest.so

[test1]
appledouble = ea
path = /tmp/afptest1
valid users = @afpusers

[test2]
appledouble = ea
path = /tmp/afptest2
valid users = @afpusers
AFP
cat <<EXT > /usr/local/etc/extmap.conf
.         "????"  "????"      Unix Binary                    Unix                      application/octet-stream
.doc      "WDBN"  "MSWD"      Word Document                  Microsoft Word            application/msword
.pdf      "PDF "  "CARO"      Portable Document Format       Acrobat Reader            application/pdf
EXT

Restart netatalk and run the tests. The IP address or domain name you pass to -h is the netatalk host to test against. It can be a remote machine, or the localhost.

When run against a remote host, only the so-called tier 1 tests are executed.

afp_spectest -h 192.168.0.2 -u atalk1 -d atalk2 -w afpafp -s test1 -S test2

When run on localhost, and you pass the local file system path to the primary shared volume with the -c parameter, the so-called tier 2 tests are executed as well. These are tests are involve making local file system modifications to set up certain test preconditions.

afp_spectest -h localhost -u atalk1 -d atalk2 -w afpafp -s test1 -S test2 -c /tmp/afptest1

For additional instructions, see the afptest man page.

The majority of the spec tests are also running in the GitHub CI workflow, the only exceptions being a handful that require special setup that is cumbersome to replicate or take too long time to run.

Inspect implemented test functions

To get an authoritative list of test methods that are actually implemented, e.g. not commented out, you can inspect the compiled test binaries.

Build the netatalk source code with testsuite enabled, then run the following commands.

cd build/test/testsuite
nm afp_logintest afp_spectest | cut -d " " -f 3 | egrep "^test[[:digit:]]+" | sort -n -k 1.5

This should give you an exhaustive list of tests.

afp_lantest

afp_lantest is a comprehensive AFP (Apple Filing Protocol) protocol performance testing tool designed to benchmark various aspects of AFP servers, including directory cache performance in Netatalk.

IO Monitoring

The afp_lantest IO Monitoring capability is an analysis mechanism to enable Netatalk developers to validate end-to-end performance across the Netatalk codebase by quantifying AFP Client Operations -to- Netatalk Server Storage IO Operations.

IO Monitoring requires the afp_lantest client to be run on the same host as the Netatalk server, so it can watch the afpd server processes disk IO via the Linux proc virtual filesystem.

NB; Distributions running systemd do not allow existing /proc filesystem to be remounted with the required permissions, therefore a secondary proc mount is required at /proc_io by default.

Example - Run as root to allow 'root' GID access to /proc_io for other UIDs (when running afp_lantest as user other than root, update gid accordingly): mkdir -p /proc_io && mount -t proc -o hidepid=0,gid=0 proc /proc_io

IO Monitoring also works with the Netatalk test Docker container (see testsuite_alp.Dockerfile), to enable simple performance and regression testing.

DSI Quantum Size Impact

The DSI (Data Stream Interface) protocol default quantum size is 1MB (1048576 bytes), which affects how many read/write operations are performed for large data transfers. Ie, for a 1MB file, with the default 1MB quantum, only 1 AFP operation and 1 disk IO operation is required to read or write 1MB.

AFP Operation Counts Analysis

The number of AFP (Apple Filing Protocol) operations performed by each test in lantest.c (afp_lantest).

The AFP operation counts are measured between starttimer() and stoptimer() calls to understand the actual workload each test generates.

1. TEST_OPENSTATREAD - Open, stat and read 512 bytes from 1000 files

Total AFP Operations: 8,000

For each of the 1000 files, the test performs:

• 1 is_there() call (FPGetFileDirParams)
• 2 additional FPGetFileDirParams calls
• 1 FPOpenFork operation
• 2 FPGetForkParam operations
• 1 FPRead operation (512 bytes)
• 1 FPCloseFork operation

Operations per file: 8

• Open operations: 1
• Stat operations (GetFileDirParams + GetForkParam): 5
• Read operations: 1
• Close operations: 1

Total: 8 × 1000 files = 8,000 AFP operations

2. TEST_WRITE100MB - Writing one large file

Total AFP Operations: 103

• 1 FPCreateExt (create file)
• 1 FPOpenFork
• 100 FPWrite operations (100MB @1MB quantum)
• 1 FPCloseFork

Total: 103 AFP operations (100MB @1MB quantum)

3. TEST_READ100MB - Reading one large file

Total AFP Operations: 102

• 1 FPOpenFork
• 100 FPRead operations (100MB @1MB quantum)
• 1 FPCloseFork

Total: 102 AFP operations (100MB @1MB quantum)

4. TEST_LOCKUNLOCK - Locking/Unlocking 10000 times each

Total AFP Operations: 20,000

• 10,000 FPByteRangeLock operations (lock)
• 10,000 FPByteRangeLock operations (unlock)

Total: 20,000 AFP operations

5. TEST_CREATE2000FILES - Creating dir with 2000 files

Total AFP Operations: 4,000

• 1 FPCreateDir (Directory creation outside timed section)
• 2,000 FPCreateExt (one per file)
• 2,000 FPGetFileDirParams (one per file after creation)

Total: 4,000 AFP operations

6. TEST_ENUM2000FILES - Enumerate dir with 2000 files

Total AFP Operations: ~51

• ~51 FPEnumerate operations (based on response packet size)
  • Each FPEnumerate can return ~40 entries
  • 2000 files ÷ 40 per call ≈ 50 calls
  • Plus 1 final call to confirm end

This test demonstrates the efficiency of enumeration through batching.

7. TEST_DELETE2000FILES - Deleting dir with 2000 files

Total AFP Operations: 2,000

• 2,000 FPDelete operations (one per file)
• 1 FPDelete (Directory deletion outside timed section)

Total: 2,000 AFP operations

8. TEST_CREATEDIR - Create directory tree with 1000 dirs

Total AFP Operations: 1,110

Creates nested structure: 10 × 10 × 10 directories + 10 top-level

• 10 FPCreateDir (level 1)
• 100 FPCreateDir (level 2: 10 × 10)
• 1,000 FPCreateDir (level 3: 10 × 10 × 10)

Total: 10 + 100 + 1,000 dirs = 1,110 AFP operations

9. TEST_DIRCACHE_HITS - Directory cache hits

Total AFP Operations: 11,100

• 100 FPCreateDir operations (10 × 10 directories)
• 1,000 FPCreateExt operations (100 dirs × 10 files per dir)
• 10,000 FPGetFileDirParams operations (100 iterations × 100 dirs)

Total: 100 + 1,000 + 10,000 dirs = 1,110 AFP operations

10. TEST_DIRCACHE_MIXED - Mixed cache operations

Total AFP Operations: 820

For each of 10 iterations (with 20 files each):

• 1 FPCreateDir
• 10 FPCreateExt (files)
• 20 FPGetFileDirParams (2 × 10 stats)
• 41 FPEnumerate (~40 entries per call + 1)
• 10 FPDelete (files)

Operations per iteration: 82 Total for 10 iterations: 820 AFP operations

11. TEST_DIRCACHE_TRAVERSE - Deep path traversal

Total AFP Operations: 3,500

Creates 20-level deep directory structure with 50 files in the deepest directory, then performs 50 traversals:

• Initial Directory creation: 20 FPCreateDir operations (outside timed section)
• Initial File creation: 50 FPCreateFile operations (outside timed section)

For each of 50 traversals (within timed section):

• 20 FPGetFileDirParams (navigating down 20 directory levels using is_there() call)
• 50 FPGetFileDirParams (accessing all 50 files in the deepest directory)

Operations per traversal: 70 Total: 50 traversals × 70 operations = 3,500 AFP operations

12. TEST_CACHE_VALIDATION - Cache validation efficiency

Total AFP Operations: 30,000

• 100 FPCreateDir
• 1,000 FPCreateExt (10 files per dir)
• 10,000 FPGetFileDirParams (validation before modifications)
• 1,000 FPSetFileParms (modify timestamps)
• 1,000 FPWrite operations (trigger metadata changes)
• 10,000 FPGetFileDirParams (revalidation after changes)
• 1,000 FPRename operations (test cache coherency)
• 6,000 FPGetFileDirParams (verify renames - partial dirs)

Total: ~30,000 AFP operations

Key Insights

1. Enumeration Efficiency: The enumeration test (TEST_ENUM2000FILES) performs only ~51 operations for 2000 files due to batching, making it the most efficient operation per item.

2. Read/Write Quantum Impact: With 1MB quantum size, the 100MB file tests perform only ~100 operations each, making them very efficient for bulk data transfer.

3. Cache Validation Overhead: The cache validation test performs the most operations (30,000), making it ideal for stress testing and cache coherency validation.

4. Lock Operations: The lock/unlock test provides a pure protocol overhead measurement with 20,000 rapid operations that don't involve actual data transfer.

5. Directory Operations: Directory creation and traversal tests (1,110-1,500 operations) provide good measurements for metadata operation performance.

These operation counts help identify which tests are best suited for:

• Protocol efficiency testing: Enumeration, large file I/O
• Stress testing: Cache validation, lock/unlock, open/stat/read
• Metadata performance: Directory operations, cache hits
• Real-world simulation: Mixed operations, open/stat/read patterns