Skip to content

Add HyperLogLog support for COUNT (NIP-45)#181

Open
alltheseas wants to merge 3 commits intohoytech:masterfrom
alltheseas:nip45-hll
Open

Add HyperLogLog support for COUNT (NIP-45)#181
alltheseas wants to merge 3 commits intohoytech:masterfrom
alltheseas:nip45-hll

Conversation

@alltheseas
Copy link
Contributor

Closes #178

Summary

  • Add HLL sketch (256 registers, precision 8) to COUNT responses for single-tag filters (#e, #p, etc.), enabling clients to merge counts from multiple relays without double-counting pubkeys
  • Algorithm is a mechanical port of go-nostr nip45/hyperloglog: extract 8 bytes from pubkey at a filter-derived offset, use first byte as register index, count leading zeros in remaining 56 bits
  • COUNT responses now include a "hll" field (512-char hex string) when the filter has exactly one tag with one value

Files

File Action What
src/HyperLogLog.h NEW Self-contained HLL struct (addPubkeyBytes, encodeHex, clz56)
src/DBQuery.h MODIFY Add hllOffset/hll fields, computeHllOffset(), update registers during scan
src/QueryScheduler.h MODIFY Pass hllHex string through onComplete callback
src/apps/relay/RelayReqWorker.cpp MODIFY Include "hll" field in COUNT JSON response
src/apps/relay/RelayNegentropy.cpp MODIFY Update onComplete signature (ignore new param)
test/hll_reference.py NEW Python oracle (exact go-nostr port) for test vector generation
test/hll_unit_test.cpp NEW Standalone C++ test validating against Python oracle
test/nip45_hll_test.py NEW E2E websocket test (5 test cases)
test/cfgs/nip45HllTest.conf NEW Test relay config

References

Test plan

  • Python oracle generates test vectors for known pubkeys
  • C++ unit test passes against Python oracle (g++ -std=c++20 -I src test/hll_unit_test.cpp)
  • E2E websocket test against live strfry on Linux
  • Full make build on Linux

🤖 Generated with Claude Code

alltheseas and others added 3 commits March 19, 2026 18:15
Add HLL sketch (256 registers, precision 8) to COUNT responses for
single-tag filters (#e, #p, etc.), enabling clients to merge counts
from multiple relays without double-counting pubkeys.

Algorithm is a mechanical port of go-nostr nip45/hyperloglog: extract
8 bytes from pubkey at a filter-derived offset, use first byte as
register index, count leading zeros in remaining 56 bits.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Python oracle is an exact port of go-nostr nip45/hyperloglog used
to generate test vectors. Standalone C++ test validates the HLL
algorithm against those vectors (buildable without strfry deps).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Five test cases against a live strfry relay: #e tag HLL, #p tag HLL,
non-eligible filter omits HLL, zero-result all-zeros HLL, and
multi-tag filter omits HLL.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet