Make regex-filtered re2 bench easier to run

Also make both re2 and regex benches run using simplified /
down-compiled regexes.

At this point, regex-filtered trounces FilteredRE2 on CPU *for the
specific job of ua-parsing*, and its memory use has gotten quite
reasonable (overhead compared to re2 is down to just 20%):

```sh
> /usr/bin/time -l target/bench_re2 \
	target/devices.regexes regex-filtered/samples/useragents.txt 100 -q
633 regexes 1630 atoms in 0.0200757s
	prefilter built in 0.00527812s
75158 user agents in 0.0353107s
       49.67 real        48.88 user         0.35 sys
            43958272  maximum resident set size
                   0  average shared memory size
                   0  average unshared data size
                   0  average unshared stack size
                3499  page reclaims
                 289  page faults
                   0  swaps
                   0  block input operations
                   0  block output operations
                   0  messages sent
                   0  messages received
                   0  signals received
                   2  voluntary context switches
               28247  involuntary context switches
        600560455436  instructions retired
        157226396942  cycles elapsed
            35309696  peak memory footprint
> /usr/bin/time -l target/release/examples/bench_regex \
	target/devices.regexes regex-filtered/samples/useragents.txt -r 100 -q
633 regexes in 0.051890332s
75158 user agents in 0.007460291s
       38.93 real        38.52 user         0.22 sys
            43958272  maximum resident set size
                   0  average shared memory size
                   0  average unshared data size
                   0  average unshared stack size
                2798  page reclaims
                  98  page faults
                   0  swaps
                   0  block input operations
                   0  block output operations
                   0  messages sent
                   0  messages received
                   0  signals received
                   0  voluntary context switches
               17680  involuntary context switches
        372376225085  instructions retired
        123339367792  cycles elapsed
            42370560  peak memory footprint
```

note: bench.rs was renamed to stop conflicting with the one in
ua-parser, and make the two bench programs easier to differentiate.

Also one day I need to look into the difference between maximum rss
and peak memory footprint on macos. It seems weird that RSS matches
between the two programs, and RSS and peak match for rust, but re2's
peak is 25% lower.

Author: masklinn

.gitignore

@@ -1,7 +1,7 @@
 /target
 .DS_Store
 *.dSYM/
-regex-filtered/re2/flake.lock
-regex-filtered/re2/bench
+flake.lock
 .tox/
+*/uv.lock
 __pycache__

Makefile

@@ -0,0 +1,24 @@
+CXXFLAGS += -std=c++20 -Wall -Werror -g -fPIC -O3
+LDFLAGS += -lre2
+
+.PHONY: bench
+
+bench: target/bench_re2 target/devices.regexes target/release/examples/bench_regex
+	/usr/bin/time -l target/bench_re2 \
+		target/devices.regexes regex-filtered/samples/useragents.txt 100 -q
+	/usr/bin/time -l target/release/examples/bench_regex \
+		target/devices.regexes regex-filtered/samples/useragents.txt -r 100 -q
+
+target/bench_re2: regex-filtered/re2/bench.cpp
+	# build re2 bench, requires re2 to be LD-able, can `nix develop` for setup
+	@mkdir -p target
+	$(CXX) $(CXXFLAGS) $^ -o $@ $(LDFLAGS)
+
+target/release/examples/bench_regex: regex-filtered/examples/bench_regex.rs regex-filtered/src/*
+	# build regex bench
+	cargo build --release --example bench_regex -q
+
+target/devices.regexes: scripts/devices ua-parser/uap-core/regexes.yaml
+	# compiles regexe.yaml to a list of just the device regex (with embedded flags)
+	@mkdir -p target
+	uv run --script $^ > $@

regex-filtered/re2/flake.nix flake.nixregex-filtered/re2/flake.nix renamed to flake.nix

@@ -0,0 +1,121 @@
+#!/usr/bin/env python
+# /// script
+# requires-python = ">=3.10"
+# dependencies = [
+#     "pyyaml",
+# ]
+# ///
+r"""Compiles regexes.yaml to just the device regexps, with rewriting:
+
+- Rust's `regex` implements perl-style character classes with full
+  unicode semantics making them much more expensive than re2's
+  ascii-only semantics, so compile down the most frequent ones down to
+  ascii classes.
+
+  regexes.yaml uses \d, \w, \s, \S, \b, and the first one is the most
+  common by two orders of magnitude (but convert \w as well because I
+  dun so already, converting \s might be a good idea too)
+
+- Both `regex` and `re2` suffer tremendously from large bounded
+  repetitions as they need to create a *ton* of states to keep track
+  of the limit. This mostly affects memory consumption (and the issue
+  compounds when captures are added to the mix), but there is a minor
+  CPU hit as well.
+
+  In regexes.yaml, large bounded repetitions were introduced only to
+  limit the risks of catastrophic backtracking in backtracking
+  engines. Which neither re2 nor regex are.
+
+  So compile large bounded repetition (where heuristically "large" is
+  3 digits in the upper bound) back to simple unbounded repetitions.
+  Note that this is only done for a lower bound of `0` or `1`, but
+  that's the case of all large bounded repetitions in regexes.yaml.
+"""
+import string
+import sys
+
+from yaml import SafeLoader, load
+
+def main() -> None:
+    with open(sys.argv[1]) as f:
+        regexes = load(f, Loader=SafeLoader)
+    for dev in regexes['device_parsers']:
+        print(
+            f'(?{f})' if (f := dev.get('regex_flag')) else '',
+            rewrite(dev['regex']),
+            sep='',
+        )
+
+def rewrite(re: str) -> str:
+    from_ = 0
+    out = []
+    it = enumerate(re)
+    escape = False
+    inclass = False
+
+    cont = True
+    while cont and (e := next(it, None)):
+        idx, c = e
+        match c:
+            case '\\' if not escape:
+                escape = True
+                continue
+            case '[' if not escape:
+                inclass = True
+            case ']' if not escape:
+                inclass = False
+            case 'd' if escape:
+                out.append(re[from_:idx-1])
+                from_ = idx+1
+                if inclass:
+                    out.append('0-9')
+                else:
+                    out.append('[0-9]')
+            case 'w' if escape:
+                out.append(re[from_:idx-1])
+                from_ = idx+1
+                if inclass:
+                    out.append('A-Za-z0-9_')
+                else:
+                    out.append('[A-Za-z0-9_]')
+            case '{' if not escape and not inclass:
+                if not idx:
+                    return re
+
+                try:
+                    _, start = next(it)
+                except StopIteration:
+                    continue
+                if start not in '01':
+                    continue
+
+                try:
+                    _, comma = next(it)
+                except StopIteration:
+                    continue
+                else:
+                    if comma != ',':
+                        continue
+
+                digits = 0
+                for ri, rc in it:
+                    match rc:
+                        case c if c in string.digits:
+                            digits += 1
+                        case '}' if digits > 2:
+                            out.append(re[from_:idx])
+                            from_ = ri + 1
+                            out.append('*' if start == '0' else '+')
+                        case _:
+                            break
+            case _:
+                pass
+        escape = False
+
+    if from_ == 0:
+        return re
+    out.append(re[from_:])
+    return ''.join(out)
+
+if __name__ == "__main__":
+    main()