[ZIPT Benchmark] Z3 c3 branch — 2026-03-12

[github-actions[bot]]

Date: 2026-03-12
Branch: c3
Z3 version: 4.17.0 (build hash: acbf5c6)
Benchmark set: QF_S (50 randomly selected files from tests/QF_S.tar.zst, total pool: 22,172 files)
Timeout: 10 seconds per benchmark (-T:10)

Note: The build had a compilation error in src/smt/seq/seq_nielsen.cpp (line 447–448): m_len_vars was accidentally placed inside the constructor body instead of the initializer list. This was automatically fixed before building.

---

Summary

Metric	seq solver	nseq solver
sat	26	6
unsat	16	6
unknown	8	38
timeout	0	0
bug/crash	0	0
Total time (s)	35.762	230.646
Avg time/benchmark (s)	0.715	4.613

Soundness disagreements (seq says sat, nseq says unsat or vice versa): 0 ✅

---

Notable Issues

Soundness Disagreements (Critical)

None detected. ✅

Crashes / Bugs

None detected. ✅

Performance Gap

nseq is significantly less complete than seq on this benchmark set:

• seq solved 42/50 (84%) benchmarks definitively (sat or unsat)
• nseq solved only 12/50 (24%) definitively
• nseq returned unknown on 38 out of 50 benchmarks, including 22 cases where it hit the 10-second timeout but seq solved the same instance quickly

Slow Benchmarks (> 8s, either solver)

File	seq time (s)	nseq time (s)
instance09598.smt2	10.025 ⏱	10.009 ⏱
bakery_sat_non_incre_equiv_trans_2_0.smt2	10.020 ⏱	0.012
instance15024.smt2	7.279	10.008 ⏱
instance14875.smt2	5.315	10.008 ⏱
instance14501.smt2	0.258	10.007 ⏱
instance00346.smt2	0.068	10.007 ⏱
pcp_instance_14.smt2	0.014	10.009 ⏱
instance00936.smt2	0.018	10.008 ⏱
instance14528.smt2	0.057	10.009 ⏱
instance05731.smt2	0.034	10.007 ⏱
instance00246.smt2	0.017	10.008 ⏱
instance10777.smt2	0.178	10.007 ⏱
instance10808.smt2	0.023	10.009 ⏱
instance09460.smt2	0.075	10.009 ⏱
instance14626.smt2	0.024	10.010 ⏱
instance03328.smt2	0.017	10.009 ⏱
instance09694.smt2	0.671	10.009 ⏱
instance12580.smt2	0.067	10.009 ⏱
instance12585.smt2	0.094	10.009 ⏱
instance14749.smt2	0.193	10.009 ⏱
instance01356.smt2	0.017	10.007 ⏱
pcp_instance_314.smt2	0.013	10.009 ⏱
instance02280.smt2	0.024	10.009 ⏱
instance12704.smt2	0.022	10.008 ⏱

---

Per-File Results

Click to expand all 50 results

#	File	seq verdict	seq time (s)	nseq verdict	nseq time (s)	Notes
1	instance14369.smt2	sat	0.070	unknown	0.013
2	instance00075.smt2	sat	0.012	sat	0.011
3	instance04182.smt2	sat	0.013	sat	0.011
4	instance14501.smt2	sat	0.258	unknown	10.007
5	instance12942.smt2	unsat	0.012	unsat	0.012
6	instance08984.smt2	unsat	0.019	unknown	0.016
7	instance00346.smt2	sat	0.068	unknown	10.007
8	instance15024.smt2	sat	7.279	unknown	10.008
9	benchmark_0166.smt2	unknown	0.015	unknown	0.012
10	pcp_instance_14.smt2	unknown	0.014	unknown	10.009
11	benchmark_0060.smt2	unknown	0.014	unknown	0.013
12	instance00936.smt2	sat	0.018	unknown	10.008
13	instance08092.smt2	unsat	0.041	unknown	0.014
14	instance14528.smt2	sat	0.057	unknown	10.009
15	instance05731.smt2	sat	0.034	unknown	10.007
16	04_track_200.smt2	sat	0.462	sat	0.116
17	instance11272.smt2	unsat	0.040	unknown	0.036
18	instance00246.smt2	sat	0.017	unknown	10.008
19	instance09834.smt2	unsat	0.011	unsat	0.011
20	slog_stranger_4311_sink.smt2	sat	0.017	unknown	0.012
21	instance10777.smt2	sat	0.178	unknown	10.007
22	query3111.smt2	sat	0.258	unknown	0.012
23	instance10808.smt2	unsat	0.023	unknown	10.009
24	instance09460.smt2	sat	0.075	unknown	10.009
25	instance10992.smt2	sat	0.040	unknown	0.014
26	instance14626.smt2	unsat	0.024	unknown	10.010
27	instance03328.smt2	sat	0.017	unknown	10.009
28	instance07963.smt2	sat	0.011	sat	0.012
29	slog_stranger_4310_sink.smt2	sat	0.018	unknown	0.012
30	instance15552.smt2	sat	0.011	sat	0.010
31	query4111.smt2	sat	0.045	unknown	0.011
32	instance09694.smt2	unsat	0.671	unknown	10.009
33	instance09598.smt2	unknown	10.025	unknown	10.009
34	instance12580.smt2	sat	0.067	unknown	10.009
35	slog_stranger_779_sink.smt2	unsat	0.010	unsat	0.010
36	instance14875.smt2	unsat	5.315	unknown	10.008
37	instance09729.smt2	unsat	0.011	unsat	0.012
38	bakery_sat_non_incre_equiv_trans_2_0.smt2	unknown	10.020	unknown	0.012
39	benchmark_0437.smt2	unknown	0.014	unknown	0.012
40	instance12585.smt2	unsat	0.094	unknown	10.009
41	query3767.smt2	sat	0.038	unknown	0.012
42	instance14749.smt2	unsat	0.193	unknown	10.009
43	03_track_12.smt2	unsat	0.010	unsat	0.010
44	instance01356.smt2	sat	0.017	unknown	10.007
45	instance14455.smt2	unsat	0.020	unsat	0.014
46	pcp_instance_314.smt2	unknown	0.013	unknown	10.009
47	instance02280.smt2	sat	0.024	unknown	10.009
48	instance13373.smt2	sat	0.012	sat	0.012
49	benchmark_0309.smt2	unknown	0.015	unknown	0.011
50	instance12704.smt2	unsat	0.022	unknown	10.008

---

Generated automatically by the QF_S Benchmark workflow on the c3 branch.

AI generated by Qf S Benchmark · history

• expires on Mar 19, 2026, 6:14 PM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Qf S Benchmark.

A newer discussion is available at Discussion #9031.