[DBIP] Validator markdown heuristics miss broken markdown when bold is present, truncate parenthesized URLs, and providers.csv bypasses rule validation entirely

[foxxx009]

Proposal type

Other — validator correctness: fix markdown-heuristic false negatives in has_unclosed_markdown / is_markdown_link, and close the rule-coverage gap that leaves references/providers/providers.csv unvalidated by the rule engine.

Affected scope

tools/validate.py on the json-tools branch (functions has_unclosed_markdown, is_markdown_link, check_validation, and the "Validate providers" section of main()).

Evidence on current json-tools (tools/validate.py @ HEAD)

1. has_unclosed_markdown misses an unclosed single-* span whenever any ** is present.

The single-star check short-circuits on s.count("**") == 0:

if s.count("*") % 2 != 0 and s.count("**") == 0:  # single * for italic
    return True

Reproduction (exact function from the branch):

has_unclosed_markdown("**bold** and *unclosed")  -> False   (expected True)

**bold** contributes 4 stars, the stray *italic start adds a 5th — odd total, but because count("**") != 0 the branch is skipped and the cell passes. The count-based fix is (s.count("*") - 2 * s.count("**")) % 2 != 0.

2. is_markdown_link silently truncates URLs containing parentheses.

The non-greedy (?P<link>.*?)\) stops at the first ):

re.match(r"(?:\[(?P<text>.*?)\])\((?P<link>.*?)\)",
         "[Docs](https://en.wikipedia.org/wiki/Chain_(blockchain))").group("link")
-> "https://en.wikipedia.org/wiki/Chain_(blockchain"     # closing paren lost

The rule only checks whether the pattern matches, so a parenthesized-URL action button passes validation even though the markdown link target is malformed. No current actionButtons cell contains a parenthesized URL, so this is a validation gap rather than live corruption (same class as #3628). A stricter check would either reject unbalanced/parenthesized targets explicitly or require a full-string match (re.fullmatch) so trailing junk like "[Buy](https://a.com) trailing text" stops passing.

3. references/providers/providers.csv never reaches the rule engine.

main() labels its second phase "Validate providers", but it loads the offers folder:

providers_data = load_csv_folder("references/offers")

references/providers/providers.csv (722 rows) is therefore never run through rule_slug_kebab_case, rule_provider_casing_consistent, rule_no_unclosed_markdown, or the other rules. The gap is not hypothetical: scanning providers.csv with the exact branch functions today flags one cell:

providers.csv, row slug 'subquery', column 'docs':
https://subquery.network/doc/subquery_network/introduction/introduction.html

The URL contains a single underscore, so the _-count heuristic reports it as "unclosed markdown" — which also demonstrates problem 4 below.

4. The _-count heuristic false-positives on legitimate URLs.

Underscores are legal in URLs and plain identifiers (snake_case), but any cell with an odd number of them is rejected as broken markdown regardless of context. The subquery docs URL above is a real example of data that this heuristic would reject if providers.csv were scanned. Either the heuristic should ignore underscores inside URLs/plain text (or be dropped in favor of checking only cells that already contain markdown syntax), or the style guide should say underscore-bearing URLs are not allowed — today the rule and the data disagree silently.

Minor, same file: check_validation returns check_schema_validation(...) and check_rules_validation(...), so when the schema fails, rule errors are not reported in the same run — contributors fix schema issues and only discover rule failures on a second CI round. Running both and aggregating would halve review cycles.

Problem

The quality gateway (discussions#41) reduces rewards by 30% for failed reviews, so contributors are penalized against the validator as the source of truth. A validator that (a) lets genuinely broken markdown through when bold is present, (b) accepts malformed link targets, and (c) never applies any of these rules to the root table of the model is a silent source of both missed defects and future false rejections (the subquery cell becomes an instant 30% penalty the day providers.csv is added to the rule sweep without fixing the heuristic first).

Detailed proposal

1. Fix the single-* branch: (s.count("*") - 2 * s.count("**")) % 2 != 0.
2. Make is_markdown_link use re.fullmatch and either handle parenthesized URL targets or reject them explicitly with a clear message.
3. Scope the _ heuristic to cells that contain markdown syntax, or exclude URL characters from the count; align the style guide with whichever behavior is chosen.
4. Actually validate references/providers/providers.csv with the rule engine (it needs its own schema mapping, since its columns differ from offer tables), and fix the subquery docs cell so the new check is green from day one.
5. Run schema and rule validation unconditionally and aggregate both error sets before deciding the exit code.

Acceptance criteria

• has_unclosed_markdown("**bold** and *unclosed") returns True.
• A parenthesized-URL action button is either accepted with a correct captured target or rejected with an explicit message — no silent truncation.
• providers.csv rows pass through the same rule set as offer tables, and the current tree is green under it.
• No legitimate underscore-bearing URL in the current tree is reported by the fixed heuristic.

Reward address

0x2409b47a530be3831158f10b08ac93f7d08c1ff2