Merge bitcoin/bitcoin#31960: seeds: add signet/testnet4, update makeseeds regex, minblocks, fixed seeds

f0b6597 seeds: update .gitignore with signet and testnet4 (Jon Atack)
48f07ac chainparams: remove hardcoded signet seeds (Jon Atack)
d4ab115 chainparams: add signet fixed seeds if default network (Jon Atack)
49f155e seeds: update fixed dns seeds (Jon Atack)
2366870 makeseeds: regex improvements (Lőrinc)
98f84d6 generate-seeds: update and add signet (Jon Atack)
c4ed23e seeds: add testnet4 seeds (Jon Atack)
60f17dd seeds: add signet seeds (Jon Atack)
2bcccaa makeseeds: align I2P column header (Jon Atack)
94e21aa makeseeds: update MIN_BLOCKS, add reminder to README (Jon Atack)
6ae7a3b makeseeds: update user agent regex (Jon Atack)
9b0d2e5 makeseeds: fix incorrect regex (laanwj)

Pull request description:

  In `makeseeds.py`:
  - fix the user agent regex (by laanwj)
  - fix the I2P column header spacing
  - update the regex (it was also not updated for the previous release)
  - update `MIN_BLOCKS` (4320 blocks/month * ~6.5 months) and add README documentation to remember to update it
  - further robustness/standardness/consistency improvements to the regexes (by l0rinc)

  Add signet and testnet4 seeds to the README and to `generate-seeds.py`

  Update the fixed seeds in `src/chainparamsseeds.h`

  In `kernel/chainparams.cpp`:
  - add signet fixed seeds if default network
  - remove hardcoded signet seeds

  Update `contrib/seeds/.gitignore` with signet and testnet4

  The previous 2 seeds updates were bitcoin/bitcoin#30008 and bitcoin/bitcoin#30695.

  mainnet:
  ```
  $ contrib/seeds$ python3 makeseeds.py -a asmap-filled.dat -s seeds_main.txt > nodes_main.txt

  Loading asmap database "asmap-filled.dat"…Done.
  Loading and parsing DNS seeds…Done.
    IPv4   IPv6  Onion    I2P  CJDNS Pass
   17252   3630  21079   3095     12 Initial
   17252   3630  21079   3095     12 Skip entries with invalid address
    8444   1742  14607   2330     10 After removing duplicates
    8194   1691  14321   2102     10 Enforce minimal number of blocks
    7838   1578  14321   2102     10 Require service bit 1
    6802   1326  14321   2102     10 Require minimum uptime
    6762   1321  14320   2102     10 Require a known and recent user agent
    6762   1321  14320   2102     10 Filter out hosts with multiple bitcoin ports
     512    485    512    512     10 Look up ASNs and limit results per ASN and per net
  ```

  signet:
  ```
  $ contrib/seeds$ python3 makeseeds.py -a asmap-filled.dat -s seeds_signet.txt -m 237800 > nodes_signet.txt

  Loading asmap database "asmap-filled.dat"…Done.
  Loading and parsing DNS seeds…Done.
    IPv4   IPv6  Onion    I2P  CJDNS Pass
     110     47     63      9      4 Initial
     110     47     63      9      4 Skip entries with invalid address
     110     47     63      9      4 After removing duplicates
      83     31     58      9      4 Enforce minimal number of blocks
      83     31     58      9      4 Require service bit 1
      83     31     57      9      4 Require minimum uptime
      83     31     57      9      4 Require a known and recent user agent
      83     31     57      7      4 Filter out hosts with multiple bitcoin ports
      42     30     57      7      4 Look up ASNs and limit results per ASN and per net
  ```

  testnet:
  ```
  $ contrib/seeds$ python3 makeseeds.py -a asmap-filled.dat -s seeds_test.txt > nodes_test.txt

  Loading asmap database "asmap-filled.dat"…Done.
  Loading and parsing DNS seeds…Done.
    IPv4   IPv6  Onion    I2P  CJDNS Pass
     204     73     96     11      5 Initial
     204     73     96     11      5 Skip entries with invalid address
     204     73     96     11      5 After removing duplicates
     204     73     96     11      5 Enforce minimal number of blocks
     204     73     96     11      5 Require service bit 1
     195     69     87      9      5 Require minimum uptime
     193     69     87      9      5 Require a known and recent user agent
     193     69     87      9      5 Filter out hosts with multiple bitcoin ports
      79     39     87      9      5 Look up ASNs and limit results per ASN and per net
  ```

  testnet4
  ```
  $ contrib/seeds$ python3 makeseeds.py -a asmap-filled.dat -s seeds_testnet4.txt -m 72600 > nodes_testnet4.txt

  Loading asmap database "asmap-filled.dat"…Done.
  Loading and parsing DNS seeds…Done.
    IPv4   IPv6  Onion    I2P  CJDNS Pass
     149    115     69     11      4 Initial
     149    115     69     11      4 Skip entries with invalid address
     149    115     69     11      4 After removing duplicates
     104     75     52      7      4 Enforce minimal number of blocks
     104     75     52      7      4 Require service bit 1
     100     73     51      7      4 Require minimum uptime
     100     73     51      7      4 Require a known and recent user agent
     100     73     51      7      4 Filter out hosts with multiple bitcoin ports
      43     46     51      7      4 Look up ASNs and limit results per ASN and per net
  ```

ACKs for top commit:
  l0rinc:
    I have mostly reviewed the regexes, for the rest it's only a very lightweight ACK f0b6597
  achow101:
    ACK f0b6597
  laanwj:
    re-ACK f0b6597

Tree-SHA512: 86f4ea247469dbb3f131f2de884e470fbf93f399744d4854fcc26511afafcec231d7eaed37f8564244bc64d917d130b314d948aa97b13020613f8e186c70e368

Author: glozow

contrib/seeds/.gitignore

@@ -1,3 +1,5 @@
 seeds_main.txt
+seeds_signet.txt
 seeds_test.txt
+seeds_testnet4.txt
 asmap-filled.dat

contrib/seeds/README.md

@@ -8,20 +8,24 @@ and remove old versions as necessary (at a minimum when SeedsServiceFlags()
 changes its default return value, as those are the services which seeds are added
 to addrman with).
 
+Update `MIN_BLOCKS` in  `makeseeds.py` and the `-m`/`--minblocks` arguments below, as needed.
+
 The seeds compiled into the release are created from sipa's, achow101's and luke-jr's
 DNS seed, virtu's crawler, and asmap community AS map data. Run the following commands
 from the `/contrib/seeds` directory:
 
 ```
 curl https://bitcoin.sipa.be/seeds.txt.gz | gzip -dc > seeds_main.txt
-curl https://mainnet.achownodes.xyz/seeds.txt.gz | gzip -dc >> seeds_main.txt
 curl https://21.ninja/seeds.txt.gz | gzip -dc >> seeds_main.txt
 curl https://luke.dashjr.org/programs/bitcoin/files/charts/seeds.txt >> seeds_main.txt
+curl https://mainnet.achownodes.xyz/seeds.txt.gz | gzip -dc >> seeds_main.txt
+curl https://signet.achownodes.xyz/seeds.txt.gz | gzip -dc > seeds_signet.txt
 curl https://testnet.achownodes.xyz/seeds.txt.gz | gzip -dc > seeds_test.txt
+curl https://testnet4.achownodes.xyz/seeds.txt.gz | gzip -dc > seeds_testnet4.txt
 curl https://raw.githubusercontent.com/asmap/asmap-data/main/latest_asmap.dat > asmap-filled.dat
 python3 makeseeds.py -a asmap-filled.dat -s seeds_main.txt > nodes_main.txt
+python3 makeseeds.py -a asmap-filled.dat -s seeds_signet.txt -m 237800 > nodes_signet.txt
 python3 makeseeds.py -a asmap-filled.dat -s seeds_test.txt > nodes_test.txt
-# TODO: Uncomment when a seeder publishes seeds.txt.gz for testnet4
-# python3 makeseeds.py -a asmap-filled.dat -s seeds_testnet4.txt -m 30000 > nodes_testnet4.txt
+python3 makeseeds.py -a asmap-filled.dat -s seeds_testnet4.txt -m 72600 > nodes_testnet4.txt
 python3 generate-seeds.py . > ../../src/chainparamsseeds.h
 ```

contrib/seeds/generate-seeds.py

@@ -9,6 +9,7 @@
 argument:
 
     nodes_main.txt
+    nodes_signet.txt
     nodes_test.txt
     nodes_testnet4.txt
 
@@ -19,9 +20,9 @@
     <onion>.onion:<port>
     <i2p>.b32.i2p:<port>
 
-The output will be two data structures with the peers in binary format:
+The output will be several data structures with the peers in binary format:
 
-   static const uint8_t chainparams_seed_{main,test}[]={
+   static const uint8_t chainparams_seed_{main,signet,test,testnet4}[]={
    ...
    }
 
@@ -170,6 +171,9 @@ def main():
     with open(os.path.join(indir,'nodes_main.txt'), 'r', encoding="utf8") as f:
         process_nodes(g, f, 'chainparams_seed_main')
     g.write('\n')
+    with open(os.path.join(indir,'nodes_signet.txt'), 'r', encoding="utf8") as f:
+        process_nodes(g, f, 'chainparams_seed_signet')
+    g.write('\n')
     with open(os.path.join(indir,'nodes_test.txt'), 'r', encoding="utf8") as f:
         process_nodes(g, f, 'chainparams_seed_test')
     g.write('\n')

contrib/seeds/makeseeds.py

@@ -26,29 +26,29 @@
     'ipv6': 10,
 }
 
-MIN_BLOCKS = 840000
+MIN_BLOCKS = 868000
 
-PATTERN_IPV4 = re.compile(r"^((\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})):(\d+)$")
-PATTERN_IPV6 = re.compile(r"^\[([0-9a-z:]+)\]:(\d+)$")
+PATTERN_IPV4 = re.compile(r"^(([0-2]?\d{1,2})\.([0-2]?\d{1,2})\.([0-2]?\d{1,2})\.([0-2]?\d{1,2})):(\d{1,5})$")
+PATTERN_IPV6 = re.compile(r"^\[([\da-f:]+)]:(\d{1,5})$", re.IGNORECASE)
 PATTERN_ONION = re.compile(r"^([a-z2-7]{56}\.onion):(\d+)$")
-PATTERN_I2P = re.compile(r"^([a-z2-7]{52}\.b32.i2p):(\d+)$")
+PATTERN_I2P = re.compile(r"^([a-z2-7]{52}\.b32\.i2p):(\d{1,5})$")
 PATTERN_AGENT = re.compile(
     r"^/Satoshi:("
-    r"0.14.(0|1|2|3|99)|"
-    r"0.15.(0|1|2|99)|"
-    r"0.16.(0|1|2|3|99)|"
-    r"0.17.(0|0.1|1|2|99)|"
-    r"0.18.(0|1|99)|"
-    r"0.19.(0|1|2|99)|"
-    r"0.20.(0|1|2|99)|"
-    r"0.21.(0|1|2|99)|"
-    r"22.(0|1|99).0|"
-    r"23.(0|1|99).0|"
-    r"24.(0|1|2|99).(0|1)|"
-    r"25.(0|1|2|99).0|"
-    r"26.(0|1|99).0|"
-    r"27.(0|1|99).0|"
-    r"28.(0|99).0|"
+    r"0\.14\.(0|1|2|3|99)"
+    r"|0\.15\.(0|1|2|99)"
+    r"|0\.16\.(0|1|2|3|99)"
+    r"|0\.17\.(0|0\.1|1|2|99)"
+    r"|0\.18\.(0|1|99)"
+    r"|0\.19\.(0|1|2|99)"
+    r"|0\.20\.(0|1|2|99)"
+    r"|0\.21\.(0|1|2|99)"
+    r"|22\.(0|1|99)\.0"
+    r"|23\.(0|1|2|99)\.0"
+    r"|24\.(0|1|2|99)\.(0|1)"
+    r"|25\.(0|1|2|99)\.0"
+    r"|26\.(0|1|2|99)\.0"
+    r"|27\.(0|1|2|99)\.0"
+    r"|28\.(0|1|99)\.0"
     r")")
 
 def parseline(line: str) -> Union[dict, None]:
@@ -216,7 +216,7 @@ def main():
     random.shuffle(ips)
     print('Done.', file=sys.stderr)
 
-    print('\x1b[7m  IPv4   IPv6  Onion  I2P    CJDNS Pass                                               \x1b[0m', file=sys.stderr)
+    print('\x1b[7m  IPv4   IPv6  Onion    I2P  CJDNS Pass                                               \x1b[0m', file=sys.stderr)
     print(f'{ip_stats(ips):s} Initial', file=sys.stderr)
     # Skip entries with invalid address.
     ips = [ip for ip in ips if ip is not None]