Support new page nav attribute changes made by SA (#1221)

* Support new page nav attribute changes made by SA
The original scrapePageDropdown method is still in place as a fallback, but the new data atributes will be used first.

See: https://forums.somethingawful.com/showthread.php?threadid=3944917&pagenumber=3#post548658773

  - The tests using the updated showthread2.html fixture (testIgnoredPost and testWeirdSizeTags) are using the new data attributes: [PageNav] Using data  attributes: page 17/18
  - The tests with older fixtures are falling back to the select menu method: [PageNav] Using select menu fallback: page X/Y

  Changes Made:

  1. Updated scrapePageDropdown function
  (AwfulCore/Sources/AwfulCore/Scraping/Helpers.swift:144-169):
    - First attempts to read the new data-current-page and data-total-pages attributes
    - Falls back to the old select menu counting method for backward compatibility
    - Added logging to show which method is being used: [PageNav] Using data attributes or [PageNav] Using select menu fallback
  2. Added new PageNavigationData struct and scrapePageNavigationData function
  (Helpers.swift:111-142):
    - Captures all four new data attributes: current-page, total-pages, base-url, and per-page
    - Available for future use when you need the base URL or per-page values
  3. Updated test fixture (showthread2.html):
    - Replaced this file with a modern sample (taken from the same thread and roughly the same page)
    - Tests confirm the new attributes are being read correctly

  Test Results:
  - All 60 tests pass successfully
  - Logging shows the new data attributes are being used when present
  - Fallback to select menu works for older fixtures without the attributes

  The implementation ensures the app won't break when the site changes the inner HTML of the pages div, as warned by the admin.

* Removed print statements, removed scrapePageDropdown() fallback and solely use the new scrapePageNavigationData()

Author: dfsm

AwfulCore/Sources/AwfulCore/Scraping/Helpers.swift

@@ -108,20 +108,35 @@ func scrapeCustomTitle(_ html: HTMLNode) -> RawHTML? {
 }
 
 
-func scrapePageDropdown(_ node: HTMLNode) -> (pageNumber: Int?, pageCount: Int?) {
+struct PageNavigationData {
+    let currentPage: Int
+    let totalPages: Int
+    let baseURL: String?
+    let perPage: Int?
+}
+
+func scrapePageNavigationData(_ node: HTMLNode) -> PageNavigationData? {
     guard let pageDiv = node.firstNode(matchingSelector: "div.pages") else {
-        return (nil, nil)
+        return nil
     }
-    let pageSelect = pageDiv.firstNode(matchingSelector: "select")
-    let pageCount = pageSelect?.childElementNodes.count ?? 1
 
-    let pageNumber = pageSelect
-        .flatMap { $0.firstNode(matchingSelector: "option[selected]") }
-        .flatMap { $0["value"] }
-        .flatMap { Int($0) }
-        ?? 1
+    if let currentPageStr = pageDiv["data-current-page"],
+       let totalPagesStr = pageDiv["data-total-pages"],
+       let currentPage = Int(currentPageStr),
+       let totalPages = Int(totalPagesStr) {
+
+        let baseURL = pageDiv["data-base-url"]
+        let perPage = pageDiv["data-per-page"].flatMap { Int($0) }
+
+        return PageNavigationData(
+            currentPage: currentPage,
+            totalPages: totalPages,
+            baseURL: baseURL,
+            perPage: perPage
+        )
+    }
 
-    return (pageNumber: pageNumber, pageCount: pageCount)
+    return nil
 }
 
 enum ForumGroupID: String {

AwfulCore/Sources/AwfulCore/Scraping/PostsPageScrapeResult.swift

@@ -34,7 +34,9 @@ public struct PostsPageScrapeResult: ScrapeResult {
 
         isSingleUserFilterEnabled = body.firstNode(matchingSelector: "table.post a.user_jump[title *= 'Remove']") != nil
 
-        (pageNumber: pageNumber, pageCount: pageCount) = scrapePageDropdown(body)
+        let pageNavData = scrapePageNavigationData(body)
+        pageNumber = pageNavData?.currentPage
+        pageCount = pageNavData?.totalPages
 
         let posts = try body
             .nodes(matchingSelector: "table.post")

AwfulCore/Sources/AwfulCore/Scraping/ThreadListScrapeResult.swift

@@ -80,7 +80,9 @@ public struct ThreadListScrapeResult: ScrapeResult {
 
         isBookmarkedThreadsPage = body.firstNode(matchingSelector: "form[name='bookmarks']") != nil
 
-        (pageNumber: pageNumber, pageCount: pageCount) = scrapePageDropdown(body)
+        let pageNavData = scrapePageNavigationData(body)
+        pageNumber = pageNavData?.currentPage
+        pageCount = pageNavData?.totalPages
     }
 }