NH: refactor to HttpSource and apiV2#11
Open
AaronDeimos wants to merge 5 commits intoyuzono:masterfrom
Open
Conversation
Reviewer's GuideRefactors the NHentai extension to use the JSON-based api/v2 via HttpSource, replaces HTML parsing with typed DTOs, reworks favorites and pagination, introduces a reusable random user agent helper library enforced by a Spotless lint check, and bumps the extension and build tooling to support context receivers and Brotli. Sequence diagram for NHentai favorites API authorization flowsequenceDiagram
participant User
participant Tachiyomi as TachiyomiApp
participant NH as NHentaiSource
participant Client as OkHttpClient
participant Interceptor as authorizationInterceptor
participant CookieMgr as CookieManager
participant API as NHentaiApiV2
User->>Tachiyomi: Open favorites for NHentai
Tachiyomi->>NH: fetchSearchManga(page, query, filters with FavoriteFilter)
NH->>NH: searchMangaRequest(...) builds https://nhentai.net/api/v2/favorites
NH->>Client: newCall(favoritesRequest)
Client->>Interceptor: intercept(request)
alt request.path contains /favorites
Interceptor->>CookieMgr: getCookie(baseUrl)
CookieMgr-->>Interceptor: cookies string
Interceptor->>Interceptor: extract access_token
Interceptor->>Interceptor: update accessToken cache
alt accessToken not blank
Interceptor->>Client: proceed(request with Authorization User accessToken)
else accessToken blank
Interceptor->>Client: proceed(original request)
end
else other paths
Interceptor->>Client: proceed(original request)
end
Client->>API: GET /api/v2/favorites
API-->>Client: HTTP 200 with JSON or HTTP 401
alt response.code == 401 and path contains /favorites
Client-->>Interceptor: Response 401
Interceptor->>Interceptor: clear accessToken
Interceptor->>Client: close response
Interceptor-->>Tachiyomi: throw IOException(Log in via WebView)
Tachiyomi-->>User: Show error "Log in via WebView to view favorites"
else success
Client-->>NH: Response 200
NH->>NH: searchMangaParse(response) using PaginatedResponse~GalleryListItem~
NH-->>Tachiyomi: MangasPage
Tachiyomi-->>User: Render favorites list
end
Sequence diagram for random user agent selection in headersBuildersequenceDiagram
participant Source as HttpSource
participant Headers as HeadersBuilder
participant Prefs as SharedPreferences
participant Helper as UserAgentPreferenceHelpers
participant UAHelper as HelperKt
participant Network as NetworkHelper.client
participant UADB as UA_DB_server
Source->>Source: headersBuilder()
Source->>Headers: super.headersBuilder()
Source->>Helper: setRandomUserAgent(headers, userAgentType=null, filterInclude=["chrome"], filterExclude=[])
Helper->>Source: getPreferences()
Source-->>Helper: SharedPreferences
Helper->>Prefs: getPrefUAType()
Prefs-->>Helper: UserAgentType
Helper->>Prefs: getPrefCustomUA()
Prefs-->>Helper: customUA or null
alt random type != OFF
Helper->>UAHelper: getRandomUserAgent(type, filterInclude, filterExclude)
alt cached userAgent exists
UAHelper-->>Helper: cached userAgent
else no cache
UAHelper->>Network: newCall(GET UA_DB_URL)
Network->>UADB: GET user-agents.json
UADB-->>Network: JSON desktop and mobile lists
Network-->>UAHelper: Response
UAHelper->>UAHelper: parseAs UserAgentList
UAHelper->>UAHelper: filter include and exclude
UAHelper-->>Helper: random user agent
end
Helper->>Headers: set(User-Agent, userAgent)
else random type OFF and customUA not null
Helper->>Headers: set(User-Agent, customUA)
else no UA
Helper-->>Headers: return without changes
end
Source-->>Source: headers with possibly randomized User-Agent
Class diagram for NHentai HttpSource refactor and DTOsclassDiagram
direction LR
class NHentai {
+String lang
+String nhLang
+String baseUrl
+String apiUrl
+Long id
+String name
-Json json
-CookieManager webViewCookieManager
-String accessToken
-SharedPreferences preferences
+OkHttpClient client
-Boolean displayFullTitle
-Regex shortenTitleRegex
+setupPreferenceScreen(screen PreferenceScreen)
+headersBuilder() Headers.Builder
+popularMangaRequest(page Int) Request
+popularMangaParse(response Response) MangasPage
+latestUpdatesRequest(page Int) Request
+latestUpdatesParse(response Response) MangasPage
+fetchSearchManga(page Int, query String, filters FilterList) Observable~MangasPage~
+searchMangaRequest(page Int, query String, filters FilterList) Request
+searchMangaParse(response Response) MangasPage
-combineQuery(filters FilterList) String
-searchMangaByIdRequest(id String) Request
-searchMangaByIdParse(response Response, id String) MangasPage
+mangaDetailsRequest(manga SManga) Request
+mangaDetailsParse(response Response) SManga
+getMangaUrl(manga SManga) String
+chapterListRequest(manga SManga) Request
+chapterListParse(response Response) List~SChapter~
+pageListRequest(chapter SChapter) Request
+pageListParse(response Response) List~Page~
+imageUrlParse(response Response) String
+getFilterList() FilterList
-authorizationInterceptor(chain Interceptor.Chain) Response
-shortenTitle(text String) String
-getThumbServer() String
-getImageServer() String
}
class HttpSource {
<<abstract>>
+Headers.Builder headersBuilder()
}
class PaginatedResponse~T~ {
+List~T~ result
+Int numPages
}
class GalleryListItem {
-Int id
-String thumbnail
-String englishTitle
-String japaneseTitle
+toSManga(displayFullTitle Boolean, shortenTitle Function, thumbServer String) SManga
}
class Hentai {
+Int id
-NHTitle title
-ImageInfo thumbnail
-Long uploadDate
+List~NHTag~ tags
+Int numPages
-Int numFavorites
+List~NHPage~ pages
+toSManga(full Boolean, displayFullTitle Boolean, shortenTitle Function, thumbServer String) SManga
+toSChapter() SChapter
}
class NHTitle {
+String english
+String japanese
+String pretty
}
class ImageInfo {
+String path
}
class NHTag {
+String type
+String name
}
class NHPage {
+String path
}
class Config {
+List~String~ imageServers
+List~String~ thumbServers
}
class NHUtils {
+getArtists(data Hentai) String
+getGroups(data Hentai) String
+getTagDescription(data Hentai) String
+getTags(data Hentai) String
}
NHentai ..|> HttpSource
NHentai --> PaginatedResponse : uses
NHentai --> GalleryListItem : uses
NHentai --> Hentai : uses
NHentai --> Config : uses
NHentai --> NHUtils : uses
PaginatedResponse o--> Hentai : result
PaginatedResponse o--> GalleryListItem : result
Hentai --> NHTitle : has
Hentai --> ImageInfo : thumbnail
Hentai --> NHPage : pages
Hentai --> NHTag : tags
NHUtils ..> Hentai : extension data
Class diagram for randomua helper library and Spotless RandomUaCheckclassDiagram
direction LR
class HttpSource {
<<abstract>>
+Headers.Builder headersBuilder()
}
class Headers_Builder {
+set(name String, value String) Headers.Builder
}
class PreferenceScreen {
}
class UserAgentPreferenceHelpers {
<<file-level functions>>
-SharedPreferences.getPrefUAType() UserAgentType
-SharedPreferences.getPrefCustomUA() String
+setRandomUserAgent(builder Headers_Builder, userAgentType UserAgentType, filterInclude List~String~, filterExclude List~String~) Headers_Builder
+addRandomUAPreference(screen PreferenceScreen)
}
class UserAgentType {
<<enum>>
MOBILE
DESKTOP
OFF
}
class HelperKt {
<<file-level functions>>
-String UA_DB_URL
-NetworkHelper client
-String userAgent
+getRandomUserAgent(userAgentType UserAgentType, filterInclude List~String~, filterExclude List~String~) String
}
class UserAgentList {
+List~String~ desktop
+List~String~ mobile
}
class NetworkHelper {
+OkHttpClient client
}
class RandomUaCheck {
+create() FormatterStep
}
class FormatterStep {
+create(name String, state Serializable, toFormatter Function) FormatterStep
}
class FormatterFunc {
}
class RandomUaCheck_State {
+toFormatter() FormatterFunc
}
HttpSource ..> UserAgentPreferenceHelpers : context(source)
Headers_Builder ..> UserAgentPreferenceHelpers : extension
UserAgentPreferenceHelpers ..> UserAgentType : uses
UserAgentPreferenceHelpers ..> HelperKt : calls getRandomUserAgent
HelperKt ..> NetworkHelper : uses
HelperKt ..> UserAgentList : parses
RandomUaCheck ..> RandomUaCheck_State : creates
RandomUaCheck_State ..> FormatterFunc : returns
RandomUaCheck ..> FormatterStep : creates
File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 1 issue, and left some high level feedback:
- The global
userAgentcache inrandomua.Helperignores the requestedUserAgentTypeand filters, so once a value is set all subsequent calls (including mobile vs desktop, different includes/excludes) will reuse the same UA; consider caching per type/filter or removing the global cache. - In
getRandomUserAgent,runBlocking(Dispatchers.IO)is used even when called from the main thread (the cache-only branch still blocks while reading/parsing), which risks UI freezes; consider deferring UA resolution off the main thread or ensuring the header builder never triggers network/cache IO on the main looper.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The global `userAgent` cache in `randomua.Helper` ignores the requested `UserAgentType` and filters, so once a value is set all subsequent calls (including mobile vs desktop, different includes/excludes) will reuse the same UA; consider caching per type/filter or removing the global cache.
- In `getRandomUserAgent`, `runBlocking(Dispatchers.IO)` is used even when called from the main thread (the cache-only branch still blocks while reading/parsing), which risks UI freezes; consider deferring UA resolution off the main thread or ensuring the header builder never triggers network/cache IO on the main looper.
## Individual Comments
### Comment 1
<location path="lib/randomua/src/keiyoushi/lib/randomua/Helper.kt" line_range="31-36" />
<code_context>
+ }
+ .build()
+
+internal fun getRandomUserAgent(
+ userAgentType: UserAgentType,
+ filterInclude: List<String>,
+ filterExclude: List<String>,
+): String? {
+ if (!userAgent.isNullOrEmpty()) return userAgent
+
+ // avoid network on main thread when webview screen accesses headers
</code_context>
<issue_to_address>
**issue (bug_risk):** User-agent caching ignores type and filters, which can produce inconsistent or unexpected results.
Since `userAgent` is a single global cache, it’s set based only on the first call’s `userAgentType` and filters. All later calls with different types or `filterInclude`/`filterExclude` values will still return that first value, which can break callers that switch between mobile/desktop or update filters. To support multiple combinations, either:
- cache by a key that includes `(UserAgentType, filterInclude, filterExclude)`, or
- remove the global cache and rely on HTTP caching (you already set `Cache-Control`).
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request refactors the NHentai extension to transition from HTML scraping to using the site's JSON API, involving a complete overhaul of the DTOs and the main source class. Additionally, it introduces a new library for managing random user agents. The review feedback points out critical issues including potential null pointer exceptions due to unsafe assertions in the new DTOs, an invalid dependency version in the build configuration, and a logic bug in the user agent helper where global state prevents the correct application of filtering parameters.
There was a problem hiding this comment.
Hey - I've found 1 issue, and left some high level feedback:
- getRandomUserAgent() caches the first returned value in a single global userAgent variable without considering UserAgentType or the include/exclude filters, so subsequent calls for different types or filters may reuse an inappropriate UA; consider caching per (type, filterInclude, filterExclude) or not caching at all.
- The preference keys PREF_KEY_RANDOM_UA and PREF_KEY_CUSTOM_UA in UserAgentPreference.kt are very generic and shared across all sources using the library; consider namespacing them (e.g., by source id or library name) to avoid collisions with other extensions that might also use random UA settings.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- getRandomUserAgent() caches the first returned value in a single global userAgent variable without considering UserAgentType or the include/exclude filters, so subsequent calls for different types or filters may reuse an inappropriate UA; consider caching per (type, filterInclude, filterExclude) or not caching at all.
- The preference keys PREF_KEY_RANDOM_UA and PREF_KEY_CUSTOM_UA in UserAgentPreference.kt are very generic and shared across all sources using the library; consider namespacing them (e.g., by source id or library name) to avoid collisions with other extensions that might also use random UA settings.
## Individual Comments
### Comment 1
<location path="src/all/nhentai/src/eu/kanade/tachiyomi/extension/all/nhentai/NHentai.kt" line_range="58-59" />
<code_context>
.build()
}
+ override fun headersBuilder() = super.headersBuilder()
+ .setRandomUserAgent(
+ filterInclude = listOf("chrome"),
+ )
</code_context>
<issue_to_address>
**issue (bug_risk):** The context receiver `setRandomUserAgent` is used without a matching `HttpSource` context in scope, which is likely to fail compilation.
Because `setRandomUserAgent` is defined with `context(source: HttpSource)`, it can only be called where an implicit `HttpSource` context is in scope (e.g., inside `with(this@NHentai)` or another `context(HttpSource)` function). Simply being in an `HttpSource` subclass is not enough. You can either wrap the call in a context, for example:
```kotlin
override fun headersBuilder() = with(this) {
super.headersBuilder()
.setRandomUserAgent(filterInclude = listOf("chrome"))
}
```
or change the helper’s signature if you meant it to be a normal extension function.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
src/all/nhentai/src/eu/kanade/tachiyomi/extension/all/nhentai/NHentai.kt
Show resolved
Hide resolved
Contributor
Author
|
@sourcery-ai dismiss |
src/all/nhentai/src/eu/kanade/tachiyomi/extension/all/nhentai/NHentai.kt
Outdated
Show resolved
Hide resolved
src/all/nhentai/src/eu/kanade/tachiyomi/extension/all/nhentai/NHentai.kt
Show resolved
Hide resolved
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Close: #10
Supersede: #9
Ai assisted
Checklist:
extVersionCodevalue inbuild.gradlefor individual extensionsoverrideVersionCodeorbaseVersionCodeas needed for all multisrc extensionsisNsfw = trueflag inbuild.gradlewhen appropriateidif a source's name or language were changedweb_hi_res_512.pngwhen adding a new extensionSummary by Sourcery
Refactor the NHentai extension to use the v2 HTTP API and HttpSource, updating models, parsing, and favorites handling while integrating shared random user agent helpers and enforcing getMangaUrl overrides.
New Features:
Bug Fixes:
Enhancements:
Build: