Releases: greysquirr3l/stygian
Releases · greysquirr3l/stygian
v0.2.1
Added
stygian-browser:Navigator.prototype.webdriverprototype-level patch — previously only the
instance property was overridden; scanners such as pixelscan.net and Akamai probe
Object.getOwnPropertyDescriptor(Navigator.prototype, 'webdriver')directly,
so the prototype getter is now also patched on every new document contextstygian-browser: Network Information API spoofing —navigator.connection(previously
nullin headless, an immediate detection signal) is replaced with a realistic
NetworkInformation-like object (effectiveType: "4g",type: "wifi", seeded
downlink/rttvalues stable within a session)stygian-browser: Battery Status API spoofing —navigator.getBattery()(previously
nullin headless) now resolves with a plausible disconnected-battery state;level,
dischargingTimeare seeded fromperformance.timeOriginto vary across sessionsstygian-browser:examples/scraper_cli.rs— generic CLI scraper usingStealthLevel::Advanced,
WaitUntil::NetworkIdle; emits structured JSON (title, description, headings, links,
text excerpt, timing); successfully scrapes Cloudflare-protected sites (CNN.com, etc.)stygian-browser:examples/pixelscan_check.rs— targeted pixelscan.net fingerprint scan
example; polls until client-side result cards settle; extracts verdict, per-card pass/fail
status, hardware/font/UA detail sections, and livenav_signalsfor stealth regression testingstygian-graph:SigningPorttrait — request-signing abstraction for attaching HMAC tokens,
AWS Signature V4, OAuth 1.0a, device attestation tokens, or any per-request auth material
without coupling adapters to signing schemestygian-graph:NoopSigningAdapter— passthrough signer for testing and optional-signer defaultsstygian-graph:HttpSigningAdapter— delegates signing to any external sidecar over HTTP POST
(e.g. a Frida RPC bridge exposing a/signendpoint); configurable timeout and retriesbook: stealth guide updated — prototype-level webdriver patch, Network Information API
spoofing, and Battery Status API spoofing sections added
Fixed
stygian-browser:outerWidth/outerHeightnow set viascreen_scriptinjection to match
the spoofed screen resolution (headless Chrome returns0without this)stygian-browser:navigator.pluginsspoofed with a realistic 5-entryPluginArray
(PDF Viewer entries +navigator.mimeTypeswith 2 entries), eliminating the
empty-plugins headless signal
v0.2.0
Added
stygian-graph:OpenApiAdapter— OpenAPI 3.x introspection adapter (kind = "openapi")- Fetches and caches parsed OpenAPI specs (JSON or YAML) per spec URL for the lifetime of the adapter
- Resolves operations by
operationId(e.g."listPets") or"METHOD /path"syntax (e.g."GET /pet/findByStatus") - Binds
params.argsto path parameters (substituted into the URL template), query parameters, and request body - Delegates all HTTP calls to the inner
RestApiAdapter, inheriting full auth support, retries, and 429 handling - Optional proactive rate limiting via
params.rate_limit(sliding-window or token-bucket, same shape as GraphQL rate limiter) params.server.urloverridesservers[0].urlfrom the spec at runtime- Two new workspace dependencies:
openapiv3 = "1",serde_yaml = "0.9"(always compiled; pure Rust)
book: OpenAPI adapter section added to Built-in Adapters
v0.1.20
Added
stygian-proxy: new crate — high-performance, resilient proxy pool with per-proxy circuit breakers, configurable rotation strategies (round-robin, random, weighted, failover), SOCKS4/5 support, health scoring, and in-memory storage;socksfeature enables SOCKS proxy types via reqweststygian-proxy:CircuitBreakerper-proxy state machine — open/half-open/closed transitions with configurable failure threshold and recovery window; wired intoProxyManager::acquire_proxyso unhealthy proxies are skipped automaticallystygian-proxy:ProxyManager— unified pool orchestrator exposingadd_proxy,remove_proxy,acquire_proxy,release_proxy, andpool_stats; every operation is traced and meteredstygian-proxy:ProxyBrowserPool—browserfeature flag wires the proxy pool intostygian-browser; browsers are launched with a live proxy from the pool and the proxy is re-evaluated on browser releasestygian-graph:RestApiAdapterproxy integration via stygian-proxy (graphfeature on stygian-proxy)book:stygian-proxymdBook module added — architecture overview, circuit breaker semantics, rotation strategies, and browser/graph integration guides
Changed
- Workspace MSRV updated to Rust 1.94.0 (aligned with stable feature usage: async closures,
LazyLock, let chains)
Fixed
stygian-proxy: eliminated TOCTOU race inadd_proxy— thecircuit_breakerswrite lock is now held for the full duration ofstorage.add(), ensuringacquire_proxycan never observe a proxy record without a corresponding circuit breakerstygian-browser: CDP protection removescdc_*anddomAutomationautomation artifact properties fromwindowon every navigation, reducing fingerprint surface for Akamai/PerimeterX detectors
Tests
stygian-graph: 12 live integration tests against crawllab.dev (all#[ignore]) — status code classification (200/404/429/500), redirect following, redirect cycle detection, JSON/text/HTML content types, 204 No Content, and paginated forum endpointstygian-browser: 4 live browser integration tests against crawllab.dev (all#[ignore]) — inline JS rendering, external script rendering, basic navigation, and JS eval with stealth verification
v0.1.19
Changed
stygian-graph:CloudflareCrawlAdapter::new()andwith_config()now returnResult<Self>instead of panicking on reqwest TLS init failure;Defaultimpl removed (breaking change for direct construction)
Fixed
stygian-graph:RestApiAdapter::parse_specbody selection usesmatchinstead ofif let/elsechain (lint)stygian-browser:BrowserPool::acquire_innerfast path drops mutex guard before spawning disposal tasks, reducing lock contention under unhealthy-pool conditions
v0.1.18
Fixed
stygian-graph:RestApiAdapternow checksbody_rawbeforebodywhen both are present, matching the documented precedence contractstygian-graph:RestApiAdapter429 responses now returnServiceError::RateLimitedwith the parsedRetry-Aftervalue;send_onehonours the server-specified delay instead of blind exponential backoffstygian-graph: token-bucket rate limiter guards againstmax_requests = 0or zero-duration window configs that previously caused a division-by-zero panic viaDuration::from_secs_f64(inf)stygian-graph:CloudflareCrawlAdapter::with_confignow panics with a clear message on TLS init failure instead of silently falling back to a misconfigured defaultreqwest::Clientbook: Cloudflare crawl adapter metadata example corrected tojob_id,pages_crawled,output_format(waspages,url_count)book:HeadlessMode::Legacydocs across configuration and env-vars pages corrected to "classic--headlessfor Chromium < 112" (was incorrectly referencingchrome-headless-shelland Chrome 132 removal)
v0.1.17
Added
stygian-browser: context-scoped browser pool segregation —BrowserPool::acquire_for(context_id)returns browsers isolated by context (bot, tenant, etc.) so multiple consumers share one pool without cross-contamination;release_context(id)drains all idle browsers for a context;context_ids()lists active contexts;BrowserHandle::context_id()exposes the owning contextstygian-browser: pool eviction now walks both shared and per-context queues, pruning empty context entries automatically
Changed
stygian-browser:PoolInnerinternal structure changed from singleVecDequeto shared +HashMap<String, VecDeque>scoped queues (no public API break —acquire()remains fully backward-compatible)
v0.1.16
Changed
- Workspace license updated to
AGPL-3.0-only OR LicenseRef-Commercialdual licensing; addedLICENSE-COMMERCIAL.md thiserror1.0 → 2.0 (unifies with chromiumoxide transitive dep)tomlpinned at 0.8 (unifies with figment; previous bump to 1.0 caused duplicate)reqwest0.12 → 0.13 (featurerustls-tlsrenamed torustls; addedqueryfeature)toml0.8 → 1.0scraper0.20 → 0.25mockall0.13 → 0.14criterion0.5 → 0.8prometheus-client0.22 → 0.24indicatif0.17 → 0.18
v0.1.15
Added
stygian-graph:RestApiAdapter— flexible REST JSON API adapter with 5 auth schemes (Bearer, Basic, API key header/query, none), 4 pagination strategies (none, offset, cursor, RFC 8288 Link header), dot-path JSON response extraction, configurable retries with exponential backoff, and 24 unit tests; registered as"rest-api"stygian-graph:CloudflareCrawlAdapter— delegates whole-site crawling to the Cloudflare Browser Rendering/crawlendpoint (open beta); polls until complete, aggregates page results, configurable poll interval and job timeout; gated behindcloudflare-crawlfeature flagexamples/rest-api-scrape.toml— example pipeline demonstrating unauthenticated GET, Bearer-auth + Link-header pagination, and API-key + cursor pagination patterns
Fixed
stygian-graph: resolved allclippy -D warningslint failures inrest_api.rsandcloudflare_crawl.rs—indexing_slicing,map_unwrap_or,manual_map,if_not_else,option_if_let_else,unnecessary_map_or,cast_possible_truncation,ignore_without_reason,panicin tests
v0.1.14
Fixed
stygian-graph: corrected broken rustdoc intra-doc links ingraphql_throttlemodule (pre_flight_delay→pre_flight_reserve, removed unresolvable link brackets from module-level prose)stygian-graph: fixedcargo fmtimport ordering ingraphql.rs(graphql_rate_limitbeforegraphql_throttle)
v0.1.13
Added
stygian-graph:RateLimitConfig(port layer) +RequestRateLimit/RequestWindowsliding-window request-count rate limiter — complements the existing leaky-bucket cost throttle; tracks up tomax_requestsin any rollingwindowduration using aVecDeque<Instant>;rate_limit_acquiresleeps until a slot is free before each request;rate_limit_retry_afterimposes a hard block to honour server-returnedRetry-Afterheaders (a shorter value can never shorten an existing block);parse_retry_afterparses integer seconds from a header stringstygian-graph:GraphQlTargetPlugin::rate_limit_config()default method — plugins opt in to per-plugin request-count limiting by returning aRateLimitConfig; operates in parallel withcost_throttle_config()and both can be active simultaneously