more readme and Dockerfile update

Author: metalwarrior665

CHANGELOG.md

@@ -6,4 +6,5 @@
 - Removed `useGoogleBotHeaders` option (we don't want to impersonate Google anyway)
 - Updated `apify` from `0.18.1` to `1.3.1`
 - `saveSnapshots` is `true` by default
+- Added recognition of Amazon's `hCaptcha`
 - `success` and `wasSuccess` metrics added to output. Success is measured by status being less than 400 and no captcha

Dockerfile

@@ -1,15 +1,5 @@
-# Dockerfile contains instructions how to build a Docker image that will contain
-# all the code and configuration needed to run your actor. For a full
-# Dockerfile reference, see https://docs.docker.com/engine/reference/builder/
 
-# First, specify the base Docker image. Apify provides the following base images
-# for your convenience:
-#  apify/actor-node-basic (Node.js 10 on Alpine Linux, small and fast image)
-#  apify/actor-node-chrome (Node.js 10 + Chrome on Debian)
-#  apify/actor-node-chrome-xvfb (Node.js 10 + Chrome + Xvfb on Debian)
-# For more information, see https://apify.com/docs/actor#base-images
-# Note that you can use any other image from Docker Hub.
-FROM apify/actor-node-chrome-xvfb
+FROM apify/actor-node-puppeteer-chrome
 
 # Second, copy just package.json and package-lock.json since they are the only files
 # that affect NPM install in the next step

README.md

@@ -1,4 +1,35 @@
 ## Website Checker
+Website checker is a simple actor that allows you to scan any website for performance and blocking.
+
+### Features
+The actor provides these useful features out of the box:
+- Collects response status codes
+- Recognizes the most common captchas
+- Saves HTML snapshots and screenshots (if Puppeteer is chosen)
+- Enables choosing between Cheerio (plain HTTP) and Puppeteer (browser) scraper
+- Enables re-scraping start URLs or enqueueing with a familiar link selector + pseudo URLs system
+- Handles different failure states like timeouts and network errors
+- Enables basic proxy and browser configuration
+
+#### Planned features
+- Usage calculation/stats
+- Better automatic workloads/workload actors
+- Add support for Playwright + Firefox
+
+### How to use
+The most common use-case is to do a quick check on how aggressively the target site is blocking. In that case just supply a start URL, ideally a category one or product one. You can either set `replicateStartUrls` or add enqueueing with `linkSelector` + `pseudoUrls`, both are good options to test different proxies. You can test a few different proxy groups and compare `cheerio` vs `puppeteer` options.
+
+In the end you will get a simple statistics about the blocking rate. It is recommended to check a few screenshots just to make sure the actor correctly recognized the page status. You can get to the detailed output (per URL) via KV store or dataset (the KV output sorts by response status while dataset is simply ordered by scraping order).
+
+#### Checker workloads
+To make your life easier, you can use other actors that will start more checker runs at once and aggregate the result. This way you can test more sites at once or different cheerio/browser and proxy combinations and compare those.
+
+All of these actors are very young so we are glad for any feature ideas:
+[lukaskrivka/website-checker-workload](https://apify.com/lukaskrivka/website-checker-workload)
+[vaclavrut/website-checker-starter](https://apify.com/vaclavrut/website-checker-starter)
+
+### Input
+Please follow the [actor's input page](https://apify.com/lukaskrivka/website-checker/input-schema) for a detailed explanation. Most input fields have reasonable defaults.
 
 ### Example output
 
@@ -10,15 +41,20 @@
     "accessDenied": 0,
     "recaptcha": 0,
     "distilCaptcha": 24,
+    "hCaptcha": 0, 
     "statusCodes": {
         "200": 3,
         "401": 2,
         "403": 5,
         "405": 24
     },
+    "success": 3,
     "total": 43
 }
 ```
 
 #### Detailed output with URLs, screenshots and HTML links
 https://api.apify.com/v2/key-value-stores/zT3zxpd53Wv9m9ukQ/records/DETAILED-OUTPUT?disableRedirect=true
+
+### Changelog
+Check history of changes in the [CHANGELOG](https://github.com/metalwarrior665/actor-website-checker/blob/master/CHANGELOG.md)

src/main.js

@@ -139,7 +139,7 @@ Apify.main(async () => {
 
         const wasSuccess = statusCode < 400 && captchas.length === 0;
         if (wasSuccess) {
-            state.wasSuccess.push({ url: request.url, screenshotUrl, htmlUrl });
+            state.success.push({ url: request.url, screenshotUrl, htmlUrl });
         }
 
         await Apify.pushData({