You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
- Updated release notes with comprehensive feature documentation
- Updated package metadata with detailed release notes
- Version 0.1.0 includes initial release with all core functionality
LinkValidator is a fast, reliable CLI tool for crawling websites and validating both internal and external links. Built with Akka.NET for high-performance concurrent crawling.
6
+
7
+
**Features:**
8
+
-**Fast Concurrent Crawling** - Leverages Akka.NET actors for efficient parallel processing of websites
9
+
-**Smart External Link Handling** - Implements intelligent rate limiting with configurable retry policies for HTTP 429 responses
10
+
-**Comprehensive Reporting** - Generates detailed markdown reports of all discovered links and their status
11
+
-**Broken Link Tracking** - Tracks broken external links and provides detailed reporting on link status
12
+
-**Advanced Error Handling** - Properly handles HTTP status codes 400+ as errors while allowing redirects and other success responses
13
+
-**Link Graph Analysis** - Tracks relationships between pages to identify where broken links originate ([#32](https://github.com/Aaronontheweb/link-validator/pull/32))
14
+
-**CI/CD Ready** - Perfect for automated testing in build pipelines with strict mode and exit codes
15
+
-**Cross-Platform** - Single-file binaries for Windows, Linux, and macOS (Intel + Apple Silicon)
16
+
-**Flexible Configuration** - CLI flags and environment variables for easy customization
17
+
18
+
**Major Components:**
19
+
-**HTTP 429 Rate Limit Handling** - Automatically detects and respects `Retry-After` headers, with configurable fallback delays and jitter to prevent thundering herd issues ([#63](https://github.com/Aaronontheweb/link-validator/pull/63))
20
+
-**External Link Validation** - Comprehensive tracking and reporting of broken external URLs separate from internal site structure ([#62](https://github.com/Aaronontheweb/link-validator/pull/62))
21
+
-**Smart HTTP Status Code Handling** - Only treats HTTP status codes 400 and above as errors, properly handling redirects and other success responses ([#61](https://github.com/Aaronontheweb/link-validator/pull/61))
22
+
-**Release Infrastructure** - Complete release automation with configuration options and build system integration ([#64](https://github.com/Aaronontheweb/link-validator/pull/64))
23
+
24
+
**Installation:**
25
+
- Install scripts available for Windows PowerShell and Linux/macOS Bash
26
+
- Pre-built single-file binaries for all major platforms
27
+
- Build from source with .NET 9 SDK
28
+
29
+
**Command Line Options:**
30
+
-`--url <URL>` - The website URL to crawl (required)
31
+
-`--output <PATH>` - Save sitemap report to file
32
+
-`--diff <PATH>` - Compare against previous sitemap file
33
+
-`--strict` - Return error code if broken links found
34
+
-`--max-external-retries <N>` - Max retries for external 429 responses (default: 3)
35
+
-`--retry-delay-seconds <N>` - Default retry delay when no Retry-After header present (default: 10)
36
+
37
+
**Environment Variable Support:**
38
+
-`LINK_VALIDATOR_MAX_EXTERNAL_RETRIES` - Configure max retry attempts
LinkValidator is a fast, reliable CLI tool for crawling websites and validating both internal and external links. Built with Akka.NET for high-performance concurrent crawling.
9
+
10
+
**Features:**
11
+
- **Fast Concurrent Crawling** - Leverages Akka.NET actors for efficient parallel processing of websites
12
+
- **Smart External Link Handling** - Implements intelligent rate limiting with configurable retry policies for HTTP 429 responses
13
+
- **Comprehensive Reporting** - Generates detailed markdown reports of all discovered links and their status
14
+
- **Broken Link Tracking** - Tracks broken external links and provides detailed reporting on link status
15
+
- **Advanced Error Handling** - Properly handles HTTP status codes 400+ as errors while allowing redirects and other success responses
16
+
- **Link Graph Analysis** - Tracks relationships between pages to identify where broken links originate ([#32](https://github.com/Aaronontheweb/link-validator/pull/32))
17
+
- **CI/CD Ready** - Perfect for automated testing in build pipelines with strict mode and exit codes
18
+
- **Cross-Platform** - Single-file binaries for Windows, Linux, and macOS (Intel + Apple Silicon)
19
+
- **Flexible Configuration** - CLI flags and environment variables for easy customization
20
+
21
+
**Major Components:**
22
+
- **HTTP 429 Rate Limit Handling** - Automatically detects and respects `Retry-After` headers, with configurable fallback delays and jitter to prevent thundering herd issues ([#63](https://github.com/Aaronontheweb/link-validator/pull/63))
23
+
- **External Link Validation** - Comprehensive tracking and reporting of broken external URLs separate from internal site structure ([#62](https://github.com/Aaronontheweb/link-validator/pull/62))
24
+
- **Smart HTTP Status Code Handling** - Only treats HTTP status codes 400 and above as errors, properly handling redirects and other success responses ([#61](https://github.com/Aaronontheweb/link-validator/pull/61))
25
+
- **Release Infrastructure** - Complete release automation with configuration options and build system integration ([#64](https://github.com/Aaronontheweb/link-validator/pull/64))
26
+
27
+
**Installation:**
28
+
- Install scripts available for Windows PowerShell and Linux/macOS Bash
29
+
- Pre-built single-file binaries for all major platforms
30
+
- Build from source with .NET 9 SDK
31
+
32
+
**Command Line Options:**
33
+
- `--url <URL>` - The website URL to crawl (required)
34
+
- `--output <PATH>` - Save sitemap report to file
35
+
- `--diff <PATH>` - Compare against previous sitemap file
36
+
- `--strict` - Return error code if broken links found
37
+
- `--max-external-retries <N>` - Max retries for external 429 responses (default: 3)
38
+
- `--retry-delay-seconds <N>` - Default retry delay when no Retry-After header present (default: 10)
39
+
40
+
**Environment Variable Support:**
41
+
- `LINK_VALIDATOR_MAX_EXTERNAL_RETRIES` - Configure max retry attempts
0 commit comments