integrate astro

Author: gildesmarais

.github/workflows/frontend.yml

@@ -55,16 +55,33 @@ jobs:
           cache: "npm"
           cache-dependency-path: frontend/package-lock.json
 
+      - name: Setup Ruby
+        uses: ruby/setup-ruby@v1
+        with:
+          bundler-cache: true
+
       - name: Install dependencies
         run: |
           cd frontend
           npm ci
+          cd ..
+          bundle install
 
       - name: Check for vulnerabilities
         run: |
           cd frontend
           npm audit --audit-level=moderate
 
+      - name: Run unit tests
+        run: |
+          cd frontend
+          npm run test:unit
+
+      - name: Run integration tests
+        run: |
+          cd frontend
+          npm run test:integration
+
       - name: Build and verify
         run: |
           cd frontend

.github/workflows/test_build_push.yml

@@ -33,10 +33,22 @@ jobs:
           cache: "npm"
           cache-dependency-path: frontend/package-lock.json
 
-      - name: Install frontend dependencies
+      - name: Setup Ruby
+        uses: ruby/setup-ruby@v1
+        with:
+          bundler-cache: true
+
+      - name: Install dependencies
         run: |
           cd frontend
           npm ci
+          cd ..
+          bundle install
+
+      - name: Run frontend tests
+        run: |
+          cd frontend
+          npm run test:ci
 
       - name: Build frontend
         run: |

.tool-versions

@@ -0,0 +1 @@
+ruby 3.4.1

.yardopts

@@ -0,0 +1,10 @@
+--markup markdown
+--charset utf-8
+--exclude spec/
+--exclude tmp/
+--exclude log/
+--exclude public/
+--exclude config/
+--exclude bin/
+--exclude .devcontainer/
+--exclude .github/

CONFIGURATION.md

@@ -0,0 +1,88 @@
+# Configuration Guide
+
+## Environment Variables
+
+### Auto Source Configuration
+
+| Variable                      | Description                            | Default           | Example                                               |
+| ----------------------------- | -------------------------------------- | ----------------- | ----------------------------------------------------- |
+| `AUTO_SOURCE_ENABLED`         | Enable auto source feature             | `false`           | `true`                                                |
+| `AUTO_SOURCE_USERNAME`        | Basic auth username                    | Required          | `admin`                                               |
+| `AUTO_SOURCE_PASSWORD`        | Basic auth password                    | Required          | `changeme`                                            |
+| `AUTO_SOURCE_ALLOWED_ORIGINS` | Allowed request origins                | Required          | `localhost:3000,example.com`                          |
+| `AUTO_SOURCE_ALLOWED_URLS`    | **URL whitelist for public instances** | `""` (allows all) | `https://github.com/*,https://news.ycombinator.com/*` |
+
+### Health Check Configuration
+
+| Variable                | Description           | Default        | Example    |
+| ----------------------- | --------------------- | -------------- | ---------- |
+| `HEALTH_CHECK_USERNAME` | Health check username | Auto-generated | `health`   |
+| `HEALTH_CHECK_PASSWORD` | Health check password | Auto-generated | `changeme` |
+
+### Ruby Integration
+
+| Variable    | Description                | Default | Example         |
+| ----------- | -------------------------- | ------- | --------------- |
+| `RUBY_PATH` | Path to Ruby executable    | `ruby`  | `/usr/bin/ruby` |
+| `APP_ROOT`  | Application root directory | `.`     | `/app`          |
+
+## URL Restriction Patterns
+
+The `AUTO_SOURCE_ALLOWED_URLS` variable supports:
+
+- **Exact URLs**: `https://example.com/news`
+- **Wildcard patterns**: `https://example.com/*` (matches any path)
+- **Domain patterns**: `https://*.example.com` (matches subdomains)
+- **Multiple patterns**: Comma-separated list
+
+### Examples
+
+```bash
+# Allow only specific sites
+AUTO_SOURCE_ALLOWED_URLS=https://github.com/*,https://news.ycombinator.com/*,https://example.com/news
+
+# Allow all subdomains of a domain
+AUTO_SOURCE_ALLOWED_URLS=https://*.example.com/*
+
+# Allow everything (for private instances)
+AUTO_SOURCE_ALLOWED_URLS=
+
+# Block everything (disable auto source)
+AUTO_SOURCE_ENABLED=false
+```
+
+## Security Considerations
+
+### Public Instances
+- **Always set** `AUTO_SOURCE_ALLOWED_URLS` to restrict URLs
+- Use strong authentication credentials
+- Monitor usage and set up rate limiting
+- Consider IP whitelisting for additional security
+
+### Private Instances
+- Leave `AUTO_SOURCE_ALLOWED_URLS` empty to allow all URLs
+- Still use authentication to prevent unauthorized access
+- Consider network-level restrictions
+
+## Deployment Examples
+
+### Public Demo Instance
+```bash
+AUTO_SOURCE_ENABLED=true
+AUTO_SOURCE_USERNAME=demo
+AUTO_SOURCE_PASSWORD=secure_password
+AUTO_SOURCE_ALLOWED_URLS=https://github.com/*,https://news.ycombinator.com/*,https://example.com/*
+```
+
+### Private Instance
+```bash
+AUTO_SOURCE_ENABLED=true
+AUTO_SOURCE_USERNAME=admin
+AUTO_SOURCE_PASSWORD=very_secure_password
+AUTO_SOURCE_ALLOWED_URLS=
+```
+
+### Disabled Auto Source
+```bash
+AUTO_SOURCE_ENABLED=false
+```

Makefile

@@ -30,8 +30,21 @@ dev-ruby: ## Start Ruby server only
 dev-frontend: ## Start Astro dev server only
 	@cd frontend && npm run dev
 
-test: ## Run tests
+test: ## Run all tests (Ruby + Frontend)
 	bundle exec rspec
+	@cd frontend && npm run test:ci
+
+test-ruby: ## Run Ruby tests only
+	bundle exec rspec
+
+test-frontend: ## Run frontend tests only
+	@cd frontend && npm run test:ci
+
+test-frontend-unit: ## Run frontend unit tests only
+	@cd frontend && npm run test:unit
+
+test-frontend-integration: ## Run frontend integration tests only
+	@cd frontend && npm run test:integration
 
 lint: ## Run linter
 	bundle exec rubocop

README.md

@@ -14,13 +14,25 @@ This web application scrapes websites to build and deliver RSS 2.0 feeds with a
 - **Pre-built Configs**: Comes with plenty of [included configs](https://html2rss.github.io/web-application/how-to/use-included-configs)
 - **Performance**: Handles request caching and sets caching-related HTTP headers
 - **Progressive Enhancement**: Works without JavaScript, enhanced with modern features
+- **Security**: URL restrictions, authentication, SSRF protection, and input validation
 
 **Architecture:**
 
 - **Backend**: Ruby + Roda for API and RSS generation
 - **Frontend**: Astro for modern, fast static site generation
 - **Core Engine**: [`html2rss`](https://github.com/html2rss/html2rss) Ruby gem for feed generation
 
+## Configuration
+
+The application can be configured using environment variables. See the [configuration guide](CONFIGURATION.md) for details.
+
+### Security Features
+
+- **URL Restrictions**: Public instances can restrict auto source to specific URLs
+- **Authentication**: Basic auth for auto source and health check endpoints  
+- **SSRF Protection**: Built-in protection against Server-Side Request Forgery
+- **Input Validation**: Comprehensive validation of all inputs
+
 ## Documentation
 
 For full documentation, please see the [html2rss-web documentation](https://html2rss.github.io/web-application/).