Spider SEO Auditor

Crawl websites and audit SEO elements — titles, meta descriptions, headings, and Open Graph tags using Spider Cloud — the fastest web crawling infrastructure.

Live Demo: https://seo-auditor.spider.cloud

Features

• Crawl any website using Spider Cloud API
• Real-time JSONL streaming results
• Supabase authentication (GitHub & Discord)
• Local IndexedDB storage for caching results
• Dark theme matching spider.cloud branding
• Configurable crawl settings (limit, format, request type)

Getting Started

Prerequisites

• Node.js 18+
• A Spider Cloud account with API credits
• Supabase project for authentication

Setup

1. Clone the repository:

git clone https://github.com/spider-rs/spider-seo-auditor.git
cd spider-seo-auditor

2. Install dependencies:

npm install

3. Configure environment variables:

cp .env.local .env

Edit .env and add your Supabase and Spider Cloud credentials:

NEXT_PUBLIC_SUPABASE_URL=your_supabase_url
NEXT_PUBLIC_SUPABASE_ANON_KEY=your_supabase_anon_key
NEXT_PUBLIC_API_URL=https://api.spider.cloud

4. Run the development server:

npm run dev

Open http://localhost:3001 in your browser.

Tech Stack

• Next.js 14 — React framework
• Tailwind CSS — Styling
• shadcn/ui — UI components
• Supabase — Authentication
• Spider Cloud — Web crawling API

License

MIT