LinkedIn Job Scraper (Node.js)

A simple and efficient Node.js script to scrape job listings from LinkedIn's guest API.

Installation

1. Clone this repository or navigate to the project folder.
2. Install dependencies:

npm install

Usage

Run the scraper from the command line with the following arguments:

node index.js -t "job title" -l "location" [-r number of results] [-h hours]

Arguments

• -t (required): Job title or search term (can be multiple words)
• -l (required): Location (can be multiple words)
• -r (optional): Number of results wanted (default: 10)
• -h (optional): Filter jobs posted within the last N hours

Examples

Search for software engineer jobs in United Kingdom, get 200 results, filter to last 168 hours:

node index.js -t software engineer -l United Kingdom -r 200 -h 168

Search for data scientist jobs in New York, get 50 results:

node index.js -t data scientist -l New York -r 50

Search for frontend developer jobs in London, filter to last 24 hours:

node index.js -t frontend developer -l London -h 24

Output

The script outputs the results as a JSON array in the terminal. Each job object contains:

• id: Job ID
• title: Job title
• company: Company name
• location: Job location
• date: Date posted
• jobUrl: Full LinkedIn job URL
• jobType: Employment type (Full-time, Part-time, Contract, etc.)
• salary: Salary information (if available)
• description: Job description

Considerations

Rate Limiting

• LinkedIn implements rate limiting to prevent abuse. The script includes automatic delays between requests (1-2 seconds for job details, 2 seconds between pages).
• If you encounter a 429 (Too Many Requests) status, the script will wait 5 seconds before retrying.
• For large result sets, scraping may take significant time due to these delays.

Legal & Ethical

• This tool is for educational purposes only.
• Scraping LinkedIn may violate their Terms of Service.
• Use responsibly and respect LinkedIn's infrastructure.
• Consider using LinkedIn's official API for production use cases.

Performance

• Each job detail requires a separate API call, which increases scraping time for large result sets.
• Network timeouts are set to 10 seconds for search requests and 5 seconds for detail requests.
• The script processes jobs sequentially to avoid overwhelming the server.

Data Accuracy

• Job data is scraped from LinkedIn's guest API and may not always be complete or up-to-date.
• Some fields (like salary) may not be available for all job postings.
• Job descriptions are cleaned to remove "See more" and "See less" text, but formatting may vary.

Error Handling

• The script continues processing even if individual job details fail to fetch.
• Network errors and API failures are logged but don't stop the entire scraping process.
• Invalid command-line arguments will cause the script to exit with an error message.