Skip to content

Commit 7086252

Browse files
Add price scraper and GitHub Actions workflow
Co-authored-by: willtheorangeguy <18339050+willtheorangeguy@users.noreply.github.com>
1 parent 97949a1 commit 7086252

File tree

5 files changed

+393
-0
lines changed

5 files changed

+393
-0
lines changed
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
name: Update PC Part Prices
2+
3+
on:
4+
schedule:
5+
# Run daily at 2 AM UTC (adjust as needed)
6+
- cron: '0 2 * * *'
7+
workflow_dispatch: # Allow manual triggering
8+
9+
jobs:
10+
update-prices:
11+
runs-on: ubuntu-latest
12+
13+
permissions:
14+
contents: write
15+
16+
steps:
17+
- name: Checkout repository
18+
uses: actions/checkout@v5
19+
with:
20+
fetch-depth: 0
21+
22+
- name: Set up Python
23+
uses: actions/setup-python@v5
24+
with:
25+
python-version: '3.12'
26+
cache: 'pip'
27+
28+
- name: Install dependencies
29+
run: |
30+
python -m pip install --upgrade pip
31+
pip install -r requirements.txt
32+
33+
- name: Run price scraper
34+
run: |
35+
python scraper.py
36+
37+
- name: Check for changes
38+
id: git-check
39+
run: |
40+
git diff --exit-code || echo "changes=true" >> $GITHUB_OUTPUT
41+
42+
- name: Commit and push changes
43+
if: steps.git-check.outputs.changes == 'true'
44+
run: |
45+
git config --local user.email "github-actions[bot]@users.noreply.github.com"
46+
git config --local user.name "github-actions[bot]"
47+
git add .
48+
git commit -m "chore: update PC part prices [automated]"
49+
git push
50+
env:
51+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.gitignore

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# Python
2+
__pycache__/
3+
*.py[cod]
4+
*$py.class
5+
*.so
6+
.Python
7+
env/
8+
venv/
9+
ENV/
10+
build/
11+
develop-eggs/
12+
dist/
13+
downloads/
14+
eggs/
15+
.eggs/
16+
lib/
17+
lib64/
18+
parts/
19+
sdist/
20+
var/
21+
wheels/
22+
*.egg-info/
23+
.installed.cfg
24+
*.egg
25+
26+
# IDE
27+
.vscode/
28+
.idea/
29+
*.swp
30+
*.swo
31+
*~
32+
33+
# OS
34+
.DS_Store
35+
Thumbs.db
36+
37+
# Testing
38+
.pytest_cache/
39+
.coverage
40+
htmlcov/
41+
42+
# Logs
43+
*.log

README.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@
5050
- Available as PCPartPicker lists, Markdown files, or on a website.
5151
- Markdown files and website show the original printed price.
5252
- Current prices are available in United States Dollar or Canadian Dollar.
53+
- **Automated daily price updates** - Prices are scraped from retailers (Newegg, Amazon, Best Buy) and updated automatically via GitHub Actions.
5354
- Cross platform.
5455

5556
## Download
@@ -157,6 +158,25 @@ Each of the issues has its builds listed in three different places, with either
157158
| October 2021 | AMD Turbo | [PCPartPicker](https://pcpartpicker.com/user/willtheornageguy/saved/VBjMFT) | [Markdown](/2021/October/AMD%20Turbo.md) | [Web](https://willtheorangeguy.github.io/Maximum-PC-Builds-Archive/2021/october/) |
158159
| October 2021 | Intel Turbo | [PCPartPicker](https://pcpartpicker.com/user/willtheornageguy/saved/F4s7wP) | [Markdown](/2021/October/Intel%20Turbo.md) | [Web](https://willtheorangeguy.github.io/Maximum-PC-Builds-Archive/2021/october/) |
159160

161+
## Automated Price Updates
162+
163+
This repository includes an automated price scraper that runs daily to keep component prices up-to-date. The scraper:
164+
165+
- Runs automatically every day at 2 AM UTC via GitHub Actions
166+
- Scrapes current prices from PCPartPicker (which aggregates prices from retailers like Newegg, Amazon, and Best Buy)
167+
- Updates the markdown files with the latest pricing information
168+
- Can be manually triggered using the "Update PC Part Prices" workflow in the Actions tab
169+
170+
The price scraper is implemented in Python and uses BeautifulSoup to parse PCPartPicker's build lists. If you want to run it manually:
171+
172+
```bash
173+
# Install dependencies
174+
pip install -r requirements.txt
175+
176+
# Run the scraper
177+
python scraper.py
178+
```
179+
160180
## Contributing
161181

162182
Please contribute using [GitHub Flow](https://guides.github.com/introduction/flow). Create a branch, add commits, and [open a pull request](https://github.com/willtheorangeguy/PyWorkout/compare).

requirements.txt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
beautifulsoup4==4.12.3
2+
requests==2.32.3
3+
lxml==5.3.0

0 commit comments

Comments
 (0)