@@ -42,6 +42,7 @@ This repository includes an automated price scraper that updates PC component pr
4242### Data Source
4343
4444The scraper uses ** PCPartPicker** as the data source because:
45+
4546- PCPartPicker already aggregates prices from multiple retailers (Newegg, Amazon, Best Buy, etc.)
4647- Each build in the repository already has a PCPartPicker list URL
4748- PCPartPicker handles the complexity of tracking product availability across retailers
@@ -75,6 +76,7 @@ python scraper.py
7576```
7677
7778The script will:
79+
78801 . Find all markdown files in year directories (2018/, 2020/, 2021/, etc.)
79812 . Extract PCPartPicker URLs from each file
80823 . Scrape current prices
@@ -102,6 +104,7 @@ The script will:
102104### Automatic Execution
103105
104106The workflow runs automatically:
107+
105108- ** Schedule** : Daily at 2:00 AM UTC
106109- ** Trigger** : Can also be manually triggered via GitHub Actions UI
107110
@@ -118,10 +121,11 @@ The workflow runs automatically:
118121``` yaml
119122# .github/workflows/update-prices.yml
120123schedule :
121- - cron : ' 0 2 * * *' # Daily at 2 AM UTC
124+ - cron : " 0 2 * * *" # Daily at 2 AM UTC
122125` ` `
123126
124127To change the schedule, modify the cron expression:
128+
125129- ` ' 0 */6 * * *' ` - Every 6 hours
126130- ` ' 0 0 * * 1' ` - Every Monday at midnight
127131- ` ' 0 12 * * *' ` - Daily at noon
@@ -135,13 +139,14 @@ The scraper expects markdown files with this table format:
135139
136140[PCPartPicker Part List](https://ca.pcpartpicker.com/list/8gGn9r)
137141
138- | Type | Item | Price | Print Price |
139- | :--- | :--- | :--- | :--- |
140- | **CPU** | [AMD Ryzen 3 1200...](https://ca.pcpartpicker.com/product/...) | $276.90 @ Amazon Canada | $110.00 |
141- | **Memory** | [Patriot Viper Elite...](https://ca.pcpartpicker.com/product/...) | - | $77.00 |
142+ | Type | Item | Price | Print Price |
143+ | :--------- | :---------------------------------------------------------------- | :---------------------- | :------- --- |
144+ | **CPU** | [AMD Ryzen 3 1200...](https://ca.pcpartpicker.com/product/...) | $276.90 @ Amazon Canada | $110.00 |
145+ | **Memory** | [Patriot Viper Elite...](https://ca.pcpartpicker.com/product/...) | - | $77.00 |
142146```
143147
144148The scraper:
149+
145150- Extracts the PCPartPicker list URL from the header
146151- Parses the table to find product names
147152- Updates the "Price" column with current prices
@@ -150,6 +155,7 @@ The scraper:
150155## Supported Retailers
151156
152157The scraper recognizes these retailers:
158+
153159- Amazon Canada (amazon.ca, amazon.com)
154160- Newegg Canada (newegg.ca, newegg.com)
155161- Best Buy Canada (bestbuy.ca, bestbuy.com)
@@ -161,13 +167,15 @@ The scraper recognizes these retailers:
161167### No Prices Found
162168
163169If the scraper reports "No prices scraped":
170+
1641711 . Check that the PCPartPicker URL is valid
1651722 . Verify the PCPartPicker page loads in a browser
1661733 . Check GitHub Actions logs for detailed error messages
167174
168175### Prices Not Updating
169176
170177Common causes:
178+
1711791 . Products are out of stock (shows as "-")
1721802 . PCPartPicker page structure changed (may need scraper update)
1731813 . Network issues during GitHub Actions run
@@ -201,6 +209,7 @@ To add support for a new retailer:
2012094 . Commit and push
202210
203211Example:
212+
204213``` python
205214trusted_retailers = {
206215 # ... existing retailers ...
@@ -219,6 +228,7 @@ trusted_retailers = {
219228## Future Improvements
220229
221230Potential enhancements:
231+
222232- [ ] Support for US pricing (pcpartpicker.com)
223233- [ ] Price history tracking
224234- [ ] Email notifications when prices drop significantly
@@ -229,6 +239,7 @@ Potential enhancements:
229239## Support
230240
231241For issues or questions:
242+
2322431 . Check existing Issues on GitHub
2332442 . Review GitHub Actions logs for errors
2342453 . Open a new Issue with detailed information
0 commit comments