Skip to content

Commit 4682dab

Browse files
Update country configuration to GB and update data
1 parent 3ba9795 commit 4682dab

File tree

4 files changed

+144
-342
lines changed

4 files changed

+144
-342
lines changed

bing-scraper/bing.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717
# bypass Bing web scraping blocking
1818
"asp": True,
1919
# set the poxy location to US to get the result in English
20-
"country": "US",
20+
"country": "GB",
2121
"proxy_pool": "public_residential_pool",
2222
}
2323

@@ -130,6 +130,7 @@ async def scrape_search(query: str, max_pages: int = None):
130130
async def scrape_keywords(query: str):
131131
"""scrape bing search pages for keyword data"""
132132
url = f"https://www.bing.com/search?{urlencode({'q': query})}"
133+
print(f"url:{url}")
133134
log.info("scraping Bing search for keyword data")
134135
response = await SCRAPFLY.async_scrape(ScrapeConfig(url, **BASE_CONFIG, render_js=True))
135136
keyword_data = parse_keywords(response)

bing-scraper/results/keywords.json

Lines changed: 24 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -1,56 +1,29 @@
11
{
2-
"FAQs": [
3-
{
4-
"query": "How to scrape emails from a website?",
5-
"answer": "Next, you will need a web scraper that can scrape emails from any website. For this example, we will download and install ParseHub, a free and powerful web scraper that works with any website. Now it’s time to get scraping. Open ParseHub and click on “New Project”. Then enter the URL of the page you will want to scrape.",
6-
"title": "How to Scrape Emails from any Website | ParseHub",
7-
"domain": "parsehub.com",
8-
"url": "https://www.parsehub.com/blog/scrape-emails/"
9-
},
10-
{
11-
"query": "How to scrape email addresses?",
12-
"answer": "Define a list of websites from which you want to scrape email addresses. This can be done manually or by using bots, chrome extensions, or any other scraping tool. Use a web scraping tool to crawl the websites in your list. The tool will extract the email addresses from the websites. Save the email addresses to a database or a file.",
13-
"title": "Email Scraping: Use Cases & Best Practices - HasData",
14-
"domain": "hasdata.com",
15-
"url": "https://hasdata.com/blog/email-scraping"
16-
},
17-
{
18-
"query": "What is email scraping?",
19-
"answer": "Email scraping, or email harvesting, refers to the ",
20-
"title": "Email scraping: How-to & top email scraping tools in 2025 - AIMultiple",
21-
"domain": "research.aimultiple.com",
22-
"url": "https://research.aimultiple.com/email-scraping/"
23-
},
24-
{
25-
"query": "Should you use a web scraper for email marketing?",
26-
"answer": "With the help of a free web scraper and by carefully selecting your lead sources, you can quickly build a high-quality email list. You can then use this list for email marketing efforts or use it as a custom audience in Google or Facebook Ads. Also, many companies web scrape email lists for their prospecting or cold email outreach.",
27-
"title": "How to Scrape Emails from any Website | ParseHub",
28-
"domain": "parsehub.com",
29-
"url": "https://www.parsehub.com/blog/scrape-emails/"
30-
},
31-
{
32-
"query": "How do I use web scrape to expand my email list?",
33-
"answer": " will collect the selected data, and you'll receive an email notification. ➡️ Download the scrape as an Excel spreadsheet or JSON file. To sum up, incorporating web scraping into your toolkit for expanding email lists can significantly boost your outreach capabilities.",
34-
"title": "How to Scrape Emails from Websites: A Practical Guideline",
35-
"domain": "webscraping.blog",
36-
"url": "https://webscraping.blog/how-to-scrape-emails-from-website/"
37-
},
38-
{
39-
"query": "What tools do you use to scrape a website?",
40-
"answer": "Scrapy is suitable for large-scale data scraping tasks. ",
41-
"title": "Email scraping: How-to & top email scraping tools in 2025 - AIMultiple",
42-
"domain": "research.aimultiple.com",
43-
"url": "https://research.aimultiple.com/email-scraping/"
44-
}
45-
],
2+
"FAQs": [],
463
"related_keywords": [
47-
"extract email from webpage",
48-
"extract all emails from website",
49-
"extract email from website",
50-
"web scraping email addresses",
51-
"scrape emails from website free",
4+
"free email scraping tool",
525
"website email scraper",
53-
"how to scrap email addresses from website",
54-
"scrape emails for free"
6+
"web email scraper",
7+
"best free online email extractor",
8+
"how to delete emails from website",
9+
"pull email addresses from website",
10+
"extract email from website free",
11+
"extract emails from website",
12+
"free email grabber",
13+
"extract email address from website",
14+
"extract email addresses from website",
15+
"email extractor from website free",
16+
"free email scraping tool",
17+
"website email scraper",
18+
"web email scraper",
19+
"best free online email extractor",
20+
"how to delete emails from website",
21+
"pull email addresses from website",
22+
"extract email from website free",
23+
"extract emails from website",
24+
"free email grabber",
25+
"extract email address from website",
26+
"extract email addresses from website",
27+
"email extractor from website free"
5528
]
5629
}
Lines changed: 6 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,8 @@
11
{
2-
"title": "Google Chrome",
3-
"link": "https://bing.com/alink/link?url=https%3a%2f%2fwww.google.com%2fchrome%2f&source=serp-rr&h=0SS6iQXs6bC8dov4LbCztjsdUlldoiy7EapBUCzhG7I%3d&p=kcoffcialwebsite",
4-
"heading": "Cross-platform web browser",
5-
"links": {
6-
"Wikipedia": "https://en.wikipedia.org/wiki/Google_Chrome",
7-
"Facebook": "https://www.facebook.com/googlechrome/",
8-
"Twitter": "https://twitter.com/googlechrome",
9-
"YouTube": "https://www.youtube.com/googlechrome"
10-
},
11-
"info": {
12-
"Developer(s)": "Google",
13-
"Written in": "C, C++, Assembly, HTML, Java (Android app only), JavaScript, Python",
14-
"Engines": "Blink (WebKit on iOS), V8 JavaScript engine",
15-
"Operating system": "Android Oreo or later · ChromeOS · iOS 15 or later · Linux · macOS 10.15 or later · Windows 10 or later"
16-
},
17-
"descrption": "Microsoft Edge has a built-in edge, but Chrome beats it out in everyday use and benchmark tests. So, which browser should you choose?computertechnicians.com.auChrome provides quick access to Google’s services. It also uses the Google search engine and displays videos on other devices. It even connects to a Chromecast device for video output.\n[84] [85] They are supported by the browser's desktop edition. [86] [86] [91] In 2014, Google started preventing some Windows users from installing extensions not hosted on the Chrome Web Store. [94]en.wikipedia.orgThe JavaScript virtual machine used by Chrome, the V8 JavaScript engine, has features such as dynamic code generation, hidden class transitions, and precise garbage collection.\nLearn more about Chrome and speed. Tabs help you stay organized, keep track of multiple pages, and multi-task.techspot.comChrome is designed to be fast in every possible way: It's quick to start up from your desktop, loads web pages in a snap, and runs complex web applications fast.\n"
2+
"title": "",
3+
"link": null,
4+
"heading": "",
5+
"links": {},
6+
"info": {},
7+
"descrption": ""
188
}

0 commit comments

Comments
 (0)