Skip to content

Commit 6fd3c02

Browse files
Blocker Update Notifications
Notification System for this repo introduced today. You can subscribe to notifications at: **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker** or send a blank email to **[email protected]** Please make sure you are subscribed to notifications to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place.
1 parent 12f9849 commit 6fd3c02

File tree

11 files changed

+117
-22
lines changed

11 files changed

+117
-22
lines changed

CHANGELOG.md

Lines changed: 18 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,41 +1,47 @@
1-
#CHANGELOG - Apache Bad Bot Blocker
1+
# CHANGELOG - Apache Bad Bot Blocker
22

3-
###2017-02-26
3+
### 2017-04-11
4+
- Introduced Repo Email Notification Mailing List
5+
- Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker**
6+
or simply send a blank email to **[email protected]** to subscribe.
7+
**Please make sure you are subscribed to notifications** to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place.
8+
9+
### 2017-02-26
410
- Added bash script for those wanting hassle free auto updating of the blocker.
511

6-
###2017-02-16
12+
### 2017-02-16
713
- V2.2017.04 - Introduce new Whitelisting of Domain names and IP addresses using an Include file. This means you no longer have to remember to insert your whitelisted IP's and domains with each update. Now you can simply pull the updated globalblacklist.conf to your server, reload Apache and it will automatically include your whitelisted IP ranges and domains. **[Please read updated configuration instructions](https://github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker/blob/master/CONFIGURATION.md)**
814

9-
###2017-02-07
15+
### 2017-02-07
1016
- V2.2017.03 - Introduced .htaccess versions of the spam referrer blocker for users without root access to their Apache Server or unable to run the full bot blocker for some or other reason
1117

12-
###2017-02-06
18+
### 2017-02-06
1319
- V2.2017.02 - Added WhiteListing of CloudFlare IP Ranges
1420

15-
###2017-01-29
21+
### 2017-01-29
1622
- Changed formatting of globalblacklist.conf, some sections like semalt and miraibot merged into other sections for easier maintenance.
1723
- Additional notes, testing instructions and commenting added in globalblacklist.conf file
1824

19-
###2016-12-17
25+
### 2016-12-17
2026
- Removed "CPython" from bad bots list. This user agent string "python-requests/2.5.3 CPython/2.7.9 Linux/3.16.0-4-amd64" is used by a valid Google Feed Parser called "UniversalFeedParser/5.2.1 +https://code.google.com/p/feedparser/"
2127

22-
###2016-12-14
28+
### 2016-12-14
2329
- Removed "python-requests" from bad bots list. This user agent string "python-requests/2.5.3 CPython/2.7.9 Linux/3.16.0-4-amd64" is used by a valid Google Feed Parser called "UniversalFeedParser/5.2.1 +https://code.google.com/p/feedparser/"
2430

25-
###2016-12-11
31+
### 2016-12-11
2632
- Added some repetitively bad IP's
2733
- Added extensive blocking for builtwith.com SEO analysis company that scrapes, accumulates and sells SEO web data. Have blocked all their domains and IP's including other domains linked to the owner. Also spent hours digging for IP's linked to this site that were very nicely hidden but through some research I uncovered them. www.builtwith.com DOES obey removal instructions at https://builtwith.com/removals and it is immediate BUT for those who have never heard of builtwith.com or even know their sites are indexed and new sites are being indexed and used by your competitors to outrank you, I have sufficient blocks in place to prevent them ever accessing your Nginx/Apache server. I suggest you FIRST see if your site is indexed, then request removal of each site which requires you to place a simple .html file in your webroot and once that is done, then update to the latest version of the script which will block them indexing new sites from thereon out. I will keep a watch on them and add new IP's as they change.
2834
- In progress on growing the BAD IP block list which will be based off repetetive 444 errors from the bad referer domains in the blocker. These bad IP's will be auto added into the blocker and I will generate plain text IP lists and IP tables rules too which can be updated frequently to block them at firewall level and keep your web logs even cleaner.
2935

30-
###2016-12-04
36+
### 2016-12-04
3137
- Added creation of a Google Disavow File - google-disavow.txt
3238
- New Bad Referers Added
3339

34-
###2016-12-03
40+
### 2016-12-03
3541
- Added Over 205 New Bad Referer Domains
3642
- Added google-exclude.txt file for stopping Ghost Spam on your Google Analytics
3743
- Readme Updated with Instructions on using google-exclude.txt
3844

39-
###2016-12-02
45+
### 2016-12-02
4046
- Added Block List for Nibble SEO
4147
- Added Block List for Wordpress Theme Detectors

CONFIGURATION.md

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,11 @@
55
##### Copyright Mitchell Krog <[email protected]>
66
### Version 2.2017.04
77

8+
## Update Notification System
9+
Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker**
10+
or simply send a blank email to **[email protected]** to subscribe.
11+
**Please make sure you are subscribed to notifications** to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place.
12+
813
## Step 1:
914

1015
**COPY THE GLOBALBLACKLIST.CONF FILE FROM THE REPO**
@@ -124,4 +129,9 @@ https://github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker/raw/master/upd
124129

125130
Relax now and sleep better at night knowing your site is telling all those baddies FORBIDDEN !!!
126131

127-
### Writing code like this makes me very thirsty [Why not buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TNCNMH8QVM78J):beer:
132+
### Writing code like this makes me very thirsty [Why not buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TNCNMH8QVM78J):beer:
133+
134+
## Update Notification System
135+
Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker**
136+
or simply send a blank email to **[email protected]** to subscribe.
137+
**Please make sure you are subscribed to notifications** to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place.

CPANEL-Instructions.md

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,11 @@
22

33
#CPANEL user instructions for the Apache Ultimate Bad Bot, User-Agent and Spam Referrer Blocker
44

5+
## Update Notification System
6+
Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker**
7+
or simply send a blank email to **[email protected]** to subscribe.
8+
**Please make sure you are subscribed to notifications** to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place.
9+
510
For people wanting to run this bad bot and spam referrer blocker on Cpanel systems, here is a sample configuration excerpt for a single vhost. This came from a user who I helped to get the Apache Bad Bot Blocker working on CPanel.
611

712
The configuration below is for an SSL site and includes the very important http (port 80) redirect to https (Port 443) which a lot of people tend to forget about.
@@ -224,4 +229,9 @@ YOUR.OWN.IP.ADDR;
224229
2a06:98c0::/29
225230
```
226231

227-
### Writing code makes me thirsty [Why not buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TNCNMH8QVM78J):beer:
232+
### Writing code makes me thirsty [Why not buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TNCNMH8QVM78J):beer:
233+
234+
## Update Notification System
235+
Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker**
236+
or simply send a blank email to **[email protected]** to subscribe.
237+
**Please make sure you are subscribed to notifications** to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place.

CentOS6/CONFIGURATION.md

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,11 @@
55
##### Copyright Mitchell Krog <[email protected]>
66
### Version 2.2017.04
77

8+
## Update Notification System
9+
Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker**
10+
or simply send a blank email to **[email protected]** to subscribe.
11+
**Please make sure you are subscribed to notifications** to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place.
12+
813
# CONFIGURATION OF THE APACHE BAD BOT BLOCKER FOR CENTOS 6.8 and APACHE 2.2.15:
914

1015
##Step 1:
@@ -136,4 +141,9 @@ https://raw.githubusercontent.com/mitchellkrogza/apache-ultimate-bad-bot-blocker
136141

137142
Relax now and sleep better at night knowing your site is telling all those baddies FORBIDDEN !!!
138143

139-
### Coding makes me very thirsty [Why not buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TNCNMH8QVM78J):beer:
144+
### Coding makes me very thirsty [Why not buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TNCNMH8QVM78J):beer:
145+
146+
## Update Notification System
147+
Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker**
148+
or simply send a blank email to **[email protected]** to subscribe.
149+
**Please make sure you are subscribed to notifications** to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place.

CentOS6/README.md

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,11 @@
1717

1818
# [Configuration instructions are here](#configuration-of-the-apache-bad-bot-blocker)
1919

20+
## Update Notification System
21+
Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker**
22+
or simply send a blank email to **[email protected]** to subscribe.
23+
**Please make sure you are subscribed to notifications** to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place.
24+
2025
#### Users Unable to run the full Bot Blocker read - https://github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker/tree/master/.htaccess
2126

2227
#### Includes the creation of a google-exclude.txt file for creating filters / segments in Google Analytics (see instructions lower down)
@@ -456,4 +461,9 @@ Many parts of the generator scripts and code running behind this project have be
456461

457462
If you believe your name should be here, drop me a line.
458463

459-
### Coding makes me very thirsty [why not buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TNCNMH8QVM78J):beer:
464+
### Coding makes me very thirsty [why not buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TNCNMH8QVM78J):beer:
465+
466+
## Update Notification System
467+
Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker**
468+
or simply send a blank email to **[email protected]** to subscribe.
469+
**Please make sure you are subscribed to notifications** to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place.

CentOS7/CONFIGURATION.md

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,11 @@
55
##### Copyright Mitchell Krog <[email protected]>
66
### Version 2.2017.04
77

8+
## Update Notification System
9+
Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker**
10+
or simply send a blank email to **[email protected]** to subscribe.
11+
**Please make sure you are subscribed to notifications** to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place.
12+
813
# CONFIGURATION OF THE APACHE BAD BOT BLOCKER:
914

1015
## FOR CENTOS 7 and APACHE 2.4.6 ONLY
@@ -140,4 +145,9 @@ https://raw.githubusercontent.com/mitchellkrogza/apache-ultimate-bad-bot-blocker
140145
Relax now and sleep better at night knowing your site is telling all those baddies FORBIDDEN !!!
141146

142147

143-
### Coding makes me very thirsty [Why not buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TNCNMH8QVM78J):beer:
148+
### Coding makes me very thirsty [Why not buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TNCNMH8QVM78J):beer:
149+
150+
## Update Notification System
151+
Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker**
152+
or simply send a blank email to **[email protected]** to subscribe.
153+
**Please make sure you are subscribed to notifications** to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place.

CentOS7/README.md

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,11 @@
1717

1818
# [Configuration instructions are here](#configuration-of-the-apache-bad-bot-blocker)
1919

20+
## Update Notification System
21+
Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker**
22+
or simply send a blank email to **[email protected]** to subscribe.
23+
**Please make sure you are subscribed to notifications** to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place.
24+
2025
#### Users Unable to run the full Bot Blocker read - https://github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker/tree/master/.htaccess
2126

2227
#### Includes the creation of a google-exclude.txt file for creating filters / segments in Google Analytics (see instructions lower down)
@@ -456,4 +461,9 @@ Many parts of the generator scripts and code running behind this project have be
456461

457462
If you believe your name should be here, drop me a line.
458463

459-
### Coding makes me very thirsty [why not buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TNCNMH8QVM78J):beer:
464+
### Coding makes me very thirsty [why not buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TNCNMH8QVM78J):beer:
465+
466+
## Update Notification System
467+
Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker**
468+
or simply send a blank email to **[email protected]** to subscribe.
469+
**Please make sure you are subscribed to notifications** to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place.

Fail2Ban/README.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,10 @@
77
# Add on for Apache Ultimate Bad Bot and Spam Referrer Blocker
88
GitHub: https://github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker
99

10+
## Update Notification System
11+
Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker**
12+
or simply send a blank email to **[email protected]** to subscribe.
13+
**Please make sure you are subscribed to notifications** to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place.
1014

1115
##### Tested On: Fail2Ban 0.9.3
1216
##### Server: Ubuntu 16.04
@@ -51,4 +55,9 @@ findtime = 604800 ; 1 week
5155
maxretry = 20
5256
```
5357

54-
### If this helped you [why not buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TNCNMH8QVM78J):beer:
58+
### If this helped you [why not buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TNCNMH8QVM78J):beer:
59+
60+
## Update Notification System
61+
Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker**
62+
or simply send a blank email to **[email protected]** to subscribe.
63+
**Please make sure you are subscribed to notifications** to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place.

Pull_Requests_Here_Please/README.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,4 +7,9 @@ as well as the Nginx Bad Bot and Spam Referrer Blocker which use this same centr
77
Thank you for contributing to help produce the most extensive bad bot and spam referrer
88
blocker script.
99

10-
### If this helps you [Why not buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TNCNMH8QVM78J):beer:
10+
### If this helps you [Why not buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TNCNMH8QVM78J):beer:
11+
12+
## Update Notification System
13+
Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker**
14+
or simply send a blank email to **[email protected]** to subscribe.
15+
**Please make sure you are subscribed to notifications** to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place.

README.md

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,11 @@
1717

1818
# [Configuration instructions are here](#configuration-of-the-apache-bad-bot-blocker)
1919

20+
## Update Notification System
21+
Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker**
22+
or simply send a blank email to **[email protected]** to subscribe.
23+
**Please make sure you are subscribed to notifications** to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place.
24+
2025
#### CPANEL Users read - https://github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker/blob/master/CPANEL-Instructions.md
2126

2227
#### Users Unable to run the full Bot Blocker read - https://github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker/tree/master/.htaccess
@@ -446,4 +451,9 @@ Many parts of the generator scripts and code running behind this project have be
446451

447452
If you believe your name should be here, drop me a line.
448453

449-
### And remember [Coding make me thirsty so why not buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TNCNMH8QVM78J):beer:
454+
### And remember [Coding make me thirsty so why not buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TNCNMH8QVM78J):beer:
455+
456+
## Update Notification System
457+
Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker**
458+
or simply send a blank email to **[email protected]** to subscribe.
459+
**Please make sure you are subscribed to notifications** to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place.

0 commit comments

Comments
 (0)