Update the data

Author: 311ba2 5453bf

README.md

@@ -1,27 +1,41 @@
 # openftp4
 
-This is a list of all (796,578) FTP servers directly connected to port 21 in the IPv4 address space that allow anonymous logins. The login must be completed in less than 5 seconds to qualify for this list.  How and why this list was created is documented in detail in my blog post [*Mass-analyzing a chunk of the Internet*](http://255.wf/2016-09-18-mass-analyzing-a-chunk-of-the-internet/).
+This is a list of all FTP servers directly connected to port 21 in the IPv4 address space that allow anonymous logins. The login must be completed in less than 15 seconds to qualify for this list.  
 
-You can do whatever you want with this data. Consider linking to this repo if you find something interesting or odd.
+How and why this list was created is documented in detail in my blog post [*Mass-analyzing a chunk of the Internet*](http://255.wf/2016-09-18-mass-analyzing-a-chunk-of-the-internet/). You can do whatever you want with this data. Consider linking to this repo if you find something interesting or odd.
 
-### Usage
+The last scan contains **796,268** servers that allow anonymous access. This is **4.31486 %** of the **18,454,087** services running on port 21 in IPv4.
 
-The file is provided as a gz and [lz5](https://github.com/inikep/lz5) compressed file.
+### Usage
 
-LZ5: Decompress the file to stdout to get a stream of IPs:
+1) Decompress the file
 
 ```sh
-lz5 -d openftp4_all_20160918.lz5
+gzip -d openftp4.txt.gz
+```
+
+2) Hack away
+
+### Format
+
+The data follows this loose format:
+
+```text
+ip|timestamp|banner
 ```
 
-will print a stream of all IP addresses to stdout. Just a hint: If you are going to interact in any way with these servers, consider piping the list through `shuf` each time you try something new so you don't hit the same server(s) over and over again. Also, don't sort the list before rescanning, because you will enounter IP slashes that belong to one network. Think about what it looks like from their perspective ;-).
+- `ip` is the IPv4 address (`^([0-9.]+)\|`).
+- `timestamp` is the unix timestamp of the exchange with that server (`^+?\|(\d+)\|`).
+- `banner` is **everything** after the second `|` and includes the full initial banner, every response code and the full login exchange (`\|\d+\|(.+)$`).
+
+Just a hint: If you are going to interact in any way with these servers, consider piping the list through `shuf` each time you try something new so you don't hit the same server(s) over and over again. Also, don't sort the list before rescanning, because you will encounter IP slashes that belong to one network. Think about what it looks like from their perspective ;-).
 
-If you want to be extra nice, provide your actual email address (or one you have access to) as the password (blog post for details), so server admins can contact you. 
+If you want to be extra nice, provide your actual email address (or one you have access to) as the password (blog post for details), so server admins can contact you.
 
 ## [![](https://news.ycombinator.com/y18.gif) Discussion](https://news.ycombinator.com/item?id=12523455)
 
 - News: [SoftPedia](http://news.softpedia.com/news/nearly-800-000-ftp-servers-accessible-online-without-authentication-508421.shtml) • [D. Pratt (German)](https://dominicpratt.de/unsichere-ftp-server/) • [IDG: NETWORKWORLD](http://www.networkworld.com/article/3121655/security/teenager-claims-to-have-accessed-ftps-downloaded-data-from-every-state-with-us-domain.html#comments) • [mob3](http://mob3.net/forum/threads/user-scans-all-open-ftp-servers-on-ipv4-posts-ip-results.6391/)
-- Discussion elsewehre: [HN](https://news.ycombinator.com/item?id=12527989) • [r/DataHoarder](https://www.reddit.com/r/DataHoarder/comments/53cyhm/list_of_all_anonymous_login_ftp_servers_worldwide/) • [r/opendirectories](https://www.reddit.com/r/opendirectories/comments/53b0ar/a_list_of_all_ftp_servers_in_the_whole_internet/) • [r/netsec](https://www.reddit.com/r/netsec/comments/53bori/massanalyzing_a_chunk_of_the_internet/) • [r/sysadmin](https://www.reddit.com/r/sysadmin/comments/53cor1/someone_just_posted_every_open_ftp_server_on_ipv4/) 
+- Discussion elsewehre: [HN](https://news.ycombinator.com/item?id=12527989) • [r/DataHoarder](https://www.reddit.com/r/DataHoarder/comments/53cyhm/list_of_all_anonymous_login_ftp_servers_worldwide/) • [r/opendirectories](https://www.reddit.com/r/opendirectories/comments/53b0ar/a_list_of_all_ftp_servers_in_the_whole_internet/) • [r/netsec](https://www.reddit.com/r/netsec/comments/53bori/massanalyzing_a_chunk_of_the_internet/) • [r/sysadmin](https://www.reddit.com/r/sysadmin/comments/53cor1/someone_just_posted_every_open_ftp_server_on_ipv4/)
 
 ### In the Wild