2.4.0 release

Author: joepie91

README.md

@@ -29,6 +29,12 @@ The manual (including install instructions) can be found in the doc/ directory.
 	* Nameservers
 * Optional WHOIS data normalization
 	* Attempts to intelligently reformat WHOIS data for better (human) readability
+	* Converts various abbreviation types to full locality names
+		* Airport codes
+		* Country names (2- and 3-letter ISO codes)
+		* US states and territories
+		* Canadian states and territories
+		* Australian states
 * `pwhois`, a simple WHOIS tool using pythonwhois
 	* Easily readable output format
 	* Can also output raw WHOIS data
@@ -39,6 +45,8 @@ The manual (including install instructions) can be found in the doc/ directory.
 
 ## Important update notes
 
+*2.4.0 and up*: A lot of changes were made to the normalization, and the performance under Python 2.x was significantly improved. The average parsing time under Python 2.7 has dropped by 94% (!), and on my system averages out at 18ms. Performance under Python 3.x is [unchanged](https://github.com/joepie91/python-whois/issues/27). `pythonwhois` will now expand a lot of abbreviations in normalized mode, such as airport codes, ISO country codes, and US/CA/AU state abbreviations. The consequence of this is that the library is now bigger (as it ships a list of these abbreviations). Also note that there *may* be licensing consequences, in particular regarding the airport code database. More information about that can be found below.
+
 *2.3.0 and up*: Python 3 support was fixed. Creation date parsing for contacts was fixed; correct timestamps will now be returned, rather than unformatted ones - if your application relies on the broken variant, you'll need to change your code. Some additional parameters were added to the `net` and `parse` methods to facilitate NIC handle lookups; the defaults are backwards-compatible, and these changes should not have any consequences for your code. Thai WHOIS parsing was implemented, but is a little spotty - data may occasionally be incorrectly split up. Please submit a bug report if you run across any issues.
 
 *2.2.0 and up*: The internal workings of `get_whois_raw` have been changed, to better facilitate parsing of WHOIS data from registries that may return multiple partial matches for a query, such as `whois.verisign-grs.com`. This change means that, by default, `get_whois_raw` will now strip out the part of such a response that does not pertain directly to the requested domain. If your application requires an unmodified raw WHOIS response and is calling `get_whois_raw` directly, you should use the new `never_cut` parameter to keep pythonwhois from doing this post-processing. As this is a potentially breaking behaviour change, the minor version has been bumped.
@@ -51,6 +59,23 @@ The manual (including install instructions) can be found in the doc/ directory.
 
 If any of those apply, don't hesitate to file an issue! The goal is 100% coverage, and we need your feedback to reach that goal.
 
+## License
+
+This library may be used under the WTFPL - or, if you take issue with that, consider it to be under the CC0.
+
+## Data sources
+
+This library uses a number of third-party datasets for normalization:
+
+* `airports.dat`: [OpenFlights Airports Database](http://openflights.org/data.html) ([Open Database License 1.0](http://opendatacommons.org/licenses/odbl/1.0/), [Database Contents License 1.0](http://opendatacommons.org/licenses/dbcl/1.0/))
+* `countries.dat`: [Country List](https://github.com/umpirsky/country-list) (MIT license)
+* `countries3.dat`: [ISO countries list](https://gist.github.com/eparreno/205900) (license unspecified)
+* `states_au.dat`: Part of `pythonwhois` (WTFPL/CC0)
+* `states_us.dat`: [State Table](http://statetable.com/) (license unspecified, free reuse encouraged)
+* `states_ca.dat`: [State Table](http://statetable.com/) (license unspecified, free reuse encouraged)
+
+Be aware that the OpenFlights database in particular has potential licensing consequences; if you do not wish to be bound by these potential consequences, you may simply delete the `airports.dat` file from your distribution. `pythonwhois` will assume there is no database available, and will not perform airport code conversion (but still function correctly otherwise). This also applies to other included datasets.
+
 ## Contributing
 
 Feel free to fork and submit pull requests (to the `develop` branch)! If you change any parsing or normalization logic, ensure to run the full test suite before opening a pull request. Instructions for that are below.

pwhois

@@ -1,6 +1,6 @@
 #!/usr/bin/env python2
 
-import argparse, pythonwhois, json, datetime
+import argparse, pythonwhois, json, datetime, sys
 try:
 	from collections import OrderedDict
 except ImportError as e:
@@ -100,7 +100,15 @@ else:
 					for key, value in data_map.items():
 						if key in contact_data and contact_data[key] is not None:
 							label = "    " + value + (" " * (widest_label - len(value))) + " :"
-							actual_data = contact_data[key]
+							if sys.version_info < (3, 0):
+								if type(contact_data[key]) == str:
+									actual_data = contact_data[key].decode("utf-8")
+								elif type(contact_data[key]) == datetime.datetime:
+									actual_data = unicode(contact_data[key])
+								else:
+									actual_data = contact_data[key]
+							else:
+								actual_data = str(contact_data[key])
 							if "\n" in actual_data: # Indent multi-line values properly
 								lines = actual_data.split("\n")
 								actual_data = "\n".join([lines[0]] + [(" " * (widest_label + 7)) + line for line in lines[1:]])