Update doc on weirdness with MAST HST URIs

Author: cam72cam

astroquery/mast/core.py

@@ -1331,6 +1331,22 @@ def _download_from_s3(self, dataProduct, localPath, cache=True):
         obs_id = dataProduct['obs_id']
 
         obs_id = obs_id.lower()
+        
+        # This next part is a bit funky.  Let me explain why:
+        # We have 2 different possible URI schemes for HST:
+        #   mast:HST/product/obs_id_filename.type (old style)
+        #   mast:HST/product/obs_id/obs_id_filename.type (new style)
+        # The first scheme was developed thinking that the obs_id in the filename
+        # would *always* match the actual obs_id folder the file was placed in.
+        # Unfortunately this assumption was false.
+        # We have been trying to switch to the new uri scheme as it specifies the 
+        # obs_id used in the folder path correctly.
+        # The cherry on top is that the obs_id in the new style URI is not always correct either!
+        # When we are looking up files we have some code which iterates through all of
+        # the possible permutations of the obs_id's last char which can be *ANYTHING*
+        #
+        # So in conclusion we can't trust the last char obs_id from the file or from the database
+        # So with that in mind, hold your nose when reading the following:
 
         # magic associations logic per Brian - 0-9/a-e we convert to 0
         magicValues = "123456789abcde"