Weird origin offset after transforming the tpk with custom zoom levels

[eguiheneuf]

Hello,

Thank you for your tool, I have used it several times and worked exactly as I expected since today

I am having trouble while transforming a tpk I was provided with, generated with 10.8, for which the tpk author made some changes in the zoom level in order to make some perfomance tweaks.

The tool does create the mbtiles, with the correct data, but the location is completely off.
I have suceeded in finding the offset with some artisanal methods (matching points of interests), but I was wondering if there was something more robust we could do about it.
If that's any help, I ended up with the following origin for the mbtiles : origin:[-20037701.37, -9834020.58] , which somewhat matches the originX in the config.xml, but is waaay off for the originY

I attached a L00, a L01 and a L02 folder, alonside the config.xml file associated with it, it would be great if you could take a look at it

_alllayers.zip

All the best

[brendan-ward]

It is my understanding that map tiles using spherical mercator have a standard origin, and that zoom level determines the x and y coordinates of each tile. If zoom levels have been altered, that could result in tile coordinates being incorrect (i.e., if the wrong zoom level is assigned to a tile of a given index).

the tpk author made some changes in the zoom level in order to make some perfomance tweaks

Are you able to elaborate on what exactly was changed? My first instinct is that this isn't an issue with tpkutils but rather with the TPK file, but I'd like to know a little bit more first.

[eguiheneuf]

Hello,

In order to reduce the size of the file, the author of the TPK implemented non-standard zoom levels, which are the following (here in resolutions) :
[ 152.87 - 76.43 - 38.21 - 19.10 - 9.55 - 4.77 - 2.39 - 1.19 - 0.59 - 0.14 ]

Note that the last two ones does not respect the 1/2 factor (we could say that one zoom level is missing).

If I serve the tpk through an Arcgis server, this one is perfectly valid and the tiles are being served exactly where I expect them to be, but maybe the changes in the tpk are too exotic to be supported by tpkutils ?

[brendan-ward]

There may indeed be assumptions that we have baked into the code that do not hold up in this case. Good to know that ArcGIS server handles this without issue. We may need to parse more out of some of the config files to handle this, or perhaps there are rounding assumptions where we assume integer tile levels.

Can you please post a subset of the TPK, with the other config files? Doesn't need all zoom levels, so could also be a zip file of the _alllayers you've already posted plus at least esriinfo/iteminfo.xml and servicedescriptions/mapserver/mapserver.json. Please include other XML / JSON files at the top level or in any of those folders.

[eguiheneuf]

Hi Brendan

Here is the full tpk, from which I deleted all layers beyond 3

exotic.zip

[brendan-ward]

It looks like the conf.xml file includes a different Y origin than I'm seeing in any of my other sample TPK files.

I'm not seeing an obvious place in the code to account for this difference. I think the issue is that the tile coordinates need to be translated between the origin in the TPK to the standard web mercator tile origin (i.e., tiles would be renumbered).

Feel free to submit a PR if you find a way to correct for this; otherwise I'm considering exotic tiling schemes out of scope for what we can support here.

[eguiheneuf]

Some seems to have encountered the same exotic origins as I did, seems like a little Esri magic origin they implemented
:
Esri/esri-leaflet#567

You seem correct with your analysis on the origin Y : your script does respect the Y being between -20 037 508,34 and + 20 037 508,34, (-0,5 circumference ; +0,5 circumference) whereas Esri seems to do adopt a 0; 40075016,69 (0, 1xcircumference)

I will look further on that, but I think I will be limited by my Python knowledge

[brendan-ward]

I think the standard is the former. Here is the relevant bit from other standard TPK conf.xml files:

        <TileOrigin xsi:type='typens:PointN'>
            <X>-20037508.342787001</X>
            <Y>20037508.342787001</Y>
        </TileOrigin>

whereas yours is

        <TileOrigin xsi:type='typens:PointN'>
            <X>-20037700</X>
            <Y>30241100</Y>
        </TileOrigin>

Feel free to suggest solutions. The ones linked (retrieve, merge, and retile) above are outside scope for this project though, so if it isn't a simple matter of assigning different tile coordinates to the existing tiles, we may not be able to do much here.

Is it possible to simply create the original TPK file using a standard tiling scheme, or is there a very clear performance win to doing so that outweighs lack of support outside ArcGIS?

[eguiheneuf]

Hi,

To my knowledge, there is no advantage in having a non standard origin.
Just asked the author of the TPK : he confirmed he left the origin with default values, I'm guessing some versions of Esri, or a plugin changes this (can't say if it is intentional or a bug), but by simply searching "30241100 3857" in google, it looks like many arcgis services adopt this origin...

I agree that it's more a arcgis-related issue, from my point of view... It isn't just offseting the tile coordinates, but rather reconstructing a new origin, which seems a lot of effort for very limited usage (it is the first tpk I have to work with that had the problem, I encountered dozen more that didn't have any issue).

If someone encounters this problem and reads his ticket, the issue can be worked around by manually setting the origin in the meta of the mbtiles (if the lib reading the mbtiles supports it).

Thank you for you time though