Allow combine_lines_below on each layer (#850)

Co-authored-by: Etilène Jourdier <[email protected]>

Author: systemed

README.md

@@ -128,11 +128,13 @@ Bug reports, suggestions and (especially!) pull requests are very welcome on the
 
 Formatting: braces and indents as shown, hard tabs (4sp). (Yes, I know.) Please be conservative about adding dependencies or increasing the memory requirement.
 
+The Makefile does not currently pick up changes to header files (.h). If you change these, you may need to run `make clean` before building with `make` and `sudo make install`.
+
 ## Copyright
 
 tilemaker is maintained by Richard Fairhurst and supported by [many contributors](https://github.com/systemed/tilemaker/graphs/contributors). We particularly celebrate the invaluable contributions of Wouter van Kleunen, who passed away in 2022.
 
-Copyright tilemaker contributors, 2015-2024.
+Copyright tilemaker contributors, 2015-2025.
 
 The tilemaker code is licensed as FTWPL; you may do anything you like with this code and there is no warranty.
 

docs/CONFIGURATION.md

@@ -33,12 +33,12 @@ It also includes these global settings:
 * `basezoom` - the zoom level for which tilemaker will generate tiles internally (should usually be the same as `maxzoom`)
 * `include_ids` - whether you want to store the OpenStreetMap IDs for each way/node within your vector tiles. This option is not compatible with the merging options defined by the `combine_xxx` settings (see the dedicated paragraph below)
 * `compress` - for mbtiles output, whether to compress vector tiles (Any of "gzip","deflate" or "none"(default)). For pmtiles output, compression is hardcoded to gzip
-* `combine_below` - whether to merge adjacent linestrings of the same type: will be done at zoom levels below that specified here (e.g. `"combine_below": 14` to merge at z1-13)
 * `name`, `version` and `description` - about your project (these are written into the MBTiles file)
 * `high_resolution` (optional) - whether to use extra coordinate precision at the maximum zoom level (makes tiles a bit bigger)
 * `bounding_box` (optional) - the bounding box to output, in [minlon, minlat, maxlon, maxlat] order
 * `default_view` (optional) - the default location for the client to view, in [lon, lat, zoom] order (MBTiles only)
 * `mvt_version` (optional) - the version of the [Mapbox Vector Tile](https://github.com/mapbox/vector-tile-spec) spec to use; defaults to 2
+* `combine_below` (deprecated) - whether to merge all linestrings in the tile with the same attributes: will be done at zoom levels below that specified here (e.g. `"combine_below": 14` to merge at z1-13). Please use the layer-specific parameter `combine_lines_below` instead.
 
 A typical config file would look like this:
 
@@ -83,6 +83,7 @@ You can add optional parameters to layers:
 * `feature_limit` - restrict the number of features written to each tile
 * `feature_limit_below` - restrict only below this zoom level
 * `combine_polygons_below` - merge adjacent polygons with the same attributes below this zoom level
+* `combine_lines_below` - whether to merge all linestrings in the tile with the same attributes. If not defined, the global setting `combine_below` will be used.
 * `combine_points` - merge points with the same attributes (defaults to `true`: specify `false` to disable)
 * `z_order_ascending` - sort features in ascending order by a numeric value set in the Lua processing script (defaults to `true`: specify `false` for descending order)
 
@@ -118,12 +119,14 @@ For example:
 
 ### Including IDs
 
-Be careful when using both the `include_ids: true` setting to include IDs and the `combine_xxx` settings to lighten tiles : they are not compatible. During the merging process, items with identical tags are combined in a collection, with only 1 ID being retained for all the merged items. If you need to have the exact ID for each item (for example, for a clickable map), you need to remove the merging settings in the target levels and layers:
+You can carry the original OpenStreetMap object ID through to each vector tile feature with the global `include_ids` option.
+
+Note that this is not compatible with the `combine_xxx` settings to reduce tilesize. These settings merge items with identical tags into a collection, so only one ID will be retained for all the merged items. If you need to have the exact ID for each item, remove the merging settings in the target levels and layers:
 
 * set `combine_points: false` (`true` is the default value) in the target layers
-* set `combine_below` and `combine_polygons_below` below the target zoom level (or remove them)
+* set `combine_lines_below` and `combine_polygons_below` in layer properties below the target zoom level (or remove them)
 
-If you need the include OSM types as well, you will need to modify the `process.lua` script, check issue [#740](https://github.com/systemed/tilemaker/issues/740) for more information.
+If you need to include OSM object types as well, you can use the `OsmType()` function in your `process.lua` script.
 
 ## Lua processing reference
 
@@ -163,7 +166,7 @@ To do that, you use these methods:
 * `OsmType()`: get the OSM type of the current object.
 * `IsClosed()`: returns true if the current object is a closed area.
 * `IsMultiPolygon()`: returns true if the current object is a multipolygon.
-* `ZOrder(number)`: Set a numeric value (default 0) used to sort features within a layer. Use this feature to ensure a proper rendering order if the rendering engine itself does not support sorting. Sorting is not supported across layers merged with `write_to`. Features with different z-order are not merged if `combine_below` or `combine_polygons_below` is used. Use this in conjunction with `feature_limit` to only write the most important (highest z-order) features within a tile. (Values can be -50,000,000 to 50,000,000 and are lossy, particularly beyond -1000 to 1000.)
+* `ZOrder(number)`: Set a numeric value (default 0) used to sort features within a layer. Use this feature to ensure a proper rendering order if the rendering engine itself does not support sorting. Sorting is not supported across layers merged with `write_to`. Features with different z-order are not merged if `combine_below`, `combine_lines_below` or `combine_polygons_below` is used. Use this in conjunction with `feature_limit` to only write the most important (highest z-order) features within a tile. (Values can be -50,000,000 to 50,000,000 and are lossy, particularly beyond -1000 to 1000.)
 * `MinZoom(zoom)`: set the minimum zoom level (0-15) at which this object will be written. Note that the JSON layer configuration minimum still applies (so `:MinZoom(5)` will have no effect if your layer only starts at z6).
 * `Length()` and `Area()`: return the length (metres)/area (square metres) of the current object. Requires Boost 1.67+.
 * `Centroid()`: return the lat/lon of the centre of the current object as a two-element Lua table (element 1 is lat, 2 is lon).

include/shared_data.h

@@ -26,11 +26,12 @@ struct LayerDef {
 	uint simplifyAlgo;
 	uint filterBelow;
 	double filterArea;
-	uint combinePolygonsBelow;
 	bool sortZOrderAscending;
 	uint featureLimit;
 	uint featureLimitBelow;
 	bool combinePoints;
+	uint combineLinesBelow;
+	uint combinePolygonsBelow;
 	std::string source;
 	std::vector<std::string> sourceColumns;
 	bool allSourceColumns;
@@ -58,8 +59,8 @@ class LayerDefinition {
 	// Define a layer (as read from the .json file)
 	uint addLayer(std::string name, uint minzoom, uint maxzoom,
 			uint simplifyBelow, double simplifyLevel, double simplifyLength, double simplifyRatio, uint simplifyAlgo,
-			uint filterBelow, double filterArea, uint combinePolygonsBelow, bool sortZOrderAscending,
-			uint featureLimit, uint featureLimitBelow, bool combinePoints,
+			uint filterBelow, double filterArea, bool sortZOrderAscending,
+			uint featureLimit, uint featureLimitBelow, bool combinePoints, uint combineLinesBelow, uint combinePolygonsBelow,
 			const std::string &source,
 			const std::vector<std::string> &sourceColumns,
 			bool allSourceColumns,

src/shared_data.cpp

@@ -139,8 +139,8 @@ void SharedData::writePMTilesBounds() {
 // Define a layer (as read from the .json file)
 uint LayerDefinition::addLayer(string name, uint minzoom, uint maxzoom,
 		uint simplifyBelow, double simplifyLevel, double simplifyLength, double simplifyRatio, uint simplifyAlgo,
-		uint filterBelow, double filterArea, uint combinePolygonsBelow, bool sortZOrderAscending,
-		uint featureLimit, uint featureLimitBelow, bool combinePoints,
+		uint filterBelow, double filterArea, bool sortZOrderAscending,
+		uint featureLimit, uint featureLimitBelow, bool combinePoints, uint combineLinesBelow, uint combinePolygonsBelow,
 		const std::string &source,
 		const std::vector<std::string> &sourceColumns,
 		bool allSourceColumns,
@@ -150,7 +150,7 @@ uint LayerDefinition::addLayer(string name, uint minzoom, uint maxzoom,
 
 	bool isWriteTo = !writeTo.empty();
 	LayerDef layer = { name, minzoom, maxzoom, simplifyBelow, simplifyLevel, simplifyLength, simplifyRatio, simplifyAlgo,
-		filterBelow, filterArea, combinePolygonsBelow, sortZOrderAscending, featureLimit, featureLimitBelow, combinePoints,
+		filterBelow, filterArea, sortZOrderAscending, featureLimit, featureLimitBelow, combinePoints, combineLinesBelow, combinePolygonsBelow,
 		source, sourceColumns, allSourceColumns, indexed, indexName,
 		std::map<std::string,uint>(), isWriteTo };
 	layers.push_back(layer);
@@ -317,10 +317,11 @@ void Config::readConfig(rapidjson::Document &jsonConfig, bool &hasClippingBox, B
 		double simplifyRatio  = it->value.HasMember("simplify_ratio" ) ? it->value["simplify_ratio" ].GetDouble() : 2.0;
 		int    filterBelow    = it->value.HasMember("filter_below"   ) ? it->value["filter_below"   ].GetInt()    : 0;
 		double filterArea     = it->value.HasMember("filter_area"    ) ? it->value["filter_area"    ].GetDouble() : 0.5;
-		int    combinePolyBelow=it->value.HasMember("combine_polygons_below") ? it->value["combine_polygons_below"].GetInt() : 0;
 		int    featureLimit   = it->value.HasMember("feature_limit"  ) ? it->value["feature_limit"  ].GetInt()    : 0;
 		int  featureLimitBelow= it->value.HasMember("feature_limit_below") ? it->value["feature_limit_below"].GetInt() : (maxZoom+1);
 		bool combinePoints    = it->value.HasMember("combine_points" ) ? it->value["combine_points" ].GetBool()   : true;
+		int combineLinesBelow = it->value.HasMember("combine_lines_below"  ) ? it->value["combine_lines_below"  ].GetInt()    : combineBelow;
+		int    combinePolyBelow=it->value.HasMember("combine_polygons_below") ? it->value["combine_polygons_below"].GetInt() : 0;
 		bool sortZOrderAscending = it->value.HasMember("z_order_ascending") ? it->value["z_order_ascending"].GetBool() : (featureLimit==0);
 		string algo           = it->value.HasMember("simplify_algorithm") ? it->value["simplify_algorithm"].GetString() : "";
 		uint simplifyAlgo = algo=="visvalingam" ? LayerDef::VISVALINGAM : LayerDef::DOUGLAS_PEUCKER;
@@ -343,7 +344,7 @@ void Config::readConfig(rapidjson::Document &jsonConfig, bool &hasClippingBox, B
 
 		layers.addLayer(layerName, minZoom, maxZoom,
 				simplifyBelow, simplifyLevel, simplifyLength, simplifyRatio, simplifyAlgo,
-				filterBelow, filterArea, combinePolyBelow, sortZOrderAscending, featureLimit, featureLimitBelow, combinePoints,
+				filterBelow, filterArea, sortZOrderAscending, featureLimit, featureLimitBelow, combinePoints, combineLinesBelow, combinePolyBelow,
 				source, sourceColumns, allSourceColumns, indexed, indexName,
 				writeTo);
 

src/tile_worker.cpp

@@ -277,13 +277,13 @@ void ProcessObjects(
 	double simplifyLevel,
 	unsigned simplifyAlgo,
 	double filterArea,
-	bool combinePolygons,
 	bool combinePoints,
+	bool combineLines,
+	bool combinePolygons,
 	unsigned zoom,
 	const TileBbox &bbox,
 	vtzero::layer_builder& vtLayer
 ) {
-
 	for (auto jt = ooSameLayerBegin; jt != ooSameLayerEnd; ++jt) {
 		OutputObjectID oo = *jt;
 		if (zoom < oo.oo.minZoom) { continue; }
@@ -330,7 +330,7 @@ void ProcessObjects(
 			}
 
 			//This may increment the jt iterator
-			if (oo.oo.geomType == LINESTRING_ && zoom < sharedData.config.combineBelow) {
+			if (oo.oo.geomType == LINESTRING_ && combineLines) {
 				// Append successive linestrings, then reorder afterwards
 				while (jt<(ooSameLayerEnd-1) && oo.oo.compatible((jt+1)->oo)) {
 					jt++;
@@ -440,7 +440,6 @@ void ProcessLayer(
 		if (zoom < ld.filterBelow) { 
 			filterArea = meter2degp(ld.filterArea, latp) * pow(2.0, (ld.filterBelow-1) - zoom);
 		}
-
 		for (size_t i=0; i<sources.size(); i++) {
 			// Loop through output objects
 			auto ooListSameLayer = getObjectsAtSubLayer(data[i], layerNum);
@@ -449,7 +448,7 @@ void ProcessLayer(
 			ProcessObjects(sources[i], attributeStore, 
 				ooListSameLayer.first, end, sharedData, 
 				simplifyLevel, ld.simplifyAlgo,
-				filterArea, zoom < ld.combinePolygonsBelow, ld.combinePoints, zoom, bbox, vtLayer);
+				filterArea, ld.combinePoints, zoom < ld.combineLinesBelow, zoom < ld.combinePolygonsBelow, zoom, bbox, vtLayer);
 		}
 	}
 	if (verbose && std::time(0)-start>3) {