Skip to content

feat: impl pre-processing encodings#2683

Draft
CommanderStorm wants to merge 3 commits intomaplibre:mainfrom
CommanderStorm:postprs
Draft

feat: impl pre-processing encodings#2683
CommanderStorm wants to merge 3 commits intomaplibre:mainfrom
CommanderStorm:postprs

Conversation

@CommanderStorm
Copy link
Copy Markdown
Member

@CommanderStorm CommanderStorm commented Apr 13, 2026

No description provided.

@CommanderStorm CommanderStorm requested a review from nyurik April 13, 2026 04:54
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 13, 2026 •
edited
Loading

Performance Comparison main → postprs

Total Elapsed Time: 69.31s → 65.15s (-6.0%)
CPU Baseline: 254.24µs → 252.39µs (-0.7%)
Benchmark ID: timing

timing - Function execution time metrics.

+----------------------------+------------------------------+--------------------------------+---------------------------------+--------------------------------+-------------------------------+
| Function                   | Calls                        | Avg                            | P95                             | Total                          | % Total                       |
+----------------------------+------------------------------+--------------------------------+---------------------------------+--------------------------------+-------------------------------+
| content::get_tile          | 2100500 → 2100500 (+0.0%)    | 42.91µs → 32.50µs (-24.3%) 🚀  | 55.94µs → 37.53µs (-32.9%) 🚀   | 90.12s → 68.26s (-24.3%) 🚀    | 130.03% → 104.77% (-19.4%)    |
+----------------------------+------------------------------+--------------------------------+---------------------------------+--------------------------------+-------------------------------+
| martin::main               | 1 → 1 (+0.0%)                | 69.29s → 65.15s (-6.0%)        | 69.32s → 65.16s (-6.0%)         | 69.31s → 65.15s (-6.0%)        | 100.00% → 100.00% (+0.0%)     |
+----------------------------+------------------------------+--------------------------------+---------------------------------+--------------------------------+-------------------------------+
| martin::start              | 1 → 1 (+0.0%)                | 69.29s → 65.15s (-6.0%)        | 69.32s → 65.16s (-6.0%)         | 69.31s → 65.15s (-6.0%)        | 99.99% → 100.00% (+0.0%)      |
+----------------------------+------------------------------+--------------------------------+---------------------------------+--------------------------------+-------------------------------+
| content::get_http_response | 2100500 → 2100500 (+0.0%)    | 33.49µs → 23.48µs (-29.9%) 🚀  | 47.52µs → 30.25µs (-36.3%) 🚀   | 70.36s → 49.33s (-29.9%) 🚀    | 101.51% → 75.72% (-25.4%) 🚀  |
+----------------------------+------------------------------+--------------------------------+---------------------------------+--------------------------------+-------------------------------+
| content::get_tile_content  | 2100500 → 2100500 (+0.0%)    | 28.80µs → 19.00µs (-34.0%) 🚀  | 43.68µs → 27.44µs (-37.2%) 🚀   | 60.49s → 39.91s (-34.0%) 🚀    | 87.27% → 61.26% (-29.8%) 🚀   |
+----------------------------+------------------------------+--------------------------------+---------------------------------+--------------------------------+-------------------------------+
| content::new               | 2100500 → 2100500 (+0.0%)    | 2.60µs → 2.55µs (-1.9%)        | 2.63µs → 2.47µs (-6.1%)         | 5.45s → 5.35s (-1.8%)          | 7.87% → 8.22% (+4.4%)         |
+----------------------------+------------------------------+--------------------------------+---------------------------------+--------------------------------+-------------------------------+
| content::recompress        | 2100500 → 2100500 (+0.0%)    | 12.37µs → 1.44µs (-88.4%) 🚀   | 22.22µs → 175.00ns (-99.2%) 🚀  | 25.99s → 3.03s (-88.3%) 🚀     | 37.49% → 4.64% (-87.6%) 🚀    |
+----------------------------+------------------------------+--------------------------------+---------------------------------+--------------------------------+-------------------------------+
| content::encode            | 100100 → 100100 (+0.0%)      | 30.34µs → 27.69µs (-8.7%)      | 45.05µs → 50.17µs (+11.4%)      | 3.04s → 2.77s (-8.9%)          | 4.38% → 4.25% (-3.0%)         |
+----------------------------+------------------------------+--------------------------------+---------------------------------+--------------------------------+-------------------------------+
| source::get_sources        | 2100500 → 2100500 (+0.0%)    | 1.34µs → 1.30µs (-3.0%)        | 2.06µs → 1.93µs (-6.3%)         | 2.81s → 2.73s (-2.8%)          | 4.05% → 4.19% (+3.5%)         |
+----------------------------+------------------------------+--------------------------------+---------------------------------+--------------------------------+-------------------------------+
| server::new_server         | 1 → 1 (+0.0%)                | 198.21µs → 192.96µs (-2.6%)    | 198.27µs → 193.02µs (-2.6%)     | 198.23µs → 193.00µs (-2.6%)    | 0.00% → 0.00% (+0.0%)         |
+----------------------------+------------------------------+--------------------------------+---------------------------------+--------------------------------+-------------------------------+

alloc-bytes - Exclusive allocation bytes by each function (excluding nested calls).

+----------------------------+------------------------------+--------------------------------+--------------------------------+--------------------------------+-----------------------------+
| Function                   | Calls                        | Avg                            | P95                            | Total                          | % Total                     |
+----------------------------+------------------------------+--------------------------------+--------------------------------+--------------------------------+-----------------------------+
| content::get_tile_content  | 2100500 → 2100500 (+0.0%)    | 95.7 KB → 95.8 KB (+0.1%)      | 180.0 KB → 180.2 KB (+0.1%)    | 191.7 GB → 192.0 GB (+0.2%)    | 84.30% → 84.31% (+0.0%)     |
+----------------------------+------------------------------+--------------------------------+--------------------------------+--------------------------------+-----------------------------+
| content::encode            | 100100 → 100100 (+0.0%)      | 344.5 KB → 344.5 KB (+0.0%)    | 347.5 KB → 347.5 KB (+0.0%)    | 32.9 GB → 32.9 GB (+0.0%)      | 14.47% → 14.44% (-0.2%)     |
+----------------------------+------------------------------+--------------------------------+--------------------------------+--------------------------------+-----------------------------+
| source::get_sources        | 2100500 → 2100500 (+0.0%)    | 932 B → 964 B (+3.4%)          | 1.2 KB → 1.2 KB (+0.0%)        | 1.8 GB → 1.9 GB (+5.6%)        | 0.80% → 0.83% (+3.8%)       |
+----------------------------+------------------------------+--------------------------------+--------------------------------+--------------------------------+-----------------------------+
| content::get_tile          | 2100500 → 2100500 (+0.0%)    | 296 B → 296 B (+0.0%)          | 296 B → 296 B (+0.0%)          | 592.9 MB → 592.9 MB (+0.0%)    | 0.25% → 0.25% (+0.0%)       |
+----------------------------+------------------------------+--------------------------------+--------------------------------+--------------------------------+-----------------------------+
| content::get_http_response | 2100500 → 2100500 (+0.0%)    | 181 B → 181 B (+0.0%)          | 180 B → 180 B (+0.0%)          | 364.5 MB → 364.5 MB (+0.0%)    | 0.16% → 0.16% (+0.0%)       |
+----------------------------+------------------------------+--------------------------------+--------------------------------+--------------------------------+-----------------------------+
| martin::start              | 1 → 1 (+0.0%)                | 3.1 MB → 3.1 MB (+0.0%)        | 3.1 MB → 3.1 MB (+0.0%)        | 3.1 MB → 3.1 MB (+0.0%)        | 0.00% → 0.00% (+0.0%)       |
+----------------------------+------------------------------+--------------------------------+--------------------------------+--------------------------------+-----------------------------+
| martin::main               | 1 → 1 (+0.0%)                | 139.8 KB → 139.8 KB (+0.0%)    | 139.9 KB → 139.9 KB (+0.0%)    | 139.8 KB → 139.8 KB (+0.0%)    | 0.00% → 0.00% (+0.0%)       |
+----------------------------+------------------------------+--------------------------------+--------------------------------+--------------------------------+-----------------------------+
| server::new_server         | 1 → 1 (+0.0%)                | 30.9 KB → 30.9 KB (+0.0%)      | 30.9 KB → 30.9 KB (+0.0%)      | 30.9 KB → 30.9 KB (+0.0%)      | 0.00% → 0.00% (+0.0%)       |
+----------------------------+------------------------------+--------------------------------+--------------------------------+--------------------------------+-----------------------------+
| content::new               | 2100500 → 2100500 (+0.0%)    | 0 B → 0 B (+0.0%)              | 0 B → 0 B (+0.0%)              | 0 B → 0 B (+0.0%)              | 0.00% → 0.00% (+0.0%)       |
+----------------------------+------------------------------+--------------------------------+--------------------------------+--------------------------------+-----------------------------+
| content::recompress        | 2100500 → 2100500 (+0.0%)    | 22 B → 0 B (-100.0%) 🚀        | 22 B → 0 B (-100.0%) 🚀        | 44.1 MB → 0 B (-100.0%) 🚀     | 0.02% → 0.00% (-100.0%) 🚀  |
+----------------------------+------------------------------+--------------------------------+--------------------------------+--------------------------------+-----------------------------+

Threads

Total Alloc: 2.7 MB → 2.7 MB (+0.0%)
Total Dealloc: 272.8 MB → 272.8 MB (+0.0%)
Mem Diff: -270.1 MB → -270.1 MB (+0.0%)

+----------+----------------------------+------------------------------+------------------------------+--------------------------------+----------------------------------+
| Thread   | CPU % Avg                  | CPU % Max                    | Alloc                        | Dealloc                        | Mem Diff                         |
+----------+----------------------------+------------------------------+------------------------------+--------------------------------+----------------------------------+
| martin   | 0.00% → 0.00% (+0.0%)      | 12.00% → 12.00% (+0.0%)      | 2.7 MB → 2.7 MB (+0.0%)      | 1.8 MB → 1.8 MB (+0.0%)        | 896.7 KB → 897.5 KB (+0.1%)      |
+----------+----------------------------+------------------------------+------------------------------+--------------------------------+----------------------------------+
| hp-mcp   | 0.00% → 0.00% (+0.0%)      | 0.00% → 0.00% (+0.0%)        | 54.7 KB → 54.7 KB (+0.0%)    | 3.4 KB → 3.4 KB (+0.0%)        | 51.4 KB → 51.4 KB (+0.0%)        |
+----------+----------------------------+------------------------------+------------------------------+--------------------------------+----------------------------------+
| hp-debug | 3.10% → 4.10% (+32.3%) ⚠️  | 15.90% → 12.00% (-24.5%) 🚀  | 10.9 KB → 10.4 KB (-4.6%)    | 271.0 MB → 271.0 MB (+0.0%)    | -271.0 MB → -271.0 MB (+0.0%)    |
+----------+----------------------------+------------------------------+------------------------------+--------------------------------+----------------------------------+

Generated with hotpath-rs

CommanderStorm
CommanderStorm commented Apr 13, 2026
View reviewed changes
Comment thread martin/src/config/file/main.rs
@CommanderStorm CommanderStorm linked an issue Apr 14, 2026 that may be closed by this pull request
RFC Post-processing (compression, encoding, conversion, pruning) #2689
Open
@CommanderStorm
feat: add tile post-processing pipeline (MVT-to-MLT conversion)
0c590d6 
Add ProcessConfig for per-source tile post-processing, enabling
automatic MVT-to-MLT conversion via `process.mlt: auto` in config.
Integrates with the existing Accept header content negotiation and
tile caching, applying processors before cache insertion.

Squash-merge of the postprs branch onto main, combining both the
post-processing pipeline and Accept header support.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nyurik nyurik Awaiting requested review from nyurik

Copilot code review Copilot Awaiting requested review from Copilot Copilot will automatically review once the pull request is marked ready for review

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

RFC Post-processing (compression, encoding, conversion, pruning)

2 participants

@CommanderStorm @nyurik