Introduce public MergedDataMap #14766

lucylq · 2025-10-03T02:14:24Z

Stack from ghstack (oldest at bottom):

Add public merged data map. Module can use this to resolve multiple named data maps.

Differential Revision: D83527299

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

pytorch-bot · 2025-10-03T02:14:27Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14766

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 66 Pending, 2 Unrelated Failures

As of commit 836dc33 with merge base a39866c ():

NEW FAILURE - The following job has failed:

trunk / test-qnn-model (fp32, ic3) / linux-job (gh)
RuntimeError: Command docker exec -t 357caab441f3843126ceedf9ba3d4d0136a4c7db9802690bb63647818f90143a /exec failed with exit code 1

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

trunk / test-qnn-optimum-model (fp32, mobilevit_v2) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)
trunk / test-torchao-huggingface-checkpoints (phi_4_mini, linux.2xlarge, executorch-ubuntu-22.04-clang12,... / linux-job (gh) (detected as infra flaky with no log or failing log classifier)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-10-03T02:14:46Z

@lucylq has exported this pull request. If you are a Meta employee, you can view the originating Diff in D83527299.

github-actions · 2025-10-03T02:15:20Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

swolchok

ping me when this is ready for re-review so you don't have to wait

extension/named_data_map/merged_data_map.cpp

extension/named_data_map/merged_data_map.h

extension/named_data_map/merged_data_map.cpp

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Pull Request resolved: #14766 Add public merged data map. Module can use this to resolve multiple named data maps. Creating as a sep dependency rather than inside module/ so it can be used independently of module. (think there may be some other internal usages soon) Add support for BUCK and CMake. ghstack-source-id: 313990892 @exported-using-ghexport Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/)

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Pull Request resolved: #14766 Add public merged data map. Module can use this to resolve multiple named data maps. Creating as a sep dependency rather than inside module/ so it can be used independently of module. (think there may be some other internal usages soon) Add support for BUCK and CMake. ghstack-source-id: 314009154 @exported-using-ghexport Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/)

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Pull Request resolved: #14766 Add public merged data map. Module can use this to resolve multiple named data maps. Creating as a sep dependency rather than inside module/ so it can be used independently of module. (think there may be some other internal usages soon) Add support for BUCK and CMake. ghstack-source-id: 314016314 @exported-using-ghexport Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/)

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Pull Request resolved: #14766 Add public merged data map. Module can use this to resolve multiple named data maps. Creating as a sep dependency rather than inside module/ so it can be used independently of module. (think there may be some other internal usages soon) Add support for BUCK and CMake. ghstack-source-id: 314018889 @exported-using-ghexport Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/)

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Pull Request resolved: #14766 Add public merged data map. Module can use this to resolve multiple named data maps. Creating as a sep dependency rather than inside module/ so it can be used independently of module. (think there may be some other internal usages soon) Add support for BUCK and CMake. ghstack-source-id: 314023106 @exported-using-ghexport Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/)

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Pull Request resolved: #14766 Add public merged data map. Module can use this to resolve multiple named data maps. Creating as a sep dependency rather than inside module/ so it can be used independently of module. (think there may be some other internal usages soon) Add support for BUCK and CMake. On CMake side, have to add the dep for everyone who uses module .. ghstack-source-id: 314029159 @exported-using-ghexport Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/)

swolchok

just a bunch of minor complaints; seems fine overall

extension/named_data_map/test/CMakeLists.txt

extension/named_data_map/test/merged_data_map_test.cpp

extension/named_data_map/CMakeLists.txt

swolchok · 2025-10-04T00:11:44Z

extension/named_data_map/CMakeLists.txt

+install(
+  TARGETS extension_named_data_map
+  EXPORT ExecuTorchTargets
+  DESTINATION lib


this should be CMAKE_INSTALL_LIBDIR from GNUInstallDirs; inconveniently I never landed #12792 so we have a bunch of bad examples laying around

extension/named_data_map/merged_data_map.cpp

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Pull Request resolved: #14766 Add public merged data map. Module can use this to resolve multiple named data maps. Creating as a sep dependency rather than inside module/ so it can be used independently of module. (think there may be some other internal usages soon) Add support for BUCK and CMake. On CMake side, have to add the dep for everyone who uses module .. ghstack-source-id: 314310581 @exported-using-ghexport Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/)

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Pull Request resolved: #14766 Add public merged data map. Module can use this to resolve multiple named data maps. Creating as a sep dependency rather than inside module/ so it can be used independently of module. (think there may be some other internal usages soon) Add support for BUCK and CMake. On CMake side, have to add the dep for everyone who uses module .. ghstack-source-id: 314321393 @exported-using-ghexport Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/)

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Pull Request resolved: #14766 Add public merged data map. Module can use this to resolve multiple named data maps. Creating as a sep dependency rather than inside module/ so it can be used independently of module. (think there may be some other internal usages soon) Add support for BUCK and CMake. On CMake side, have to add the dep for everyone who uses module .. ghstack-source-id: 314330666 @exported-using-ghexport Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/)

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Pull Request resolved: #14766 Add public merged data map. Module can use this to resolve multiple named data maps. Creating as a sep dependency rather than inside module/ so it can be used independently of module. (think there may be some other internal usages soon) Add support for BUCK and CMake. On CMake side, have to add the dep for everyone who uses module .. ghstack-source-id: 314332815 @exported-using-ghexport Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/)

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Pull Request resolved: #14766 Add public merged data map. Module can use this to resolve multiple named data maps. Creating as a sep dependency rather than inside module/ so it can be used independently of module. (think there may be some other internal usages soon) Add support for BUCK and CMake. On CMake side, have to add the dep for everyone who uses module .. ghstack-source-id: 314332815 @exported-using-ghexport Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/)

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Pull Request resolved: #14766 Add public merged data map. Module can use this to resolve multiple named data maps. Creating as a sep dependency rather than inside module/ so it can be used independently of module. (think there may be some other internal usages soon) Add support for BUCK and CMake. On CMake side, have to add the dep for everyone who uses module .. ghstack-source-id: 314332815 @exported-using-ghexport Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/)

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Pull Request resolved: #14766 Add public merged data map. Module can use this to resolve multiple named data maps. Creating as a sep dependency rather than inside module/ so it can be used independently of module. (think there may be some other internal usages soon) Add support for BUCK and CMake. On CMake side, have to add the dep for everyone who uses module .. ghstack-source-id: 314332815 @exported-using-ghexport Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/)

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Introduce public MergedDataMap

d5a5295

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 3, 2025

This was referenced Oct 3, 2025

JNI support for multiple ptd files #14168

Merged

Swift support for multiple ptd files #14754

Merged

Use merged data map in module #14767

Open

facebook-github-bot added fb-exported meta-exported labels Oct 3, 2025

Update on "Introduce public MergedDataMap"

a00bab7

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

lucylq mentioned this pull request Oct 2, 2025

Export lora weights to sep file #14756

Open

swolchok requested changes Oct 3, 2025

View reviewed changes

Update on "Introduce public MergedDataMap"

3aad6a9

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

lucylq requested review from larryliu0820 and kirklandsign as code owners October 3, 2025 21:51

Update on "Introduce public MergedDataMap"

a9c4c55

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Update on "Introduce public MergedDataMap"

3843756

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Update on "Introduce public MergedDataMap"

f19b922

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Update on "Introduce public MergedDataMap"

cb5f30d

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

lucylq requested a review from cccclai as a code owner October 3, 2025 23:30

Update on "Introduce public MergedDataMap"

e2592ae

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

lucylq requested review from GregoryComer and SS-JIA as code owners October 4, 2025 00:02

swolchok approved these changes Oct 4, 2025

View reviewed changes

Update on "Introduce public MergedDataMap"

041a65a

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Update on "Introduce public MergedDataMap"

a79a486

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Update on "Introduce public MergedDataMap"

4896344

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Update on "Introduce public MergedDataMap"

75ebd26

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Update on "Introduce public MergedDataMap"

bae6c44

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Update on "Introduce public MergedDataMap"

8f475c1

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

lucylq added 3 commits October 6, 2025 15:45

Update on "Introduce public MergedDataMap"

4fe31ba

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Update on "Introduce public MergedDataMap"

47383bf

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Update on "Introduce public MergedDataMap"

9f1378c

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

lucylq added 2 commits October 6, 2025 15:48

Update on "Introduce public MergedDataMap"

53faa4a

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Update on "Introduce public MergedDataMap"

836dc33

Add public merged data map. Module can use this to resolve multiple named data maps. Differential Revision: [D83527299](https://our.internmc.facebook.com/intern/diff/D83527299/) [ghstack-poisoned]

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Introduce public MergedDataMap #14766

Introduce public MergedDataMap #14766

lucylq commented Oct 3, 2025 •

edited

Loading

Uh oh!

pytorch-bot bot commented Oct 3, 2025 •

edited

Loading

Uh oh!

facebook-github-bot commented Oct 3, 2025

Uh oh!

github-actions bot commented Oct 3, 2025

Uh oh!

swolchok left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

swolchok left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

swolchok Oct 4, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Introduce public MergedDataMap #14766

Are you sure you want to change the base?

Introduce public MergedDataMap #14766

Conversation

lucylq commented Oct 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Oct 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14766

❌ 1 New Failure, 66 Pending, 2 Unrelated Failures

Uh oh!

facebook-github-bot commented Oct 3, 2025

Uh oh!

github-actions bot commented Oct 3, 2025

This PR needs a release notes: label

Uh oh!

swolchok left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

swolchok left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

swolchok Oct 4, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lucylq commented Oct 3, 2025 •

edited

Loading

pytorch-bot bot commented Oct 3, 2025 •

edited

Loading

This PR needs a `release notes:` label