Skip to content

Commit 5f56af2

Browse files
mike4gitlukadschaakjdreesencoderabbitai[bot]
authored
[Feature][ImportExport] Assets and Documents in a common way (#13)
* [Feature] enable exporting and importing several pages by ONE yaml file * [Fix] use ['page'] instead of config * [Feature] Add integration tests for PageExporter Add Menu and Controller route for export page incl. subpages * [Feature] Moved import menu to main menu toolbar * [Feature] Test PageImporter and adapt way of saving * [Export All] new command for export all pages different converters for folder, snippet & pages * [Chore] fix general code styles * [Chore] readme: edit how tests must be executed * [Export All][Refactoring] remove duplicated code * Update src/Controller/Admin/PageImportController.php Co-authored-by: Luka Dschaak <l.dschaak@neusta.de> * Update tests/Integration/Documents/Export/PageExporterTest.php thx for translations Co-authored-by: Luka Dschaak <l.dschaak@neusta.de> * Update tests/Integration/Documents/Export/PageImporterTest.php thx for translations Co-authored-by: Luka Dschaak <l.dschaak@neusta.de> * Update tests/Integration/Documents/Export/PageImporterTest.php thx for translations Co-authored-by: Luka Dschaak <l.dschaak@neusta.de> * Update translations/admin.de.yaml thx for translations Co-authored-by: Luka Dschaak <l.dschaak@neusta.de> * Update translations/admin.en.yaml thx for translations Co-authored-by: Luka Dschaak <l.dschaak@neusta.de> * Update tests/Integration/Documents/Export/PageImporterTest.php thx for translations Co-authored-by: Luka Dschaak <l.dschaak@neusta.de> * Update tests/Integration/Documents/Export/PageImporterTest.php thx for translations Co-authored-by: Luka Dschaak <l.dschaak@neusta.de> * [Export All][Tests] add expected yaml files * [Export All][Tests] fix it * [Export All][Refactoring] (#12) * [Export All][Refactoring] fix it * [Export All][Refactoring] check import by path inheritance except parentId * [Export All][Refactoring] add option for output file * [Export All][Refactoring] rename property for Luka * [Export All][Refactoring] add some docs * [Export All][Refactoring][New Commands] for import and export * [Export All][Refactoring][New Commands] review by Bundle Dev Group * [Export All][Refactoring][New Commands] some refactorings and an additional menu for the export filename * [Export All][Refactoring][New Commands] change inlining for better readability and remove old export method * [Export All][Refactoring][New Commands] generalize serialization and add JSON input/output * [Export All][Refactoring][New Commands] fix tests * [Export All][Refactoring][New Commands] add parameter for format to Controllers and js Scripts * [Export All][Refactoring][New Commands] add test results * Update tests/Integration/Documents/Export/PageExporterTest.php Co-authored-by: Jacob Dreesen <jacob@hdreesen.de> * [Export All][Refactoring][New Commands] adapt naming import to export * [Export All][Refactoring][New Commands] fix test * Update src/Controller/Admin/PageImportController.php Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/Command/ExportPagesCommand.php Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/Command/ImportPagesCommand.php Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/Command/ImportPagesCommand.php Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * [Import][Export] Assets and Documents in common way * [Import][Export] fix tests - export refactoring done. * [Import][Export] fix tests - import refactoring done. * [Import][Export] Export Assets via Menu - done. * [Import][Export] Export Assets via Command - done. * [Import][Export] Import Assets via Command - done. * [Import][Export] Three menu items under Extras Main Menu for Import * [Import][Export] adapted icons for menus * [Import][Export] export Data Objects as well * [Import][Export] provide all Importers as Controllers * [Import][Export] provide mechanisms for Data Objects import * [Import][Export] provide mechanisms for Assets import * [Import][Export] cs fixed and phpstan corrected * [Import][Export] tests fixed * Update src/Populator/PageImportPopulator.php Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/Serializer/JsonSerializer.php Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * [Import][Export] CodeRabbit review fix * [Import][Export] CodeRabbit review fix * [Import][Export] Implement additional export commands * [Import][Export] CodeRabbit fix * [Import][Export] cs fixed, phpstan corrected and doc added * [Import][Export] Code Rabbit review fixed * [Import][Export] fix code style php * [Import][Export] remove unnecessary CSS * [Import][Export] code review by Luka * [Import][Export] remove unnecessary $sourceArrayKey declaration * [Import][Export] early access * [Import][Export] add red borders to context menu doc images * [Import][Export] update readme * [Import][Export] simplify routing.yaml * [Import][Export] integration test for ParentRelationResolver * [NSDTRINITY-17] rename common findAllInTree methods and pull up to ExportRepositoryInterface * Update src/Command/ExportDataObjectsCommand.php Co-authored-by: Luka Dschaak <l.dschaak@neusta.de> * Update src/Command/ExportDocumentsCommand.php Co-authored-by: Luka Dschaak <l.dschaak@neusta.de> * Update src/Command/ImportDocumentsCommand.php Co-authored-by: Luka Dschaak <l.dschaak@neusta.de> * [NSDTRINITY-17] correct export of Data Objects and cut relations of related objects to avoid circular reference * [NSDTRINITY-17] partial correct import of Data Objects (e.g. DataLink will not be imported atm) * [NSDTRINITY-17] cs fixed, phpstan corrected * [NSDTRINITY-17] fix parentResolver * [NSDTRINITY-17] add help to commands * [NSDTRINITY-17] code rabbit review --------- Co-authored-by: Luka Dschaak <l.dschaak@neusta.de> Co-authored-by: Jacob Dreesen <jacob@hdreesen.de> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
1 parent e213f62 commit 5f56af2

File tree

99 files changed

+4398
-552
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

99 files changed

+4398
-552
lines changed

README.md

Lines changed: 144 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -3,57 +3,165 @@
33
## Installation
44

55
1. **Require the bundle**
6-
6+
77
```shell
88
composer require teamneusta/pimcore-import-export-bundle
99
```
1010

11-
2. **Enable the bundle**
12-
11+
2. **Enable the bundle**
12+
1313
Add the Bundle to your `config/bundles.php`:
14-
15-
```php
16-
Neusta\Pimcore\ImportExportBundle\NeustaPimcoreImportExportBundle::class => ['all' => true],
17-
```
14+
15+
```php
16+
Neusta\Pimcore\ImportExportBundle\NeustaPimcoreImportExportBundle::class => ['all' => true],
17+
```
1818

1919
## Usage
2020

21-
After enabling the bundle you should see a new menu item in the context menu of Pimcore Admin Backend - Section Documents:
21+
### Pimcore Admin Backend
22+
23+
#### Export
24+
25+
After enabling the bundle you should see a new menu item in the context menu of Pimcore Admin Backend:
2226

23-
![context_menu_import_export.png](docs/images/context_menu_import_export.png)
27+
| Documents | Assets | Data Objects |
28+
|--------|-------------------------------------------------------------------------------------|---------------------------------------------------------------------------------|
29+
|![context_menu_export_assets.png](docs/images/context_menu_export_assets.png)| ![context_menu_export_documents.png](docs/images/context_menu_export_documents.png) | ![context_menu_export_objects.png](docs/images/context_menu_export_objects.png) |
30+
31+
Currently only YAML Export is supported by menu
32+
33+
After clicking one of the menu items you will be asked for a file name and the export will start:
34+
35+
![filename_dialog.png](docs/images/filename_dialog.png)
2436

2537
(german translation)
2638

39+
##### Special case: Assets Export
40+
41+
Because Assets are mostly assigned to physical files (images, videos, documents, etc.) the export will create a ZIP file containing the binary data and a YAML file with the metadata.
42+
The same structured zip file can be used for the import of Assets as well.
43+
44+
#### Import
45+
For the import you can have a look into the Tools Menu:
46+
47+
![import_menu.png](docs/images/import_menu.png)
48+
49+
50+
### Symfony Commands
51+
52+
This bundle provides several commands to export and import data into Pimcore.
53+
54+
Fur usage, run the commands with the `--help` option to see all available options and arguments.
55+
56+
- `neusta:pimcore:export:documents`
57+
- Export Pimcore documents (e.g., pages, snippets) to a YAML file. Export either the entire document tree or specific document IDs.
58+
- `neusta:pimcore:export:assets`
59+
- Exports Pimcore assets (e.g., images, videos, PDFs) into two files: A YAML file along with a ZIP file containing the actual binary files.
60+
- Both files share the same base name (e.g., `assets.yaml` and `assets.zip`).
61+
- `neusta:pimcore:export:objects`
62+
- Exports Pimcore DataObjects into a YAML file. The objects can be filtered by ID or exported in full.
63+
- `neusta:pimcore:import:documents`
64+
- Imports Pimcore documents (e.g., pages, snippets) from a YAML file.
65+
- `neusta:pimcore:import:assets`
66+
- Imports assets (e.g., images, PDFs, videos) into Pimcore based on a YAML definition.
67+
- The command expects a YAML file describing the assets and a ZIP archive containing the corresponding files. The ZIP file must be located in the same directory as the YAML file and must have the same base name (e.g., `assets.yaml` and `assets.zip`)
68+
- `neusta:pimcore:import:objects`
69+
- Imports Pimcore DataObjects from a YAML file.
70+
71+
#### Common Options
72+
73+
All import commands follow a similar structure and support the following common options:
74+
75+
- `--input` or `-i`: Path to the input YAML file (required for import commands).
76+
- `--dry-run`: Perform the operation without persisting data (only available for import commands).
77+
78+
## Concepts
79+
2780
### Page Export
81+
2882
The selected page will be exported into YAML format:
83+
2984
```yaml
30-
page:
31-
id: 123
32-
parentId: 1
33-
type: page
34-
published: true
35-
path: /
36-
language: de
37-
navigation_name: my-site
38-
navigation_title: 'My Site'
39-
key: my-site
40-
title: 'My Site'
41-
controller: 'App\DefaultController::indexAction'
42-
editables:
43-
main:
44-
type: areablock
45-
name: main
46-
data: [{ key: '1', type: text-editor, hidden: false }]
47-
...
85+
elements:
86+
-
87+
Pimcore\Model\Document:
88+
id: 123
89+
parentId: 1
90+
type: page
91+
published: true
92+
path: /
93+
language: de
94+
navigation_name: my-site
95+
navigation_title: 'My Site'
96+
key: my-site
97+
title: 'My Site'
98+
controller: 'App\DefaultController::indexAction'
99+
editables:
100+
main:
101+
type: areablock
102+
name: main
103+
data: [ { key: '1', type: text-editor, hidden: false } ]
104+
# ...
48105
```
49106
50107
In the same way you can re-import your yaml file again by selecting: `Import from YAML` in the context menu.
51108
52-
## Configuration
53-
54109
### Page Import
55110
56-
The import process will create a new page with the given data.
111+
The import process will create pages with the given data.
112+
113+
The following rule applies:
114+
115+
If the parseYaml method of the `PageImporter` is not called with `forcedSave`, the data from the provided YAML will be
116+
adopted, regardless of whether it makes sense or not, and without checking whether the page could be saved that way.
117+
118+
* If `forcedSave` is set to `true`, the ID will be retained (Caution – this can overwrite an existing page).
119+
* If `forcedSave` is set to `true` and no ID has been set, it will be generated by Pimcore (Creating new page).
120+
121+
* If a `parentId` is specified, the corresponding document will be searched for.
122+
* If it exists, it will be set as the parent (Note: This may override the `path` specification).
123+
* If the `parentId` does not exist, an attempt will be made to find a parent using the `path` specification.
124+
* If such a parent exists, the `parentId` will be set accordingly and saved.
125+
* If neither is found, an InvalidArgumentException will be thrown, and the save operation will be aborted.
126+
127+
![parent_flow.png](docs/images/parent_flow.png)
128+
129+
If multiple pages are imported and a path specification changes by the applied rules, this path specification will be
130+
replaced with the new, correct path specification in all provided page configurations.
131+
132+
### Parameterize your yaml files
133+
134+
You can parameterize your yaml files with placeholders. The placeholders will be replaced by the values you provide in your fixtures.
135+
136+
```yaml
137+
elements:
138+
-
139+
Pimcore\Model\Document:
140+
id: 2
141+
parentId: 1
142+
# ...further properties
143+
editables:
144+
# ...several editables
145+
'main:1.img:1.image':
146+
type: image
147+
data:
148+
id: %IMAGE_ID%
149+
'main:1.img:1.title':
150+
# ...
151+
```
152+
153+
In the case above an image has been assigned to an `Editable/Image` editable. The image id is a placeholder `%IMAGE_ID%`.
154+
155+
You can use now a `Neusta\Pimcore\ImportExportBundle\Documents\Import\Filter\SearchAndReplaceFilter` instance to replace the placeholder with the actual image id (e.g. 1234).
156+
157+
```php
158+
$yamlContent = (new SearchAndReplaceFilter(['%IMAGE_ID%' => 1234]))->filterAndReplace($yamlContent);
159+
```
160+
161+
If you want to change your yaml in a more complex way you can use the `Neusta\Pimcore\ImportExportBundle\Documents\Import\Filter\YamlFilter` interface to implement your own filter.
162+
163+
With that technique you can export test pages for Fixtures, change values into placeholders (e.g. for assets and data objects) and replace them with the actual values in your tests.
164+
57165
58166
## Contribution
59167
@@ -74,5 +182,10 @@ We use composer scripts for our main quality tools. They can be executed via the
74182
```shell
75183
bin/composer cs:fix
76184
bin/composer phpstan
77-
bin/composer tests
185+
```
186+
187+
For the tests there is a different script, that includes a database setup.
188+
189+
```shell
190+
bin/run-tests
78191
```

compose.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ services:
1616
timeout: 10s
1717

1818
php:
19-
image: pimcore/pimcore:php8.3-latest
19+
image: pimcore/pimcore:php8.3-debug-latest
2020
volumes:
2121
- ./:/var/www/html/
2222
environment:

composer.json

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,9 @@
1212
],
1313
"require": {
1414
"php": "~8.1.0 || ~8.2.0 || ~8.3.0",
15+
"ext-zip": "*",
1516
"pimcore/pimcore": "^11.0",
16-
"teamneusta/converter-bundle": "^1.7"
17+
"teamneusta/converter-bundle": "^1.8.0"
1718
},
1819
"require-dev": {
1920
"ergebnis/composer-normalize": "^2.42.0",
@@ -25,6 +26,7 @@
2526
"phpstan/phpstan-symfony": "^1.3.8",
2627
"phpunit/phpunit": "^9.5",
2728
"pimcore/admin-ui-classic-bundle": "^1.6",
29+
"spatie/phpunit-snapshot-assertions": "^4.2",
2830
"teamneusta/pimcore-testing-framework": "^0.12"
2931
},
3032
"autoload": {

config/pimcore/config.yaml

Lines changed: 3 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -1,63 +1,3 @@
1-
###########################################################
2-
# Neusta Converter Bundle
3-
###########################################################
4-
neusta_converter:
5-
converter:
6-
###########################################################
7-
# Import Converter (YamlExportPage -> Page)
8-
###########################################################
9-
neusta_pimcore_import_export.import_page:
10-
target: Pimcore\Model\Document\Page
11-
populators:
12-
- Neusta\Pimcore\ImportExportBundle\Documents\Import\PageImportPopulator
13-
properties:
14-
id:
15-
source: id
16-
skip_null: true
17-
default: 0
18-
key: ~
19-
title:
20-
source: title
21-
default: 'no title'
22-
controller:
23-
source: controller
24-
default: 'no controller'
25-
type:
26-
source: type
27-
default: 'page'
28-
published:
29-
source: published
30-
default: false
31-
path:
32-
source: path
33-
default: '/'
34-
parentId:
35-
source: parentId
36-
default: 0
37-
38-
###########################################################
39-
# Export Converter (Page -> YamlExportPage)
40-
###########################################################
41-
neusta_pimcore_import_export.export_page:
42-
target: Neusta\Pimcore\ImportExportBundle\Documents\Export\YamlExportPage
43-
populators:
44-
- neusta_pimcore_import_export.page.property.language.populator
45-
- neusta_pimcore_import_export.page.property.navigation_title.populator
46-
- neusta_pimcore_import_export.page.property.navigation_name.populator
47-
- neusta_pimcore_import_export.page.editables.populator
48-
properties:
49-
id: ~
50-
key: ~
51-
title: ~
52-
controller: ~
53-
type: ~
54-
published: ~
55-
path: ~
56-
parentId: ~
57-
58-
neusta_pimcore_import_export.editable_converter:
59-
target: Neusta\Pimcore\ImportExportBundle\Documents\Export\YamlExportEditable
60-
properties:
61-
type: ~
62-
name: ~
63-
data: ~
1+
imports:
2+
- { resource: 'import/**/*' }
3+
- { resource: 'export/**/*' }
Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
###########################################################
2+
# Neusta Converter Bundle
3+
###########################################################
4+
neusta_converter:
5+
converter:
6+
###########################################################
7+
# Export Converter (Pimcore Asset -> Folder)
8+
###########################################################
9+
neusta_pimcore_import_export.export_asset_folder:
10+
target: Neusta\Pimcore\ImportExportBundle\Model\Asset\Asset
11+
properties:
12+
id: ~
13+
key: ~
14+
type: ~
15+
path: ~
16+
parentId: ~
17+
18+
services:
19+
_defaults:
20+
autowire: true
21+
autoconfigure: true
22+
23+
###########################################################
24+
# Export Converter (Pimcore Asset -> Asset)
25+
###########################################################
26+
neusta_pimcore_import_export.strategy.export_asset:
27+
class: Neusta\Pimcore\ImportExportBundle\Converter\TypeStrategyConverter
28+
arguments:
29+
$typeToConverterMap:
30+
Pimcore\Model\Asset\Archive: '@neusta_pimcore_import_export.export_asset_archive'
31+
Pimcore\Model\Asset\Audio: '@neusta_pimcore_import_export.export_asset_audio'
32+
Pimcore\Model\Asset\Document: '@neusta_pimcore_import_export.export_asset_document'
33+
Pimcore\Model\Asset\Image: '@neusta_pimcore_import_export.export_asset_image'
34+
Pimcore\Model\Asset\Text: '@neusta_pimcore_import_export.export_asset_text'
35+
Pimcore\Model\Asset\Video: '@neusta_pimcore_import_export.export_asset_video'
36+
Pimcore\Model\Asset\Folder: '@neusta_pimcore_import_export.export_asset_folder'
37+
Pimcore\Model\Asset\Unknown: '@neusta_pimcore_import_export.export_asset_unknown'
38+
39+
neusta_pimcore_import_export.export_asset_archive:
40+
class: Neusta\Pimcore\ImportExportBundle\Converter\ExtendedConverter
41+
arguments:
42+
$converter: '@neusta_pimcore_import_export.export_asset_folder'
43+
$postPopulators:
44+
- '@neusta_pimcore_import_export.asset.filename.populator'
45+
46+
neusta_pimcore_import_export.export_asset_audio:
47+
class: Neusta\Pimcore\ImportExportBundle\Converter\ExtendedConverter
48+
arguments:
49+
$converter: '@neusta_pimcore_import_export.export_asset_folder'
50+
$postPopulators:
51+
- '@neusta_pimcore_import_export.asset.filename.populator'
52+
53+
neusta_pimcore_import_export.export_asset_document:
54+
class: Neusta\Pimcore\ImportExportBundle\Converter\ExtendedConverter
55+
arguments:
56+
$converter: '@neusta_pimcore_import_export.export_asset_folder'
57+
$postPopulators:
58+
- '@neusta_pimcore_import_export.asset.filename.populator'
59+
60+
neusta_pimcore_import_export.export_asset_image:
61+
class: Neusta\Pimcore\ImportExportBundle\Converter\ExtendedConverter
62+
arguments:
63+
$converter: '@neusta_pimcore_import_export.export_asset_folder'
64+
$postPopulators:
65+
- '@neusta_pimcore_import_export.asset.filename.populator'
66+
67+
neusta_pimcore_import_export.export_asset_text:
68+
class: Neusta\Pimcore\ImportExportBundle\Converter\ExtendedConverter
69+
arguments:
70+
$converter: '@neusta_pimcore_import_export.export_asset_folder'
71+
$postPopulators:
72+
- '@neusta_pimcore_import_export.asset.filename.populator'
73+
74+
neusta_pimcore_import_export.export_asset_video:
75+
class: Neusta\Pimcore\ImportExportBundle\Converter\ExtendedConverter
76+
arguments:
77+
$converter: '@neusta_pimcore_import_export.export_asset_folder'
78+
$postPopulators:
79+
- '@neusta_pimcore_import_export.asset.filename.populator'
80+
81+
neusta_pimcore_import_export.export_asset_unknown:
82+
class: Neusta\Pimcore\ImportExportBundle\Converter\ExtendedConverter
83+
arguments:
84+
$converter: '@neusta_pimcore_import_export.export_asset_folder'
85+
$postPopulators: []
86+
87+
###########################################################
88+
# Export Populator (Pimcore Asset -> Asset)
89+
###########################################################
90+
neusta_pimcore_import_export.asset.filename.populator:
91+
class: Neusta\ConverterBundle\Populator\PropertyMappingPopulator
92+
arguments:
93+
$sourceProperty: filename
94+
$targetProperty: filename
95+

0 commit comments

Comments
 (0)