Skip to content

Latest commit

 

History

History
651 lines (438 loc) · 22.5 KB

File metadata and controls

651 lines (438 loc) · 22.5 KB
description page_type month_change
Import data into your repository from prepared YAML files.
reference
false

Importing data

To import data from YAML migration files into repository, you run the ibexa:migrations:migrate command.

The ibexa:migrations:import command automatically places migration files in the correct folder.

Alternatively, you can place the files manually in the src/Migrations/Ibexa/migrations folder or in a custom folder that you configure, and specify the file name within this folder as parameter. If you don't specify the file, all files within this directory are used.

php bin/console ibexa:migrations:migrate --file=my_data_export.yaml --siteaccess=admin

Migrations store execution metadata in the ibexa_migrations database table. This allows incremental upgrades: the ibexa:migrations:migrate command ignores files that it had previously executed.

The --siteaccess option usage can be relevant when multiple languages or multiple repositories are used.

Migration step

A data migration step is a single operation in data migration process that combines a mode (for example: create, update, delete) and a type (for example: content, section, currency), with optional additional information depending on the specific step.

In a migration file, a step is an array item starting with the mandatory properties type and mode, for example:

-
    type: content
    mode: create

Then, the step is described by additional properties depending on its type and mode.

Available migrations

The following data migration step modes are available:

type create update delete swap trash
action_configuration
attribute
attribute_group
content_type
content_type_group
content
currency
customer_group
discount
discount_code
language
location
object_state
object_state_group
payment_method
product_asset
product_availability
product_price
product_variant
role
section
segment
segment_group
setting
user
user_group

Additionally, the following special migration types are available:

type execute
repeatable
sql
try_catch

Repeatable steps

You can run a set of one or more similar migration steps multiple times by using the special repeatable migration type.

A repeatable migration performs the defined migration steps as many times as the iterations setting declares.

[[= include_file('code_samples/data_migration/examples/repeatable_step.yaml', 0, 5) =]]

!!! tip

You can use repeatable migration steps, for example, to quickly generate large numbers of content items for testing purposes.

You can vary the operations using the iteration counter.

For example, to create five Folders, with names ranging from "Folder 0" to "Folder 4", you can run the following migration using the iteration counter i:

[[= include_file('code_samples/data_migration/examples/repeatable_step.yaml', 0, 16) =]]

To vary the content name, the migration above uses Symfony expression syntax.

In the example above, the expression is enclosed in ### and the repeated string SSS.

!!! note

Iteration counter is assigned to `i` by default, but you can modify it in the `iteration_counter_name` setting.

Generating fake data

You can also generate fake data with the help of FakerPHP.

To use it, first install Faker on your system:

composer require fakerphp/faker

Then, you can use faker() in expressions, for example:

[[= include_file('code_samples/data_migration/examples/repeatable_step.yaml', 16, 19) =]]

This step generates field values with fake personal names.

SQL migrations

You can execute raw SQL queries directly in migrations using the sql migration type. Use it for custom database operations that don't fit into standard entity migrations, such as creating custom tables or performing bulk updates.

Each query requires a driver property that specifies which database system the query is for. The migration system automatically filters queries and executes only those matching your current database driver.

[[= include_file('code_samples/data_migration/examples/sql_execute.yaml') =]]

The supported database drivers are:

  • mysql - MySQL/MariaDB
  • postgresql - PostgreSQL
  • sqlite - SQLite

You can define queries for multiple database drivers in a single migration step. The system executes only the queries that match your configured database platform. If no matching queries are found, the migration throws an error.

!!! caution

SQL migrations bypass the content model abstraction layer and directly modify the database.
Use them with caution and ensure your queries are compatible with your target database system.

Error handling with try-catch

You can wrap one or more migration steps with a try_catch step to handle exceptions gracefully.

Use it for migrations that may fail under specific conditions but should not halt the entire migration process.

For example, you can ensure a language creation migration succeeds even if the language already exists. If the migration fails for this reason, the exception is suppressed, allowing the remaining migrations to proceed without interruption.

A try_catch migration requires the steps property and accepts optional allowed_exceptions and stop_after_first_exception settings.

[[= include_file('code_samples/data_migration/examples/try_catch_step.yaml') =]]

When an exception is thrown within a try-catch step, it's compared against the list of allowed_exceptions. If the exception matches, it's caught and the migration continues or stops depending on the stop_after_first_exception configuration setting. The default value of stop_after_first_exception is true.

Expression syntax

You can use [Symfony expression syntax]([[= symfony_doc =]]/reference/formats/expression_language.html) in data migrations, like in repeatable steps, where you can use it to generate varied content in migration steps.

The expression syntax uses the following structure: ###<IDENTIFIER> <EXPRESSION> <IDENTIFIER>###

The IDENTIFIER can be any repeated string that encloses the actual expression.

Built-in functions

Built-in expression language functions that are tagged with ibexa.migrations.template.expression_language.function:

  • to_bool, to_int, to_float, to_string - convert various data types by passing them into PHP casting functions (like floatval, intval, and others).
                -   fieldDefIdentifier: show_children
                    languageCode: eng-US
                    value: '###XXX to_bool(i % 3) XXX###'

                -   fieldDefIdentifier: quantity
                    languageCode: eng-US
                    value: '###XXX to_int("42") XXX###'

                -   fieldDefIdentifier: price
                    languageCode: eng-US
                    value: '###XXX to_float("19.99") XXX###'

                -   fieldDefIdentifier: description
                    languageCode: eng-US
                    value: '###XXX to_string(123) XXX###'
  • reference - references a specific object or resource within your application or configuration. Learn more about migration references.
                -   fieldDefIdentifier: some_field
                    languageCode: eng-US
                    value: '###XXX reference("example_reference") XXX###'
  • project_dir - retrieves the project's root directory path, for example to construct file paths or access project-specific resources.
                -   fieldDefIdentifier: project_directory
                    languageCode: eng-US
                    value: '###XXX project_dir() XXX###'
  • env - retrieves the value of an environmental variable.
[[= include_file('code_samples/data_migration/examples/update_user.yaml') =]]

Custom functions

To add custom functionality into Migration's expression language declare it as a service and tag it with ibexa.migrations.template.expression_language.function.

Example:

ibexa.migrations.template.to_bool:
    class: Closure
    factory: [ Closure, fromCallable ]
    arguments:
        - 'boolval'
    tags:
        -   name: 'ibexa.migrations.template.expression_language.function'
            function: to_bool

ibexa.migrations.template.faker:
    class: Closure
    factory: [ Closure, fromCallable ]
    arguments:
        - 'Faker\Factory::create'
    tags:
        -   name: 'ibexa.migrations.template.expression_language.function'
            function: faker

Service-based functions can be also added, but they must be callable, requiring either an __invoke function or a wrapping service with one.

Migration examples

The following examples show what data you can import using data migrations.

Content types

The following example shows how to create a content type with two field definitions.

The required metadata keys are: identifier, mainTranslation, contentTypeGroups and translations.

The default values of field definition properties mirror the underlying PHP API, for example:

  • translatable defaults to true
  • required defaults to false
[[= include_file('code_samples/data_migration/examples/create_blog_post_ct.yaml') =]]

Content items

The following example shows how to create two content items: a folder and an article inside it.

When creating a content item, three metadata keys are required: contentType, mainTranslation, and parentLocationId.

To use the location ID of the folder, which is created automatically by the system, you can use a reference. In this case you assign the parent_folder_location_id reference name to the location ID, and then use it when creating the article.

[[= include_file('code_samples/data_migration/examples/create_parent_and_child_content.yaml') =]]

Use the delete mode to delete content items:

[[= include_file('code_samples/data_migration/examples/delete_content.yaml') =]]

Images

The following example shows how to migrate an example-image.png located in public/var/site/storage/images/3/8/3/0/383-1-eng-GB without manually placing it in the appropriate path.

To prevent the manual addition of images to specific DFS or local locations, such as public/var/site/storage/images/ you can move image files to, for example src/Migrations/images. Adjust the migration file and configure the image field data as follows:

        -   fieldDefIdentifier: image
            languageCode: eng-GB
            value:
                alternativeText: ''
                fileName: example-image.png
                path: src/Migrations/images/example-image.png

This migration copies the image to the appropriate directory, in this case public/var/site/storage/images/3/8/3/0/254-1-eng-GB/example-image.png, enabling swift file migration regardless of storage (local, DFS).

Roles

The following example shows how to create a role. A role requires the identifier metadata key.

For each policy assigned to the role, you select the module and function, with optional limitations.

The following example shows the creation of a Contributor role:

[[= include_file('code_samples/data_migration/examples/create_role.yaml') =]]

To update an existing role, two policies' modes are available:

  • replace: (default) All existing policies are replaced by the ones from the migration.
  • append: Migration policies are added while already existing ones are kept.

The following example shows how to replace the policies of the existing Editor role:

[[= include_file('code_samples/data_migration/examples/update_role.yaml', 0, 16) =]]

The following example shows the addition of a policy to the Anonymous role:

[[= include_file('code_samples/data_migration/examples/update_role.yaml', 18, 32) =]]

The following example shows how to delete the Contributor role:

[[= include_file('code_samples/data_migration/examples/delete_role.yaml') =]]

Locations

The following example shows how to swap content items assigned to given locations.

[[= include_file('code_samples/data_migration/examples/swap_location.yaml') =]]

The metadata keys for Location are optional.

The following example shows how to trash locations.

[[= include_file('code_samples/data_migration/examples/trash_location.yaml') =]]

Users

The following example shows how to create a user.

The required metadata keys are: login, email, password, enabled, mainLanguage, and contentType. You also need to provide the user group's remote content ID.

You can use an action to assign a role to the user.

[[= include_file('code_samples/data_migration/examples/create_user.yaml') =]]

You can also update user information, including passwords:

[[= include_file('code_samples/data_migration/examples/update_user.yaml') =]]

Languages

The following example shows how to create a language.

The required metadata keys are: languageCode, name, and enabled.

[[= include_file('code_samples/data_migration/examples/create_language.yaml') =]]

Product catalog

Attributes and attribute groups

The following example shows how to create an attribute group with two attributes:

[[= include_file('code_samples/data_migration/examples/create_attribute_group.yaml') =]]

You can also update attributes, including changing which attribute group they belong to:

[[= include_file('code_samples/data_migration/examples/update_attribute.yaml') =]]

You can't change the attribute type of an existing attribute.

Date and time attributes

You can manage the date and time attribute type through the migrations, for example:

[[= include_file('code_samples/data_migration/examples/create_datetime_attribute.yaml') =]]

Product types

The following example shows how to create a product type.

The main part of the migration file is the same as when creating a regular content type.

A product type must also contain the definition for an ibexa_product_specification field. fieldSettings contains information about the product attributes.

[[= include_file('code_samples/data_migration/examples/create_product_type.yaml') =]]

Products

The following example shows how to create a product:

[[= include_file('code_samples/data_migration/examples/create_product_variant.yaml', 0, 18) =]]

Product variants

The following example shows how to create variants for a product identified by its code:

[[= include_file('code_samples/data_migration/examples/create_product_variant.yaml', 19, 29) =]]

Product assets

The following example creates an image content item from a local image file, and then uses it as a product asset for a variant (created in previous example):

[[= include_file('code_samples/data_migration/examples/create_product_asset.yaml') =]]

This migration uses a reference to store the created image content ID, and then uses it while creating the asset. It uses an expression syntax to [concat (~)]([[= symfony_doc =]]/reference/formats/expression_language.html#string-operators) the mandatory scheme ezcontent:// and the image content ID through the reference function used on the reference's name.

Product prices

The following example shows how to create a price for a product identified by its code:

[[= include_file('code_samples/data_migration/examples/create_product_price.yaml') =]]

Customer groups

The following example shows how to create a customer group with a defined global price discount:

[[= include_file('code_samples/data_migration/examples/create_customer_group.yaml') =]]

Currencies

The following example shows how to create a currency:

[[= include_file('code_samples/data_migration/examples/create_currency.yaml') =]]

Commerce [[% include 'snippets/commerce_badge.md' %]]

Payment methods

The following example shows how to create a payment method:

[[= include_file('code_samples/data_migration/examples/create_payment_method.yaml') =]]

Shipping methods

The following example shows how to create a shipping method:

[[= include_file('code_samples/data_migration/examples/create_shipping_method.yaml') =]]

Segments [[% include 'snippets/experience_badge.md' %]] [[% include 'snippets/commerce_badge.md' %]]

The following example shows how to create a segment group and add segments in it:

[[= include_file('code_samples/data_migration/examples/create_segment.yaml', 0, 17) =]]

When updating a segment group or segment, you can match the object to update by using its numerical ID or identifier:

[[= include_file('code_samples/data_migration/examples/create_segment.yaml', 18, 24) =]]

Settings

The following example shows how you can create and update a setting stored in the database:

[[= include_file('code_samples/data_migration/examples/create_update_setting.yaml') =]]

Taxonomies

The following example shows how you can create a "Car" tag in the main Taxonomy:

[[= include_file('code_samples/data_migration/examples/create_tag.yaml') =]]

The field identifiers must match the identifiers used in the ibexa_taxonomy configuration file.

If the content type associated with the tags is changed, the configuration should be adjusted when creating migrations.

!!! note If there are multiple taxonomies, the taxonomy field is then necessary here (line 21).

You can use the following example to assign tags to a Content (content type Article has an additional field):

[[= include_file('code_samples/data_migration/examples/assign_tag.yaml') =]]

When updating a content type, use:

[[= include_file('code_samples/data_migration/examples/update_tag.yaml') =]]

AI action configurations

  • The following example shows how you can create a new action configuration in your system:
[[= include_file('code_samples/data_migration/examples/ai/action_configuration_create.yaml') =]]
  • Use the update mode to modify an existing action configuration:
[[= include_file('code_samples/data_migration/examples/ai/action_configuration_update.yaml') =]]
  • Use the delete mode to delete an existing action configuration:
[[= include_file('code_samples/data_migration/examples/ai/action_configuration_delete.yaml') =]]

Discounts

  • The following example shows how you can create a new discount in your system:
[[= include_file('code_samples/data_migration/examples/discounts/discount_create.yaml') =]]
  • Use the update mode to modify an existing discount as in the example below. The provided conditions overwrite any already existing ones.
[[= include_file('code_samples/data_migration/examples/discounts/discount_update.yaml') =]]

For a list of available conditions, see Discounts API.

Discount codes

You can create a discount code as in the following example:

[[= include_file('code_samples/data_migration/examples/discounts/discount_code_create.yaml') =]]

Criteria

When using update or delete modes, you can use criteria to identify the objects to operate on.

!!! caution

Criteria only work with objects related to the product catalog.
type: currency
mode: update
criteria:
    type: field_value
    field: code
    value: EUR
    operator: '=' # default
code: EEE
subunits: 3
enabled: false

Available operators are:

  • =
  • <>
  • <
  • <=
  • >
  • >=
  • IN
  • NIN
  • CONTAINS
  • STARTS_WITH
  • ENDS_WITH

You can combine criteria by using logical criteria and and or:

type: or
criteria:
    -   type: field_value
        field: code
        value: EUR
    -   type: field_value
        field: code
        value: X
        operator: STARTS_WITH

Criteria can be nested.