| shorty | MTX Migration | |||
|---|---|---|---|---|
| synopsis | Explains how to migrate from <code>@sap/cds-mtx</code> (aka Old MTX) to 'streamlined' <code>@sap/cds-mtxs</code>. | |||
| breadcrumbs |
|
|||
| status | released | |||
| impl-variants | true |
Towards new multitenancy capabilities {.subtitle}
::: warning Separate model changes from migration
We strongly recommended to separate any model changes from the migration. If you need to do model changes for the migration, please deploy the application
based on @sap/cds-mtx and upgrade all tenants using the upgrade endpoint before you do the migration.
:::
::: warning Deprecated! Update all modules
Make sure that you always use the latest version of the CAP modules using npm outdated. For Java, also check the versions configured in pom.xml files. Since
@sap/cds-mtx is deprecated for quite some time now and will no longer run with, for example, the latest version of @sap/cds, updating the versions and adapting your application to it
can only be done together with the migration to @sap/cds-mtxs. Please also read all release notes carefully and check it for changes that need to be made to the configuration.
:::
Before you start to migrate to @sap/cds-mtxs, read about the differences compared to the old MTX.
With @sap/cds-mtxs, the persistence has been simplified. There's no second container needed (META-tenant) any longer. Instead, tenant-specific metadata, such as extensions, are stored in the same container as the application data.
In addition, @sap/cds-mtxs also uses a dedicated tenant t0 to store some runtime data, such as job logs.
In contrast to @sap/cds-mtx, with @sap/cds-mtxs, the extensions are no longer stored as sources, but only as compiled csn files. Instead of running a build on the server with each extension activation, the build is now run locally before the extension is deployed. The extensions are then stored as csn files with a tag as key. When using cds push, the tag is derived from the name of the extension project in package.json.
Example package.json of extension project:
{
"name": "@capire/orders-ext",
"extends": "@capire/orders",
...
}
When the extension is pushed, it is stored with the tag @capire/orders-ext.
Also check the Push API.
As mentioned previously, cds push only uploads compiled extensions as CSN files. Thus, it's no longer possible to download the CDS sources from the server. Source control is expected to be done by the SaaS application provider using his own repository.
Some of the roles have changed with @sap/cds-mtxs.
| @sap/cds-mtx | @sap/cds-mtxs |
|---|---|
ExtendCDS |
cds.ExtensionDeveloper |
ExtendCDSdelete |
w/o replacement |
- Diagnose API isn't available.
-
Scopes aren't configurable.
-
It isn't possible to have tenant-specific model versions.
-
Use of SAP HANA hdbmigrationtable is only possible for entities that aren't to be extended.
-
Upload of arbitrary custom files together with extensions is no longer available.
To switch to @sap/cds-mtxs, you need to change your project configuration, your custom handlers, and you might need to update the database content.
{style="width:500px"}
To switch your Node.js project to @sap/cds-mtxs, perform the following steps:
-
Remove
@sap/cds-mtx:npm remove @sap/cds-mtx
-
Add
@sap/cds-mtxs:npm add @sap/cds-mtxs
-
Open your package.json and add the following:
"cds": { "requires": { "multitenancy": true } }
If your project supports extensibility, you need to enable extensibility in your configuration.
To do so, you only need to add extensibility: true to your cds configuration in .cdsrc.json or package.json.
"requires": {
"multitenancy": true,
"extensibility": true
}To create a sidecar based on @sap/cds-mtxs, you can use the following command:
cds add multitenancyIt creates a new sidecar folder mtx/sidecar and also modifies other files, including mta.yaml.
Currently, as cds add multitenancy is meant to be used with new projects, the best way is to revert the changes that have been made to mta.yaml and to make a few manual changes instead.
The global build section can be removed. The necessary build script has moved to the sidecar module.
# build-parameters:
# before-all:
# - builder: custom
# commands:
# - npm install --production
# - npx -p @sap/cds-dk cds build --productionTo switch the runtime module to @sap/cds-mtxs, you need to add the corresponding environment variable:
requires:
...
- name: mtx-sidecar
properties:
CDS_MULTITENANCY_MTXS_ENABLED: true # Only required for cds-services version 2
CDS_MULTITENANCY_SIDECAR_URL: ~{url}To enable the newly created sidecar, you need to change the path of your existing sidecar to the new path.
You only need to adapt the path to mtx/sidecar and add a custom build section.
::: code-group
modules:
- name: bookshop-mtx
type: nodejs
path: mtx/sidecar # adapted path
build-parameters: # added build section
builder: custom
build-result: gen
commands:
- npm run build
requires:
- name: bookshop-srv
parameters:
memory: 256M
disk-quota: 1G
requires:
- name: bookshop-auth
- name: bookshop-db
provides:
- name: mtx-api
properties:
mtx-url: ${default-url}:::
To make the @sap/cds-mtxs models part of the installation, add a workspace to the root package.json to include the sidecar dependencies.
"workspaces": [
"mtx/sidecar"
]::: tip Freeze Sidecar Dependencies
To prepare the build of the MTA archive (mbt build), you need to generate a package-lock.json for the sidecar by executing this in the project root:
npm i --package-lock-only --prefix mtx/sidecar:::
cds add multitenancy also adapts the build tasks in .cdsrc.json or package.json.
You only need to remove the mtx build task.
If your project uses the default project layout, all build tasks can be removed from the build configuration as follows:
{
"build": {
"target": "."
},
"profiles": ["with-mtx-sidecar", "java"],
"requires": {
"multitenancy": true
}
}If your project supports extensibility, you need to enable extensibility in your configuration.
To do so, you only need to add extensibility: true to your cds configuration in .cdsrc.json or package.json.
"requires": {
"multitenancy": true,
"extensibility": true
}The scopes needed by extension developers have changed.
Scopes ExtendCDS and ExtendCDSdelete have changed to cds.ExtensionDeveloper.
Make sure to adapt all occurrences in your security configuration (xs-security.json).
Communicate to customer admins and extension developers to add the new scope to their role collection. Also adjust the documentation for the SaaS application accordingly if available.
A typical handler registration in server.js now looks like
cds.on('served', async () => {
const { 'cds.xt.SaasProvisioningService': provisioning } = cds.services
const { 'cds.xt.DeploymentService': deployment } = cds.services
await provisioning.prepend(() => {
provisioning.on('UPDATE', 'tenant', async (req, next) => { ... })
provisioning.on('dependencies', async (req, next) => { ... })
...
})
await deployment.prepend(() => {
// previously this was `upgradeTenant`
deployment.on('upgrade', async (req) => {
// HDI container credentials are not yet available here
})
// previously this was `deployToDb`
deployment.on('deploy', async (req) => {
const { tenant, options: { container } } = req.data
...
})
...
})
})Here's what has changed:
ProvisioningServicechanged tocds.xt.SaasProvisioningServiceDeploymentServicechanged tocds.xt.DeploymentService- Use
cds.on('served')instead ofcds.on('mtx').
For Node.js, the saas-registry endpoints in mta.yaml need to be changed to .../-/cds/saas-provisioning/...:
parameters:
service: saas-registry
config:
appUrls:
getDependencies: ~{mtx-api/mtx-url}/-/cds/saas-provisioning/dependencies
onSubscription: ~{mtx-api/mtx-url}/-/cds/saas-provisioning/tenant/{tenantId}@sap/cds-mtx offers some additional configuration that you can also set in @sap/cds-mtxs.
In @sap/cds-mtx, you can configure the HDI container creation as follows:
"mtx": {
"provisioning": {
"lazymetadatacontainercreation": true,
"container": {
"provisioning_parameters": {
"database_id": "<ID>"
},
"binding_parameters": {
"key": "value"
}
},
"metadatacontainer": {
"provisioning_parameters": {
"database_id": "<ID_META>"
}
}
}
}In @sap/cds-mtxs, you can do the same configuration for the cds.xt.DeploymentService:
"requires": {
"cds.xt.DeploymentService": {
"lazyT0": true,
"hdi": {
"create": {
"database_id": "<ID>"
},
"bind": {
"key": "value"
}
},
"for": {
"t0": {
"hdi": {
"create": {
"database_id": "<ID_META>"
}
}
}
}
},
}See also Deployment configuration
This configuration allows to set what extensions are allowed.
With @sap/cds-mtx:
"mtx" : {
"extension-allowlist": [
{
"for": ["my.bookshop.Authors", "my.bookshop.Books"],
"new-fields": 2
},
{
"for": ["CatalogService"]
}
]
}With @sap/cds-mtxs, the same configuration has moved to the cds.xt.ExtensibilityService configuration:
"requires": {
"cds.xt.ExtensibilityService": {
"extension-allowlist": [
{
"for": ["my.bookshop.Authors", "my.bookshop.Books"],
"new-fields": 2
},
{
"for": ["CatalogService"]
}]
}
}See also Extensibility configuration
As first verification of your configuration changes, you can try to run your application locally in hybrid mode. To bind all the service
that are bound to your existing application, you can call cds bind --to-app-services <your application>. Afterwards, you can run cds run --profile hybrid --resolve-bindings.
Depending on the MTX features that your existing application has used, you need to execute some steps to move your data to the persistence used by @sap/cds-mtxs.
In case you only used the multitenancy features such as subscription/unsubscription, you just need to make the configuration changes described earlier.
::: tip When does this scenario apply?
- Your application doesn't support extensibility.
- You don't need to read all tenant IDs or the tenant metadata using
GET /-/cds/saas-provisioning/tenant/orGET /-/cds/saas-provisioning/tenant/<tenantId>.
The tenant metadata is the data that is sent to the MTX API by the SAP BTP SaaS Provisioning Service on subscription, similar to this:
{
"subscriptionAppId": "...",
"subscriptionAppName": "..." ,
"subscribedTenantId": "...",
...
}:::
If your application needs access to the tenant list or tenant metadata, you need to update this data for @sap/cds-mtxs.
::: tip When does this scenario apply?
- Your application doesn't support extensibility.
- Your application needs to read all tenant IDs or the tenant metadata using
GET /-/cds/saas-provisioning/tenant/orGET /-/cds/saas-provisioning/tenant/<tenantId>. :::
In order to copy the metadata from existing subscriptions to the new persistence of @sap/cds-mtxs, you need to run a migration script that comes with @sap/cds-mtxs.
If your application supports extensibility, you also need to update the existing extensions for @sap/cds-mtxs. You can do this with the same migration script.
::: tip When does this scenario apply?
- Your application supports extensibility. :::
The migration script is part of @sap/cds-mtxs. You can run it locally or during application deployment. Before running the script, you need to make the configuration changes mentioned earlier.
The script has to run in the (Node.js) application environment resulting from cds build --production to correctly simulate the execution in the deployment environment.
For Node.js applications, this result is the gen/srv folder generated in the application root, for Java applications, this result is the gen folder of the new @sap/cds-mtxs sidecar (mtx/sidecar/gen).
It also needs access to the application bindings. That means, when running locally, it has to run in hybrid mode.
You also need to add the production profile to ensure that the models are resolved correctly.
::: tip
Make sure, that the sources you want to migrate have the exact same version on your local machine as the sources that are deployed to the @sap/cds-mtx application .
:::
Example:
cds migrate "*" --dry --profile hybrid,production --resolve-bindingsTo run the migration for all or a set of tenants, you need to run:
cds migrate <tenant>[,<tenant>]|"*"The option --dry allows you to perform a dry run, without changing the database content.
Keep in mind that, depending on the number of tenants, the script requires some time to run. This is important when you consider to run it in combination with the application deployment.
If the migration was successful, tenants are marked as migrated. When running the migration a second time, these tenants are ignored. If you want to rerun the migration also for the already migrated tenants, you can do so by using parameter --force.
You can use the migration script to save the content of the subscribers' extension projects.
With parameter -d, you can specify a directory that is used by the script to store the existing, migrated extension projects.
cds migrate <tenant>[,<tenant>]|"*" -d <your directory>To really access the saved extension projects, you need access to the file system, of course. So, the easiest way is to run the script locally for that.
You can add the migration script as a hook to your Node.js server module (application or sidecar) in mta.yaml.
For that, you can use the script cds-mtx-migrate that also comes with the @sap/cds-mtxs but doesn't require @sap/cds-dk to be installed.
Example:
- name: bookshop-mt-sidecar
type: nodejs
path: mtx/sidecar
...
hooks:
- name: migrate-tenants
type: task
phases:
# - blue-green.application.before-start.idle
- deploy.application.before-start
parameters:
name: migration
memory: 512M
disk-quota: 768M
command: cds-mtx-migrate "*"See also Module Hooks ::: warning Warning: In case you already run an upgrade as task and your project supports extensions, make sure that the upgrade is run AFTER the migration. Otherwise, the content of extended tables can get lost. :::
The concept of extensions has slightly changed with @sap/cds-mtxs. Extensions sources are no longer stored in the backend. Instead, each extension gets a tag and the extension is stored as csn with the tag as key.
When running the migration script, all extension files are compiled to one csn and are stored with a default tag: migrated.
You can change the default tag by passing your own tag using the --tag parameter:
cds migrate "*" -d migrated_projects --tag "mytag"In addition, you can separate your extensions into several csn-files with different tags. For example, if your original extension files follow a pattern, you can do so by passing parameter --tagRule with a regular expression.
Let's use the following extension project structure:
old-bookshop-ext/
├── db/
│ ├── extension_id_1.cds
│ └── extension_id_2.cds
│ └── order_ext_id_1.cds
│ └── order_ext_id_2.cds
├── srv/
└── package.jsonYou can split your extensions as follows:
cds migrate "*" -d migrated_projects --tagRule "(?:ext_|extension_)(.*)\.cds"As a result, you get two extensions with tags id_1 and id_2. The tag is taken from the first captured group of the regular expression.
::: tip Find the right regular expression To verify if the result meets your expectations, you can make a dry run:
cds migrate "*" -d migrated_projects --tagRule "(?:ext_|extension_)(.*)\.cds" --dryYou can find the result in the folder migrated_projects. :::
To verify the result of the migration script, check the tenant's content of the HDI container. You can use any database client that can access SAP HANA databases.
To see the content of an HDI Container, you can add the tenant container to the SAP HANA Database Explorer.
You can find the migrated extensions in table CDS_XT_EXTENSIONS. The table contains:
- extensions parsed as
csnstrings in column csn - key column tag
{ .adapt style="width:800px; box-shadow: 1px 1px 5px #888888"}
As mentioned in Save Existing Extension Projects, you can store existing extension projects locally.
We recommend to upload the projects to a source repository (e. g. github), because with @sap/cds-mtxs the content of extension projects is no longer stored in the tenant database. With that setup you can change and push the extension again later.
The content of extension projects is usually the property of the customer (subscriber). So, alternatively, the customer can download the extension projects himself and upload them to his own source repository.
As described in the extensibility guide, you usually start with an empty extension project and pull the base model of the application using cds pull.
When starting with a migrated extension project, you need to make some adaptations after running cds pull. Previously, extension projects were using the full set of CDS files whereas extension projects based on @sap/cds-mtxs are using a compiled index.csn of the base model. This affects the references in the extension sources of the migrated project. So these references need to be adapted.
Recommended steps:
- Run
cds pullto fetch the latest version of the base model asindex.csn. - Fix the references in your extension sources. All references to the base model must use the name specified in the
cds.extendsentry of the extension package.json, omitting any additional subfolders. Example:using sap.capire.bookshop from '_base/db/schema';must be replaced byusing sap.capire.bookshop from 'base-model';
You can see all broken references as error messages when using the CDS Editor.
As long as the metadata containers (TENANT-<id>-META) created by @sap/cds-mtx still exist, the customer extension projects can be downloaded using the CDS client. The user running the download command needs to have the scope cds.ExtensionDeveloper assigned:
cds extend <url> --download-migrated-projectsThe command downloads an archive named migrated_projects.tgz that contains the existing extensions that are ready to be used with @sap/cds-mtxs.