Skip to content

LIBRARIES.md

Latest commit

 

History

History
164 lines (120 loc) · 4.62 KB

LIBRARIES.md

File metadata and controls

164 lines (120 loc) · 4.62 KB

Module libraries [ALPHA]

The Toolkit can import modules from external libraries. This is an ALPHA feature.

Usage

A user can add packages to cdf.toml. At the moment, the Toolkit only loads one library.

[alpha_flags]
external-libraries = true


[library.package_1]
url = "https://raw.githubusercontent.com/cognitedata/toolkit-data/librarian/builtins.zip"

When running cdf modules init or cdf modules add, the packages will be imported and selectable for the user.

Publishing a library

The library must be available over https. Authentication is not currently supported.

To publish a library, create a repository that contains one or more downloadable zip files. The zip file must have the structure and content described below.

<package_1>.zip

Zip file content must be structured like this:

<package_1>.zip
├── packages.toml
└── module_1
    ├── <module content>
    ├── module.toml
    └── default.config.yaml
└── module_2
    ├── <module content>
    ├── module.toml
    └── default.config.yaml

packages.toml

The packages.toml in the root folder is required to make the library discoverable by the Toolkit. It allows you to describe the content of the library and bundle modules into packages. A module can belong to several packages.

The packages.toml file should contain the following information:

[library]
description = "<Description for the end user>"
toolkit-version = ">=0.6.0" # Recommended version of the Toolkit required to use this library
canCherryPick = true # Set to false if the user should not be able to pick individual modules in this package

[packages.quickstart]
id = "tk:quickstart"
title = "Quickstart"
description = "Get started with Cognite Data Fusion in minutes."
canCherryPick = false
modules = [
    "cdf_ingestion",
    "sourcesystem/cdf_pi",
    "sourcesystem/cdf_sap_assets",
    "sourcesystem/cdf_sap_events",
    "sourcesystem/cdf_sharepoint",
    "models/cdf_process_industry_extension",
    "contextualization/cdf_connection_sql",
    "contextualization/cdf_p_and_id_parser",
    "industrial_tools/cdf_search"
]

[packages.infield]
title = "InField"
id = "infield"
description = "Put real-time data into the hands of every field worker."
modules = [
    "infield/cdf_infield_location",
    "infield/cdf_infield_second_location",
    "infield/cdf_infield_common",
    "common/cdf_apm_base"
]

module.toml

Each module must have a module.toml file in its root folder. This file describes the module and its content.

The module.toml file should contain the following information:

[module]
title = "OSIsoft/AVEVA PI Data Source" # Title displayed to the user
is_selected_by_default = false # If true, the checkbox is selected by default
id = "aveva"
package_id = "quickstart"

[dependencies]
modules = [] # List of modules that this module depends on

default.config.yaml

A default.config.yaml file must be present in each module folder if the module contains any value placeholders.

In other words, if your module has any curly bracket placeholders in the resource file my.<Kind>.yaml file, you must provide a default.config.yaml file with those values:

#<root folder>/module_1/data_models/my.Space.yaml
space: sp_{{ location }}_instances
name: {{ location_name }}_instances
description: 'Contains instances for {{ location_name }}.'

#<root folder>/module_1/data_models/my.DataSet.yaml
externalId: ds_{{ location }}_functions
name: '{{location_name}} Functions'
description: This dataset contains Functions for the {{location_name}}.

#root folder>/module_1/default.config.yaml
location: <not set>
location_name: <not set>

When cdf modules init or cdf modules add is run, the default values from default.config.yaml will be automatically added to config.dev.yaml and config.prod.yaml so that the user can maintain the correct values.

environment:
  project: <cdf project>
  type: dev
  selected:
  - modules/

variables:
  modules:
    module_1:
      location: <not set>
      location_name: <not set>

Verification

To verify that the library is correctly structured:

  1. Add your library path ending with the root folder to cdf.toml:
[libraries.my_library]
url = https://example.com/my-library/my-root-folder
  1. Run cdf modules init or cdf modules add to verify that packages and modules are shown correctly
  2. Verify that the module content is copied into the local modules/ folder

Rate limiting

Certain cloud services, like GitHub, may rate limit requests to the library URL. Consider using a CDN or similar if concurrent requests are expected.