[Connectors] DataGalaxy connector

A DataGalaxy integration can be configured to synchronize dictionaries, usages, data products, and other defined workspace elements with YOOI instances.

It is possible to do so via Admin settings > Integration.

Initialisation on YOOI

1. In Admin settings > Integration, create a new Integration and give it a name, for example “DataGalaxy Integration”.

2. Generate an API Token and copy the value. Unless there is a very specific need, the DataGalaxy connector token must be chosen preferentially with no expiration date.

The token will never be accessible again, so copy the value in a secure place

3. Give to YOOI the API token.

4. YOOI needs to re-deploy your app with this information and add a feature-flag.

5. Create or have these concepts on YOOI :

   1. (mandatory) A concept representing the DataGalaxy instance you want to map, which must have an External key field.

Only one DataGalaxy connector is possible per platform.

Instances synchronization behaviour

Creation

DataGalaxy → YOOI

• Trigger : Every X minutes.

• Actions : Creates the YOOI instance based on mapping and populates the fields mapped in the connector configuration.

YOOI → DataGalaxy

• Trigger : YOOI instance is in a concept mapped with a DataGalaxy objet with a configuration that is non-flat and has:

  • A name

  • A technical name

  • Embedded-by field (if applicable)

    • If this field is filled, the instance is considered embedded under its parent.

    • If not filled, the instance can be considered a first-class object (top-level).

If any of those above isn’t set, the instance will not be created in DataGalaxy.

• Actions : Creates the DataGalaxy instance and populates the fields mapped in the connector configuration.

Update

DataGalaxy → YOOI

• Trigger : Every X minutes (see polling).

• Actions : Updates the YOOI instance based on mapping and populates the fields mapped in the connector configuration.

YOOI → DataGalaxy

• Trigger : Every update on a mapped field in YOOI.

• Actions : Updates the DataGalaxy instance based on mapping and populates the fields mapped in the connector configuration.

Not all fields are editable on DataGalaxy. See details here.

Deletion

DataGalaxy → YOOI

Behavior

• Nothing happens; the instance will no longer be synchronized.

YOOI → DataGalaxy

• Trigger : Every deletion on a mapped instance in YOOI.

• Actions :

  • If makeObsoleteInsteadOfDeletingis set to true(default option), the corresponding DataGalaxy instance status is set to Obsolete (instance is not removed).

  • If makeObsoleteInsteadOfDeletingis set to false, the corresponding DataGalaxy instance is deleted from DataGalaxy.

Connector restart

The connector restarts when the YOOI app restarts or the JSON configuration is changed.

On connector restart, we re-fetch all DataGalaxy instances that are in mapped projects.

Configuration

JSON Configuration

You need to create a JSON as below, more information on the JSON structure is available below :

{
  "apiUrl": "https://url.api.datagalaxy.com",
  "credentials": {
    "token": "myIntegrationTokenOrPersonalToken"
  },
  "polling": {
     "cron": "0 23 * * *", //Synchronization done every day at 23:00 
     "timezone": "Europe/Paris",
     "forceSyncOnConfigurationUpdate": true  
  },   
  "configuration": {
    "notObsoleteContainer": {
      "name": "Container",
      "type": "flat-dictionary",
      "workspace": "workspaceId",
      "dataType": "containers",
      "containers" : {
        "filters": [ //All filters will be applied as an AND filters
          {
            "attribute": "EntityStatus",
            "condition": "NOT_EQUALS", // EQUALS, NOT_EQUALS, IS_NOT_EMPTY or IS_EMPTY 
            "value": "Obsolete"
          },
          {
            "attribute": "Stewards", // Multi-value relation attribute
            "condition": "CONTAIN", // CONTAIN, CONTAIN_SOME, DO_NOT_CONTAIN, EQUALS, NOT_EQUALS, IS_NOT_EMPTY or IS_EMPTY 
            "value": ["admin@datagalaxy.com"]
          }
        ],
        "attributes": [ //Optional, if no configuration is mapped all fields will be added to the mapping. As DataGalaxy has a lot of fields, it is recomended to filter them.
          {
            "key": "DisplayName",
            "label": "Name", //Optional, if empty the Attribute name will be used
            "mode": "sync" //sync/read/write,
          },
          {
            "key": "TechnicalName",
            "label": "Technical Name",
            "mode": "write"
          }
        ]
      },
    "notObsoleteReport": {
      "name": "Report",
      "type": "usages",
      "workspace": "workspaceId",
      "usages" : {
        "attributes": [ //Optional, if no configuration is mapped all fields will be added to the mapping. As DataGalaxy as a lot of field, it is recommended to filter them.
          {
            "key": "DisplayName",
            "label": "Name", //Optional, if empty the Attribute name will be used
            "mode": "sync" //sync/read/write,
          },
          {
            "key": "TechnicalName",
            "label": "Technical Name",
            "mode": "read"
          }
        ]
      }
    }
  }
}

API URL

To retrieve the DataGalaxy API URL, follow these steps:

1. Go to your company's DataGalaxy platform.

2. Click on your profile in the top right corner.

3. Select DataGalaxy API.

4. Copy the provided URL.

Polling

The polling configuration take three options:

• cron: A cron-like string that defines when polling is done.

  • Use a website like crontab.guru to help create the configuration

• timezone(optional):

  • If not specified, UTC will be used by default.

  • Valid timezones are:

• forceSyncOnConfigurationUpdate(optional):

  • If empty or set to true, synchronization occurs each time the configuration is updated.

  • If set to false, synchronization will only happen according to the cron schedule.

Workspace Id

To find out the workspaceId:

1. Go to your company's DataGalaxy platform.

2. Navigate to the workspace you need the ID for.

3. Check the URL in your browser:

   • The workspaceId is the UUID that appears right after space/ in the URL.

   • For example, if the URL is:

     CODE

     https://dkw.datagalaxy.com/#/client/ccd4ddc5-2236-4df2-b028-c7041a48e5ef/space/17eefa1c-9ca2-4083-b4ae-d5c7a7ebd094/49fc80fa-6837-4791-9768-e4771a973bb8/home
     

     then the workspaceId is 17eefa1c-9ca2-4083-b4ae-d5c7a7ebd094.

Links

If getLinks is set to true (optional, false by default) an association will be added.

An n-n association corresponding to the links in the instance will be added to the mapping.

Only links with the following characteristics will be included:

• There is only one type of target instance.
  e.g.: for use case, only links with domain and objective will be added.

• The target type must only appear once in the configuration.

The linkModeoption (optional, ‘sync’ by default) allows you to choose how these associations will be synchronized :

• sync: association value will be synchronized between YOOI and DataGalaxy, meaning that both YOOI → DataGalaxy and DataGalaxy → YOOI will be handled

• write: (DataGalaxy → YOOI) only updates from DataGalaxy will be considered for this association

• read: (YOOI → DataGalaxy) only updates from YOOI will be considered for this association

Configuration type

Two types of configurations are supported: flat and non-flat.

Flat Configuration

• Description:
  Used when the tree (parent/child representation) of DataGalaxy is not synchronized with YOOI. Instances are synchronized independently, without representing their position in the DataGalaxy hierarchy.

• Filters:
  Supported to determine which DataGalaxy instances should be synchronized.

• Limitations:
  Creation from YOOI → DataGalaxy is not supported because DataGalaxy requires a tree representation.

Non-Flat Configuration

• Description:
  Used when the tree structure is synchronized and mandatory in YOOI. The tree is represented by an embedding field on the parent, which must be mapped in YOOI to the corresponding field.

  • If this field is not mapped, no children of the given parent will be synchronized.

• Example:
  To synchronize Use Cases (children) with their Initiatives (parents), both objects must be declared in the JSON configuration. Since the connector recognizes that Use Cases can be embedded within Initiatives, an embedding field will automatically be proposed in the Initiatives mapping.

• Filters:
  Not supported.

• Scope:
  All parents and children must belong to the same workspace.

Data Types

• The dataType is required when applicable.

• It tells the connector which type of instances will be synchronized.

• In some types (usages), the type is implied, so you don’t need to specify dataType.

The configuration is declared with the followed model:

{
  "configurationKey": {
  "name": "Container",
  "type": "flat-dictionary",
  "workspace": "workspaceId",
  "dataType": "containers",
  "makeObsoleteInsteadOfDeleting": false, //true per default
  "getLinks": true, // false per default
  "linkMode": 'sync', // 'sync', 'write' or 'read'  
  "{name of the dataType}" : {
    "filters": [], // only applicable if type is flat
    "attributes": []
  }
}

The configurationKey needs to be unique for each configuration, as it is used internally as an id for the configuration. The name is the actual display name used in the mapping.

The filters (only supported if flat) and attributes properties are described in the next sections.

Filters

Only supported in flat configuration.

DataGalaxy instance count tend to be high, and you probably don’t want to add all of them to your YOOI instance. To avoid that, the configuration allows you to define a list of filters to apply to each of your configurations. Filters take three parameters:

• attribute: the attribute key you want to filter on

• condition:

  • either EQUALS, NOT_EQUALS, IS_EMPTY, or IS_NOT_EMPTY for most attributes,

  • or EQUALS, NOT_EQUALS, CONTAIN, CONTAIN_SOME, DO_NOT_CONTAIN, IS_EMPTY or IS_NOT_EMPTYfor multi-value relation attributes (ManagedTag, MultiValueList, UserReference, and StewardUserReference)

• value: the value to be tested (array of values for multiple value relations attributes, number for number attributes, etc.).

All the filters on the list will be applied as an AND filters.

Instances that have already been synchronised will still be updated even if they no longer match the filter. To prevent synchronisation, key must be removed from the YOOI instance.

Attributes

The configuration can be enriched with an optional list of attributes to synchronized.

• key: this is the attribute key of DataGalaxy, to get it you need to make an API call to GetAttributes endpoint

• label: optional, only used to generate mapping label, if empty the attribute label will be used

• mode: optional, default value is sync if attribute is editable on DataGalaxy, read if not.

  • sync: attribute value will be synchronized between YOOI and DataGalaxy, meaning that both YOOI → DataGalaxy and DataGalaxy → YOOI will be handled

  • write: (DataGalaxy → YOOI) only updates from DataGalaxy will be considered for this attribute

  • read: (YOOI → DataGalaxy) only updates from YOOI will be considered for this attribute

If no attributes list is configured, all DataGalaxy attributes will be displayed in the mapping section. As there is a lot of attribute in an DataGalaxy workspace, it is encouraged to configure an attribute list in order to make the mapping more readable, and the connector more efficient.

For non-flat configurations, embedding fields will automatically be added for mapping on the parent concepts.

Mapping

DataGalaxy dictionaries and usages can be mapped to YOOI instances.

To display the mapping section for the first time, a valid JSON configuration is required.

Supported field types

Currently, the following field types can be mapped:

• Text

• EntityLogicalPathString

• Boolean

• Number

• ValueList

• SystemEntityType

• ManagedTag

• UserReference

• StewardUserReference

• MultiValueList

• HtmlLink

• FormattedText

• Embedding field when applicable

Single relation mapping

For single-value relation attributes (such as ValueList and SystemEntityType), two types of mappings can be defined: one for mapping the value to an n-1 relation field, and another for mapping it within a workflow. You should choose only one of these mappings and disregard the other. Mapping both is not recommended, as it can lead to unpredictable behavior when syncing from YOOI to DataGalaxy.

Association field mapping

For each configured association field, the connector creates a mapping section to match instances. This matching process ensures that each DataGalaxy instance corresponds to the correct instance in YOOI.

The primary tool for this mapping is the externalKey field.

The table serves two purposes:

• A visual aid to see which instance is mapped to which value.

• An edition tool, as linking an instance in the table (selecting a YOOI instance) will edit the key field with the correct value.

If the keys are already filled in YOOI, the mapping happens automatically.

Editable attributes in DataGalaxy

Not all attributes can be updated in DataGalaxy.

Below is a list of attributes that can be modified depending on the data type. Other attributes will be ignored when doing an update from YOOI to DataGalaxy, even if there are set to sync or read.

TechnicalName needs to be unique per dataType in DataGalaxy.

For Stewards and Owners, only specific users are allowed to hold these roles in DataGalaxy. If a user assigned from YOOI is not eligible, the update will be rejected.

Sources

• Name

• Description

• Owners

• Status

• Stewards

• Summary

• Tags

• Custom properties

Type and TechnicalName are not editable, but are required to identify and update a DataGalaxy source instance. They must always be included in the attribute mapping with YOOI for the connector to function properly.

Containers

• Name

• Description

• Owners

• Status

• Stewards

• Summary

• Tags

• TechnicalName

• Custom properties

Structures

• Name

• Description

• Owners

• Status

• Stewards

• Summary

• Tags

• TechnicalName

• Custom properties

Fields

• Name

• Description

• Owners

• Status

• Stewards

• Summary

• Tags

• TechnicalName

• ColumnDataType (required)

• Size(required)

ColumnDataType and Size are required when updating a Field instance. They must always be included in the attribute mapping with YOOI for the connector to function properly and be set to sync or read.

Usages

• Name

• Description

• Owners

• Status

• Stewards

• Summary

• Tags

• TechnicalName
  

Objectives

• Name

• Description

• Owners

• Status

• Stewards

• Summary

• Tags

• TechnicalName

• Custom properties

Initiatives

• Name

• Description

• Owners

• Status

• Stewards

• Summary

• Tags

• TechnicalName

• Custom properties

Use Cases

• Name

• Description

• Owners

• Status

• Stewards

• Summary

• Tags

• TechnicalName

• Custom properties

Data Products

• Name

• Description

• Owners

• Status

• Stewards

• Summary

• Tags

• TechnicalName

• Custom properties

AI Products

• Name

• Description

• Owners

• Status

• Stewards

• Summary

• Tags

• TechnicalName

• Custom properties

Policies

• Name

• Description

• Owners

• Status

• Stewards

• Summary

• Tags

• TechnicalName

• Custom properties

Rule Groups

• Name

• Description

• Owners

• Status

• Stewards

• Summary

• Tags

• TechnicalName

• Custom properties

Rules

• Name

• Description

• Owners

• Status

• Stewards

• Summary

• Tags

• TechnicalName

• Custom properties

Monitor Groups

• Name

• Description

• Owners

• Status

• Stewards

• Summary

• Tags

• TechnicalName

• Custom properties

Monitors

• Name

• Description

• Owners

• Status

• Stewards

• Summary

• Tags

• TechnicalName

• Custom properties