[Feature] Existing attribute filter for DDOs

[moritzkirstein]

Motivation / Problem

We want to be able to query the catalogue based on specific attributes existing in the DDO. Example: list only DDOs that have a Gaia-X service credential connected.

Solution

We need a new filter function that allows dynamically adding these filters via the filters.config.js file. Something like:

{
  filters: [
    // move existing filter config into this file, so it is all in one location
    {
      id: 'serviceType',
      label: 'Service Type',
      type: 'filterList',
      // add a "global" query path that gets used as fallback if options have none defined
      queryPath: 'metadata.type', 
      options: [
        { label: 'datasets', value: FilterByTypeOptions.Data },
        { label: 'algorithms', value: FilterByTypeOptions.Algorithm },
        { label: 'saas', value: FilterByTypeOptions.Saas }
      ]
    },
    {
      id: 'gaiax',
      label: 'Gaia-X Service',
      type: 'filterList',
      options: [
        // a new filter value for a MUST_EXIST type could be added to handle this new functionality
        { label: "Service SD", value: FilterValues.MUST_EXIST, queryPath: "additionalInformation.gaiaXInformation.serviceSd.url" },
        // options can have their own queryPath defined that gets validated against the defined value
        { label: "Terms and Conditions", value: true, queryPath: "additionalInformation.gaiaXInformation.termsAndConditions" } 
        // options can have their own queryPath defined that gets validated against the defined value
        { label: "Verified", value: true, queryPath: "additionalInformation.gaiaXInformation.serviceSd.isValid" } 
      ]
    }
  ]
}

Additional context

We will need to handle filters based on the value that should be checked. In cases where the value is any user defined string, such as specific tags or search queries, we want to use the existing filter approach: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-filter-context.html#filter-context
When handling the to be added MUST_EXIST filters, we need to change to a elasticsearch bool context, via exists:
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html

export enum FILTER_VALUES = {
  MUST_EXIST,
  MUST_EXISTS_AND_NON_EMPTY,
  //...
}

Example query:

{
  "query": {
    "bool": {
      "filter": [
        {
          "bool": {
          // MUST_EXIST & MUST_EXIST_AND_NON_EMPTY
           "must": {
             "exists": {
		"field": "metadata.additionalInformation.gaiaXInformation.serviceSD.url"
              }
            },
          // MUST_EXIST_AND_NON_EMPTY
            "must_not": {
               "term": {
		"metadata.additionalInformation.gaiaXInformation.serviceSD.url.keyword": ""
	        }
              }
          }
        }
      ]
    }
  }
}

It is probably necessary to wrap the src/utils/aquarius/index.ts:getFilterTerm() function in a new getFilter() fucntion and then decide based on the value to filter for what to generate, e.g. (pseudo-code):

var filter {}
if value is MUST_EXIST or MUST_EXIST_AND_NON_EMPTY:
  filter = {...filter, bool: { ...filter.bool, ...getMustExistQuery(field) } }
if value is MUST_EXIST_AND_NON_EMPTY:
  filter = {...filter, bool: { ...filter.bool, ...getNotEmptyQuery(field) } }
// check other potential pre-defined values
else:
  filter = getFilterTerm(field, value, key)