[GPDSC Spec clarification]: Do "csolution" uses the GPDSC file content to override the PDSC definition of components ?

[BadreddineBJA]

GPDSC Spec clarification

1. Assumptions:

• The GPDSC .xsd schema is the same as the PDSC schema, hence any PDSC element declared in the spec should be supported by the GPDSC.
• The GPDSC is parsed by csolution and used as a mean to override the PDSC definition of components.

2. Scenario example description:

a. Use a packA PDSC definition like the following:

<?xml version="1.0" encoding="UTF-8" ?>
<package xmlns:xs="http://www.w3.org/2001/XMLSchema-instance" schemaVersion="1.7.2" xs:noNamespaceSchemaLocation="https://github.com/Open-CMSIS-Pack/Open-CMSIS-Pack-Spec/blob/main/schema/PACK.xsd">
    <vendor>STMicro</vendor>
    <name>packA</name>
    <description>Pack A description</description>
    <generators>
        <generator id="GEN-1">
            <description>GEN1 generator</description>
            <exe>
                <command host="win">gen.bat</command>
                <command host="linux">gen.sh</command>
                <!-- Generator Id -->
                <argument>GEN-1</argument>
                <!-- Generator Input File absolute path -->
                <argument>$G</argument>
                <!-- GPDSC Template File Name -->
                <argument>template_model.gpdsc.hbs</argument>
            </exe>
            <gpdsc name="$P/template_model.gpdsc"/>
        </generator>
    </generators>
    <components>
        <component Cclass="Device" Cgroup="generated" Csub="compA" Cversion="0.0.1" generator="GEN-1">
            <description>compA description</description>
            <RTE_Components_h>
                <!-- the following content goes into file 'RTE_Components.h' -->
				#define _RTE_COMPA_FROM_PDSC
            </RTE_Components_h>
            <Pre_Include_Global_h>
                <!-- the following content goes into file 'Pre_Include_Global.h' -->
				#define PRE_INCLUDE_GLOBAL_COMPA_FROM_PDSC
            </Pre_Include_Global_h>
            <Pre_Include_Local_Component_h>
                <!-- the following content goes into file 'Pre_Include_Local_<ComponentId>.h' -->
				#define PRE_INCLUDE_LOCAL_COMPA_FROM_PDSC
            </Pre_Include_Local_Component_h>
            <files>
                <file category="genAsset" name="templates/compA/stm32_compA_template.c"/>
                <file category="genAsset" name="templates/compA/stm32_compA_template.h"/>
            </files>
        </component>
    </components>
</package>

b. Generate a GPDSC file with the following content:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- ******************************************************************************
 * File Name   : GPDSC_template.gpdsc
 * Description : Generator PDSC File generated (DO NOT EDIT!)
 ****************************************************************************** -->

<package xmlns:xs="http://www.w3.org/2001/XMLSchema-instance" schemaVersion="1.0"
    xs:noNamespaceSchemaLocation="https://github.com/Open-CMSIS-Pack/Open-CMSIS-Pack-Spec/blob/main/schema/PACK.xsd">
    <vendor>STMicro</vendor>
    <name>packA</name>
    <description>Pack A description</description>
    <url>path</url>
    <generators>
        <generator id="GEN-1">
            <description>GEN1 generator</description>
            <exe>
                <command host="win">gen.bat</command>
                <command host="linux">gen.sh</command>
                <!-- Generator Id -->
                <argument>GEN-1</argument>
                <!-- Generator Input File absolute path -->
                <argument>$G</argument>
                <!-- GPDSC Template File Name -->
                <argument>template_model.gpdsc.hbs</argument>
            </exe>
            <gpdsc name="$P/template_model.gpdsc" />
        </generator>
    </generators>
    <components>
    	<component Cclass="Device" Cgroup="generated" Csub="compA" Cversion="1.0.0" generator="GEN-1">
            <description>compA description</description>
            <RTE_Components_h>
                <!-- the following content goes into file 'RTE_Components.h' -->
                #define USING_CompA_GPDSC
            </RTE_Components_h>
            <Pre_Include_Global_h>
                <!-- the following content goes into file 'Pre_Include_Global.h' -->
                #define PRE_INCLUDE_GLOBAL_compA_GPDSC
            </Pre_Include_Global_h>
            <Pre_Include_Local_Component_h>
                <!-- the following content goes into file 'Pre_Include_Local_<ComponentId>.h' -->
                #define PRE_INCLUDE_LOCAL_compA_GPDSC
            </Pre_Include_Local_Component_h>
            <files>
                <file category="source" name="stm32_compA.c" />
                <file category="header" name="stm32_compA.h" />
            </files>
        </component>
    </components>    
</package>

3. Observed Result:
The content of the RTE_<context_id> Generated files is the PDSC list of defines, the GPDSC definition of <RTE_Components_h>, <Pre_Include_Global_h>, <Pre_Include_Local_Component_h> for the same component is not taken into consideration by csolution.

RTE_Components.h generated file:

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.0.0
 * Target:  'debug+main' 
 */
#ifndef RTE_COMPONENTS_H
#define RTE_COMPONENTS_H
/*
 * Define the Device Header File: 
 */
#define CMSIS_device_header "stm32u5xx.h"

/* STMicro::Device:generated:compA:0.0.1 */
#define _RTE_COMPA_FROM_PDSC**

#endif /* RTE_COMPONENTS_H */

Pre_Include_Global.h

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.0.0
 * Target:  'debug+main' 
 */
#ifndef PRE_INCLUDE_GLOBAL_H
#define PRE_INCLUDE_GLOBAL_H
/* STMicro::Device:generated:compA:0.0.1 */
#define PRE_INCLUDE_GLOBAL_COMPA_FROM_PDSC**
#endif /* PRE_INCLUDE_GLOBAL_H */

Pre_Include_Local_compId.h

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.0.0
 * Target:  'debug+main' 
 */

#ifndef PRE_INCLUDE_DEVICE_GENERATED_COMPA_H
#define PRE_INCLUDE_DEVICE_GENERATED_COMPA_H

/* STMicro::Device:generated:compA:0.0.1 */
#define PRE_INCLUDE_LOCAL_COMPA_FROM_PDSC

#endif /* PRE_INCLUDE_DEVICE_GENERATED_COMPA_H */

4. What is expected ?

• GPDSC "component A" tags definitions ( <RTE_Components_h>, <Pre_Include_Global_h>, <Pre_Include_Local_Component_h> ) should override the PDSC definitions of the same component.
• General use case: Any GPDSC component definition should override the same PDSC component definition at the convert step.

[BadreddineBJA]

Tests re-executed using csolution v2.3.0, I get the same result as csolution v2.0.0, only the PDSC defines are taken into consideration, GPDSC declared defines are not taken into account at the convert step

RTE_Components.h generated file:

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.3.0
 *
 * Project: 'app6_SW.debug+main' 
 * Target:  'debug+main' 
 */

#ifndef RTE_COMPONENTS_H
#define RTE_COMPONENTS_H


/*
 * Define the Device Header File: 
 */
#define CMSIS_device_header "stm32u5xx.h"

/* STMicro::Device:generated:compA@0.0.1 */
#define _RTE_COMPA_FROM_PDSC


#endif /* RTE_COMPONENTS_H */

Pre_Include_Global.h

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.3.0
 *
 * Project: 'app6_SW.debug+main' 
 * Target:  'debug+main' 
 */

#ifndef PRE_INCLUDE_GLOBAL_H
#define PRE_INCLUDE_GLOBAL_H

/* STMicro::Device:generated:compA@0.0.1 */
#define PRE_INCLUDE_GLOBAL_COMPA_FROM_PDSC


#endif /* PRE_INCLUDE_GLOBAL_H */

Pre_Include_Local_compId.h

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.3.0
 *
 * Project: 'app6_SW.debug+main' 
 * Target:  'debug+main' 
 */

#ifndef PRE_INCLUDE_DEVICE_GENERATED_COMPA_H
#define PRE_INCLUDE_DEVICE_GENERATED_COMPA_H

/* STMicro::Device:generated:compA@0.0.1 */
#define PRE_INCLUDE_LOCAL_COMPA_FROM_PDSC

#endif /* PRE_INCLUDE_DEVICE_GENERATED_COMPA_H */

Observed behavior:

GPDSC definition of the same component is not overriding the PDSC component element.

Extra tests:

When updating the pack vendor name of the GPDSC template, or the component cgroup like displayed in the snapshot:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- ******************************************************************************
 * File Name   : GPDSC_template.gpdsc
 * Description : Generator PDSC File generated by CodeGen (DO NOT EDIT!)
 ****************************************************************************** -->

<package xmlns:xs="http://www.w3.org/2001/XMLSchema-instance" schemaVersion="1.0"
    xs:noNamespaceSchemaLocation="https://github.com/Open-CMSIS-Pack/Open-CMSIS-Pack-Spec/blob/main/schema/PACK.xsd">
    <vendor>STMicro</vendor>
    <name>packA</name>
    <description>Pack A description</description>
    <url>path</url>
    <releases>
        <release version="0.0.1">
            {{lowercase "- First GPDSC template Model Release"}}
        </release>
    </releases>
    <conditions>
        <condition id="conditionC">
            <description>condition C</description>
            <require Cclass="fake" Cgroup="generated" Csub="compF" />
        </condition>
        <condition id="filecondition1">
            <description>Condition 1</description>
            <require Cclass="fake" Cgroup="generated" Csub="compB" />
        </condition>
        <condition id="filecondition2">
            <description>Condition 2</description>
            <require Cclass="fake" Cgroup="generated" Csub="compC" />
        </condition>
    </conditions>
    <generators>
        <generator id="GEN-1">
            <description>GEN1 generator</description>
            <exe>
                <command host="win">gen.bat</command>
                <command host="linux">gen.sh</command>
                <!-- Generator Id -->
                <argument>GEN-1</argument>
                <!-- Generator Input File absolute path -->
                <argument>$G</argument>
                <!-- GPDSC Template File Name -->
                <argument>template_model.gpdsc.hbs</argument>
                <!-- Flag to indicate that this Generator is dry-run capable  -->
                <argument mode="dry-run">--dry-run</argument>
            </exe>
            <gpdsc name="$P/template_model.gpdsc" />
        </generator>
    </generators>
    {{!-- SELECTED COMPONENTS LIST --}}

    <components>
    	<component Cclass="Device" Cgroup="generated-updated" Csub="compA" Cversion="1.0.0" generator="GEN-1">
            <description>TO BE DEFINED</description>
            <RTE_Components_h>
                <!-- the following content goes into file 'RTE_Components.h' -->
                #define USING_CompA_GPDSC
            </RTE_Components_h>
            <Pre_Include_Global_h>
                <!-- the following content goes into file 'Pre_Include_Global.h' -->
                #define PRE_INCLUDE_GLOBAL_compA_GPDSC
            </Pre_Include_Global_h>
            <Pre_Include_Local_Component_h>
                <!-- the following content goes into file 'Pre_Include_Local_<ComponentId>.h' -->
                #define PRE_INCLUDE_LOCAL_compA_GPDSC
            </Pre_Include_Local_Component_h>
            <files>
                <file category="source" name="stm32_compA.c" />
                <file category="header" name="stm32_compA.h" />
            </files>
        </component>
    </components>    
</package>

the new updated component description is taken into consideration
this is what we get in the RTE_Components.h file

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.3.0
 *
 * Project: 'app6_SW.debug+main' 
 * Target:  'debug+main' 
 */

#ifndef RTE_COMPONENTS_H
#define RTE_COMPONENTS_H


/*
 * Define the Device Header File: 
 */
#define CMSIS_device_header "stm32u5xx.h"

/* STMicro::Device:generated-updated:compA@1.0.0 */
#define USING_CompA_GPDSC
/* STMicro::Device:generated:compA@0.0.1 */
#define _RTE_COMPA_FROM_PDSC


#endif /* RTE_COMPONENTS_H */

Main Question

• For the same component Exact PDSC / GPDSC definition, what are the rules applied ?
• Should the GPDSC override the PDSC definition ?
• Is there any specific handling in the toolbox for the GPDSC components definition ? (Ex: components with higher versions in the GPDSC should override the PDSC ones ?)

[jkrech]

I reproduced a similar scenario in MDK uVision 5.40 and do see the expected behavior, that the Cversion attribute of the component definition in the GPDSC file needs to be larger or equal to the component version in the PDSC file to override it.
This includes all aspects of the component including RTE_Components.h, Global and Local Pre-Include definitions.

I converted the uvprojx to csolution and repeated the test and I can see the defines from GPDSC if the component defined in the gpdsc is greater or equal to the Cversion of the component specified in the PDSC file. Otherwise I see the defines coming from the PDSC
I used CMSIS-Toolbox 2.4.0 and 2.3.0 but was unable to reproduce the behavior you are describing above.

[BadreddineBJA]

Thanks Joachim for the tests, I will re-check the tests following your described scenario about the component version in the GPDSC that should be greater or equal to the PDSC version and confirm the behavior.
i used the csolution "v2.3.0" from the toolbox on "windows" for previous described scenarios,

[BadreddineBJA]

• Tests re-applied using the toolbox v2.4.0.
• Mocked packs and project example used for test: shared in this private link
• Test results:

Following the above recommendations from @jkrech (The Cversion attribute of the component definition in the GPDSC file needs to be larger or equal to the component version in the PDSC file to override it), at the convert step, i can see in the RTE folder that the defines from GPDSC are taken into consideration if the component defined in the gpdsc is greater or equal to the Cversion of the component specified in the PDSC file. Otherwise I see the defines coming from the PDSC.

. I think that this behavior should be documented in the GPDSC spec

For the rest of the applied tests, the GPDSC template manual editing is very error prone since it can cause GPDSC .xsd schema inconsistancies which seems implicitly not detected by csolution (maybe warnings messages to the end user should be added to ease the debug step)

[BadreddineBJA]

Hi @jkrech,

Thank you for your feedback
Regarding your question, Do you make use of config files in your component descriptions for the generator component and the overwrite in the generated gpdsc?

• No, currently In the applied tests, I am not using the attr=”config” in the gpdsc template.

Following our discussion about the strange behaviors observed and described in this ticket, I re-adjusted the GPDSC template following your recommendation:

• 1. I added the same node in the GPDSC template.
• 2. I Checked for any open / malformed tag in the GPDSC (applied a packchk on the gpdsc .xsd schema).
• 3. I verified if the GPDSC component used version is greater or equal to the PDSC component version.

After following this scenario:

1. Compose a project using 2 components: compA, compB .
2. Generate a GPDSC file with the same PDSC definition of this components (Cclass, Cgroup, Cversion,..) (Same exact version).
3. Apply a conversion step: ‘csolution convert -s .csolution.yml)

Observed results:

• Am observing the expected behavior (the GPDSC components definitions with their associated tags (RTE_components.h, Pre_Include_Global, Pre_Include_local) are winning (overriding) the PDSC component definitions only if they have the version greater or equal to the PDSC declared one.
  NB: tests where not applied on files coming from the GPDSC

• Based on the applied tests, csolution seems to take into consideration the GPDSC file for component override actions.

• The major challenge that remains and that we are discovering with trials and errors on the GPDSC is that missing attributes in the GPDSC (like the generators tag) or a malformed GPDSC template could cause the tools to crash.

• Maybe we should enhance the GPDSC Spec by providing a small guideline to describe the list of (PDSC elements) required to exist in the GPDSC template and what is optional:

this would confirm our assumption that every PDSC element described in the spec (https://open-cmsis-pack.github.io/[Open-CMSIS-Pack](https://open-cmsis-pack.github.io/Open-CMSIS-Pack-Spec/main/html/index.html)-Spec/main/html/index.html) is supported via the GPDSC.