CLI: `--dnp-field kicad_dnp` does not work as expected

[schlimmchen]

Note that this is a possible duplicate of #519.

I noticed that parts marked as "do not place" on the board are not recognized as such in the iBOM. This means they are not outlined when the respective option is enabled and they do appear in the table.

Please have a look at this dummy project, which includes iBOMs that I expect to be the same, but they are not: 2025-11-10_kicad_ibom_dnp.zip

Setup:

• KiCad debian native package 9.0.2+dfsg-1 from and on trixie.
• Clone of this repo with the following diff:

diff --git a/InteractiveHtmlBom/core/config.py b/InteractiveHtmlBom/core/config.py
index b39d0b6..94105c6 100644
--- a/InteractiveHtmlBom/core/config.py
+++ b/InteractiveHtmlBom/core/config.py
@@ -257,6 +257,7 @@ class Config:
         self.board_variant_blacklist = list(
             dlg.fields.boardVariantBlacklist.GetCheckedStrings())
         self.dnp_field = dlg.fields.dnpFieldBox.Value
+        print(f"DNP field from dialog: '{self.dnp_field:s}'")
         if self.dnp_field == dlg.fields.NONE_STRING:
             self.dnp_field = ''
 
@@ -443,7 +444,7 @@ class Config:
         self.show_silkscreen = not args.hide_silkscreen
         self.highlight_pin1 = args.highlight_pin1
         self.redraw_on_drag = not args.no_redraw_on_drag
-        self.board_rotation = math.fmod(args.board_rotation // 5, 37)
+        self.board_rotation = int(math.fmod(args.board_rotation // 5, 37))
         self.offset_back_rotation = args.offset_back_rotation
         self.checkboxes = args.checkboxes
         self.bom_view = args.bom_view
@@ -475,6 +476,7 @@ class Config:
         self.board_variant_whitelist = self._split(args.variants_whitelist)
         self.board_variant_blacklist = self._split(args.variants_blacklist)
         self.dnp_field = args.dnp_field
+        print(f"DNP field from args: '{self.dnp_field:s}'")
 
     def get_html_config(self):
         import json

• Python virtual environment: python3 -m venv --system-site-packages <venvpath>
• Then source <venvpath>/bin/activate; pip install .

I can now run the iBOM generator and observe the following, while setting up to exclude "do not populate" parts:

1. Parts excluded from the BOM as per checkbox in KiCad are indeed excluded, no matter their "do not populate" checkbox state.
2. Parts with the "do not populate" checkbox are
   • excluded from the BOM when using the GUI: python3 -m InteractiveHtmlBom.generate_interactive_bom --show-dialog /home/schlimmchen/Documents/2025-11-10_kicad_ibom_dnp/kicad_ibom_dnp.kicad_pcb
   • still included when using --dnp-field kicad_dnp: python3 -m InteractiveHtmlBom.generate_interactive_bom --dark-mode --highlight-pin1 selected --bom-view left-right --layer-view F --dest-dir /home/schlimmchen/Documents/2025-11-10_kicad_ibom_dnp/bom/ --name-format ibom_cli --group-fields Value,Footprint --dnp-field kicad_dnp --no-browser --show-fields Value,Footprint,Reference --normalize-field-case /home/schlimmchen/Documents/2025-11-10_kicad_ibom_dnp/kicad_ibom_dnp.kicad_pcb --board-rotation 0 --variant-field ''

The print statements show me that the DNP field is "kicad_dnp" in both cases.

...
DNP field from dialog: 'kicad_dnp'
...
DNP field from args: 'kicad_dnp'
...

I also see how the code works and that ibom.main does the heavy lifting in both cases, so this is really puzzling to me.

Note that the GUI iBOM has the "Reference" (not "References") column populated, but the CLI version does not. Is that a hint to something or a different issue?

More digging using printf-debugging reveals that the component's extra_fields does not have kicad_dnp:

diff --git a/InteractiveHtmlBom/core/ibom.py b/InteractiveHtmlBom/core/ibom.py
index 178467c..d64345b 100644
--- a/InteractiveHtmlBom/core/ibom.py
+++ b/InteractiveHtmlBom/core/ibom.py
@@ -67,6 +67,15 @@ def skip_component(m, config):
         return True
 
     # skip components with dnp field not empty
+    if config.dnp_field:
+        print(f"{m.ref:s} config.dnp_field evaluates to boolean True")
+        if config.dnp_field in m.extra_fields:
+            print(f"{m.ref:s} dnp_field is in extra_fields")
+            if m.extra_fields[config.dnp_field]:
+                print(f"{m.ref:s} m.extra_fields[config.dnp_field] evaluates to boolean True: {m.extra_fields[config.dnp_field]:s}")
+        else:
+            print(f"{m.ref:s} dnp_field NOT available in extra_fields")
+
     if config.dnp_field \
             and config.dnp_field in m.extra_fields \
             and m.extra_fields[config.dnp_field]:
@@ -328,6 +337,7 @@ def main(parser, config, logger):
     if not pcbdata and not components:
         raise ParsingException('Parsing failed.')
 
+    print(f"DNP field in main is '{config.dnp_field}'")
     pcbdata["bom"] = generate_bom(components, config)
     pcbdata["ibom_version"] = config.version

2025-11-10 22:42:25,279 WARNING Ignoring extra fields related config parameters since no netlist/xml file was specified.
DNP field in main is 'kicad_dnp'
R3 config.dnp_field evaluates to boolean True
R3 dnp_field NOT available in extra_fields
R4 config.dnp_field evaluates to boolean True
R4 dnp_field NOT available in extra_fields
R2 config.dnp_field evaluates to boolean True
R2 dnp_field NOT available in extra_fields
R1 config.dnp_field evaluates to boolean True
R1 dnp_field NOT available in extra_fields
2025-11-10 22:42:25,332 INFO Compressing pcb data
2025-11-10 22:42:25,344 INFO Dumping pcb data
2025-11-10 22:42:25,346 INFO Created file /home/schlimmchen/Documents/2025-11-10_kicad_ibom_dnp/bom/ibom_cli.html

Hm: "WARNING Ignoring extra fields related config parameters since no netlist/xml file was specified." Why is that required when using the CLI but not when using the dialog?

[schlimmchen]

So the CLI needs --extra-data-file <path to .kicad_pcb>...

Have not seen this yet as that's not part of the ini-file, which would have told me that difference.

New question: Why isn't the default for --extra-data-file the positional argument (which seems to be the case in the GUI).

I have a feeling that

        config.netlist_initial_directory = os.path.dirname(parser.file_name)
        extra_data_file = parser.latest_extra_data(
            extra_dirs=[config.bom_dest_dir])

should move from ibom.run_with_dialog() to generate_interactive_bom.main() (and then extra_data_file needs to be transported into the config).

[qu1ck]

Why is that required when using the CLI but not when using the dialog?

It is required in GUI too, without specifying the file it won't even show the fields. As you found out, GUI automaticlly picks latest valid extra data source for convenience.

Historically kicad used to not have the exta field information in the pcb file so you had to do the manual step of exporting netlist/xml file from eeschema to be able to use them in ibom. Nowadays that option is there for backward compatibility and for rare cases where you want to programmatically generate pcb variants and feed that data into ibom cli.

I intended to make cli do the same thing GUI does and assume extra data source is the pcb file unless specified otherwise but haven't gotten around to it. You seem to know your way around python, PRs are welcome.

[qu1ck]

should move from ibom.run_with_dialog() to generate_interactive_bom.main() (and then extra_data_file needs to be transported into the config).

No, the latest_extra_data will find any valid extra data source with latest modification time, it's usually the pcb file but not necessarily. In CLI behavior should be explicit, but defaulting to the pcb file is fine.

It shouldn't be in config because config is (usually) global while extra data file is project specific.