Error processing file 'Marktrollen.xml': 'Index(['MastrNummer'], dtype='object')'

[terminator-997]

I am encountering a reproducible error while running a bulk download using the latest released version of the open-mastr (0.16.1).
The error occurs during parsing of the Marktrollen.xml file and causes the entire bulk ETL to fail.
This is happening for todays run (20 Jan 2026)

ERROR Message:
[ERROR] Error processing file 'Marktrollen.xml':
KeyError: Index(['MastrNummer'], dtype='object')

Any suggestion on how can I tackle this?

[pt-kkraemer]

I reproduced your error:

2026-01-20 15:19:46,012 [INFO] Processing file 'Marktakteure_1.xml'...
2026-01-20 15:19:46,013 [INFO] Creating table 'market_actors'...
2026-01-20 15:20:19,519 [INFO] Processing file 'Marktrollen.xml'...
2026-01-20 15:20:19,520 [INFO] Creating table 'market_roles'...
2026-01-20 15:20:19,706 [INFO] From the downloaded xml files following new attribute was introduced: market_roles.Wert
2026-01-20 15:20:19,716 [INFO] From the downloaded xml files following new attribute was introduced: market_roles.Id
2026-01-20 15:20:19,753 [ERROR] Error processing file 'Marktrollen.xml': 'Index(['MastrNummer'], dtype='object')'
2026-01-20 15:20:19,757 [INFO] Processing file 'Marktakteure_2.xml'...

[pt-kkraemer]

It also happens with the file from 01.01.2026:

[pt-kkraemer]

We create the market_roles table as follows:

class MarketRoles(ParentAllTables, Base):
    __tablename__ = "market_roles"

    MastrNummer = Column(String, primary_key=True)
    MarktakteurMastrNummer = Column(String)
    Marktrolle = Column(String)
    Marktpartneridentifikationsnummer_nv = Column(Boolean)
    BundesnetzagenturBetriebsnummer = Column(String)
    BundesnetzagenturBetriebsnummer_nv = Column(Boolean)
    Marktpartneridentifikationsnummer = Column(String)
    KontaktdatenMarktrolle = Column(String)
    DatumLetzteAktualisierung = Column(DateTime(timezone=True))

The Marktrollen.xml looks like this:

<Marktrollen>
    <Marktrolle>
    <Id>1</Id>
    <Wert>Übertragungsnetzbetreiber</Wert>
    </Marktrolle>
    <Marktrolle>
    <Id>2</Id>
    <Wert>Anschlussnetzbetreiber</Wert>
    </Marktrolle>
...

So none of the columns we expect are actually in the downloaded .xml.

@FlorianK13 @nesnoj @Simon-Will this would be fixed when we use the .xsd files to create the tables. Do we want to provide a hotfix until we implement that?

[pt-kkraemer]

@terminator-997 does it actually stop the whole bulk download? In my case it just parsed the next .xml file.

[FlorianK13]

I'm definetly against providing a hotfix here to save us some time.

[terminator-997]

After further investigation, the container job failure I mentioned earlier was due to an issue in my container setup, not caused by open-mastr.
Although an error occurs while processing this particular XML file, it does not stop the bulk download or overall processing — the job completes successfully.
Sorry for the confusion, and thanks for the support.

[pt-kkraemer]

Thanks for the clarification @terminator-997! :)

As we will solve this problem in the future with another pull request I close this issue now.