Skip to content

Batteries are marked as broken when trying to charge beyond the bounds #1321

@llucax

Description

@llucax

What happened?

I've been seeing errors with failed preconditions that end up marking batteries as non-working. But the error is actually a bounds error (trying to charge beyond the current bounds), so the batteries are actually working. Marking them as broken might not be the best option, as we might lose battery capacity unnecessarily.

2025-12-08T05:25:36+0000 WARNING frequenz.sdk.microgrid._power_distributing._component_managers._battery_manager:696: Set power for battery frozenset({ComponentId(175)}) failed, mark it as broken. Error: Failed calling 'SetPowerActive' on 'grpc://[::0]:61060': The operation was rejected because the system is not in a required state <status=FAILED_PRECONDITION>: { "kind": "InvalidState", "message": "Controller API failed to run command", "source": { "kind": "InvalidState", "message": "Cannot charge when current SoC 95.9% >= upper bound 95%", "source": "" } } (UNKNOWN:Error received from peer  {grpc_status:9, grpc_message:"{ \"kind\": \"InvalidState\", \"message\": \"Controller API failed to run command\", \"source\": { \"kind\": \"InvalidState\", \"message\": \"Cannot charge when current SoC 95.9% >= upper bound 95%\", \"source\": \"\" } }"})
2025-12-08T06:17:53+0000 WARNING frequenz.sdk.microgrid._power_distributing._component_managers._battery_manager:696: Set power for battery frozenset({ComponentId(178)}) failed, mark it as broken. Error: Failed calling 'SetPowerActive' on 'grpc://[::0]:61060': The operation was rejected because the system is not in a required state <status=FAILED_PRECONDITION>: { "kind": "InvalidState", "message": "Controller API failed to run command", "source": { "kind": "InvalidState", "message": "Power 5806W (AC power 5806W - self consumption 0W) outside battery's DC power bounds: [Bounds { lower: Some(-140000.0), upper: Some(4764.328) }]", "source": "" } } (UNKNOWN:Error received from peer  {grpc_status:9, grpc_message:"{ \"kind\": \"InvalidState\", \"message\": \"Controller API failed to run command\", \"source\": { \"kind\": \"InvalidState\", \"message\": \"Power 5806W (AC power 5806W - self consumption 0W) outside battery\'s DC power bounds: [Bounds { lower: Some(-140000.0), upper: Some(4764.328) }]\", \"source\": \"\" } }"})

What did you expect instead?

Ideally I would expect that:

  1. The battery is not marked as broken the pool keeps using it
  2. Bounds are updated in the tracker to the new actual bounds, but given the current microgrid API exposes the actual bounds as a string, it might be difficult

Affected version(s)

No response

Affected part(s)

Data pipeline (part:data-pipeline)

Extra information

I also wonder if this is happening because of some race, and the reported bounds via streaming data are too old by the time we issue the set power command, or if there is a bug somewhere, either in the microgrid API sending wrong data, or the battery pool not tracking the bounds properly.

/cc @cyiallou @tiyash-basu-frequenz @shsms

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority:❓We need to figure out how soon this should be addressedtype:bugSomething isn't working

    Type

    Projects

    Status

    To do

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions