Request for Beginner Support

[jaroslawsky]

Hello,

Would it be possible to give me some tutoring to get the examples here up and running? I seem to be missing the overview. It’s not a problem for me to start the devices-app. I can also start the remote module with appropriate entries in the eebusConfiguration in its main function. I understand this module to be the central eebus. Is that correct? The modules hems and controlbox from the examples directory can also be started in principle and are displayed in the web interface of the devices-app, if not using the SKI of the eebus module.

And here are my questions: With what values should the parameters serverport, remoteski, crtfile, and keyfile be filled? serverport is apparently not the port of the eebus, but each ship node gets its own port? And what needs to be specified for remoteski? If I use the eebus (remote) SKI, I get the error message: "The remote service denied trust. Exiting." Previously, I had started the individual modules without parameters and made corresponding files from the generated cert and key data. I then specify their respective paths when calling each module. Is that correct?

Actually, I'm not surprised by the error message mentioned. How is a ship node supposed to know that it should trust the other one if the certificates were created completely independently of each other? I seem to have a misunderstanding here.

It would be nice if someone could help me out.

Best regards,
Klaus

[sthelen-enqs]

Hi,

it really depends on what you're trying to do. The examples provide a short overview of how to use the eebus-go library for various use cases such as for writing/integrating with a home energy system or creating a controlbox etc.

I think the better approach would be to look at what you're trying to accomplish and then guiding you on the right direction from there. Are you trying to connect to a device in your house to control it or retrieve data, or are you interested in integrating EEBus support into an existing or new device, or ...?

[jaroslawsky]

Hello,

Thank you very much for the extremely quick response. Yes, you are certainly right. I should describe what I intend to do. I operate a self-written EMS with which I control and monitor all the consumption in the house and a wallbox. I would like to expand this with an EEBUS interface so that it can communicate with a control box from our network operator in the future, allowing individual consumers to be dimmed by the network operator when needed. Due to a recent change in legislation (§14a EnWG), this should significantly reduce our network fees as part of the electricity costs.

To this end, I first wanted to create a kind of test setup to learn the details of EEBUS. Afterwards, my goal would be to write an interactive control box simulator and have my EMS communicate with it. The examples "hems" and "controlbox" seemed like a good foundation for this. Subsequently, I got the impression that the "remote" module is required as the central EEBUS ship node for their communication. And the devices-app seemed like an ideal way to control such a simple test system.

I have worked my way into the Go language to the point where debugging the various code parts in Visual Studio Code and thereby gaining deeper understanding is also not a problem. But, as I said, I still lack an understanding of the big picture.

Best regards,
Klaus

[sthelen-enqs]

First off, that should be possible to implement using the eebus-go stack so in that sense you're working in the correct direction. In such a system your EMS would act similarly to the examples/hems system by accepting LimitationOfPowerConsumption commands (using the usecase in github.com/enbility/eebus-go/usecases/cs/lpc) and then ensuring those limitations are applied to your wallbox (and other §14a systems in your house). In production your network operator would install a component similar to the examples/controlbox in your house and use that to send LPC commands to your EMS to "dim" your consumption.

Now to try and clear up some misconceptions, the "remote" module is not required in any of these scenarios as long as your EMS is written in Go or you're using/expanding on the above mentioned examples. The remote module exists to allow EMS not written in Go to use the EEBus functionality provided by the eebus-go stack by exposing the API of eebus-go via JSON-RPC. While you're creating a simple test setup and getting to know the eebus-go stack I would recommend sticking with expanding the hems and controlbox examples, and then once you're more comfortable with the stack you can look into trying to connect your existing EMS with the remote (assuming your EMS is not written in Go) so that you can then migrate the functionality you need from the hems example to your EMS. If you have questions regarding how to integrate with the remote, we can have a chat when you get to that point.

Regarding the devices-app, as far as I'm aware that just provides an overview of the available EEBus devices in the local network, and which EEBus usecases they support. This is mainly interesting if you're looking to e.g. integrate a device with your EMS and you want to know which usecases it supports etc.

Regarding the controlbox and hems examples and how to use them. In the default configuration most of the examples take 4 arguments:

1. the TCP Port on which the example should listen. This can be chosen freely as long as it's reachable by all other EEBus devices you want to connect with each other (firewall etc). The ports can be different for every device (e.g. you're running multiple examples on the same computer for testing)
2. The Remote SKI which this EEBus device will trust. EEBus devices will only connect to other EEBus devices if this device trusts the SKI of the other device, and the other device trusts the SKI of this device (mutual authentication). The SKI is an attribute of the TLS Private Key (sha1 hash of the public key) and is therefore often used in EEBus when establishing trust as it's shorter than sending the full TLS Certificate (albeit arguably less secure).
3. The TLS Certificate for this device.
4. The TLS Private key for this device.

I hope this answers most of your questions, but feel free to ask if something is still unclear.

[jaroslawsky]

Hello Simon,

Once again, thank you very much for your detailed response. That will definitely help me. So, I had misunderstood the role of the remote module. But I had already come to realize that hems and controlbox need to be paired. I see that there is a heartbeat every second. A good sign.

What I am now wondering is how it will work with a control box from the network operator. Is it ensured that I can then communicate the SKI of my EMS (or that of the remote module as an interface to my EMS written in C#) to that control box and that this box will also trust my EMS? My network operator will hardly provide me with such a control box just for testing.

When I look into the SHIP specifications, I find various verification modes in chapter 12.3. Will a hardware control box support all these modes? Including the one where you only pass the SKI of the own EMS? And most importantly, will such a control box be satisfied with my self-generated certificate? After all, this certificate will not be recognized as trustworthy by any known CA.

Best regards,
Klaus

[sthelen-enqs]

Hi,

Pairing will depend on your specific network operator, but it's generally expected that they will (somehow) provide you with the SKI of their controlbox and you will have to somehow provide them with the SKI of your EMS. Bilateral trust for an initial pairing will then be derived from both sides verifying the other side's SKI and then persisting some combination of the SKI, the public key/certificate and the ShipId.

According to SHIP 12.1.1 interoperability should not depend on using a certain PKI, self-signed certificates are equally valid and I can't see any reason for a network operator to require certificates to be signed by any particular PKI.

[DerAndereAndi]

PKI is not used in EEBUS. The SteuVE or EMS needs to provide the SKI and the SHIP ID. This maybe provided by scanning a QR code. The stack has an API for that.
The SKI and SHIP ID of the EMS may not change, otherwise the control box will reject connections.

[jaroslawsky]

That sounds really good. I hope it will turn out that way. It would of course be nice to get information about their control boxes from the network operator (here: Bayernwerk-Netz GmbH). However, they themselves do not seem to know which ones they might start delivering as early as April. I will get to work and will surely get back to you. But it will take a while. Until then, many thanks for the competent information.

Best regards,
Klaus

[jaroslawsky]

Hi,

Since I'm lying in bed with a cold and therefore have a lot of time, I already have something to show.

A really stupid question: is there maybe a possibility to commit the source code to your github repository in a separate branch? It would be really nice if someone could take a look at it to see if it's going in the right direction. If that's the case, I would be happy if you could include the result in your development. In other words, I would like to give you the code as a gift.

What already works quite well is the communication of the frontend with the controlbox module. This forwards the data to the hems module and back to the frontend. Fetching the nominal maxima is not working yet. However, this seems to be due to the existing example code.

I've also found and fixed a few small errors in the examples.

The next steps would include incorporating the remaining use cases and adding a frontend to the hems example as well, so that it becomes an interactive EMS dummy. At least, those are my thoughts.

Best regards,
Klaus

[DerAndereAndi]

This looks good :)

I would love to get a Pull Request with the fixes for the small errors in the examples code. But I do think that the simulator should go into its own repository. The examples should be kept simple and small to provide an idea on how to use the stack. But your simulator is much more than that: Also others might assume that for every possible device there should be such an example then, which isn't feasible ;) I hope this makes sense.

[jaroslawsky]

Well,

where something is checked in is currently also quite unimportant to me. The main thing is that you can take a look at my stuff and evaluate it. So I created a fork on GitHub (https://github.com/vollautomat/eebus-go). In the dev branch, you can see two small bug fixes. Nothing groundbreaking. And in the simulators branch, I put everything I've written so far. The commit comment says the most important things. I'm looking forward to your opinion, comments, and criticism...

You wrote that it might create the expectation that simulators are available for all possible devices. Quite honestly, I consider that to be very sensible. At the moment, software in this area can practically only be developed if you have the appropriate hardware available on the lab table or at home. This significantly limits the number of people who can contribute to the energy transition.

Imagine some open source hardware projects, wall boxes, inverters, etc., also offering suitable simulators. These products would have a crystal-clear competitive advantage. The interoperability with other components could be tested before investing in such a product. People could develop and try out new ideas just with their notebook on their lap.

If you now say that such simulators would never be good enough to achieve this, I have to disagree. I think I can claim to have some practical experience in this area. This includes the realization that simulators are even better than reality! With them, critical scenarios can be tested much more easily than with real hardware. And even automatically. My opinion is that simulators are an absolute must for software development.

I would be happy if we could have a dialogue about this soon. Preferably, to be honest, in German and not via a web forum.

Best regards,
Klaus

[DerAndereAndi]

It looks like there is a misunderstanding:

I meant that this repository is not meant to have simulators or full blown implementations for all kind of devices. This repository is about providing an EEBUS stack/framework implementation, so that everyone can create their own tools/systems/simulators. But these should not be part of this repository. Separation between stack/API and applications.

Feel free to join the Slack linked on the contact page of enbility.net.

[jaroslawsky]

No, no, you're absolutely right if you want to keep the existing repository reserved for simple examples. That's fine.

Unfortunately, I made the mistake of not including new files when committing yesterday. Now I have made some changes and this time (hopefully) checked in everything.

I am trying to set up the simulator in such a way that it can later be used with other devices besides an EMS.

[meisel2000]

Hi Klaus,
I'm very interested in what you do.
I would like to test my installation of evcc before installing of hardware control box. The connection between evcc and the example control box works. I'm currently not able to set limits.
Maybe you can share your code in a separate repo.

[meisel2000]

@jaroslawsky : Vielen Dank für deinen Simulator-Code. Leider habe ich keine Erfahrung mit Websockets. Kannst du in deinen Branch ggf. auch den Code für den Client veröffentlichen? Oder habe ich da etwas übersehen?

[jaroslawsky]

@meisel2000: Ich hatte ja schon geschrieben, dass ich beim Einchecken einen Fehler gemacht hatte, das aber nachgeholt hab. Allerdings musst Du in den simulators Branch wechseln. Unter examples/controlbox ist jetzt auch der Client Code zu finden.

[meisel2000]

@jaroslawsky Ich habe es gefunden und schaue mir das am Wochenende mal im Detail an.

[jaroslawsky]

Das ganze ist noch bei weitem nicht fertig. Auch das bisherige ist noch nicht in sich konsistent. Bevor ich da aber noch mehr Zeit reinstecke, wäre es schön von kompetenter Seite zu hören, ob diese Ansätze in die richtige Richtung gehen. Ideen hätte ich noch viele und wenn die EEBUS Dokumentation besser verständlich wäre, könnte ich einfach damit arbeiten.

[malcolmx67]

I executed the first call to create the certificate, key and SKI. I then copied the certificate and the key to separate files and called it once more:

root@steuerbox:/opt/eebus-go-simulators/examples/controlbox# go run ./main.go 8888 <generated SKI> my.cert my.key

What am I supposed to do next ? When trying to access https://:8888 I get the following error:

2025-02-06 15:11:36 INFO  Local SKI: <SKI>
2025/02/06 15:12:56 http: TLS handshake error from 10.x.x.x:57804: remote error: tls: unknown certificate
2025/02/06 15:13:00 http: TLS handshake error from 10.x.x.x:57809: remote error: tls: unknown certificate
2025/02/06 15:13:00 http: TLS handshake error from 10.x.x.x:57810: EOF
2025/02/06 15:13:06 http: TLS handshake error from 10.x.x.x:57802: read tcp 10.x.x.x:8888->10.x.x.x:57802: i/o timeout
2025/02/06 15:13:31 http: TLS handshake error from 10.x.x.x:57816: tls: client didn't provide a certificate
2025/02/06 15:20:21 http: TLS handshake error from 10.x.x.x:61942: EOF
2025/02/06 15:20:24 http: TLS handshake error from 10.x.x.x:61929: read tcp 10.x.x.x:8888->10.x.x.x:61929: i/o timeout

resp. error on the client side: ERR_BAD_SSL-CLIENT_AUTH_CERT

COuld you please advise on how to get this simulator working ? Thanks in advance !!

[malcolmx67]

@DerAndereAndi you are absolutely right, I should have used a discussion in the repo of @jaroslawsky
Maybe I can ask another question directly related to evcc:
what exactly is evcc's shipid meant to be ? Obviously it's not it's own SKI. Do I have to add the simulator's SKI as shipid in evcc.yaml ?
Thanks to both of you for this great support!

[malcolmx67]

@jaroslawsky could you please open discussions for your repo ?

[DerAndereAndi]

@malcolmx67 the SHIP ID is the id value of the mDNS txt record of the evcc service.

[malcolmx67]

Thx, and how can I configure the simulator's SKI in evcc ?

[jaroslawsky]

@malcolmx67 discussions are now enabled in the simulator repo.
Diskussionen dort dürfen auch gerne auf Deutsch geführt werden. Dieses Repo ist eh nur als temporäres gedacht, um die ersten Schritte zu programmieren.

[jaroslawsky]

Hi @DerAndereAndi, @sthelen-enqs,

I have been working with your EEBUS APIs for a few weeks now and have made some progress. The controlbox simulator now independently detects available EEBUS devices and connects to them after selection, without needing to provide an SKI value at startup. The evcc GUI displays the current limiting state, making the response to the controlbox simulator commands clearly visible. So, the initial steps look quite promising.

However, I have some fundamental questions about EEBUS and your specific implementation.

For the LPC use case, the interface ucapi.EgLPCInterface includes the methods ConsumptionLimit and ConsumptionNominalMax. When I call these for a connected peer, they do not seem to reach the peer at all. ConsumptionNominalMax always returns 0, and ConsumptionLimit appears to return some cached values and an automatic countdown of the remaining duration. This seems to me to be a fundamental design flaw. In my understanding, these values should be queried from the peer. This issue becomes particularly noticeable when you connect the controlbox simulator to 2 peers, namely evcc and the hems example. Switching between both, one would naturally expect to see the current states given in each device. Any caching or calculating of a duration makes no sense here.

Could you please give me some tips on this matter?

If it is due to the implementation of your APIs still being incomplete, I would be happy to help change that. However, I would need a starting point from you. A personal conversation would probably be the most efficient way here.

Overall, I fear that with the current state of the APIs, a truly functional system cannot be built. By functional, I mean a system that works reliably with a real control box of a network operator, ensuring that this network operator will definitely not cause any trouble.

Best regards from Bavarian Forrest,
Klaus

PS: I'm glad that my small bug fixes in the controlbox and hems examples have already been merged into the master. I have also found a bug in evcc, which I should communicate there. The failsafe state is only exited after a heartbeat is received AND the time has expired. Of course, this has to be a || condition instead of a && condition. :-)

[sthelen-enqs]

Hi,

for ConsumptionLimit as well as ConsumptionNominalMax you pass as the singular
argument the EntityRemoteInterface of the peer for which you want the value.
Since the spine logic is based on subscriptions and notifications, calling
either of those functions may not cause any network traffic because the current
values are already cached locally. When the remote entity updates its limit it
will send a notification to all subscribed devices which should then trigger an
e.g. DataUpdateLimit event containing the ski, device and entity for which the
ConsumptionLimit changed. You can then call ConsumptionLimit with that entity
and receive the updated cached value. If this doesn't work correctly in some scenario,
please do create an issue (preferably including a way to reproduce it).

ConsumptionNominalMax is not a mandatory scenario of the LPC use case (see e.g.
#157) for the tracking issue, if the
controllable service doesn't implement this scenario this will always currently
return (0, api.ErrDataNotAvailable).

[jaroslawsky]

Hello @sthelen-enqs ,

First of all, thank you for the quick explanation. At least it shows me that with my limited understanding of the EEBUS mechanisms, but based on the observations I have made, I have correctly concluded that caching is involved here.

However, your argument that a ConsumptionLimit call will not cause a network action because EEBUS is based on subscriptions and notifications does not convince me. Okay, it is apparently implemented that way - clear. But does it have to be like that? And shouldn't it rather lead to faulty behavior?

A WriteConsumptionLimit call also causes a direct action on the other side. Subscriptions and notifications do not stand in the way of this. Thus, a ConsumptionLimit call could also ensure that initially cached values are returned, but subsequently the other side sends a DataUpdateLimit event with the current actual values.

Especially with these limit values, caching makes no sense at all. Such a limit consists not only of a power value, but also of an active flag and a duration. Counting down this duration inside the cache, even when the limit is not active and without knowing whether there is currently a failsafe status on the other side or whether a desired limit has been activated, shows the futility of this undertaking. Caching can only make sense in a one-to-one relationship between the EEBUS actors without any possibility of status change from the outside. But precisely with the consumption limit, this happens when the heartbeat is disturbed.

I am writing this without knowing whether the current implementation of your APIs is as it is because the EEBUS specifications dictate it or whether it is your interpretation of the specifications. I don't know - the specs are unfortunately very difficult to read and, due to the lack of suitable sequence diagrams, not only hard to understand but also very easy to misunderstand. But either way, I consider it a mistake.

If I find the time, I will try to construct a reproducible scenario in which what is described here can be made visible.

[DerAndereAndi]

Hi,

I'd like to make a general comment: EEBUS is supposed to work the way it is implemented in this stack. The code is used in multiple projects, some for years, with real devices. It has also been tested with control boxes including passing the EEBUS qualification tests.

The stack does not implement everything that could be done, because I focused on what is really needed to work with existing products.

The stack will maintain a digital twin of all the data received by a remote device. That's also how others stack are implemented. The initial process to a server feature is always: subscribe to the feature, do an initial read. From then on all data changes will be provided via notifications and the stack will merge the data (if supported) and notify the application as @sthelen-enqs described.

A control box does not implement a server feature to store the limits, it will write to it on a remote device providing the loadcontrol server feature as part of implementing the LPC use case as a controllable system. Therefor a write will trigger a request to the remote device, while a read will not, as the stack already has all the data needed.

Hope this helps,
Andreas

[DerAndereAndi]

An additional note: some devices do forget to initialize their server feature with meaningful initial data. Stacks alone can not do that.

Regarding the implementation of LPC, LPP and MPC the EEBUS initiative also provides a set of test cases which they use for the qualification testing.