Workflow for deactivating edges.

[FHell]

It is very easy right now to accidentally pass in the wrong parameter and initial conditions.

We first work with the nw object to initialize a new state, then have to make a fresh NWState then call pflat on that.

This exposes a ton of redundant state to the user.

THis might be revisited most easily when we look at the initialization workflow again.

[FHell]

One option: solve_powerflow returns a valid power flow, initialize(nw, powerflow) returns a u0 and a p0 rather than (or on top of) setting defaults.

[hexaeder]

I'd like to have less state in the nw object and more state passed around. I am not sure on the correct interface though.

initialize! builds on top of initialize_component! which takes a single VertexModel or EdgeModel and tries to initialize it. It is written quite generally, it considers every state, input, output and parameter as free unless it has a default property. I then solves for all the free ones.

With defaults/guesses stored within the model, this is rather easy because then we don't need to construct some datastructure which contains the defaults/guesses. The results of the initialization is stored using the init metadata.
The current approach requires, that the output of the powerflow algorithm is stored as a default in the component metadata before.

In order to make the initialization of a single component stateless we would need to have something like

inits = inititialize_component(comp, defaults, guesses)

where defaults, and guesses are something like Dict(:x => 1, :y=>2) which are internally merged with the defaults/guesses stored in comp metadata.
The return value would be a dict again, either of all initialized variables, or maybe even of all variables/parameters/inputs/outputs which have a value now.

It is totally possible to take a bunch of those init dicts together with a network to generate a fully initialized NWState without having touched the components metadata. But I am not to sure if the mental model for that is actually simpler than...

[hexaeder]

The fundamental problem here is, that the information required for initialization is richer and less homogeneous, than a NWState. The state contains u and p, and knows how to construct all the observables.
The initialization routine knows a few u, a few p and some observables and tries to reconstruct u and p. This inverse problem is generally harder and requires a more flexible datastrucuture (guesses on top of defaults and defaults, more flexibility regarding which states/p/observables have those defaults and guesses). So doing it stateless requires introducing yet another way of passing around state on top of the already confusing system...

[FHell]

Maybe I don't understand the current workflow. Does the initialization do anything that is not captured in u0 or p0?

Conceptually couldn't we do something like:

function get_steady_state_from_powerflow(nw, u_r, u_i)
  nw_init = deepcopy(nw)
  set_powerflow!(nw_init, u_r, u_i)
  initialize!(nw_init)
  return NWState(nw_init)

[hexaeder]

Following up on our discussion last week I've added a few sub-issues for this. In sum, this will result in the following workflow which does not write any of the metadata and is therefore completely non-mutating.

pf_object = powerflow(nw, s_pf=get_defaults(get_static_nw(nw_static)))
pf_interface_values = get_interface_states(pf_object)
s0 = initialize(nw, pf_interface_values, [defaults], [guesses])

You end up with a NWState object, which holds the initialized state according to the powerflow solution.

Problems:

• the optional arguments defaults and guesses to initialize cannot be NWState objects, instead they'd need to be nested dict Dict(VIndex(1) => Dict(:u_r => 1.0, :p => 7.2), ...), because in general defaults and guesses also apply to inputs and outputs of components, which are not part of the flat state representation.
• It is not worked out yet, how to change the parameters of a fl

Points for Discussion

• The flat array wrappers NWState/NWParameter arn't really usefull during initialization. Their logic only really makes sense for concrete states, where everything is defined. This is mainly because they cannot hold values for inputs and outputs of components, which are crucial for initialization/partial defined states.
• The shortcomings of NWState/NWParameter for rich initialization state lead to the introduction of metadata, i.e. let the Network-Object itself behave like a container for initial conditions and so on, only using the flat representation for time domain simulations.
• Currently, copy(::ComponentModel) is fast. Much faster than reconstructing the component, because it references the same functions and such but dealiases the metadata.
• It would be possible to define copy(::Network) in a way, that it is reasonably fast too. This would enable the following workflow:

base_nw = load_network(...) # costly

for s in scenarios
        nw = copy(base_nw) #create cheap copy for scenario
        
        # define syntactic sugar for "working" with nw objects like with `NWState` 
        nw[EIndex(7), :active] = 0
        nw[EIndex(7, :active)] = 0
        set_default!(nw, EIndex(7, :active), 0)
        set_pf_default!(nw, EIndex(7, :active), 0)
        
        # solve system (either the mutating or non mutating way as describe above, since the Network copy has scenario local metadata) 
end

Internally, the symbolic metadata is relatively simple: each component holds a dict of named tuples ,something like

symmetadata = Dict(
    :u_r => (; default=1.0, guess=0.0, description="foobar")
    ...
)

An alternativ to metadata included in the network would be to externalize the metadata: i.e. have something like

struct RichNWState
    d::Dict
end

richstate = RichNWState(nw) # extracts all the metadata from the components and bundles them outside the network

which wraps a dict like

Dict(
    VIndex(1) => Dict(
        :u_r => (; default=1.0, guess=0.0, description="foobar")
        ...
    ),
    VIndex(2) => Dict(
        :p=> (; guess=0.0, init=0.1209, description="some initialized state")
        ...
    )
)

and essentially write functions to derive the flat states from the rich states:

s0 = FlatNWState(r::RichNWState) # extracts defaults, init, ... from rich state

---

In my view, the highlevel problem is the following: some metadata (here I mainly refer to state and parameter values), e.g. machine parameters, are best specified at the component level. This naturally leads to a design, where the metadata is attached to the individual components rather than the network level.
Some other metadata, e.g. line failure state, is much better defined on the network level and does not really make sense in the component level. This naturally leads to a design where the states/parameters aren't hidden within the network but explicitly defined alongside.
Somewhere in the simulation workflow we need to make the transition.

[FHell]

I think this is going in the right direction, the observation that NWState is not rich enough looks correct to me. One question though, is it crucial to keep all the information used in the component wise initialization at the component level? E.g. Couldn't we directly work with the NWState object by defining, e.g. nws[VIndex(1)] as a view on the States that belong to component 1, and then combining that with the richer data we need?

[hexaeder]

With the proposed non-mutating component initialization #254 something like this would be possible:

nws = NWState(...)
defaults_from_nw_state = nws[VIndex(1)]
# this would be a Dict(:u_r => 1.0), :p => 4) for all u and p, NaN/Missing/Nothing possibly filtered out
# possible source of confusion: changing of his dict would not be reflected in the flat state anymore

v1model = nws[VIndex(1)] # extract the actual model

# prefill defaults and guesses from component metadata
defaults_from_metadata = get_defaults(v1_model)
guesses_from_metadata = get_guesses(v1_model)

# overwrite some defualts by the values extracted from nws
defaults = merge(defaults_from_metadata, defaults_from_nw_state)

# initialize the component, do not set any metadata but get the init dict back
init, residual = initialize_component(v1_model, defaults, guesses_from_metadata)

# init itself is a Dict(:sym => 1.2) for all symbols which only had guesses before (i.e. where free during initialisation)
# we could then "refill" missing parts of the `NWState` from init
set_values!(nws, VIndex(1), init)

So in short: yes, but the "view" part is kinda hard to achive i think (see comment about confusion above)

[hexaeder]

nw = Network(..)
pfnw = get_pf_model(nw)
pfs = NWState(pfnw)
# preinit with default where available
# set all active values <- free line parameters
# adjust P and q values

pfsol::NWState(bezogen auf pf) = find_fixpoint(pfnw, pfs)

set_interface_values!(nw, pfsol) # bad!


initialialize_component!(nw[EIndex(1)]) # bad!
init_states = initialialize_component(nw[EIndex(1), additonal_defaults, additiona_guesses)

# NetworkDynamcis API
additional_defaults = get_interface_states(pfsol)
s0 = initialize(nw, additional_defaults)

# OpPoDyn API
s0 = initialize_from_powerflow(nw, pfsol; defaults_overrides, guess_overrides)
# -> auch edges werde initialisiert
# eventuell skipe init für models ohne freie variablen aber checke dotx = 0 für gesamtsystem
# -> interface variables match
get_interface_states(s0) == get_interface_state(pfsol)

[FHell]

The more I look at this the more I like it.

One additional aspect: We don't need a pfnw or a pfsol to initialize. If our system comes with a valid powerflow we can directly extract the "additional_defaults"/interface variables from this powerflow, and don't need to know what the static model used to derive this was.