Expandable operators do not scale well in large models

[casella]

I discussed this case with @GallLeo some time ago, but I forgot to open a ticket about it. Doing it now. One of his customers has a model of a complicated system where many signals were exchanged through interconnected expandable connectors, representing one single data bus that spans the entire system. The size of the model unexpectedly blew up.

The problem can be formalized as follows: you have $N$ components with an expandable connector, each defining $k$ variables on the expandable connector via internal connect statements of input and output connectors of sub-components to the expandable connector; all the expandable connectors are connected to the same connection set. Doing so, you end up with $N$ instances of the expandable connector, each containing the entire set of $kN$ connector variables, so overall you are defining $kN^2$ connector variables in your model. As the size $N$ of the model grows, the number of such variables grows quadratically with $N$, which is not sustainable if you have more than a few hundred components.

For example, if you have 1000 components with 10 variables each, of which 2 are created on the expandable connector by each model, you have $(10-2)\times 1000 = 8,000$ component variables and equations (a reasonably sized model) and $2\times 1000 \times 1000 = 2,000,000$ connector variables and equations, which is just crazy. That's because each expandable connector instance contains the whole set of connector variables.

The corresponding 2,000,000 equations are just a bunch of trivial alias equations. Unfortunately, the Modelica workflow requires to first flatten the model, hence creating the 2 millions variables and alias equations. Then, you have to spend a lot of effort time identifying and removing 2 million alias equations and alias variables. This is really not efficient in terms of memory and time usage and for sure not a scalable solution.

The only way to avoid this quadratic proliferation in the number of connector variables is to add to the system an outer component with an expandable connector, and then to connect the k local variables of each component to the expandable connector of the outer component. Since the outer components are just a pointers and are not instantiated in the model that uses them, the overall system model eventually only gets to have one expandable connector with $kN = 2,000$ connector variables, which is just fine.

This is OK, but is not really very natural or intuitive.

Could there be a way to define expandable connectors that avoids the quadratic explosion of connector variables as the size of the system grows?

[casella]

Adding @perost and @adrpo to the loop.

[HansOlsson]

I agree that this seems problematic, but even if the semantics are defined in that way, I believe that tools could already handle it in a more optimized way.