Explore namespace ideas

[odow]

I just watched @blnicho's JuMP-dev talk https://youtu.be/G1tW68vrOBM. (Thanks Bethany! It was a very useful talk and exactly what I was after.)

It has come up in various discussions over the years, but one of the biggest differences between JuMP and Pyomo is how we handle namespacing.

JuMP

JuMP uses a single global namespace, and all objects are stored in the object_dictionary(model):

model = Model()
@variable(model, x)
model[:x] === x  # true
@variable(model, x)  # Errors. x already exists.

The downside is that for unrelated model parts to compose, you need to use anonymous variables, but this has poorer printing, and the macro syntax is more limited.

model = Model()
x = model[:x] = @variable(model, base_name = "x")
model[:x] === x  # true
x = model[:x2] = @variable(model, base_name = "x")

Pyomo

Pyomo has blocks. At a high level, blocks create a new namespace, and can be nested inside each other. You can delete/deactive entire blocks as a single component.

In Julia-land, we might do:

model = Model()
@block(model, blocks[1:2])
@variable(blocks[1], x)
blocks[1][:x] === x  # true
@variable(blocks[2], x)
blocks[1][:x] === x  # false: Julia binding `x` has been replaced in this scope
blocks[2][:x] === x  # true
@objective(model, Min, sum(b[:x] for b in blocks))

Next steps

I don't have a concrete syntax proposal, or believe that we should necessarily implement this, but I'd be open to exploring possibilities, and I'd very much like to come up with a few examples where JuMP's current syntax is limiting and the block/namespace would be beneficial.

I don't think this would change anything at the MOI level. This is strictly a JuMP-level feature.

I also don't know if this is that easy to explore in a JuMP-extension, but maybe it is. A prototype could store everything in model.ext[:block_extension] for now.

Some questions:

• What exactly is the struct of block[1]?
• Are blocks added to the model lazily or eagerly?
• You could imagine a block being a self-contained JuMP model with no solver backing it, and when block is referenced in the model scope everything gets copied across.
• Dropping a breadcrumb to https://github.com/geoffleyland/rima and https://github.com/geoffleyland/rima/blob/master/papers/Rima-ORSNZ-paper-2010.pdf

[metab0t]

I think a straightforward way to implement this is to regard block as a proxy to model, otherwise JuMP needs to traverse the tree-like structure (or even a graph) to retrieve some item, which is quite complex. Maintaining model.ext[:block_extension] as a flat dict with key as block identifier is easier.

mutable struct Block
    model::Union{Model, Nothing}
    identifier::Symbol
end

[odow]

otherwise JuMP needs to traverse the tree-like structure (or even a graph) to retrieve some item, which is quite complex

Pyomo keeps things as trees, and yes, they do tree-traversal operations.

[metab0t]

I realize that maintaining a tree structure is necessary to record the hierarchical relations of blocks, but maybe their contents should still be stored in a flat dict for easier manipulation.

[odow]

Another option is to explicitly be able to pass in a namespace:

struct Namespace
    data::Dict{Symbol,Any}
end
model = Model()
@variable(model, x >= 0)
@expression(model, blocks[1:2], Namespace())
@variable(model, x >= 1, namespace = blocks[1])
@variable(model, x >= 2, namespace = blocks[2])
@objective(model, Min, sum(b[:x] for b in model[:blocks]))

[odow]

Maybe we don't even need to provide explicit Namespace objects, we just need a way for the user to choose where to store things in macros, and that it will use setindex!(object, value, ::Symbol)