Modules

[gcrois]

This is in reference to discussions had by Alexander @7h3kk1d, Andrew @disconcision, Matt @Negabinary, and Myself @gcrois on 10 March, 2025.

Whiteboard:

We need a module system in Hazel -- primarily for three applications:

1. World Tree stdlib / UI
2. User-defined livlits
3. Constructor namespaces

There has already been significant development work towards this, visible in this PR: #1020. Unfortunately, there have been substantial changes to the Hazel codebase, so much of these changes should be reworked. We also wanted to take this opportunity to ensure that we are in agreement on the semantics and syntax of modules.

Considerations:

1. (a) Should we restrict module definitions to being upper case, or (b) do not restrict the casing of modules?
2. (a) Should we change the semantics of let for
3. (a) module... in... vs. (b) module... end / module { ... }
4. (a) let... in... in module definition body (closure capture) vs. (b) ... = ... / ... := ... (record-like but mutually recursive)

Here are some examples of how we might use modules:

Livelit definition

/* probably built-in */
forall ModelT, ExpT ->
let Livelit = {
    type Model = ModelT in
    type Expansion = ExpT in
    name = String in
    model_default = ModelT in
    projector = ModelT -> Widget in
    update = ModelT -> Action -> ModelT in
    size = Int in
    explain_this = String in
}

/* User space */
type Color = (r: Int, g: Int, b: Int , a: Int)
let ^Color : Livelit = {
    type Model = Color in
    type Expansion = [Int, Int, Int, Int] in
    let name = "color selection" in
    model_default = (0, 0, 0, 255) in
    projector = ((r, g, b, a)) -> /* insert some cool html widget here */ in
    update = ((r, g, b, a), Action) -> /* insert some awesome update function that consumes widget actions and returns an updated Model */ in
    size = 200 in
    explain_this = "Ever wanted to select a color? Do I have a Livelit for you!"
}

We would like to further discuss this, including development strategies, at lunch on Wed, 12 March 2025.

[7h3kk1d]

A couple added notes:

• What's the story for mutual recursion in modules?
  • My opinion is having distinct syntax for module bindings could allow for implicit mutual recursion
  • in general we need to decide on if we're using let expressions for bindings or not
• How does our approach to first class modules allow for future data abstraction? As far as I know the current designs don't have opaque types and I just want to make sure our approach to first class modules at least makes them possible to get coherent.
• if we do different namespacing/case sensitive modules when they evaluate through polymorphic variables these will error in the stepper.

[7h3kk1d]

I also forgot to add if we're planning on modules being the basis for program organization and the world tree I don't think it's sustainable to require users to manually linearize the dependency graph. Something like a top level module registry where modules could reference each other and we deal with the dependency order seems preferable.

[disconcision]

re: linearization: def a concern for me as well... cyrus and I have talked about this a few times but not sure we reached any conclusions. what's stopping us from making the top-level mutually recursive? i guess it borks separate compilation, but i dont think thats a proximate priority. maybe it messes with straightforward approaches to statics incrementalization

[7h3kk1d]

I talked to @Negabinary a bit about it this morning and he thinks user control is at least in part a feature. Then talked to Cyrus about it in our meeting. The main issue he was concerned about was ensuring incrementality but seemed interested/open to it as the top level/default. I think where we left off was figuring out a prudent approach to get some modularlized namespaces in without solving the whole world of problems.

[7h3kk1d]

Knowing nothing of the difficulty my intuition is top level mutual recursion and having a simple way to fully functorize everything to allow for dependency inversion kind of like newspeak.

[7h3kk1d]

Another idea I had post-lunch was to combine the concept of modules/blocks by allowing for modules to "return" expressions. We had discussed having an optional expression in the last position in a module so that could be the default viewed value in a buffer. If we had this be the full evaluated value you get something that may supplant blocks/let expressions.

In the following examples val and type are module bindings and not expressions.

val result : int = {  
  type t = int;  
  val y : t = 7;  
  val x : t = 9;  
  val z : t = x + y;  
  z * 2  
};  

val final_result = result + 3;

This would be behaviorally the same as

val result : int = {  
  type t = int;  
  val y : t = 7;  
  val x : t = 9;  
  val z : t = x + y;  
  val returned = z * 2  
}.returned;  

val final_result = result + 3;

If there was no final expression it would instead be a module binding:

val result : {type t : int, val y: t, val x: t, val z: t}  = {  
  type t = int;  
  val y : t = 7;  
  val x : t = 9;  
  val z : t = x + y;
}

Pros

• Collapses syntactic constructs to learn/implement
  • Removes let expressions and type alias expressions by making them module bindings
• Blocks may be familiar syntax to users
• If we decided to do mutually recursive bindings for modules this would allow for mutually recursive bindings in expression land for free
• Behaves quite similarly to Reason's blocks with regards to module open/include

Cons

• The type of the expression changes if there's a final expression in the module syntax
• No syntactic requirement to include a final expression
  • We could introduce a return or yield keyword to make intent explicit
  • Need to decide whether requiring an explicit final expression is desirable
• Need to add curly braces if you want to define a variable in an expression: if x then let y = 3 in y * 5 else 6 becomes if x then {val y = 3; y*5} else 6.
• Stepping for variable bindings is no longer just substitution
  • This may be true for modules in general though so unclear if more work. It's also a push on whether it's beneficial to keep around the local bindings under stepping.

[gcrois]

Writeup forthcoming

[cyrus-]

@7h3kk1d interesting idea but I think there are situations where we want a trailing expression without eliminating the module, e.g. you could put some test expressions at the end of a module, or just manually be testing some things using probes.

maybe that semantics does not happen when you bind a module expression using module but does in other situations?

[7h3kk1d]

@cyrus-
I think it's useful to contextualize why you'd want to have the trailing expression without eliminating the module:

• To maintain current scratch buffer style editors while defining a module so you have something to show in the evaluation on the bottom
• To have examples included in a module

For the first use case I think having the buffers define an arbitrary value and display it is fine. If you had a trailing expression we'd display that in the result panel as we do now. For examples and test expressions I think you want those colocated with the bindings they're for; so we shouldn't limit them to tail position anyways. We could obviously require users to do let _ = expr but that's annoying so maybe we could add another keyword for expressions that you can probe but aren't intended to be bound anywhere:

module String = {  
  val string_concat : (String, [String]) -> String = ...  
  eval string_concat(" ", ["Hello ", "world", "!"]);  
  test string_concat(",", ["bar", "baz"]) == "bar,baz";  

  val string_trim : String -> String = ...  
  eval string_trim("   padded   ");  
  test string_trim("  spaced  ") == "spaced";  

  val string_split : (String, String) -> [String] = ...  
  eval string_split("one,two,three", ",");  
  test string_split("a;b;c", ";") == ["a", "b", "c"];  

  val string_sub : (String, Int, Int) -> String = ...  
  eval string_sub("functional", 0, 4);  
  test string_sub("substring", 3, 3) == "str";  
};

I'm using eval as the keyword to represent a dropped expression. I realize this is already used in stepper filters but felt best to me. We could also use drop, ignore, or exec. Parser permitting we could just use the presence of the semicolon to determine that it's a unused expression rather than the whole module yielding an expression:

module String = {  
  val string_concat : (String, [String]) -> String = ...  
  string_concat(" ", ["Hello ", "world", "!"]);  
  test string_concat(",", ["bar", "baz"]) == "bar,baz";
  "still a module";  
};

As a last option we can do something a la ruby blocks and have an explicit yield statement where

# This is a module #
{
 val a = 1;
1
}

# This is a integer #
{
  val a = 1;
  yield 1
}

I find this last option aesthetically unpleasing. My preference would be to have a keyword to signify that a unused value is being deopped in the module.

[7h3kk1d]

maybe that semantics does not happen when you bind a module expression using module but does in other situations?

My main concern around this is having to reason as the stepper passes modules as values into let statements. We could transform those into module bindings but I'd rather not make it confusing if we can do something more explicit like above.

[7h3kk1d]

I don't want to just keep throwing out random ideas, but I was thinking about the type punning this past weekend where:

module T = ...

Would implicitly do

module T = ...
type T = T.T

and I was wondering if there's a more general way to accomplish this goal. One issue with the above is you have to keep your local aliases in line with the internal type declaration for it to work and I'm not sure what this would look like in the hazel world tree approach:

If we have a bunch of top-level modules are they implicitly available in all other modules to make their types available or do I have to write something akin to the following to get the type to show up

module Int = Int in ...

Self-typed module

I'm wondering instead of opening a type alias based on having the same name if it were possible to have modules contain an optional self/representative type that would allow them to be used as a type:

module Peano = {
 selftype +Z +S(self); # self would be a keyword. This is essentially formalizing the idiom of `t` for the canonical type #

 val add : (Peano, Peano) -> Peano = ...;
}

let x : Peano = S(Z) in 
module PeanoAlias = Peano in
let y : PeanoAlias = S(S(Z)) in # The idea here being that you can use a module as a type if it has a self type being exposed #
...

This seems really useful for the cases where you're defining modules for a specific type with a bunch of helper functions which seems quite often as it allows you to alias the module and type together. I'm not done reading Modular Type Classes but it seems like they also require the .t idiom and this seems like a more explicit formulation of that.

[7h3kk1d]

Had ChatGPT generate a more full example:

# Define a module with a canonical type #
module Vector2D = {
  selftype (x: float, y: float);  # Vectors can be used as a type #

  val add : (Vector2D, Vector2D) -> Vector2D = fn (a, b) =>
    { x = a.x + b.x; y = a.y + b.y };

  val scale : (Vector2D, float) -> Vector2D = fn (v, s) =>
    { x = v.x * s; y = v.y * s };

  val dot : (Vector2D, Vector2D) -> float = fn (a, b) =>
    (a.x * b.x) + (a.y * b.y);
};

# Using Vector2D as a type #
val v1 : Vector2D = { x = 1.0; y = 2.0 };
val v2 : Vector2D = { x = 3.0; y = 4.0 };
val v3 : Vector2D = Vector2D.add(v1, v2);

# Aliasing while keeping type association #
module Vec = Vector2D;
val v4 : Vec = Vector2D.scale(v3, 2.0);
val dot_product = Vec.dot(v1, v4);

# A float-specific utility module #
module FloatOps = {
  selftype float;

  val square = fn x => x * x;
  val sqrt = fn x => ...;  # Placeholder for sqrt implementation #
};

# Inner module for working with unit vectors #
module UnitVector = {
  val normalize : Vector2D -> Vector2D = fn v =>
    let magnitude = FloatOps.sqrt (Vec.dot(v, v)) in
    Vec.scale(v, 1.0 / magnitude);
  
  module Basis = {
    val i : Vector2D = { x = 1.0; y = 0.0 };
    val j : Vector2D = { x = 0.0; y = 1.0 };
  };
};

val unit_v1 : Vector2D = UnitVector.normalize(v1);
val unit_i : Vector2D = UnitVector.Basis.i;
val unit_j : Vector2D = UnitVector.Basis.j;

[cyrus-]

Not sure I see the problem with having the type share the name of the module. The type alias would be in the same scope as the module so presumably it would be available everywhere the module is available.

I think it's more readable to look at signatures of values inside a module if the type name is used rather than Self.

[Negabinary]

I think the main issue is module aliasing - if the module gets imported under a different name you can no longer do the punning

[7h3kk1d]

Talked with Cyrus a bit about this today. To summarize:

Module/Type Aliasing

Cyrus seemed in favor of not having a special self or module type as a builtin/firstclass feature. For the module aliasing itself he suggested using something similar to ocaml's with type := to rename a type within the module. We could automatically insert that after a module alias. I'm still somewhat concenered what this would do in rare situations that you have collisions, but I think my hesitance more comes from a place of not wanting to have specialized behavior based on a name match.

collision concern:

module S = {
  type S; 
  type T;
}
module T = S; # Does this rename S to T and if so is it a static error? #

Blocks as modules with trailing expressions

There's concern around accidentally turning modules into expressions by having an expression in trailing position. Having an explicit yield keyword could also be noisy if you had to do it every time you wanted a nested scope.

Long term it seems like the strategy is to just have the same syntax in modules/expressions but use the curly brackets to indicate module and parens or whitespace be a regular expression scope.

Very rough examples from our meeting

type S = {
  type HashMap = Int
  let x : HashMap
}

module HashMap : S = {
  type HashMap = Int
  let x = 0
}

// (S renaming HashMap = Map) = {
  type Map = Int
  let x : Map
}

rename Map = HashMap

module Map : (S renaming HashMap = Map) = (HashMap renaming HashMap = Map)
type Map = Map.Map

Map.x : Map.Map


type Comparable(a) = {
  let compare : (a, a) -> int
}

module IntComparable : Comparable(Int) = {
  let compare = Int.compare
}

let f<a>(x : Comparable(a)) 


module x = {
  type T = Int;
  val y = 3;
}

let x = 
  type T = Int;
  let y = 3;
  y + 1

let f(x) = {
  let y = 3;
  y + 1
}