Limit maximum memory use of string.inspect

[lpil]

We have the string.inspect function which can turn any data into a Gleam-sync-like representation of that data. This may be useful for debugging and for error reporting with unknown data (such as data from FFI, OTP exits, inside the Erlang logger, etc).

While it will detect reference cycles on JavaScript to avoid attempting to make a string of infinite size, there is no upper limit to how large the constructed string can be; It will convert the entire input data structure. With large values, or values that use a lot of duplicated data or static data (Erlang literal pool, JS static variables), this can result in very high memory usage, especially if string.inspect is being called frequently, such when logging a frequent error. This can cause performance problems or even crashes due to memory exhaustion.

Elixir has a similar function, Kernel.inspect, and it has a limit and printable_limit options that help with this by limiting how many elements to render per-collection.

:limit - limits the number of items that are inspected for tuples, bitstrings, maps, lists and any other collection of items, with the exception of printable strings and printable charlists which use the :printable_limit option. If you don't want to limit the number of items to a particular number, use :infinity. It accepts a positive integer or :infinity. Defaults to 50.

:printable_limit - limits the number of characters that are inspected on printable strings and printable charlists. You can use String.printable?/1 and List.ascii_printable?/1 to check if a given string or charlist is printable. If you don't want to limit the number of characters to a particular number, use :infinity. It accepts a positive integer or :infinity. Defaults to 4096.

https://hexdocs.pm/elixir/1.12/Inspect.html
https://hexdocs.pm/elixir/1.12/Inspect.Opts.html

note: The limit option also applies to the depth that the function will descend into nested containers.

I think it would be useful to have similar functionality for Gleam's string.inspect, to avoid accidental memory spikes.

Proposal

Add a limit to the size of the output of string.inspect, limiting the size byte-size of the output rather than Elixir's limit on the number of elements in the data structure and byte size of strings. When the limit is reached we could either append ... and return the string.

The advantage of this over an Elixir style depth limit is that the maximum size is more predictable, however there would be more situations in which values in the parent container that come after a large earlier sibling value would not be printed at all that in Elixir. I think this is an acceptable drawback as data this large is already impractical to read, we don't have reason to especially prioritise those higher-but-later values, and the programmer could raise the limit if they want those ones to be printed.

string.inspect is used in a lot of code, with GitHub search returning 1,300 modules using it. To avoid a highly disruptive breaking API change we can add a second function that takes the limit as an argument. string.bounded_inspect? Name TBD.

string.inspect would be implemented as a call to the other function with some default limit that is large enough to produce output is too big to practically read, but not so large that it enables memory use problems. It's OK for the output of string.inspect to change like this as the output is not guaranteed to be stable.

[isaacharrisholt]

Looks reasonable. I'm not sure bounded_inspect is the right name for the function - that suggests to me that plain inspect is unbounded. I might be tempted to call it custom_inspect or something along those lines.

[lpil]

Yeah I'm not sure about the name. Custom seems a bit vague to me, or suggests that the output format could be configured in some fashion. Not sure what other options there are...

[tomkennedy513]

I'm just learning Gleam and haven't seen enough to know the naming conventions, but my mind jumped to inspect_with_limit or similar.

[giacomocavalieri]

The convention for stdlib is to usually not have with suffixes and instead use labels, for example:

pub fn map(list: List(a), with fun: fn(a) -> b) -> List(b)