RFC 0004: HEAP escape analysis

[ysbaddaden]

Add an optimization pass to the compiler to detect places where a class instantiation could happen on the stack rather than the GC HEAP.

Preview: https://github.com/crystal-lang/rfcs/blob/rfc-heap-escape-analysis/text/0004-heap-escape-analysis.md

[RX14]

I think it might be possible to reuse a lot of the escape analysis architecture to implement a shadow stack for each fiber which contains only roots. This could be both precise and a far smaller stack than the actual program stack for the GC to scan. Especially given the stack usage will grow when escape analysis is turned on.

I don't know how that tradeoff compares to the LLVM built in stackmap generator, but it's an idea.

[straight-shoota]

An additional use case of escape analysis could be detecting leakage of stack pointers. Ensuring code integrity could even be considered a higher goal than just performance. A correct program is worth more than a fast one.

[straight-shoota]

This article comparing borrow checker / lifetimes handling in Rust and C# gives a short introduction how these languages handle escape analysis: https://em-tg.github.io/csborrow/

[crysbot]

This pull request has been mentioned on Crystal Forum. There might be relevant details there:

https://forum.crystal-lang.org/t/indexable-mutable-methods-acting-on-part-of-the-sequence/7911/10

[BlobCodes]

I don't really understand why this is an RFC tbh.
As I understand RFCs, they should discuss changes to the core language itself, not an implementation thereof.
The RFC rather seems to focus on one specific implementation usable to decrease allocation overhead.

I think it would be better to rephrase the RFC so the compiler simply makes no guarantees about the allocation method used for allocating a reference object (or that it may explicitly destroy an object if the compiler can reasonably determine the end of an object's lifetime).

LLVM already has a heap2stack optimization pass that could be used to implement the stack-allocation mechanism this RFC is asking for, but it relies on explicit free() calls to analyze the object's lifetime. Adding explicit GC.free(ptr) calls could also help reduce GC pressure and memory usage, but the RFC only focuses on one aspect of heap escape analysis.

EDIT: If the wording was too harsh, I still stink this would be a very beneficial improvement to the compiler, I just think the RFC should be written in a more generic, "standards"-esque way.

[straight-shoota]

For the record, RFCs are meant to discuss and document changes to any part of Crystal, not just the core language. The README states that already, but let me know if you think it needs to be expressed more clearly.

So the scope of this RFCs should be fine. It is in a very early stage, and certainly needs a lot of exploration to figure out the implementation. Maybe it makes some premature assumptions, but I'm sure that'll be refined in the future.
We should probably mark this as a draft because right now this is not much more than a letter of intent.

[straight-shoota]

It's a nice idea if we could offload escape analysis to LLVM. However, I'm not sure if LLVM can make decisions on livetimes as informed as the compiler can with full semantic information, or what would be necessary to do that. I doub LLVM is currently even aware of which functions allocate, because that is abstracted away in libgc.
Anyway, delegating this to LLVM would mean heap escape analysis is unavailable in non-LLVM backends (such as the interpreter). This might be acceptable, but it's still worth considering.
I figure it would also make it impossible (?) to use escape analysis for protecting against dangling stack pointers (mentioned under Future possibilities).

Adding explicit GC.free(ptr) calls could also help reduce GC pressure and memory usage

It could, but doing this automatically requires essentially the same lifetime analysis as heap escape analysis: The compiler can only statically free an object if it can prove it's not accessible from anywhere else.

[HertzDevil]

LLVM itself performs capture analysis on pointers during some optimization passes, and LLVM 20+ function parameters may have a captures attribute, but they seem too low-level for the IR-level changes required here

[crysbot]

This pull request has been mentioned on Crystal Forum. There might be relevant details there:

https://forum.crystal-lang.org/t/mixed-memory-model-for-crystal/8627/6