A Modular Render System for Cleanroom

[tttsaurus]

Summary

To provide a clean way to do rendering and prevent modders from leaking OpenGL.

Goals

• Provide an structured and clean rendering abstraction layer to replace the current chunk/entity/item rendering.
• Independent from Minecraft, with Minecraft functions gradually redirected to this render system over time.
• Ensure GL state management is centralized.
• Organize data and logic using Entity-Component-System (ECS) to improve modularity, scalability, and maintainability.
• Support modern rendering techniques.
• Provide clean API for modders to use.

Non Goals

• This system is not intended to serve as a shader pack loader or a direct Optifine replacement.
• Although implementing shader-based rendering, dynamic light, and other advanced features is inevitable, fundamental groundwork must be completed first.

Motivation

• Modders leak GL state
• Minecraft rendering is unstructured, making it hard to maintain and extend

Description

Overview

Here's an overview of what we need to work on:

• Abstraction of GL concepts
  • I think this can be divided into two categories:
    • One is the abstraction of raw GL concepts, like Shaders, VBOs, etc
    • The other is a higher-level abstraction, like Materials, Meshes, etc
  • This abstraction will become a standard to prevent us from messing with raw GL calls
• Entity-Component-System to organize rendering
  • ECS will be a system that holds all entities and data.
  • ECS is independent from Minecraft, so we wrap Minecraft classes and pass them to ECS.
• Rendering Itself

Low-level GL Abstraction

• Shader / ShaderProgram
• Buffers
  • VAO, VBO, EBO, PBO (pack/unpack), etc
  • Special ones: UBO, SSBO, TBO, etc
• Framebuffer
  • Attachment
  • Multi-Render Target
• Vertex Attribute Layout
• Debug (KHR_debug)
• GPU-CPU Sync / Fence
• Texture / Image

I'm just listing them based on my experience. We may need more or less while designing the system.

High-level GL Abstraction

• Framebuffer (or also called RenderTexture; can be both low- and high-level abstraction)
• Renderer
• Model / Bone
• Mesh
• Material
• Render Pass

Similar to Low-level GL Abstraction, we may need more or less of them.

ECS

// a world holds all entities
World:
  entity_0 {com_0, com_1}
  entity1 {com_2}
  ...

Components:
  com_0(type: a)
  com_1(type: b)
  com_2(type: a)
  ...

System_0:
  pass our world to system
  collect entities with com type a & b -> entity_0, ... (archetype)
  process...

System_1:
  pass our world to system
  collect entities with com type a -> entity_1, ... (archetype)
  process...

Key clarifications:

• We don't do class composition. Entity & Component & System are independent and they're driven by IDs
• Component only contains pure data (value types) so they can be continuous in memory
• Processing part of a system can be parallelized due to the nature of ECS

When designing this library, we can discuss and possibly implement an ECS variant to accommodate Minecraft.

Render Commands

Using ECS, we implement a Render System that processes entities and generate render commands. This approach allows us to:

• Batch and optimize those commands before drawing
  • Instancing, Multi-Draw etc
  • Order them to avoid excessive state switch
• Collect commands and separate them into different passes, like transparent pass etc

Overall Pipeline

• Wrap Minecraft stuff and pass them to our ECS
• Tick systems
• Generate render commands
• Batch commands and re-order them
• Split commands into different passes (Opaque, Cutout, Transparent, First Person, UI, etc)
• Command decoration stage
  • e.g. If I want to render an overlay on a tile entity, I register it with the decoration manager, and the system will handle the rest (like a blackbox)
• Opaque Pass, Cutout Pass, and so on
• Lighting Pass
• Post-processing Pass

Summary

This is my first time designing and outlining a system like this, so some technical details may be incomplete, and I'm not yet certain about the challenges before implementing it. However, the main idea is still clear and simple 1. use ECS for logic & data; 2. build a structured pipeline.

Dependencies

• JOML ?

References

No response

Guidelines

• I have and will continue to adhere to the guidelines

[tttsaurus]

Conceptual API Design

ECS-based Rendering Framework

CleanWorld world = new CleanWorld();

RenderSystem renderSystem = new RenderSystem(world);
// systems can be generic but RenderSystem must be a special one
// and systems need an order of execution
world.registerSystem(new AnimationSystem());
world.registerSystem(new AISystem());

// we register com types ahead-of-time
// so it'll be easier to alloc offheap mem and manage
world.registerComponent(MeshComponent.class);
world.registerComponent(MaterialComponent.class);
world.registerComponent(TransformComponent.class);

// render passes should be generic
renderSystem.registerPass(new OpaquePass());
renderSystem.registerPass(new TransparentPass());
renderSystem.registerPass(new PostProcessPass());

void gameTick(float deltaTime)
{
    world.tickSystems(deltaTime);
    
    renderSystem.generateRenderCommands();
    
    renderSystem.executeRenderPasses();
}

// this is not a Minecraft entity & world!
CleanEntity entity = cleanWorld.createEntity();

// addComponent is giving a composition feeling
// but com data should be flattened and continuous in mem
// we can do hybrid tho, like offheap + traditional oop
entity.addComponent(new TransformComponent(data));
entity.addComponent(new MeshComponent(meshData));
entity.addComponent(new MaterialComponent(materialData));

class RenderCommand
{
    int meshID, ... ? // currently no idea how we design RenderCommand
    // maybe we should record the owner of commands for the decoration purpose
}

Minecraft-bridge and Mod Dev Part

public class MyModTileEntityRenderer implements ITileEntityRenderer
{
    @Override
    public List<RenderCommand> createRenderCommands(TileEntity tileEntity, RenderContext ctx)
    {
        // maybe we don't use ID. not certain
        int meshID = ctx.meshFactory.cube();
        int matID = ctx.materialFactory.mat("my_mod:block/custom_texture");
        return Collections.singletonList(new RenderCommand(...));
    }
}

// this should be a powerful thing
// like it'll be totally possible to add a bloom layer or something
public class ChestRenderingDecorator implements IRenderPassDecorator
{
    @Override
    public void decorate(Decorator deco)
    {
        deco.match(Chest).insert((tileEntity, ctx) ->
        {
            return Collections.singletonList(...);
        });
    }
}

[tttsaurus]

Progress on ECS

PositionComponent positionComponent = new PositionComponent();
positionComponent.xyz = new Vector3f(1, 2, 3);

// components will be flattened and stored in archetype data pools
// and entities are represented as data rather than objects
// -> reduce per entity object allocations and gc pressure
ECS_RUNTIME.entityManager.createEntity(positionComponent);
ECS_RUNTIME.entityManager.createEntity(positionComponent);
ECS_RUNTIME.entityManager.createEntity(positionComponent);

// the side effects are deferred. guarantees thread safety
ECS_RUNTIME.entityManager.flush();

// the query is archetype oriented not entity oriented. faster query
List<ArchetypeDataPool> archetypes = ECS_RUNTIME.entityManager.startQuery(ECS_RUNTIME.entityManager.newQuery().with(PositionComponent.class));
LOGGER.info("archetype num: " + archetypes.size()); // assert: 1

// the array is fetched from archetype data pools directly. zero copy
// providing a friendly structure to parallel jobs
INativeArray<?> array = archetypes.get(0).getArray(PositionComponent.class, "xyz", "z");
ArrayRange range = archetypes.get(0).getArrayRange();
for (int i = range.start; i < range.end; i++) {
    // skip deleted entities
    if (range.deprecatedIndexes.contains(i)) {
        continue;
    }
    LOGGER.info("debug: " + array.get(i));
} // assert: 3.0 3.0 3.0

[Rongmario]

One of the goals of Kirino should be to slowly deprecate and remove the way to access raw GL calls provided from LWJGL.

Few things to consider:

1. Warning log messages when raw GL calls are made
   a. Warn caller from inside of GL calls w/ StackWalker (can degrade overall platform performance by quite a bit)
   OR
   b. Warn callers by wrapping their calls in synthetic methods with a logger call
2. What do we do with GLStateManager?
3. Physically limiting the devs from accessing GL classes
4. Warnings at compile-time with an annotation processor to detect GL calls before full access removal?

[tttsaurus]

1. Regarding runtime warnings, I would exclude option (a) to preserve performance, and are you considering class transforming mods to implement (b) btw? that would cause log spam though. I think a better approach is to collect all GL/GlStateManager references from a mod and log them only once at the start.
2. Treat GlStateManager like raw GL calls, and also TextureManager#bindTexture, TextureUtil, etc. We should prevent clients from touching any GL state.
3. I don't know how is that possible
4. That's a good idea, but I don't know how to implement it. We could also introduce @UnsafeGL to suppress warnings etc. Considering the fact that not all GL calls modify states and some of them are harmless, it's valid to use them occasionally.

At the very beginning, I think (1) (2) (4) are enough.