Add documentation for libprocessing project

Author: tychedelia

docs/principles.md

@@ -0,0 +1,116 @@
+# What is libprocessing?
+
+In short, a new experimental effort to abstract the Processing API into a new native, cross-platform library written in Rust and powered by the [Bevy](bevy.org) game engine. Processing is historically a few different renderers, but most recently (as in 15 years ago) an OpenGL renderer using [JOGL](https://jogamp.org/jogl/www/). Because of [deprecations](https://developer.apple.com/documentation/macos-release-notes/macos-mojave-10_14-release-notes#Open-GL-and-Open-CL) and lack of support for modern graphics development techniques, it's become desirable for us to prototype a new renderer based around [WebGPU](https://webgpu.org/) as a new cross-platform backend for future Processing efforts, both in Java and other languages.
+
+## Why Rust/Bevy?
+
+Bevy is an open-source game engine written in the Rust programming language. Bevy and Rust more generally have a number of characteristics that are desirable for a project like Processing:
+- Rust is a systems programming language which means that it can easily be deployed and run on a variety of different platforms and be easily embedded in most programming languages which target the C ABI. For example, see [Bevy running on an ESP32 microcontroler].
+- Unlike C/C++, Rust provides modern build and development tooling which is more suited to beginner and intermediate contributors. The Rust ecosystem places a high emphasis on learning material and is generally more inclusive towards those who are less familiar with systems programming.
+- Bevy is based around the [entity component system](https://en.wikipedia.org/wiki/Entity_component_system) design pattern, which focuses on modularity and composability. Rather than being a monolithic game engine like Unity, Unreal, or Godot, it emphasizes being able to pick and choose which components any given application uses. This makes it particularly well suited to bridging Processing's immediate-mode API with a full-fledged engine.
+- Bevy is committed to the WebGPU and open-source graphics ecosystem and an active participant in the WebGPU standards process.
+- As a community, Bevy is strategically interested in use cases that go beyond games, including art, CAD applications, scientific computing, etc. 
+
+# Technology stack
+
+There are several layers required to make this work. Starting from the outermost layer of the onion:
+
+1. `bindgen`, a rust project for generating C headers from Rust code. This is what Java and other languages bind to.
+2. Our Rust FFI library, which wraps libprocessing. FFI rust code mostly means declaring the public interface for `bindgen` using `extern "C"` functions and mapping types compatible with the C ABI to call our libprocessing API.
+3. libprocessing is our Rust library that exposes the primary Processing API.
+4. Bevy is a fully featured application framework and game engine that helps structure our Rust code and provides a variety of graphics related features.
+5. `wgpu` is the Rust implementation of the WebGPU standard and is used by Bevy as the rendering hardware interface (RHI) for Bevy.
+6. Vulkan/Metal are the low-level graphics APIs that actually interact with the user's hardware. For the most part, we just need to know these exist and are what ultimately runs our graphics code.
+# Development principles 
+
+## Library design
+
+We want to expose the Processing API in a kind of procedural style (i.e. imperative, function driven) that mirrors the immediate-mode idiom of Processing sketches.
+
+### Hew to the existing API, but not at the cost of baking in historical mistakes
+
+We want to closely model the existing Processing API, but strategically fix issues that may be a consequence of legacy design. We also want to expose features that Bevy provides that are trivial to do so. For example, Processing does not support different texture formats, but this is super easy in Bevy, so we should make that possible in the API even if Java Processing can't fully exercise it.
+
+We also want to take learnings from p5 where it makes sense! There are some cases where they may have settled on a better naming convention or api style for something and we should feel free to take their learnings, documenting differences where we do.
+
+### Accessible for intermediate users
+
+While this is a lower-level library that has fewer guardrails than the high-level Processing API, we still want it to be something that feels accessible to users who are interested in getting more involved with development of the library. In that respect, we should always prioritize internal documentation, examples, and bread crumbs to facilitate learning. Graphics programming is hard, and in some cases we may need to express patterns that are complicated, but should always make sure that we describe the "why" even where the "how" may elude less experienced contributors.
+
+### Naming
+
+All exposed C functions should begin with the `processing_` namespace and should be further qualified by the type of data they operate on, e.g. `processing_graphics`, `processing_geometry`, etc. Because this is a lower-level library, long names are fine and it's better to be descriptive rather than terse. As such, we prefer multiple functions that accomplish similar tasks rather than overloading a single function with complicated parameters, for example we have both `processing_graphics_background_color` and `processing_grpahics_background_image` although these are presented in the user-facing API via an overload.
+
+### Handles, not pointers
+
+We never return pointers to data that lives in libprocessing. Because we use the ECS to manage Rust data, where longer lifetimes are necessary we return an `Entity` id, which can be returned to the user as a `u64` containing both the index and generation of the ECS entity. Any data not representing an API-level object should be returned on the stack and where an allocation is necessary (e.g. buffers for pixel data), it's the responsibility of the consumer to allocate a provide the pointer to the requisite allocation.
+
+API-level objects should wrap ids and they should never be exposed to the user as a first-class concept. All data which is returned via an `Entity` id should have a corresponding destructor function which removes the entity from the ECS and frees any associated resources. It's the responsibility of the API wrapper object to call this function when being destroyed.
+
+### Bridging Bevy and immediate-mode APIs
+
+By default, Bevy is highly optimized for throughput and includes automatic parallelization, pipelined rendering, and sophisticated rendering batching algorithms. This poses a problem for Processing which wants to present an immediate-mode API where the user can flush their current draw state to a given surface at any time.
+
+We work around this in the following manner:
+- By default, all `Camera` entities should be disabled via the `active` field, which ensures calls to `app.update()` will not do any rendering work unless a camera has been specifically enabled for rendering.
+- When a `Camera` is desired to be rendered, it's `active` field should be set and the `Flush` marker component should be added to its parent surface. All systems which produce renderable data should check for the `Flush` component to ensure they only work on the specific surface that is being rendered.
+- A `Camera` should set `CameraWriteMode::Skip` to ensure that its intermediate texture is not written to the final `RenderTarget` until the user calls `endDraw`.
+
+In this way,  as long as `Camera` state is managed correctly, it's totally fine to call `app.update()` or run individual systems as they will not trigger unnecessary renders and presents to the surface. 
+
+### Working with systems and borrows
+
+Working with raw `&mut World` in Bevy can lead to frustrating situations with respect to borrows, as many methods require mutable world access which then prevents doing other operations. Because of our immediate mode style, we much more frequently are imperatively modifying the world rather than adding "normal" Bevy systems that run in a schedule.
+
+There are several strategies that can help work around this:
+
+1. You can call systems on `World` that accept and return data! This looks like the following:
+
+```rust
+fn my_system(In(arg_a, arg_b): In<(u32, u32)>, mut my_res: ResMut<MyResource>, a_cool_query: Query<&Foo, &Bar>) -> u32 {
+	return 1234
+}
+
+// ... later
+world  
+    .run_system_cached_with(1, 2)?;
+```
+
+2. Collect results from queries into intermediate collections. This can resolve the borrow for a query at the cost of a bit of inefficiency.
+3. Use `world.resource_scope` or otherwise temporarily remove certain resources from the world (making sure to add them back later). 
+
+### Thread safety
+
+As of now, libprocessing is intended to be used in a single-threaded manner on the main thread of the application (which is a specific requirement for macOS window rendering and could be loosened). This is primarily for implementation simplicity; Bevy itself is designed to be highly parallel and does not have this restriction. In the future, we may decide to fully embrace multi-threading if the implementation complexity is worth the performance benefits.
+
+What this means concretely is:
+- Our `App` instance lives on a single thread.
+- Calling `app.update()` will run the entire main and render schedule in a blocking manner, including presenting the frame if configured to do so.
+- Asset loads via Bevy's `AssetServer` are blocking.
+- Non-send resources are guaranteed to live on the same thread as `App`.
+
+## Error handling
+
+Due to working at the FFI boundary, libprocessing requires interaction with unsafe code. We strive to be memory and panic-safe in all instances, which means that a consumer of libprocessing should not be able to trigger undefined behavior or crash their process simply by calling into libprocessing. However, libProcessing is not intended as a high level API and incorrect use may result in unexpected results, memory leaks, or other undesirable behavior.
+
+In general, our assumption is that errors encountered by consumers indicate exceptional situations in which the correct behavior is to halt program execution. In other words, errors are not intended for user feedback and where necessary the caller is expected to validate input prior to calling into libprocessing. More specifically, we want to expose validation as data, so have functions like `validate_shader` which returns friendly, user oriented text, rather than something like `compile_shader` which provides a validation error. The latter should also still validate but without the assumption of surfacing feedback to the user.
+
+### Rust errors
+
+We have a `ProcessingError` enum that uses the [`thiserror`](https://docs.rs/thiserror/latest/thiserror/) library to help manage error variants (see the documentation for more info). In general, we encourage adding new error variants that are unique to a given situation. As error states are considered exceptional, it's okay if these messages are better oriented towards a bug report that a user may file than informative to the user. Our expectation is that users should never see them in ordinary practice.
+
+We also expose a `Result<T, ProcessingError>` type alias that should be used in most cases as the return type for top-level functions.
+
+In our rendering code, `unwrap` and `expect` should only be used in situations that indicate the renderer itself has a bug, but is totally acceptable where invariants should be held. See [this blog post](https://burntsushi.net/unwrap/) by Burntsushi for some general guidelines here.
+
+### Cross-boundary error conditions
+
+Currently, we store the errors state in a thread-local `CString`, where any non-null value indicates the presence of an error. Consumers are required to call `processing_check_error`
+after every operation and throw an exception if any value is present.
+
+Inside the FFI library, all expose functions must:
+- Clear any existing error state using `clear_error` at the top of their function.
+- Check any results and set the error state using `set_error`.
+- Catch any panics using `catch_unwind`.
+
+The convenience function `check` is provided for working with `ProcessingError` and should be used in most cases to help with error handling.