Skip to content

Electron ideas

Jump to bottom
Christian Pérez-Llamas edited this page Dec 27, 2018 · 3 revisions

Introduction

Nowadays UI development is strongly developed around web technologies. Electron allows to develop desktop application using those same technologies, which involve using the JavaScript language. But our application is written with Rust, so we need to find the way to bridge those worlds, as well as to figure out the way to write as much code in rust as possible.

Requirements

  • Performance: There will be plenty of visual indicators changing state all the time (vu meters, leds, scrolling clips, ...)
  • Layouts: The layout of the UI will be complex and I don't wan't to make layout calculations myself.
  • Vector graphics vs Canvas: Ideally it integrates vector graphics support easily, but canvas rendering would also be ok.
  • Communication/linking with the rust logic: There are plenty of options that need to be considered: RPC, IPC, cross-linking, WebSockets, ...)
  • Custom widgets: It should support creating custom components/widgets, as well as to customise very common ones such as buttons, labels and so on.

State of the art

There is an existing case that proves that it is possible to use Electron for Music composition.

AudioNodes

There is no much information about how it is implemented, but they say something in their Credits page. According to that they would be using things such as React and MaterialUI.

Update: After running it locally I realised how many problems does it have. A lot of audio overruns, and the UI is not very smooth. Worth looking at the blog linked in the Performance section.

Performance

There is an interesting blog about how to make Electron apps for 60fps: https://www.nearform.com/blog/architecting-electron-applications-for-60fps/

Bridging with Rust code

As a subprocess/server

The idea would be to launch a subprocess with the rust core from Electron, and have some kind of bi-directional communication channel between the UI process (Electron) and the Core process (written in rust). The UI part would implement some kind of model related to the representation that would be updated from events fired from the core process, while the user interactions would be sent to the core process as events too. In terms of implementation there is some overhead in terms of communication (synchronisation, UI model + Core model, De/Serialisation of Data Transfer Objects, ...). About the communication channel, there are plenty of options to consider too:

  • stdin/stdout
  • unix socket
  • UTP sockets
  • WebSockets

Linked as a native library in NodeJS

There is a project to build Rust bindings to NodeJS called Neon.

Example project: https://github.com/ubnt-intrepid/neon-electron

The building/linking complexity would increase, but the communication between the UI and the core would be much more simple and there would be less duplication of logic for dealing with the UI model.

Compiled into WASM

The rust code could be compiled into wasm and be used fromElectron as if it was any other JavaScript module (assuming that NodeJS already support wasm). The implication is that now the Audio and MIDI logic would live in the Web side, so there would be no support for VST or AU plug-ins.

UI libraries

  • React: It can generate SVG or canvas for parts that require custom drawing (leds, meters, arrange, clips, ...)
  • Inferno: They claim to be compatible with JSX but faster.
  • Vue.js: Not sure if this is going to be faster than React.
  • PixiJS: Focused on game development. High performance?
  • two.js: 2D drawing API for svg, canvas, and webgl. Scene graph based.
  • Nexus UI: Audio interfaces.
  • MUI Elements: Widgets for sound and music.
  • web-audio-controls
  • WebSynthUI
  • interact.js: JavaScript drag and drop, resizing, and multi-touch gestures for modern browsers.
  • ...

Clone this wiki locally