Architecture

The VHToolkit is powered by the Rapid Integration & Development Environment (RIDE). RIDE is a research and development (R&D) middleware platform that combines government, industry, and academic technologies to provide a rapid prototyping sandbox for real-time applications. RIDE is developed at USC ICT, in collaboration with its partners.

RIDE follows a three-layered architecture, see the figure below. The foundation is the Engine Layer, with a current focus on the Unity game engine. This provides core capabilities, including rendering, user interface (UI), and audio. The Middleware Layer adds common prototyping functionalities (e.g., networking and cloud interfaces) and VH-specific technologies, including speech recognition, nonverbal behavior generation, and text-to-speech generation. Both Engine and Middleware Layer functionality are incorporated in the RIDE API. Individual projects are developed in the Project Layer, leveraging the RIDE API. This layer can act as an incubation area where proven technology can be incorporated back into the Middleware Layer, benefiting all users. RIDE functionality can be freely mixed and matched with native Unity and custom code.

The VHToolkit leverages RIDE’s architecture, API, and a subset of the functionality, in particular the cloud services interface and VH-related technologies (e.g., speech, NLP). For most capabilities, a technical implementation exists for both a cloud and a local option, see Figure below. For instance, for NLP, developers can choose between ChatGPT and Anthropic Claude cloud services or a local RASA end point.