|
| 1 | +# MoveToCode: A Mixed Reality Visual Programming Language |
| 2 | + |
| 3 | +MoveToCode is an open source mixed reality visual programming language (VPL) created by the [Interaction Lab](http://robotics.usc.edu/interaction/) led by [Maja J Matric](https://robotics.usc.edu/~maja/index.html). The VPL is used to study [student kinesthetic curiostiy](https://tgroechel.github.io/kin_cur.html). The VPL can be thought of similar to [Scratch](https://scratch.mit.edu/) in 3D. |
| 4 | + |
| 5 | +MoveToCode is built for the [Microsoft Hololens 2](https://www.microsoft.com/en-us/hololens/hardware) using the [Mixed Reality Toolkit](https://github.com/microsoft/MixedRealityToolkit-Unity). Currently it is set up to work with the [Mayfield Kuri robot](https://www.heykuri.com/explore-kuri/) but can be modified for any [ROS](https://www.ros.org/) supported robot via [ROS#](https://github.com/siemens/ros-sharp) and [rosbridge](http://wiki.ros.org/rosbridge_suite). |
| 6 | + |
| 7 | +Here is a [video](https://www.youtube.com/watch?v=s7udZXa2wEw) of MoveToCode + a robot tutor. |
| 8 | + |
| 9 | +## Study Commits |
| 10 | +Currently the study commit for all papers: [0765f1d1ab46373f1bc1be90d9d5b07fafc2a533](https://github.com/interaction-lab/MoveToCode/commit/0765f1d1ab46373f1bc1be90d9d5b07fafc2a533) |
| 11 | + |
| 12 | +## Projects in Progress |
| 13 | +- Arrays |
| 14 | +- CodeBlock to Python Text |
| 15 | +- Robot speech via Amazon Polly |
| 16 | +- [Adding virtual robot arms](https://github.com/interaction-lab/KuriAugmentedRealityArmsPublic) |
| 17 | + |
| 18 | +## General Structure |
| 19 | + |
| 20 | +### Interpreter |
| 21 | +- Singleton class that reads/interprets connected codeblocks line-by-line from top to bottom, beginning at the "Start" codeblock. |
| 22 | +- Utilizes an instruction stack to ensure that the reaching the end of a codeblock control flow statement (if, while) does not end the program unless the stack is empty. |
| 23 | + |
| 24 | +### Memory Manager |
| 25 | +- Singleton class that holds a list of buttons which spawn variables created by the user. |
| 26 | +- Visualizes changes in the values stored in variables as interpreter reads/runs code. |
| 27 | + |
| 28 | +### IArgument |
| 29 | +- Abstract class from which all visual/block coding components are derived. |
| 30 | +- Of all the classes that inherit from IArgument, only the bottom classes/leaf nodes are non-abstract |
| 31 | +- Child classes - main partition for code blocks based on function |
| 32 | + - IDataType: All data types (primitive and derived) inherit from this class |
| 33 | + - int, float, string, bool, char, array |
| 34 | + - Instruction: All instructions inherit from this class |
| 35 | + - print, set variable value, conditionals, arithmetic |
| 36 | + |
| 37 | + |
| 38 | +## External Dependencies |
| 39 | +Versions of packages are pushed directly to the repository to avoid versioning issues, licensing permitting. |
| 40 | +- [ROS#; dwhit UWP branch](https://github.com/dwhit/ros-sharp/commit/4ccf45fc94827132397afeaa210afc01834d1dec) ([Apache 2.0 License](http://www.apache.org/licenses/LICENSE-2.0)) |
| 41 | + - [Newtonsoft.Json](https://github.com/JamesNK/Newtonsoft.Json) ([MIT License](https://en.wikipedia.org/wiki/MIT_License)) |
| 42 | + - [Newtonsoft.Json.Bson](https://github.com/JamesNK/Newtonsoft.Json.Bson) ([MIT License](https://en.wikipedia.org/wiki/MIT_License)) |
| 43 | + - [websocket-sharp](https://github.com/sta/websocket-sharp) ([MIT License](https://en.wikipedia.org/wiki/MIT_License)) |
| 44 | +- [Mixed Reality Toolkit v2.1](https://github.com/microsoft/MixedRealityToolkit-Unity)([MIT License](https://en.wikipedia.org/wiki/MIT_License)) |
| 45 | +- [ProBuilder](https://github.com/Unity-Technologies/com.unity.probuilder) ([Unity Companion License](https://unity3d.com/legal/licenses/Unity_Companion_License)) |
| 46 | +- [TextMeshPro](https://docs.unity3d.com/Packages/com.unity.textmeshpro@2.0/manual/index.html) ([Unity Companion License](https://unity3d.com/legal/licenses/Unity_Companion_License)) |
| 47 | + |
| 48 | +## Tested Platform Support |
| 49 | +The following are currently tested versions of required software. Higher versions of each software may work but have not been tested. |
| 50 | +- [Unity 3D](https://github.com/siemens/ros-sharp/tree/master/Unity3D) v2018.4.14f |
| 51 | +- .NET Framework 4.6 and Visual Studio 2017 |
| 52 | +- Built for [Hololens 2](https://www.microsoft.com/en-us/hololens/hardware) |
| 53 | + |
| 54 | +## Licensing |
| 55 | +MoveToCode is open source under the [MIT License](https://en.wikipedia.org/wiki/MIT_License) |
| 56 | + |
| 57 | + |
| 58 | +## Questions or Contributions |
| 59 | +For any questions, you can contact Thomas Groechel at groechel@usc.edu. Feel free to fork for your own projects and/or submit pull requests. |
0 commit comments