Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
107 commits
Select commit Hold shift + click to select a range
317bf53
new version od debug draw without linking problems
goatchurchprime Jan 25, 2025
bc836b3
change ConstrainedTriangulation to be instantiated by classdb
goatchurchprime Jan 25, 2025
1d53ceb
now with rooms loading at startup and null pointers fixed
goatchurchprime Jan 25, 2025
b05ae0a
- Added in a button when not connected
goatchurchprime Jan 25, 2025
2130fea
Added chevron entity
pcurranjones Jan 27, 2025
26a6d0f
UID update & JSON extraction
pcurranjones Feb 5, 2025
e6e5ed8
Added label device, fixed tooltip device
pcurranjones Feb 6, 2025
68ec92e
add in the new openxrvendors and xr-autohands
goatchurchprime Feb 8, 2025
08fc6f9
committing rest needed to get the hand tracking working
goatchurchprime Feb 8, 2025
0fa0224
Added tooltip editing
pcurranjones Feb 8, 2025
8fa4102
Fixed some tooltip editing issues
pcurranjones Feb 9, 2025
b77a047
MQTT tracking, tooltip changes, keyboard changes
pcurranjones Feb 11, 2025
4967a8c
added debug MQTT elements
pcurranjones Feb 12, 2025
f2d640a
Boundary snap checkers and Tooltip stuff
pcurranjones Feb 15, 2025
c544e65
enable the connection to home assistant (with a huge number of guid u…
goatchurchprime Feb 15, 2025
927ee8f
Made tooltips successfully save/load text
pcurranjones Feb 17, 2025
3474928
start being able to load on laptop independentlhy
goatchurchprime Feb 18, 2025
91e4644
put the CDT replacer in so it works on nix
goatchurchprime Feb 18, 2025
3eff403
update the house if signal connected too late for it
goatchurchprime Feb 18, 2025
754048d
we call the update house after connecting to make the devices with ca…
goatchurchprime Feb 18, 2025
77c219a
Tooltips renamed to stations, CDT error fixed,
pcurranjones Feb 19, 2025
94fbd36
remove bad startups
goatchurchprime Feb 19, 2025
4de682a
more print statements in sliders and filter down to the
goatchurchprime Feb 20, 2025
fe3a3f4
fix export presets that were on pico4
goatchurchprime Feb 20, 2025
7c2b45d
now with iid added to the database
goatchurchprime Feb 20, 2025
d8a1d52
update the house on open as well as on connect
goatchurchprime Feb 21, 2025
47f14f9
Stations sequencing added, Debug distance hmd element added
pcurranjones Feb 21, 2025
80c0b6c
pushed fix to house.gd problem, removed "vector3" from hmd
pcurranjones Feb 21, 2025
2df8929
fix up the home assistant integration on hass
goatchurchprime Feb 21, 2025
c3d2465
add magic tinsel and change off switch to light blue
goatchurchprime Feb 22, 2025
87cc55e
tolerates loading entities without being connected
goatchurchprime Feb 22, 2025
3871b6a
KEY_C does connect to HASS
goatchurchprime Feb 22, 2025
62d798e
Merge branch 'main' of https://github.com/DynamicDevices/Immersive-Home
pcurranjones Feb 22, 2025
daed2c1
Menu updates, removed the "Vector3" from the decimetre tracker
pcurranjones Feb 22, 2025
d937c44
attempting to access the browsemedia on hass
goatchurchprime Feb 22, 2025
ba79f34
keyboard place now works
goatchurchprime Feb 22, 2025
33852e7
magic tinsel happens when you type tilde
goatchurchprime Feb 22, 2025
9c8136e
starting to unpick the media browser
goatchurchprime Feb 22, 2025
4aabb93
Station Debug Mode, View Menu Icon Edit
pcurranjones Feb 24, 2025
50bbd59
Made stations add icons instead of changing visibility
pcurranjones Feb 24, 2025
839a587
mediabrowser node now does something
goatchurchprime Feb 22, 2025
bf4cf17
media browser object working again
goatchurchprime Feb 24, 2025
cf45a47
delete skydome
goatchurchprime Feb 24, 2025
c6d7815
working on the custom_label and the nodered pattern coming through
goatchurchprime Feb 25, 2025
a77184f
made the custom label perform
goatchurchprime Feb 25, 2025
598e086
mediabrowser uid
goatchurchprime Feb 26, 2025
b22d251
Merge branch 'customlabel' oops
goatchurchprime Feb 26, 2025
5b6d96b
add in fizzle souund on the magictinsel
goatchurchprime Feb 26, 2025
5e6ab00
Added reset to wrist options, made mini-view logo congruent
pcurranjones Feb 26, 2025
243ab66
Added resizable backdrop panel to custom label entity, removed Debug3…
pcurranjones Feb 27, 2025
58d37c4
Added a 2 room 1 door json file
pcurranjones Feb 28, 2025
aa01e92
fix the door selection and deletion due to json error
goatchurchprime Mar 1, 2025
fd7b8b3
Began making panel resizable
pcurranjones Mar 1, 2025
177ff75
Merge branch 'main' of https://github.com/DynamicDevices/Immersive-Home
pcurranjones Mar 1, 2025
8a945f2
Remerged custom label to work
pcurranjones Mar 1, 2025
94cbf50
Custom label merged again but better this time
pcurranjones Mar 1, 2025
5080970
doors fixed with extra checkings done
goatchurchprime Mar 1, 2025
8838da1
Began setting up save menu
pcurranjones Mar 1, 2025
6dcc55f
Work on the nodered extracting of names and entities
goatchurchprime Mar 2, 2025
9cf6062
Made user view less cluttered, made panel collisions scale
pcurranjones Mar 3, 2025
ede0e0e
Experimental Menu, Reference Mode Re-Implemented
pcurranjones Mar 3, 2025
ade78bc
Fixed experimental tab not hiding on dev mode
pcurranjones Mar 3, 2025
4e8e676
we can flip between two videos live
goatchurchprime Mar 4, 2025
b230a41
julianbuttons now reposition a choice of 2 videos
goatchurchprime Mar 4, 2025
33b2ade
new openxr vendors put in
goatchurchprime Mar 4, 2025
4f6ad2b
all; features with the video player now good
goatchurchprime Mar 4, 2025
2a737cb
exoplayer update
goatchurchprime Mar 5, 2025
09a29d9
commit of video moving
goatchurchprime Mar 6, 2025
dd37266
gix up vid places and all the next_stations
goatchurchprime Mar 6, 2025
e32e160
Separated out the wall and reference buttons
pcurranjones Mar 7, 2025
026d9b2
make the move alignment code work
goatchurchprime Mar 9, 2025
758fa74
Merge branch 'main' of https://github.com/DynamicDevices/Immersive-Home
pcurranjones Mar 9, 2025
bd73c9f
Added debug no depth test to stations,
pcurranjones Mar 10, 2025
79bb347
fix roods to doors
goatchurchprime Mar 11, 2025
addf0a3
delete the debug_draw addon
goatchurchprime Mar 11, 2025
bd7a6cc
reconnect wallsbutton
goatchurchprime Mar 11, 2025
04602c8
another change to the room alignment to handly doors bugs
goatchurchprime Mar 12, 2025
eefc705
alignment bug
goatchurchprime Mar 12, 2025
f38b597
trash accepts stations and expoplayer pauses
goatchurchprime Mar 12, 2025
467adfe
stations start up with the name in the blank text
goatchurchprime Mar 12, 2025
e9acd01
disable photo taking
goatchurchprime Mar 12, 2025
f250d52
Tidied up the overview menu
pcurranjones Mar 12, 2025
cd1b4ca
Added a compass
pcurranjones Mar 13, 2025
17d1953
Fixed compass and room menu
pcurranjones Mar 14, 2025
168424d
Compass updates
pcurranjones Mar 14, 2025
7dcae36
debug trails from stations
goatchurchprime Mar 16, 2025
0bf114d
add in next button container
goatchurchprime Mar 16, 2025
e5d7a64
next station preview line goes from correct place
goatchurchprime Mar 16, 2025
0ad58b9
move the dev strip to the top with the activate button
goatchurchprime Mar 16, 2025
e15295b
friendlyname label also shows in dev mode
goatchurchprime Mar 16, 2025
007107f
add in the rest of the exoplayer addon code, though we don't use it
goatchurchprime Mar 17, 2025
528d42d
big changes to allow mmultiple buttons and developer mode to show the…
goatchurchprime Mar 17, 2025
8fbbb60
more fixes to the wiring and reobustness
goatchurchprime Mar 17, 2025
23582c8
fix up the magic tinsel with a no depth test containing sphere
goatchurchprime Mar 19, 2025
ba23e63
make the stations shrink to size of the text
goatchurchprime Mar 19, 2025
a344f3f
changed wisp sound
pcurranjones Mar 21, 2025
88c6d1c
Update README.md
goatchurchprime Mar 24, 2025
31cd093
Update README.md
goatchurchprime Mar 24, 2025
786f17f
up to 4.4.1
goatchurchprime Apr 19, 2025
4772e51
align reference updates from database and bring close works
goatchurchprime Apr 19, 2025
76c11fe
now does not update room
goatchurchprime Apr 19, 2025
dede535
bodged on the photo option for video sections
goatchurchprime Apr 19, 2025
8f20620
allow media jpg to be upper case
goatchurchprime Jun 17, 2025
4ca36ce
put cybertruck into magic tinsel
goatchurchprime Jun 17, 2025
a45a88d
add a light to the cybertruck
goatchurchprime Jun 17, 2025
406f0eb
sort out directing cybertruck in correct direction and it disappearing
goatchurchprime Jun 17, 2025
8d4d5dc
fix up te spatial scanning and rescanning of room capabilities that w…
goatchurchprime Jun 27, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions HouseSetups/twoRoomOneDoor/house.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{ "align_position1": { "_type": "Vector3", "x": 0.0, "y": 0.0, "z": 0.0 }, "align_position2": { "_type": "Vector3", "x": 1.0, "y": 0.0, "z": 0.0 }, "doors": [{ "id": 1.0, "room1": "Room 1", "room1_position1": { "_type": "Vector3", "x": 3.64302, "y": 2.09712, "z": 3.94425 }, "room1_position2": { "_type": "Vector3", "x": 2.51741, "y": 2.08876, "z": 4.38976 }, "room2": "Room 2", "room2_position1": { "_type": "Vector3", "x": 3.7647, "y": 2.09712, "z": 4.25167 }, "room2_position2": { "_type": "Vector3", "x": 2.62933, "y": 2.08876, "z": 4.67251 } }, { "id": 2.0, "room1": "Room 2", "room1_position1": { "_type": "Vector3", "x": -2.10225, "y": 2.24698, "z": -0.838626 }, "room1_position2": { "_type": "Vector3", "x": -1.52879, "y": 2.25353, "z": 0.758217 }, "room2": "Room 1", "room2_position1": { "_type": "Vector3", "x": -1.88003, "y": 2.24698, "z": -0.918432 }, "room2_position2": { "_type": "Vector3", "x": -1.38362, "y": 2.25353, "z": 0.706083 } }, { "id": 3.0, "room1": "Room 2", "room1_position1": { "_type": "Vector3", "x": 4.30383, "y": 2.87975, "z": 2.83884 }, "room1_position2": { "_type": "Vector3", "x": 2.85471, "y": 2.83083, "z": 4.36584 }, "room2": "Room 1", "room2_position1": { "_type": "Vector3", "x": 3.90969, "y": 2.87975, "z": 2.4648 }, "room2_position2": { "_type": "Vector3", "x": 2.45215, "y": 2.83083, "z": 3.98382 } }, { "id": 4.0, "room1": "Room 1", "room1_position1": { "_type": "Vector3", "x": 0.453031, "y": 2.50941, "z": -0.638411 }, "room1_position2": { "_type": "Vector3", "x": 0.82357, "y": 2.50475, "z": 0.938254 }, "room2": "Room 2", "room2_position1": { "_type": "Vector3", "x": 0.557533, "y": 2.50941, "z": -0.66297 }, "room2_position2": { "_type": "Vector3", "x": 0.869165, "y": 2.50475, "z": 0.927538 } }], "entities": [], "rooms": [{ "corners": [{ "_type": "Vector2", "x": 0.896214, "y": 1.24732 }, { "_type": "Vector2", "x": -0.101211, "y": -2.99672 }, { "_type": "Vector2", "x": -2.05804, "y": -2.27154 }, { "_type": "Vector2", "x": -1.30174, "y": 1.48384 }, { "_type": "Vector2", "x": -0.981536, "y": 1.42643 }, { "_type": "Vector2", "x": -0.972009, "y": 1.6222 }], "height": 2.80828, "name": "Room 1" }, { "corners": [{ "_type": "Vector2", "x": 0.936535, "y": 1.27123 }, { "_type": "Vector2", "x": 2.51998, "y": 0.945766 }, { "_type": "Vector2", "x": 1.68187, "y": -3.6648 }, { "_type": "Vector2", "x": 0.165848, "y": -2.66228 }], "height": 2.81258, "name": "Room 2" }] }
61 changes: 19 additions & 42 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,60 +1,37 @@
![logo](app/assets/banner.png)
# Important Introduction

# 🏠 Introduction
This project is called **SafeAR** (also known as **QuestWalk**), an app deployed on the stand-alone [Meta Quest 3](https://en.wikipedia.org/wiki/Meta_Quest_3) to easily create on-boarding, safety training and instructed procedures that can be used on-site in Augmented Reality. Similar projects include [The Ghent Alterpiece](https://www.alfavision.be/project/ghent-altarpiece) by AlfaVision.

Immersive Home is project to bring Smart Home and Mixed Reality technologies together for an intuitive and immersive experience.
It is the result of work done between [18 February 2025](https://www.ukri.org/news/daresbury-laboratory-welcomes-first-businesses-to-new-5g-ecosystem/) and 21 March 2025 (not including time spent writing the bid application) for the [5G Ecosystem - Proof of Concept call](https://iuk-business-connect.org.uk/opportunities/5g-ecosystem-proof-of-concept-call/).

<a href="https://immersive-home.org" target="_blank" style="text-decoration: none !important">
<img height="32" src="app/assets/badges/website.png" alt="Badge linking to our website" />
</a>
We could not have attempted this project in such a short time frame without starting from the incredible open source work of [**Immersive-Home**](https://github.com/Nitwel/Immersive-Home) and [Solar-XR](https://github.com/TU-Dublin-Computer-Science/Solar-XR).

<a href="https://discord.gg/DWWtDfVeus" target="_blank" style="text-decoration: none !important">
<img height="32" src="app/assets/badges/discord.png" alt="Badge linking to our discord" />
</a>
# Demonstration Video

<a href="https://www.oculus.com/experiences/quest/7533875049973382/" target="_blank" style="text-decoration: none !important">
<img height="32" src="app/assets/badges/meta.png" alt="Badge linking to Meta App Lab" />
</a>
This is an example of guiding someone to refill the coffee machine in the [DoESLiverpool hackspace](https://doesliverpool.com/).

<a href="https://github.com/Nitwel/Immersive-Home/releases/latest/download/Android.zip" target="_blank" style="text-decoration: none !important">
<img height="32" src="app/assets/badges/github.png" alt="Badge linking to GitHub" />
</a>
[![Making Coffee in AR](https://github.com/user-attachments/assets/c20b158d-0c3e-4a36-b9cd-3a809a6a7bd1)](http://www.youtube.com/watch?v=QzLRFOMqT2g "SafeAR mixed reality training demonstration: How to make coffee at DoES Liverpool")

<a href="https://nitwel.itch.io/immersive-home" target="_blank" style="text-decoration: none !important">
<img height="32" src="app/assets/badges/itch.png" alt="Badge linking to itch.io" />
</a>
# Technical components

<a href="https://sidequestvr.com/app/26827/immersive-home" target="_blank" style="text-decoration: none !important">
<img height="32" src="app/assets/badges/sidequest.png" alt="Badge linking to sidequest" />
</a>
## Home Assistant

## Documentation
[Home Assistant](https://www.home-assistant.io/) is a huge modular home automation system that runs on an RaspberryPI. [Immersive-Home](https://immersive-home.org/) is able to log into it to access its location-based sensors and controls and overlay them into Augmented Reality.

For users: [https://docs.immersive-home.org/](https://docs.immersive-home.org/)
For developers: [https://docs.immersive-home.org/development/](https://docs.immersive-home.org/development/)
## Godot engine

The [Godot](https://godotengine.org/) engine is a free, lightweight open source Game Engine that supports Virtual Reality features such as pass-through cameras and hand-tracking.

## Features
## Exoplayer plugin

- **Fast and Intuitive control over IoT devices**
- **Live overview over your smart home**
- **Build in Voice Assistant for more control**
- **Miniature view to manage your whole house**
- **Natural interactions with hands**
- **Advanced automations based on position in room**
The Meta Quest 3 Virtual Reality headset runs Android which means that many the of the internal features of this operating system potentially available to an application running on it. Partway through the project the [Godot-ExoPlayer](https://github.com/bnjmntmm/godot-exoplayer/) feature appeared, which made it suddenly very easy to download and playback videos.

## Support the Project
## Node-RED

💕 If you like to see the project grow to support more devices and interaction methods, consider supporting the project using these methods:
[Node-RED](https://nodered.org/) is an optional component of Home Assistant that provides a visual flow-based coding system. It turns out to be pretty useful for planning and drawing flow charts for walk-throughs to be deployed to the Augmented Reality app. The ability to make and plan content is in some ways more important than the software to show it.

- Donate using [PayPal]( https://paypal.me/nitwel) or [GitHub Sponsors](https://github.com/sponsors/Nitwel)
- Buying the app on [Meta App Lab](https://www.oculus.com/experiences/quest/7533875049973382/) or [itch.io](https://nitwel.itch.io/immersive-home)
- Contributing to the project by creating issues or pull requests
![image](https://github.com/user-attachments/assets/367fcd70-ede1-41b4-a6de-bee537beb455)

## Supported Devices
# Future work

**Smart Home Platforms**
- [Home Assistant](https://www.home-assistant.io/)

**Mixed Reality Headsets**
- Meta Quest 2 / Pro / 3
SafeAR/QuestWalk is looking for real world uses and applications that could pay for and drive its future development. You could theoretically download it and try it out yourself, but given the necessary sketchiness of the prototype product made on this time scale, you will need our help. Please contact [Dynamic Devices](https://www.dynamicdevices.co.uk/) for details.
23 changes: 23 additions & 0 deletions app/3DGlass.gdshader
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
shader_type spatial;

uniform vec4 albedo: source_color = vec4(1.0, 1.0, 1.0, 0.4);


varying vec3 vertex_pos;

void vertex() {
vertex_pos = VERTEX;
}

float fresnel(float amount, vec3 normal, vec3 view)
{
return pow((1.0 - clamp(dot(normalize(normal), normalize(view)), 0.0, 1.0 )), amount);
}

void fragment()
{
float basic_fresnel = fresnel(3.0, NORMAL, VIEW);
ALBEDO = albedo.xyz + basic_fresnel;
ALPHA = albedo.w + basic_fresnel;

}
1 change: 1 addition & 0 deletions app/3DGlass.gdshader.uid
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
uid://dcot5nai3uxa5
21 changes: 0 additions & 21 deletions app/addons/debug_draw_3d/LICENSE

This file was deleted.

130 changes: 0 additions & 130 deletions app/addons/debug_draw_3d/README.md

This file was deleted.

90 changes: 0 additions & 90 deletions app/addons/debug_draw_3d/debug_draw_3d.gdextension

This file was deleted.

Empty file.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

Loading