Skip to content

README.md

Latest commit

 

History

History
62 lines (48 loc) · 2.99 KB

README.md

File metadata and controls

62 lines (48 loc) · 2.99 KB

Tactile Game Level Creator (TGLC)

Tactile Game Level Creator (TGLC) turns tabletop drawings and everyday objects into playable 2D platformer levels. Capture a scene, customize settings, then play the generated level in Unity. It aims to lower the barrier to level authoring for non-experts by using familiar hands-on interactions.

TGLC Gameplay

Demo video: https://drive.google.com/file/d/1St-wbC0KPDumuw-_we3f-Gaty572lLuQ/view?usp=sharing

System Pipeline

TGLC Pipeline

  1. Capture (webcam): you take a photo of the physical tabletop layout.
  2. Segmentation + filtering: the perception step isolates the objects and extracts clean contours. It also produces session artifacts like an object-only RGBA (visual layer), a contour JSON (geometry layer), and planned spawn positions.
  3. Unity build: Unity consumes the contours/JSON to build the platform geometry and generate one-way edge colliders.
  4. Customize & preview: the system uses the same geometry plus the planned spawns so users can adjust visuals/gameplay and preview placement.
  5. Gameplay runtime: Unity instantiates the final level from those artifacts and runs the playable scene.

Repository Layout

.
├── images/
│   ├── tglc_pipeline.png
│   └── capture_customize_play.png
├── segmentation/
│   ├── .venv/                                 # local python environment
│   ├── fastsam_segmentation_for_unity.py
│   ├── models/
│   │   ├── FastSAM-s.pt (or FastSAM-x.pt)     # downloaded pretrained weights
│   └── archive/
│       ├── sam2_segmentation_for_unity.py
│       └── cv_segmentation_for_unity.py
└── unity/
    └── TactileGameLevelCreator/
        ├── Assets/
        ├── Packages/
        └── ProjectSettings/

Setup and Run

  1. Open the Unity project from: unity/TactileGameLevelCreator

  2. Create a Python environment in segmentation/: python3 -m venv segmentation/.venv

  3. Install dependencies: segmentation/.venv/bin/python -m pip install ultralytics opencv-python pillow numpy

  4. Download FastSAM-s.pt or FastSAM-x.pt and place it at: segmentation/models/

    Note: this project currently uses FastSAM-s.pt as the default.

    FastSAM pretrained weights: https://docs.ultralytics.com/models/fast-sam/#available-models-supported-tasks-and-operating-modes

  5. In Unity (Capture scene), set:

    • pythonExePath -> absolute path to segmentation/.venv/bin/python
    • segmenterScriptPath -> absolute path to segmentation/fastsam_segmentation_for_unity.py

  6. Run capture flow in Unity. Outputs are written to the run directory created by CaptureController.

References