Site Detection and Tracking 3D (SDT3D)

Overview

SDT3D Dataset is a large-scale 3D LiDAR dataset specifically curated for robust perception and tracking of dynamic objects in highly complex and unstructured construction environments.

Key Features:

• Large scale: The initial release of SDT3D includes over 10,000 instances of human workers and more than 3,000 instances of large construction machinery (e.g., excavators, bulldozers, loaders). This vast quantity provides a solid foundation for training and validating deep learning models, particularly for challenging object categories.
• Dynamic object: Unlike generic outdoor datasets, SDT3D is designed around the movement and interaction of active, dynamic entities within construction zones. Each frame is enriched with precise 3D bounding box annotations for all observed workers and major machinery, enabling detailed inspection, tracking, and behavioral analysis.
• Advanced LiDAR: Data acquisition uses the cutting-edge RoboSense M1 solid-state, matrix-scanning LiDAR sensor. The M1's unique scanning pattern and high point density ensure comprehensive spatial coverage and fine-grained geometric detail.
• Continuous expansion: We are committed to the continuous development of SDT3D. A significant reserve of unannotated raw LiDAR data is currently being processed, with plans to systematically expand the dataset's size and diversity. Future releases will integrate additional samples, enriching the dataset further and providing an even more comprehensive benchmark for the community.

Download Dataset

The complete dataset is available for download from the cloud storage:

🔗 : Download Link

After downloading, extract the files and you will get the following directory structure.

Directory Structure

.
├── README.md                # This file
├── samples_info.json        # Frame sequence information
├── lidar/                   # Point cloud data files (*.bin)
└── labels/                  # Annotation files (*.json)

Point Cloud Data

File Format

Point cloud files are stored in the lidar/ directory as binary files with .bin extension. Each file corresponds to a single frame.

Data Structure

• Format: Binary file (.bin)
• Data Type: float32
• Shape: N × 4, where N is the number of points
• Channels: Each point contains 4 values: [x, y, z, intensity]
  • x, y, z: 3D coordinates in meters
  • intensity: Reflection intensity value

Loading Point Cloud Data

Python example:

import numpy as np

# Load point cloud
points = np.fromfile('lidar/0001-0002.bin', dtype=np.float32)
points = points.reshape(-1, 4)  # Reshape to (N, 4)

# Extract coordinates and intensity
xyz = points[:, :3]      # (N, 3) - coordinates
intensity = points[:, 3]  # (N,) - intensity values

Annotation Labels

File Format

Annotation files are stored in the labels/ directory as JSON files. Each file corresponds to a point cloud frame with the same token name.

Label Structure

Each annotation file contains an array of objects with the following structure:

[
  {
    "obj_id": "1",
    "obj_type": "Worker",
    "obj_attr": "squatting",
    "psr": {
      "position": {"x": 9.131, "y": -2.367, "z": -0.816},
      "rotation": {"x": 0, "y": 0, "z": 2.415},
      "scale": {"x": 0.778, "y": 0.666, "z": 0.968}
    }
  }
]

Field Descriptions

• obj_id: Unique identifier for tracking the object across frames
• obj_type: Object category (e.g., "Worker", "Vehicle", etc.)
• obj_attr: Object attribute or state (e.g., "walking", "squatting", "stooping")
• psr: Position, Scale, and Rotation (3D bounding box parameters)
  • position: Center point of the bounding box in global coordinates (meters)
    • x: Forward distance
    • y: Left distance
    • z: Height (vertical distance)
  • rotation: Orientation of the bounding box
    • x, y: Always 0 (rotation only around z-axis)
    • z: Yaw angle in radians, representing the angle between the box's heading and the positive x-axis (counter-clockwise is positive when viewed from above)
  • scale: Dimensions of the bounding box (meters)
    • x: Length (along the object's heading direction)
    • y: Width (perpendicular to heading)
    • z: Height (vertical dimension)

Coordinate System

The global coordinate system follows the LiDAR sensor convention:

• X-axis: Points forward (along the LiDAR central emission direction)
• Y-axis: Points to the left (horizontal left-hand direction)
• Z-axis: Points upward (vertical direction)
• Origin: Center of the LiDAR sensor

Rotation Convention:

• Rotation angle z is measured from the positive x-axis
• Counter-clockwise rotation (when viewed from above) is positive
• Unit: radians (rad)

Sequence Information

The samples_info.json file contains temporal relationships between frames, enabling tracking and sequential analysis.

Structure

[
  {
    "token": "0001-0002",
    "pre": "",
    "next": "0001-0007"
  },
  {
    "token": "0001-0007",
    "pre": "0001-0002",
    "next": "0001-0012"
  }
]

Fields

• token: Unique identifier for the current frame
• pre: Token of the previous frame in the sequence (empty string if first frame)
• next: Token of the next frame in the sequence (empty string if last frame)

Usage Examples

Loading Annotations

Python example:

import json

# Load annotation file
with open('labels/0001-0002.json', 'r') as f:
    annotations = json.load(f)

# Access bounding boxes
for obj in annotations:
    obj_id = obj['obj_id']
    obj_type = obj['obj_type']
    center = [obj['psr']['position']['x'], 
              obj['psr']['position']['y'], 
              obj['psr']['position']['z']]
    dimensions = [obj['psr']['scale']['x'],
                  obj['psr']['scale']['y'],
                  obj['psr']['scale']['z']]
    yaw = obj['psr']['rotation']['z']
    
    print(f"Object {obj_id} ({obj_type}): center={center}, size={dimensions}, yaw={yaw}")

Loading Sequence Information

Python example:

import json

# Load sequence information
with open('samples_info.json', 'r') as f:
    sequences = json.load(f)

# Build a lookup dictionary
seq_dict = {item['token']: item for item in sequences}

# Navigate through a sequence
current_token = "0001-0002"
while current_token:
    print(f"Processing frame: {current_token}")
    # Load and process point cloud and labels
    # ...
    
    # Move to next frame
    current_token = seq_dict[current_token]['next']

Citation

If you use this dataset in your research, please cite:

• [1] M. Zhang, W. Guo, J. Zhang, S. Han, H. Li, H. Yue, Excavator 3D pose estimation from point cloud with self-supervised deep learning, Computer-Aided Civil and Infrastructure Engineering n/a (2025). https://doi.org/10.1111/mice.13500.
• [2] M. Zhang, Y. Yang, S. Han, H. Li, D. Han, X. Yang, N. Guo, LiDAR-Based Framework for Accurate Positioning and Robust Tracking of Multiple Construction Workers, Journal of Computing in Civil Engineering 39 (2025) 04025027. https://doi.org/10.1061/JCCEE5.CPENG-6138.