• [Sept, 2025] SpatialLM-Dataset is now available on Hugging Face.
• [Sept, 2025] SpatialLM accepted at NeurIPS 2025.
• [Jun, 2025] Added finetuning instructions in FINETUNE.md.
• [Jun, 2025] Check out our new models: SpatialLM1.1-Llama-1B and SpatialLM1.1-Qwen-0.5B, now available on Hugging Face. SpatialLM1.1 doubles the point cloud resolution, incorporates a more powerful point cloud encoder Sonata and supports detection with user-specified categories.
• [Jun, 2025] SpatialLM Technical Report is now on arXiv.
• [Mar, 2025] We're excited to release the SpatialLM-Llama-1B and SpatialLM-Qwen-0.5B on Hugging Face.
• [Mar, 2025] Initial release of SpatialLM!

SpatialLM is a 3D large language model designed to process 3D point cloud data and generate structured 3D scene understanding outputs. These outputs include architectural elements like walls, doors, windows, and oriented object bounding boxes with their semantic categories. Unlike previous methods that require specialized equipment for data collection, SpatialLM can handle point clouds from diverse sources such as monocular video sequences, RGBD images, and LiDAR sensors. This multimodal architecture effectively bridges the gap between unstructured 3D geometric data and structured 3D representations, offering high-level semantic understanding. It enhances spatial reasoning capabilities for applications in embodied robotics, autonomous navigation, and other complex 3D scene analysis tasks.

Tested with the following environment:

• Python 3.11
• Pytorch 2.4.1
• CUDA Version 12.4