M³: Dense Matching Meets Multi-View Foundation Models for Monocular Gaussian Splatting SLAM

Kerui Ren, Guanghao Li, Changjian Jiang, Yingxiang Xu, Tao Lu, Linning Xu, Junting Dong, Jiangmiao Pang, Mulin Yu^†, Bo Dai^†

📢 News

[2026.10.29] 🚀 Code and Pi3 / Pi3X matching checkpoints are released!

[2026.09.02] 🎉 We are happy to announce that M³ has been accepted to SIGGRAPH Asia 2026.

Overview

Pipeline of M³. Our framework consists of joint tracking and global optimization for pose estimation and a mapper for scene reconstruction. For monocular sequences, Pi3X processes retrieved historical keyframes and new frames in one inference to facilitate factor graph construction and keyframe selection. Following the Neural Gaussian and LOD architecture of ARTDECO, Gaussians are initialized via Laplacian norm and optimized jointly with camera poses.

M³ supports Pi3 and Pi3X matching frontends with a Gaussian-splatting mapper for monocular scene reconstruction.

System architecture

1. Tracking: process incoming RGB frames together with retrieved reference keyframes; predict geometry and dense correspondences with a matching model.
2. Geometric optimization: use the correspondence constraints in the SLAM backend to estimate camera motion and optimize the factor graph.
3. Mapping: initialize and optimize neural Gaussians using multi-resolution image information, with view-dependent selection and appearance modeling.
4. Rendering and evaluation: export the reconstructed scene, camera poses, and held-out renders.

Installation

Requirements

• Linux, an NVIDIA CUDA-capable GPU, a CUDA toolkit with nvcc, and a C++17 compiler.
• Python 3.11 and a compatible CUDA-enabled PyTorch / torchvision pair.
• Sufficient VRAM for the foundation model and the growing Gaussian map.

Development environment: A800 80 GB, Python 3.11, PyTorch 2.10.0+cu128, torchvision 0.25.0+cu128, gsplat 1.5.3, torch-scatter 2.1.2, and PyPose 0.9.5.

Environment setup

git clone https://github.com/InternRobotics/M3.git
cd M3
conda create -n m3 python=3.11
conda activate m3

# Install PyTorch matching your CUDA setup first.
python -m pip install torch==2.10.0 torchvision==0.25.0 \
  --index-url https://download.pytorch.org/whl/cu128
python -m pip install -r requirements.txt

Native extensions and auxiliary dependencies

# Optional: set this to your GPU's compute capability before compilation.
# export TORCH_CUDA_ARCH_LIST=8.0  # A100/A800 example
bash scripts/install_native.sh

The installation script builds the following native extensions:

• VSLAM/: the mast3r_slam_backends CUDA extension used by geometric optimization.
• Reconstruct/submodules/diff-gaussian-rasterization: Gaussian optimizer.
• Reconstruct/submodules/simple-knn: nearest-neighbor search.
• Reconstruct/submodules/fused-ssim: the fused SSIM loss.

For headless Ubuntu environments, install the graphics runtime dependencies:

sudo apt-get install libegl1 libgl1 libopengl0 libglib2.0-0 libxrender1

Checkpoint preparation

Choose a matching model (Pi3 or Pi3X) and prepare the SALAD retrieval checkpoint:

Model	Checkpoint	Configuration
Pi3	checkpoints/pi3_matching.pt	config/yaml/pi3/pi3match.yaml
Pi3X	checkpoints/pi3x_matching.pt	config/yaml/pi3/pi3xmatch.yaml
SALAD	checkpoints/dino_salad.ckpt	—

Download the matching checkpoints from Hugging Face:

mkdir -p checkpoints
wget https://huggingface.co/cskrren/M3/resolve/main/Pi3.pt -O checkpoints/pi3_matching.pt
wget https://huggingface.co/cskrren/M3/resolve/main/Pi3X.pt -O checkpoints/pi3x_matching.pt

Download the retrieval checkpoint from SALAD and place it at checkpoints/dino_salad.ckpt.

Quick start

Input layout

data/my_scene/
├── images/
│   ├── 000000.png
│   ├── 000001.png
│   └── ...

GS with Pi3 matching

bash run.sh data/my_scene checkpoints/pi3_matching.pt pi3 \
  -m results/my_scene_pi3

GS with Pi3X matching

bash run.sh data/my_scene checkpoints/pi3x_matching.pt pi3x \
  -m results/my_scene_pi3x

To use known camera intrinsics, append --calib data/my_scene/intr.yaml to either command.

Outputs

results/my_scene_pi3/
├── slam/frames.txt          # frontend camera trajectory
├── metadata.json           # online mapper state and metrics
├── point_clouds/           # online Gaussian/point exports
└── <finetune_epoch>/
    ├── metadata.json       # final metrics and mapper-optimized cameras
    ├── point_clouds/
    ├── test_images/
    └── colmap/

Contact and citation

For questions, contact Kerui Ren. Please cite our paper:

@article{ren2026m,
  title={M\^{} 3: Dense Matching Meets Multi-View Foundation Models for Monocular Gaussian Splatting SLAM},
  author={Ren, Kerui and Li, Guanghao and Jiang, Changjian and Xu, Yingxiang and Lu, Tao and Xu, Linning and Dong, Junting and Pang, Jiangmiao and Yu, Mulin and Dai, Bo},
  journal={arXiv preprint arXiv:2603.16844},
  year={2026}
}