MACRO: Training-free Multi-plane Attention for Closeup Render Optimization

Nitzan Hodos, Roy Amoyal, Lior Fritz, Ianir Ideses, Sagie Benaim, Netalee Efrat

🌐 Project Page  |  📄 arXiv  |  💾 Data

This code is being released solely for academic and scientific reproducibility purposes, in support of the methods and findings described in the associated publication. Pull requests are not being accepted in order to maintain the code exactly as it was used in the paper.

Wide→close→wide fly-in on a DL3DV-Closeup scene: raw 3DGS vs. DiFix vs. MACRO (ours).

MACRO enhances close-up 3DGS renders by fixing the scale gap in reference-conditioned diffusion - it decomposes the close-up view into depth planes and matches each to rescaled reference crops, via depth-aware cross-attention. It is training-free (no fine-tuning, no architectural changes) and sets state of the art on two new close-up novel view synthesis benchmarks.

The pipeline: train 3DGS on the sparse wide-shot images → run the eval driver, which internally renders each close-up + its depth from the trained Gaussians, MACRO-enhances them, and scores against the ground-truth close-ups.

---

Installation

conda create -n macro python=3.9 -y
conda activate macro
# torch first, from the wheel index matching your driver (we used the CUDA 12.6 wheels)
pip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu126
pip install -r requirements.txt
# gsplat needs a matching CUDA toolkit (export CUDA_HOME=/path/to/cuda); if the pip wheel
# fails, build from source:
#   pip install git+https://github.com/nerfstudio-project/gsplat.git@v1.4.0

# basicsr imports torchvision.transforms.functional_tensor, which torchvision 0.23 no longer has:
BASICSR=$(python -c "import importlib.util, os; print(os.path.dirname(importlib.util.find_spec('basicsr').origin))")
sed -i 's/torchvision.transforms.functional_tensor/torchvision.transforms.functional/' $BASICSR/data/degradations.py

gsplat trainer (Step 1 below uses gsplat v1.4.0's examples/simple_trainer.py):

git clone --branch v1.4.0 --recursive https://github.com/nerfstudio-project/gsplat.git
pip install viser nerfview==0.0.2 "tyro>=0.8.8" tensorboard tensorly pyyaml
pip install --no-build-isolation "git+https://github.com/rahul-goel/fused-ssim@1272e21a282342e89537159e4bad508b19b34157"
export GSPLAT_EXAMPLES=$PWD/gsplat/examples

PFT-SR (reference super-resolution, part of the pipeline): the PFT architecture and its CUDA op come from the official PFT-SR repo:

git clone https://github.com/CVL-UESTC/PFT-SR.git
cp PFT-SR/basicsr/archs/pft_arch.py $BASICSR/archs/
(cd PFT-SR/ops_smm && pip install --no-build-isolation .)   # builds smm_cuda

Then get the pretrained models from the same repo - we use 103_PFT_light_SRx4_finetune.pth and 101_PFT_light_SRx2_scratch.pth - put them in one folder and point an env var at it:

export PFT_SR_WEIGHTS=/path/to/pft_sr_weights   # folder with the two PFT .pth files

DreamSim metric: its released LoRA config has a key that the pinned peft==0.9.0 does not accept. Download the weights once (this first load ends with a layer_replication error - expected) and drop that key:

export DREAMSIM_CACHE=~/.cache/dreamsim
python -c "from dreamsim import dreamsim; dreamsim(pretrained=True, device='cpu', cache_dir='$DREAMSIM_CACHE')" || true
sed -i '/"layer_replication"/d' $DREAMSIM_CACHE/ensemble_lora/adapter_config.json

---

Data

MACRO is evaluated on two close-up benchmarks. A scene folder is what the eval driver consumes:

<scene>/
  images/          # training wide-shot photos (COLMAP-registered)
  closeup_gt/      # ground-truth close-up photos (for scoring)
  sparse/0/        # COLMAP model: cameras.bin, images.bin, points3D.bin, ...
  split.json       # train/closeup split, close-up poses, intrinsics

split.json keys: training_frames (DL3DV) or train_frames (MobileClose) - the code accepts either - plus closeup_eval_pairs, closeup_poses, intrinsics.

DL3DV-Closeup (44 scenes)

DL3DV-Closeup is a set of close-up splits over scenes from the public DL3DV-10K Benchmark. The 44 split.json files are available at data/DL3DV-Closeup/. To evaluate, assemble a scene folder from your local DL3DV-10K Benchmark copy so it matches the scene-folder layout above, using the split as the manifest:

• split.json → copy our file as is.
• images/ → the frames listed in training_frames, taken from the scene's DL3DV images (we used nerfstudio/images_4, a.k.a. data_factor: 4).
• closeup_gt/ → the CLOSEUP frames named in closeup_poses / closeup_eval_pairs (closeup_frame), same source.
• sparse/0/ → a COLMAP model covering those views (the scene's DL3DV nerfstudio/colmap, or COLMAP re-run on the selected frames).

Then python verify_scene.py DL3DV-Closeup/<scene_id> to confirm the folder is complete. See data/DL3DV-Closeup/README.md for the exact field reference.

MobileClose-10 (10 scenes)

We captured these ourselves (photos + COLMAP), so full scene folders are provided:

📦 MobileClose-10 (Google Drive) - MobileClose-10.tar.gz

MobileClose-10 is released by Amazon under the CC-BY-4.0 license (the full text is in LICENSE inside the archive). If you use it, please credit Amazon as the source and cite our paper.

tar xzf MobileClose-10.tar.gz               # -> MobileClose-10/<scene>/...
python verify_scene.py MobileClose-10 --all

The drive also includes smoke_ckpt_cactus.tar.gz - a pre-trained 3DGS checkpoint for the cactus scene. To immediately test enhancement, extract it so that MobileClose-10/cactus/results/ckpts/ckpt_29999.pt exists, then skip Step 1 below.

---

Usage

1. Train 3DGS on the scene's sparse wide shots (the stock gsplat v1.4.0 trainer, examples/simple_trainer.py, run through a thin launcher):

export GSPLAT_EXAMPLES=/path/to/gsplat/examples   # see Installation
CUDA_VISIBLE_DEVICES=0 python scripts/train_3dgs.py default \
  --data-dir MobileClose-10/cactus \
  --data-factor 1 \
  --result-dir MobileClose-10/cactus/results \
  --disable-viewer \
  --no-normalize-world-space

scripts/train_3dgs.py runs simple_trainer.py unmodified, with the same CLI and hyperparameters. It changes only which views are trained on: exactly the frames in split.json (train_frames / training_frames). Stock gsplat would hold out every 8th image, and would also train on the close-ups, which are in images/ and the COLMAP model. World-space normalization must be off (--no-normalize-world-space), because the eval driver renders the close-ups in the raw COLMAP frame.

Training writes MobileClose-10/cactus/results/ckpts/ckpt_*.pt.

MACRO's eval driver loads that checkpoint (ckpt["splats"] with means/quats/scales/opacities/sh0/shN - the stock gsplat format) and renders the close-up + depth from it; no MACRO-specific training is involved.

2. Render the close-ups, MACRO-enhance, and score:

export PFT_SR_WEIGHTS=/path/to/pft_sr_weights
python src/evaluate.py \
  --scene-dir MobileClose-10/cactus \
  --gpu 0 \
  --configs macro \
  --results-dir results_out

This one command renders each close-up's RGB and depth from the trained Gaussians (saved as an intermediate plus_renders/), runs the enhancement, and scores against closeup_gt/. No manual rendering or pre-computed depth is required.

By default all of a scene's close-up pairs (the entries in split.json → closeup_eval_pairs) are evaluated; pass --pair-index N (0-based, 0 ≤ N < len(closeup_eval_pairs)) to run just one pair.

3. Aggregate a results table across scenes:

python scripts/aggregate_results.py --results-dir results_out

GPU note: set SR_GPU_ID to a GPU distinct from --gpu so PFT-SR doesn't contend for memory (we also set PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True), and run one evaluation per GPU pair at a time. evaluate.py exits normally even if a step runs out of memory, so check the log: a PFT-SR FAILED or FAILED line means that pair's metrics are not valid; re-run it on idle GPUs.

---

Citation

@misc{macro2026,
  title         = {MACRO: Training-free Multi-plane Attention for Closeup Render Optimization},
  author        = {Nitzan Hodos and Roy Amoyal and Lior Fritz and Ianir Ideses and Sagie Benaim and Netalee Efrat},
  year          = {2026},
  eprint        = {2607.03875},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2607.03875}
}

Security

See CONTRIBUTING for more information.

License

Code: this repository is released under the CC-BY-NC-4.0 license. It includes code adapted from DiFix (NVIDIA License, non-commercial), gsplat and diffusers (Apache-2.0), and uses PFT-SR (Apache-2.0); see THIRD-PARTY-LICENSES. Model weights are not included and are subject to their own licenses.

Data: the MobileClose-10 dataset is © Amazon and released under the CC-BY-4.0 license.