GaussianAttractLoss

A 3D semantic attraction loss for weakly supervised occupancy estimation

VIPLAB, Gachon University

Camera-only 3D occupancy models trained purely by rendering losses spend almost all of their capacity on road, buildings and vegetation, and almost none on the objects that matter for driving. This repository adds one term that fixes the allocation directly in 3D, on top of GaussianFlowOcc (Boeder et al., ICCV 2025).

The problem

GaussianFlowOcc represents a scene as a set of 3D Gaussians and supervises them only through a rendering loss, which is weighted by pixel area. A pedestrian occupies a tiny fraction of a camera image, so getting it wrong is nearly free, while the same Gaussian spent on the road surface covers a large one. The rendering loss is optimising correctly, just not for occupancy of small objects. The effect is visible in where the Gaussians end up: counting how many Gaussian centres fall inside ground-truth voxels of each class, the ten object classes together receive roughly an order of magnitude fewer Gaussians than the surface classes, even though they are what an occupancy map is wanted for.

The loss

SemanticAttractionLoss (mmdet3d/models/losses/semantic_attraction_loss.py) adds an explicit 3D term. Pseudo-label pixels of the important classes are unprojected into ego coordinates with the pseudo-depth, and for each target point the K nearest Gaussians are pulled toward it under a Huber penalty, weighted by a soft nearest-neighbour term and by how much each Gaussian already predicts that class. Gradients reach the Gaussian means and the semantic head at once, so a recruited Gaussian both moves and changes what it claims to be.

Three choices are documented in the module docstring because the code alone does not explain them. Recruitment is not hard-gated on the predicted class, otherwise a class with no Gaussian nearby can never acquire one. The distance penalty is Huber rather than squared, because the monocular pseudo-depth is metres off on small objects and one outlier under a squared penalty would drag Gaussians to nonsense. And the term is applied before the temporal module, so it does not compete with the temporal offsets for the same displacement.

The setting stays weakly supervised. No 3D ground truth is used for training.

Usage

Install and prepare data exactly as the base method describes below, then train with the config that enables the loss:

./tools/dist_train.sh configs/gaussianflowocc_attract.py num_gpu
python tools/test.py configs/gaussianflowocc_attract.py work_dirs/gaussianflowocc_attract/epoch_18_ema.pth --eval mIoU

configs/gaussianflowocc_attract.py inherits the baseline config and adds the loss, so the architecture, schedule and supervision are otherwise identical.

What this repository changes

Everything outside the following is the upstream code, byte for byte.

file	change
mmdet3d/models/losses/semantic_attraction_loss.py	new, the loss
configs/gaussianflowocc_attract.py	new, config enabling it
mmdet3d/models/detectors/gaussianflowocc.py	accepts and calls the loss
mmdet3d/models/losses/__init__.py	registers the loss

---

Setup

The code is a full fork, so the upstream instructions apply unchanged. Follow the GaussianFlowOcc README for these steps:

1. Installation -- conda env, PyTorch with CUDA 11.3, mmcv, mmdet, gsplat and GroundedSAM.
2. Data preparation -- nuScenes, nuScenes-panoptic and the Occ3D-nuScenes ground truth, then python tools/create_data_bevdet.py.
3. Pseudo-labels -- Metric3D depth with tools/generate_m3d_nusc.py and GroundedSAM semantics with groundedsam/generate_grounded_sam.py.

Then train with the attraction config as shown under Usage above. Evaluation, RayIoU and resuming work exactly as upstream documents them.

The base method

GaussianFlowOcc: Sparse and Weakly Supervised Occupancy Estimation using Gaussian Splatting and Temporal Flow. Simon Boeder, Fabian Gigengack and Benjamin Risse. Bosch Research and University of Münster, ICCV 2025. paper · arXiv · code

All of the modelling in this repository is theirs. This fork adds one loss term on top of it.

Copyright

Copyright (c) 2022 Robert Bosch GmbH Copyright (c) 2026 VIPLAB, Gachon University SPDX-License-Identifier: AGPL-3.0

This repository is a modified fork of boschresearch/GaussianFlowOcc. Modifications made in 2026 add SemanticAttractionLoss, register it, wire it into the detector, and add configs/gaussianflowocc_attract.py. The modified files are listed under "What this repository changes" above. Licensed under AGPL-3.0, the same terms as the original.