Code for point cloud segmentation visualization

Hi authors. Thank you for the amazing work.

Is there a python script for visualizing the point cloud semantic segmentation result?

After training the model on SemanticKITTI or nuScenes dataset, the trained model is saved (e.g. "best_miou_iter_6000.pth"), and I want to use this model parameter file to visualize the point cloud segmentation result on the validation set of SemanticKITTI / nuScenes dataset for better understanding and comparisons.

[ldkong1205]

Hi @iwajef, thanks for your interest in our work!

I can provide you a script for drawing the visualizations shown in the paper later. For now, you can refer to the visualization tool in MMDetection3D:
https://mmdetection3d.readthedocs.io/en/latest/user_guides/visualization.html#drawing-3d-semantic-mask

You can also take a look at the visualization interface from Open3D-ML:
https://github.com/isl-org/Open3D-ML

I can provide you a script for drawing the visualizations shown in the paper later. For now, you can refer to the visualization tool in MMDetection3D: https://mmdetection3d.readthedocs.io/en/latest/user_guides/visualization.html#drawing-3d-semantic-mask

Thank you for replying! I tried to follow the visualization guideline in MMDection3D, with the command:
python tools/test.py configs/lasermix/lasermix_cy3d_semi_nuscenes_10.py work_dirs/lasermix_cy3d_semi_nuscenes_10/best_miou_iter_6000.pth --show --show-dir ./visualize_result --task lidar_seg

But it exited with some error after showing first few frames as follows:

06/16 21:20:17 - mmengine - WARNING - The prefix is not set in metric class SegMetric.
Loads checkpoint by local backend from path: work_dirs/lasermix_cy3d_semi_nuscenes_10/best_miou_iter_6000.pth
06/16 21:20:21 - mmengine - INFO - Load checkpoint from work_dirs/lasermix_cy3d_semi_nuscenes_10/best_miou_iter_6000.pth
[Open3D WARNING] [ViewControl] ConvertFromPinholeCameraParameters() failed because window height and width do not match.
[Open3D WARNING] [ViewControl] ConvertFromPinholeCameraParameters() failed because window height and width do not match.
[Open3D WARNING] [ViewControl] ConvertFromPinholeCameraParameters() failed because window height and width do not match.
[Open3D WARNING] [ViewControl] ConvertFromPinholeCameraParameters() failed because window height and width do not match.
[Open3D WARNING] [ViewControl] ConvertFromPinholeCameraParameters() failed because window height and width do not match.
Traceback (most recent call last):
File "tools/test.py", line 162, in
main()
File "tools/test.py", line 158, in main
runner.test()
File "/home/user/anaconda3/envs/lasermix/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1823, in test
metrics = self.test_loop.run() # type: ignore
File "/home/user/anaconda3/envs/lasermix/lib/python3.8/site-packages/mmengine/runner/loops.py", line 445, in run
self.run_iter(idx, data_batch)
File "/home/user/anaconda3/envs/lasermix/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/home/user/anaconda3/envs/lasermix/lib/python3.8/site-packages/mmengine/runner/loops.py", line 466, in run_iter
self.runner.call_hook(
File "/home/user/anaconda3/envs/lasermix/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1839, in call_hook
getattr(hook, fn_name)(self, **kwargs)
File "/home/user/code/gits/1_sota/LaserMix/mmdet3d/engine/hooks/visualization_hook.py", line 228, in after_test_iter
self._visualizer.add_datasample(
File "/home/user/anaconda3/envs/lasermix/lib/python3.8/site-packages/mmengine/dist/utils.py", line 427, in wrapper
return func(*args, **kwargs)
File "/home/user/code/gits/1_sota/LaserMix/mmdet3d/visualization/local_visualizer.py", line 1042, in add_datasample
self._draw_pts_sem_seg(data_input['points'],
File "/home/user/code/gits/1_sota/LaserMix/mmdet3d/visualization/local_visualizer.py", line 771, in _draw_pts_sem_seg
pts_color = palette[pts_sem_seg]
IndexError: index 16 is out of bounds for axis 0 with size 16`

And when trying to visualize on SemanticKITTI, I got another error:

06/16 21:34:24 - mmengine - WARNING - The prefix is not set in metric class SegMetric.
Loads checkpoint by local backend from path: work_dirs/lasermix_cy3d_semi_semantickitti_10/best_miou_iter_3000.pth
06/16 21:34:27 - mmengine - INFO - Load checkpoint from work_dirs/lasermix_cy3d_semi_semantickitti_10/best_miou_iter_3000.pth
Traceback (most recent call last):
File "tools/test.py", line 162, in
main()
File "tools/test.py", line 158, in main
runner.test()
File "/home/user/anaconda3/envs/lasermix/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1823, in test
metrics = self.test_loop.run() # type: ignore
File "/home/user/anaconda3/envs/lasermix/lib/python3.8/site-packages/mmengine/runner/loops.py", line 445, in run
self.run_iter(idx, data_batch)
File "/home/user/anaconda3/envs/lasermix/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/home/user/anaconda3/envs/lasermix/lib/python3.8/site-packages/mmengine/runner/loops.py", line 466, in run_iter
self.runner.call_hook(
File "/home/user/anaconda3/envs/lasermix/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1839, in call_hook
getattr(hook, fn_name)(self, **kwargs)
File "/home/user/code/gits/1_sota/LaserMix/mmdet3d/engine/hooks/visualization_hook.py", line 228, in after_test_iter
self._visualizer.add_datasample(
File "/home/user/anaconda3/envs/lasermix/lib/python3.8/site-packages/mmengine/dist/utils.py", line 427, in wrapper
return func(*args, **kwargs)
File "/home/user/code/gits/1_sota/LaserMix/mmdet3d/visualization/local_visualizer.py", line 980, in add_datasample
keep_index = data_sample.gt_pts_seg.pts_semantic_mask != ignore_index # noqa: E501
AttributeError: 'PointData' object has no attribute 'pts_semantic_mask'

I think I might be missing some details to visualize the results correctly and I'll try to fix the errors. Anyway thanks for the suggestions and I'm looking forward to the visualization script :)

[Odelllll]

Hi, I encountered a similar problem when I followed the visualization guide in MMDection3D to try to visualize on the semantickitti dataset: AttributeError: 'PointData' object has no attribute 'pts_semantic_mask', What should I do to deal this problem?

Thank you very much.

[ldkong1205]

Hi @Odelllll, for the mentioned problem, you can try to resolve it by checking:

• Whether you are using the provided .pkl files? If not, you can download them from here.
• Whether you have added the following in your config file:

dict(type='LoadAnnotations3D',
     with_bbox_3d=False, with_label_3d=False, with_seg_3d=True, seg_3d_dtype='np.uint8', backend_args=backend_args
),

If the above two do not solve your problem, I recommend you consider using the following codebase for LiDAR visualization:
https://github.com/PRBonn/lidar-visualizer

[Odelllll]

Thank you very much for your suggestion, I will try it!