Panoptic-CUDAL: Rural Australia Point Cloud Dataset in Rainy Conditions

Tzu-Yun (Darryl) Tseng¹, Alexey Nekrasov², Malcolm Burdorf², Bastian Leibe², Julie Stephany Berrio Perez¹, Mao Shan¹, Zhenxing Ming¹, Stewart Worrall¹

¹The University of Sydney ²RWTH Aachen University

Panoptic Autonomous Driving Dataset for Rural Rainy Environments. This paper is accepted to IEEE ITSC 2025.

About

Existing autonomous driving datasets are predominantly oriented towards well-structured urban settings and favourable weather conditions, leaving the complexities of rural environments and adverse weather conditions largely unaddressed. Although some datasets encompass variations in weather and lighting, bad weather scenarios do not appear often. Rainfall can significantly impair sensor functionality, introducing noise and reflections in LiDAR and camera data and reducing the system’s capabilities for reliable environmental perception and safe navigation. This paper introduces the Panoptic-CUDAL dataset, a novel dataset purpose-built for panoptic segmentation in rural areas subject to rain. By recording high-resolution LiDAR, camera, and pose data, Panoptic-CUDAL offers a diverse, information-rich dataset in a challenging scenario. We present the analysis of the recorded data and provide baseline results for panoptic, se- mantic segmentation, and 3D occupancy prediction methods on LiDAR point clouds.

Data

• Images, calib.txt, metadata (.pkl) on Hugging Face.
• LiDAR, labels, poses on RWTH server.

The dataset is organized in a directory structure in SemanticKITTI format:

├── 30/
│   ├── poses.txt
│   ├── calib.txt
│   ├── labels/
│   │     ├ 000000.label
│   │     └ 000001.label
|   ├── port_a_cam_0/
|   |     ├ 000000.png
|   |     ├ 000001.png
|   .
|   .
│   └── port_b_cam_1/
|   |     ├ 000000.png
|   |     ├ 000001.png
|   ├── port_d_cam_1/
|   |     ├ 000000.png
|   |     ├ 000001.png
|   |
│   └── velodyne/
│         ├ 000000.bin
│         └ 000001.bin
├── 31/
├── 32/
.
.
.
└── 41/

We closely follow the Semantic KITTI dataset structure and MMdetection3D to create .pkl files, and it should be possible to load it as is.

Usage

To project point clouds on images, please run our code implemented on SemanticKITTI API.

python proj_label.py

For inference, our code is based on FRNet.

python visualize.py --velodyne_dir /path/to/PanopticCUDAL/sequences/34/velodyne --label_dir /path/to/FRNet/predictions/34 --vis pred --out_dir /path/to/output/vis/04 --video_name pred_04.mp4 --fps 10

Acknowledgement

We gratefully acknowledge the open-source communities whose tools, datasets and code underpinned this work.

BibTeX

@article{tseng2025panoptic,
  title={Panoptic-CUDAL: Rural Australia Point Cloud Dataset in Rainy Conditions},
  author={Tseng, Tzu-Yun and Nekrasov, Alexey and Burdorf, Malcolm and Leibe, Bastian and Berrio, Julie Stephany and Shan, Mao and Ming, Zhenxing and Worrall, Stewart},
  journal={IEEE International Conference on Intelligent Transportation Systems (ITSC)},
  url={https://arxiv.org/abs/2503.16378v2}
  year={2025}
}