Automated CAO segmentation

News

• 2025-03-06: Integrated Mobileone_S1 in the framework.
• 2025-03-05: Integrated a modified SAM2 in the framework.
• 2025-01-21: Automated segmentation framework with generic U-Net is built.

Workflow for monocular endoscopic segmentation and depth estimation

1. preprocess raw videos
2. use pretrained models to generate pseudo-labels, such as our pretrained model for segmentation and depth maps or pretrained foundation models (sam and depth-anything)
3. clean pseudo-labels
4. train

Getting ready

Installation

git clone https://github.com/ALISS-ARPAH/CAO_seg
cd CAO_seg
conda create -n cao_seg python=3.9
conda activate cao_seg
pip install -r requirements.txt

before training

1. preprocess data
2. generate data split for training
3. If you want to use pretrained weights of SAM2, download and place it under sam2_checkpoints folder

Get started

Train

python train.py --name <YOUR_RUNNING_NAME> --json_path <YOUR_SPLIT_PATH>

Test

python test.py --test_data_dir <YOUR_DATA_FOLDER> --name <YOUR_RUNNING_NAME>

To save the predictions --save_results --save_results_dir <YOUR_SAVE_RESULTS_DIR>

Either put pretrained file in the checkpoints/<--name>/cp/ (I suggest running train.py first for 1-2 minutes and folders will be created automatically)

or use --pretrained_path

Batch testing

python test_batch.py

You need to manually modify the arguments in this Python file. Currently, it only supports the frames stored in subfolders of a given parent folder.

parent_folder/
├── video1/
│   ├── frames/
├── video2/
│   ├── frames/
├── video3/
│   ├── frames/

Get it done

1. Check Python files in the post_analysis folder for generating overlayed videos or images.

License

The model is licensed under the Apache 2.0 license

Questions

Please send an email to hao.li.1@vanderbilt.edu or Slack me for any questions, and I am always happy to help! :)

If you find this repository useful, please consider citing this paper:

@inproceedings{li2025automated,
  title={Automated segmentation of central airway obstruction from endoscopic video stream with deep learning},
  author={Li, Hao and Wang, Jiacheng and Kumar, Nithin and d’Almeida, Jesse and Lu, Daiwei and Acar, Ayberk and Han, John and Yang, Qingyun and Ertop, Tayfun Efe and Wu, Jie Ying and others},
  booktitle={Medical Imaging 2025: Image-Guided Procedures, Robotic Interventions, and Modeling},
  volume={13408},
  pages={113--119},
  year={2025},
  organization={SPIE}
}