Annotation with Rerun and Gradio

Performing annotation using Rerun and Gradio. This works for both single view and multiview camera setups

Uses Rerun to visualize, Gradio for an interactive UI, and Pixi for a easy installation

Installation

Using Pixi

Make sure you have the Pixi package manager installed

This is Linux only with an NVIDIA GPU.

git clone https://github.com/rerun-io/annotation-example.git
cd annotation-example
pixi run app

All commands can be listed using pixi task list

Usage

Gradio App - Single View

Single View mask tracking and depth estimation

pixi run app

Gradio App - Multi View

Multiview mask and triangulation

pixi run multiview-app

You can see all tasks by running pixi task list

Acknowledgements

Thanks to the original Sam2, and DepthAnything repos!

SAM2

@article{ravi2024sam2,
  title={SAM 2: Segment Anything in Images and Videos},
  author={Ravi, Nikhila and Gabeur, Valentin and Hu, Yuan-Ting and Hu, Ronghang and Ryali, Chaitanya and Ma, Tengyu and Khedr, Haitham and R{\"a}dle, Roman and Rolland, Chloe and Gustafson, Laura and Mintun, Eric and Pan, Junting and Alwala, Kalyan Vasudev and Carion, Nicolas and Wu, Chao-Yuan and Girshick, Ross and Doll{\'a}r, Piotr and Feichtenhofer, Christoph},
  journal={arXiv preprint arXiv:2408.00714},
  url={https://arxiv.org/abs/2408.00714},
  year={2024}
}

DepthAnythingV2

@article{depth_anything_v2,
  title={Depth Anything V2},
  author={Yang, Lihe and Kang, Bingyi and Huang, Zilong and Zhao, Zhen and Xu, Xiaogang and Feng, Jiashi and Zhao, Hengshuang},
  journal={arXiv:2406.09414},
  year={2024}
}

@inproceedings{depth_anything_v1,
  title={Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data}, 
  author={Yang, Lihe and Kang, Bingyi and Huang, Zilong and Xu, Xiaogang and Feng, Jiashi and Zhao, Hengshuang},
  booktitle={CVPR},
  year={2024}
}