Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. Deformation-aware Unpaired Image Translation for Pose Estimation on Laboratory Animals
 
conference paper

Deformation-aware Unpaired Image Translation for Pose Estimation on Laboratory Animals

Li, Siyuan
•
Günel, Semih  
•
Ostrek, Mirela
Show more
September 1, 2020
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Conference on Computer Vision and Pattern Recognition (CVPR)

Our goal is to capture the pose of neuroscience model organisms, without using any manual supervision, to be able to study how neural circuits orchestrate behaviour. Human pose estimation attains remarkable accuracy when trained on real or simulated datasets consisting of millions of frames. However, for many applications simulated models are unrealistic and real training datasets with comprehensive annotations do not exist. We address this problem with a new sim2real domain transfer method. Our key contribution is the explicit and independent modeling of appearance, shape and poses in an unpaired image translation framework. Our model lets us train a pose estimator on the target domain by transferring readily available body keypoint locations from the source domain to generated target images. We compare our approach with existing domain transfer methods and demonstrate improved pose estimation accuracy on Drosophila melanogaster (fruit fly), Caenorhabditis elegans (worm) and Danio rerio (zebrafish), without requiring any manual annotation on the target domain and despite using simplistic off-the-shelf animal characters for simulation, or simple geometric shapes as models. Our new datasets, code, and trained models will be published to support future neuroscientific studies.

  • Files
  • Details
  • Metrics
Type
conference paper
DOI
10.1109/CVPR42600.2020.01317
Author(s)
Li, Siyuan
Günel, Semih  
Ostrek, Mirela
Ramdya, Pavan P  
Fua, Pascal  
Rhodin, Helge  
Date Issued

2020-09-01

Published in
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Start page

13155

End page

13165

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
CVLAB  
UPRAMDYA  
Event name
Conference on Computer Vision and Pattern Recognition (CVPR)
Available on Infoscience
June 3, 2020
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/169090
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés