Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. DyNCA: Real-time Dynamic Texture Synthesis Using Neural Cellular Automata
 
conference paper

DyNCA: Real-time Dynamic Texture Synthesis Using Neural Cellular Automata

Pajouheshgar, Ehsan  
•
Xu, Yitao  
•
Zhang, Tong  
Show more
January 1, 2023
2023 Ieee/Cvf Conference On Computer Vision And Pattern Recognition (Cvpr)
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Current Dynamic Texture Synthesis (DyTS) models can synthesize realistic videos. However, they require a slow iterative optimization process to synthesize a single fixed-size short video, and they do not offer any post-training control over the synthesis process. We propose Dynamic Neural Cellular Automata (DyNCA), a framework for real-time and controllable dynamic texture synthesis. Our method is built upon the recently introduced NCA models and can synthesize infinitely long and arbitrary-sized realistic video textures in real time. We quantitatively and qualitatively evaluate our model and show that our synthesized videos appear more realistic than the existing results. We improve the SOTA DyTS performance by 2 similar to 4 orders of magnitude. Moreover, our model offers several real-time video controls including motion speed, motion direction, and an editing brush tool. We exhibit our trained models in an online interactive demo that runs on local hardware and is accessible on personal computers and smartphones.

  • Details
  • Metrics
Type
conference paper
DOI
10.1109/CVPR52729.2023.01987
Web of Science ID

WOS:001062531305008

Author(s)
Pajouheshgar, Ehsan  
Xu, Yitao  
Zhang, Tong  
Süsstrunk, Sabine  
Date Issued

2023-01-01

Publisher

Ieee Computer Soc

Publisher place

Los Alamitos

Published in
2023 Ieee/Cvf Conference On Computer Vision And Pattern Recognition (Cvpr)
ISBN of the book

979-8-3503-0129-8

Start page

20742

End page

20751

Subjects

Technology

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
IVRL  
Event nameEvent placeEvent date
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Vancouver, CANADA

JUN 17-24, 2023

FunderGrant Number

Swiss National Science Foundation via the Sinergia

CRSII5180359

Available on Infoscience
February 19, 2024
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/204028
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés