Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. Mesh Neural Cellular Automata
 
conference paper

Mesh Neural Cellular Automata

Pajouheshgar, Ehsan  
•
Xu, Yitao  
•
Mordvintsev, Alexander
Show more
July 19, 2024
ACM SIGGRAPH Conference

Texture modeling and synthesis are essential for enhancing the realism of virtual environments. Methods that directly synthesize textures in 3D offer distinct advantages to the UV-mapping-based methods as they can create seamless textures and align more closely with the ways textures form in nature. We propose Mesh Neural Cellular Automata (MeshNCA), a method that directly synthesizes dynamic textures on 3D meshes without requiring any UV maps. MeshNCA is a generalized type of cellular automata that can operate on a set of cells arranged on non-grid structures such as the vertices of a 3D mesh. MeshNCA accommodates multi-modal supervision and can be trained using different targets such as images, text prompts, and motion vector fields. Only trained on an Icosphere mesh, MeshNCA shows remarkable test-time generalization and can synthesize textures on unseen meshes in real time. We conduct qualitative and quantitative comparisons to demonstrate that MeshNCA outperforms other 3D texture synthesis methods in terms of generalization and producing high-quality textures. Moreover, we introduce a way of grafting trained MeshNCA instances, enabling interpolation between textures. MeshNCA allows several user interactions including texture density/orientation controls, grafting/regenerate brushes, and motion speed/direction controls. Finally, we implement the forward pass of our MeshNCA model using the WebGL shading language and showcase our trained models in an online interactive demo, which is accessible on personal computers and smartphones and is available at https://meshnca.github.io/.

  • Files
  • Details
  • Metrics
Type
conference paper
DOI
https://doi.org/10.1145/3658127
Author(s)
Pajouheshgar, Ehsan  

EPFL

Xu, Yitao  

EPFL

Mordvintsev, Alexander

Google (Switzerland)

Niklasson, Eyvind
Zhang, Tong  

EPFL

Süsstrunk, Sabine  

EPFL

Date Issued

2024-07-19

Publisher

Association for Computing Machinery (ACM)

Published in
ACM Transactions on Graphics
Volume

43

Issue

4

Start page

1

End page

16

Subjects

Neural Cellular Automata

•

3D Texture Synthesis

•

Mesh Texturing

•

Dynamic Textures

•

Interactive Texture Synthesis

•

Self-Organization

•

Texture Interpolation and Grafting

•

Real-Time Texturing

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
IVRL  
Event nameEvent acronymEvent placeEvent date
ACM SIGGRAPH Conference

SIGGRAPH

Denver, Colorado, USA

2024-07-28 - 2024-08-01

FunderFunding(s)Grant NumberGrant URL

Swiss National Science Foundation

Sinergia grant

CRSII5-180359

Available on Infoscience
November 11, 2024
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/242002
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés