Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. InNeRF360: Text-Guided 3D-Consistent Object Inpainting on 360° Neural Radiance Fields
 
conference paper

InNeRF360: Text-Guided 3D-Consistent Object Inpainting on 360° Neural Radiance Fields

Wang, Dongqing  
•
Zhang, Tong  
•
Abboud, Alaa
Show more
June 16, 2024
2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition CVPR 2024

We propose InNeRF360, an automatic system that accurately removes text-specified objects from 360 • Neural Radiance Fields (NeRF). The challenge is to effectively remove objects while inpainting perceptually consistent content for the missing regions, which is particularly demanding for existing NeRF models due to their implicit volumetric representation. Moreover, unbounded scenes are more prone to floater artifacts in the inpainted region than frontal-facing scenes, as the change of object appearance and background across views is more sensitive to inaccurate segmentations and inconsistent inpainting. With a trained NeRF and a text description, our method efficiently removes specified objects and inpaints visually consistent content without artifacts. We apply depth-space warping to enforce consistency across multiview text-encoded segmentations, and then refine the inpainted NeRF model using perceptual priors and 3D diffusion-based geometric priors to ensure visual plausibility. Through extensive experiments in segmentation and inpainting on 360 • and frontal-facing NeRFs, we show that our approach is effective and enhances NeRF's editability. Project page: https://ivrl.github.io/ InNeRF360/.

  • Files
  • Details
  • Metrics
Type
conference paper
DOI
10.1109/CVPR52733.2024.01205
Author(s)
Wang, Dongqing  

EPFL

Zhang, Tong  

EPFL

Abboud, Alaa

École Polytechnique Fédérale de Lausanne

Süsstrunk, Sabine  

EPFL

Date Issued

2024-06-16

Publisher

IEEE

Publisher place

Los Alamitos, CA

Published in
2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
DOI of the book
10.1109/CVPR52733.2024
ISBN of the book

979-8-3503-5300-6

Start page

12677

End page

12686

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
IVRL  
Event nameEvent acronymEvent placeEvent date
2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition CVPR 2024

CVPR

Seattle, WA, USA

2024-06-17 - 2024-06-21

FunderFunding(s)Grant NumberGrant URL

Swiss National Science Foundation

CRSII5-180359

Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/241439
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés