A Deep Learning-based 6DoF Video Synthesizing Method Using Instant-NGPs

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

This paper introduces a new method for synthesizing six degree of freedom (6DoF) videos using neural radiance fields, which allows training from plain 2D images to render 3D scene at arbitrary viewports. Neural network model representing a previous timepoint is fine-Tuned to train models for subsequent timepoints. Additionally, instant neural graphics primitives (Instant-NGP) is applied for speed improvement. The proposed method achieved both improved objective quality for same number of training iterations and enhanced consistency between frames. Furthermore, it shows superiority over other methods for generating 6DoF videos, in terms of quality and time efficiency.

Original languageEnglish
Title of host publication2023 IEEE International Conference on Visual Communications and Image Processing, VCIP 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9798350359855
DOIs
StatePublished - 2023
Event2023 IEEE International Conference on Visual Communications and Image Processing, VCIP 2023 - Jeju, Korea, Republic of
Duration: 4 Dec 20237 Dec 2023

Publication series

Name2023 IEEE International Conference on Visual Communications and Image Processing, VCIP 2023

Conference

Conference2023 IEEE International Conference on Visual Communications and Image Processing, VCIP 2023
Country/TerritoryKorea, Republic of
CityJeju
Period4/12/237/12/23

Keywords

  • 6DoF
  • Deep learning
  • Immersive video
  • Instant-NGP
  • MIV
  • NeRF
  • Virtual reality

Fingerprint

Dive into the research topics of 'A Deep Learning-based 6DoF Video Synthesizing Method Using Instant-NGPs'. Together they form a unique fingerprint.

Cite this