Cheng Sun

About me

I am a research scientist at NVIDIA Research Taiwan, specializing in 3D reconstruction, understanding, and their practical applications. I earned my Ph.D. from the Institute of Information Systems and Applications at National Tsing Hua University.

Experience

2023/09 - Now NVIDIA research scientist, Taipei
2022/06 - 2022/12 Meta research scientist intern, Redmond
2018/09 - 2023/07 National Tsing Hua University, Hsinchu

Publications

Sparse Voxels Rasterization: Real-time High-fidelity Radiance Field Rendering

Cheng Sun, Jaesung Choe, Charles Loop, Wei-Chiu Ma, Yu-Chiang Frank Wang

Sparse Voxel, Rasterization, Volume Rendering, Volume Fusion

[Paper] [Code WIP]

Faithfully representing scenes by adaptive sparse voxels. Speedy rendering by rasterization. Seamlessly compatible with grid-based algorithms like volume fusion and marching cubes.

Neural-PBIR Reconstruction of Shape, Material, and Illumination

Cheng Sun*, Guangyan Cai*, Zhengqin Li, Kai Yan, Cheng Zhang, Carl Marshall, Jia-Bin Huang, Shuang Zhao, Zhao Dong

ICCV 2023

Object Shape, Physically-based Material, Lighting

[Paper] [Project Page]

Hashing Neural Video Decomposition with Multiplicative Residuals in Space-Time

Cheng-Hung Chan, Cheng-Yang Yuan, Cheng Sun, Hwann-Tzong Chen

ICCV 2023

Video Editing, Test-time Training

[Paper] [Code] [Project Page]

ImGeoNet: Image-induced Geometry-aware Voxel Representation for Multi-view 3D Object Detection

Tao Tu, Shun-Po Chuang, Yu-Lun Liu, Cheng Sun, Ke Zhang, Donna Roy, Cheng-Hao Kuo, Min Sun

ICCV 2023

Point Cloud, Object Detection, 2D-3D Fusion

[Project Page]

Multiview Regenerative Morphing with Dual Flows

Chih-Jung Tsai, Cheng Sun, Hwann-Tzong Chen

ECCV 2022

3D Morphing, Grid-based NeRF

[Paper] [Code] [Project Page]

Data Efficient 3D Learner via Knowledge Transferred from 2D Model

Ping-Chung Yu, Cheng Sun, Min Sun

ECCV 2022

Point Cloud, Semantic Segmentation, Pre-training, Semi-supervised Learning

[Paper] [Code]

Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction

Cheng Sun, Min Sun, Hwann-Tzong Chen

CVPR 2022 (oral)

Grid-based NeRF

[Paper] [Paper v2] [Code] [Project Page] [Video]

It's one of the first method to speedup NeRF optimization using grid-based models. NeRF takes 20+ hours to optimize a scene while our v2 takes only 5 minutes with comparable quality.

Specialize and Fuse: Pyramid Output Representation for Semantic Segmentation

Chi-Wei Hsiao*, Cheng Sun*, Hwann-Tzong Chen, Min Sun

ICCV 2021

Semantic Segmentation

[Paper]

HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Features

Cheng Sun, Min Sun, Hwann-Tzong Chen

CVPR 2021

360 Panorama, Semantic Segmentation, Depth, Room layout

[Paper] [Code] [Project Page] [Video]

Indoor Panorama Planar 3D Reconstruction via Divide and Conquer

Cheng Sun, Chi-Wei Hsiao, Ning-Hsu Wang, Min Sun, Hwann-Tzong Chen

CVPR 2021 (oral)

360 Panorama, Planes Detection, New Dataset

[Paper] [Code] [Video]

Leveraging Auxiliary Information from EMR for Weakly Supervised Pulmonary Nodule Detection

Hao-Hsiang Yang, Fu-En Wang, Cheng Sun, Kuan-Chih Huang, Hung-Wei Chen, Yi Chen, Hung-Chih Chen, Chun-Yu Liao, Shih-Hsuan Kao, Yu-Chiang Frank Wang, Chou-Jin Lan

MICCAI 2021

MRI, 3D Detection, Weakly-supervised Learning

[Paper]

360-Indoor: Towards Learning Real-World Objects in 360° Indoor Equirectangular Images

Shih-Han Chou, Cheng Sun, Wen-Yen Chang, Wan-Ting Hsu, Min Sun, Jianlong Fu

WACV 2020

360 Panorama, Object Detection, New Dataset

[Paper] [Dataset] [Project Page]

Flat2Layout: Flat Representation for Estimating Layout of General Room Types

Chi-Wei Hsiao, Cheng Sun, Min Sun, Hwann-Tzong Chen

Room Layout

[Paper]

HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentation

Cheng Sun, Chi-Wei Hsiao, Min Sun, Hwann-Tzong Chen

CVPR 2019

360 Panorama, Room layout, Data Augmentation

[Paper] [Code] [Project Page]

We introduced a novel 360 room layout parameterization, which is now widely adopted in this task. It won the 1st place in ECCV 2020 Holistic 3D Vision Challenge. It was also adopted as the strong baseline in Kujiale's Structure3D and Zillow's ZInD official benchmarks.

DLWV2: a Deep Learning-based Wearable Vision-system with Vibrotactile-feedback for Visually Impaired People to Reach Objects

Meng-Li Shih, Yi-Chun Chen, Chia-Yu Tung, Cheng Sun, Ching-Ju Cheng, Liwei Chan, Srenivas Varadarajan, Min Sun

IROS 2018

Object Detection, HCI

[Paper]