Shunsuke Saito

Reality Labs Research

Name Shunsuke Saito (齋藤 隼介)
Interest Effective Data Representations for Deep Human Digitization
Email {firstname}.{lastname}16 (at)

Shunsuke Saito

I am a Research Scientist at Reality Labs Research in Pittsburgh, where I work on next generation digital humans (relighting, faces, hands, full-body, hair, clothing, and accessories). For example, I am one of the core contributers of Codec Avatar 2.0, which was featured in Meta Connect 2022 and Lex Fridman's metaverse interview with MZ. Before that, I completed my Ph.D. at University of Southern California. If you are interested in an internship with us, feel free to email/DM me!

Diffusion Shape Prior for Wrinkle-Accurate Cloth Registration
3DV 2024    [Project (code coming soon)]
Jingfan Guo, Fabian Prada, Donglai Xiang, Javier Romero, Chenglei Wu, Hyun Soo Park, Takaaki Shiratori, Shunsuke Saito

RelightableHands can be useful
for boosting monocular two-hand estimation!

A Dataset of Relighted 3D Interacting Hands
NeruIPS 2023 (Datasets and Benchmark Track)    [Project (code, data)]
Gyeongsik Moon, Shunsuke Saito, Weipeng Xu, Rohan Joshi, Julia Buffalini, Harley Bellan, Nicholas Rosen, Jesse Richardson, Mallorie Mize, Philippe de Bree, Tomas Simon, Bo Peng, Shubham Garg, Kevyn McPhail, Takaaki Shiratori

Diffusion Posterior Illumination for Ambiguity-aware Inverse Rendering
ACM Transaction on Graphics (SIGGRAPH Asia 2023)    [Project (video, code)]
Linjie Lyu, Ayush Tewari, Marc Habermann, Shunsuke Saito, Michael Zollhoefer, Thomas Leimkuehler, and Christian Theobalt

Single-Image 3D Human Digitization with Shape-guided Diffusion
SIGGRAPH Asia 2023      [Project]
Badour AlBahar, Shunsuke Saito, Hung-Yu Tseng, Changil Kim, Johannes Kopf, and Jia-Bin Huang

NCHO : Unsupervised Learning for Neural 3D Composition of Humans and Objects
ICCV 2023      [Project (video, code, data)]
Taeksoo Kim, Shunsuke Saito, Hanbyul Joo

CT2Hair: High-Fidelity 3D Hair Modeling using Computed Tomography
ACM Transaction on Graphics (SIGGRAPH 2023)      [Project (video, code, data)]
Yuefan Shen, Shunsuke Saito, Ziyan Wang, Olivier Maury, Chenglei Wu, Jessica Hodgins, Youyi Zheng, Giljoo Nam

RelightableHands: Efficient Neural Relighting of Articulated Hand Models
CVPR 2023      [Project (video)]
Shun Iwase, Shunsuke Saito, Tomas Simon, Stephen Lombardi, Timur Bagautdinov, Rohan Joshi, Fabian Prada, Takaaki Shiratori, Yaser Sheikh, Jason Saragih

MEGANE: Morphable Eyeglass and Avatar Network
CVPR 2023    [Project (video)]
Junxuan Li, Shunsuke Saito, Tomas Simon, Stephen Lombardi, Hongdong Li, Jason Saragih

Dressing Avatars: Deep Photorealistic Appearance for Physically Simulated Clothing
ACM Transaction on Graphics (SIGGRAPH Asia 2022)    [Project (video, supmat)]
Donglai Xiang, Timur Bagautdinov, Tuur Stuyck, Fabian Prada, Javier Romero, Weipeng Xu, Shunsuke Saito, Jingfan Guo, Breannan Smith, Takaaki Shiratori, Yaser Sheikh, Jessica Hodgins, Chenglei Wu

KeypointNeRF: Generalizing Image-based Volumetric Avatars using Relative Spatial Encoding of Keypoints
ECCV 2022      [Project (video, code)]
Marko Mihajlovic, Aayush Bansal, Michael Zollhöfer, Siyu Tang, Shunsuke Saito

AutoAvatar: Autoregressive Neural Fields for Dynamic Avatar Modeling
ECCV 2022    [Project (video, code)]
Ziqian Bai, Timur Bagautdinov, Javier Romero, Michael Zollhöfer, Ping Tan, Shunsuke Saito

Neural Strands: Learning Hair Geometry and Appearance from Multi-View Images
ECCV 2022      [Project (video)]
Radu Alexandru Rosu, Shunsuke Saito, Ziyan Wang, Chenglei Wu, Sven Behnke, Giljoo Nam

Authentic Volumetric Avatars From a Phone Scan
ACM Transaction on Graphics (SIGGRAPH 2022)    
Chen Cao, Tomas Simon, Jin Kyu Kim, Gabe Schwartz, Michael Zollhoefer, Shunsuke Saito, Stephen Lombardi, Shih-en Wei, Danielle Belko, Shoou-i Yu, Yaser Sheikh, Jason Saragih

Drivable Volumetric Avatars Using Texel-aligned Features
SIGGRAPH 2022 (Conference Track)    [Code]
Edoardo Remelli, Timur Bagautdinov, Shunsuke Saito, Chenglei Wu, Tomas Simon, Shih-En Wei, Kaiwen Guo, Zhe Cao, Fabian Prada, Jason Saragih, Yaser Sheikh

COAP: Compositional Articulated Occupancy of People
CVPR 2022      [Project (code)]
Marko Mihajlovic, Shunsuke Saito, Aayush Bansal, Michael Zollhoefer, Siyu Tang

Neural Fields in Visual Computing and Beyond
Eurographics STAR 2022    [Search Engine]
Yiheng Xie, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, Srinath Sridhar

ARCH++: Animation-Ready Clothed Human Reconstruction Revisited
ICCV 2021      [Project (results)]
Tong He*, Yuanlu Xu*, Shunsuke Saito, Stefano Soatto, Tony Tung (* - equal contribution)

Deep Relightable Appearance Models for Animatable Faces
SIGGRAPH 2021      [Project (video, paper)]
Sai Bi, Stephen Lombardi*, Shunsuke Saito*, Tomas Simon, Shih-En Wei, Kevyn McPhail, Ravi Ramamoorthi, Yaser Sheikh, Jason Saragih (* - equal contribution)

SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks
CVPR 2021 (Oral Presentation)      [Project (video, code)]
CVPR Best Paper Nominee!!
Shunsuke Saito, Jinlong Yang, Qianli Ma, Michael J. Black

SCALE: Learning to Model Clothed 3D Humans with a Surface Codec of Articulated Local Elements
CVPR 2021      [Project (video, code)]
Qianli Ma, Shunsuke Saito*, Jinlong Yang*, Siyu Tang, Michael J. Black (* - equal contribution)

Pixel-Aligned Volumetric Avatars
CVPR 2021    [Project (video)]
Amit Raj, Michael Zollhöfer, Tomas Simon, Jason Saragih, Shunsuke Saito, James Hays, Stephen Lombardi

Monocular Real-Time Volumetric Performance Capture
ECCV 2020, SIGGRAPH RTL (Best in Show Award)    [Project (video, code)]
Ruilong Li*, Yuliang Xiu*, Shunsuke Saito, Zeng Huang, Kyle Olszewski, Hao Li (* - equal contribution)

PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization
CVPR 2020 (Oral Presentation)    [Project (video, code)]
Shunsuke Saito, Tomas Simon, Jason Saragih, Hanbyul Joo

Learning to Infer Implicit Surfaces without 3D Supervision
NeurIPS 2019    
Shichen Liu, Shunsuke Saito, Weikai Chen, Hao Li

PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization
ICCV 2019      [Project]
Shunsuke Saito*, Zeng Huang*, Ryota Natsume*, Shigeo Morishima, Angjoo Kanazawa, Hao Li (* - equal contribution)

SiCloPe: Silhouette-Based Clothed People
CVPR 2019 (Oral Presentation - Best Paper Finalist)      [Supplemental]
Ryota Natsume*, Shunsuke Saito*, Zeng Huang, Weikai Chen, Chongyang Ma, Hao Li, Shigeo Morishima (* - equal contribution)

3D Hair Synthesis Using Volumetric Variational Autoencoders
ACM Transaction on Graphics (SIGGRAPH Asia 2018)    
Shunsuke Saito, Liwen Hu, Chongyang Ma, Hikaru Ibayashi, Linjie Luo, Hao Li

paGAN: Real-time Avatars Using Dynamic Textures
ACM Transaction on Graphics (SIGGRAPH Asia 2018)    
Koki Nagano, Jaewoo Seo, Jun Xing, Lingyu Wei, Zimo Li, Shunsuke Saito, Aviral Agarwal, Jens Fursund, Hao Li

High-Fidelity Facial Reflectance and Geometry Inference From an Unconstrained Image
ACM Transaction on Graphics (SIGGRAPH 2018)    
Shugo Yamaguchi*, Shunsuke Saito*, Koki Nagano, Yajie Zhao, Weikai Chen, Kyle Olszewski, Shigeo Morishima, Hao Li (* - equal contribution)

Mesoscopic Facial Geometry inference Using Deep Neural Networks
CVPR 2018 (Spotlight Presentation)  
Loc Huynh, Weikai Chen, Shunsuke Saito, Jun Xing, Koki Nagano, Andrew Jones, Hao Li, Paul Debevec

Avatar Digitization from a Single Image for Real-Time Rendering
ACM Transaction on Graphics (SIGGRAPH Asia 2017)    
Liwen Hu*, Shunsuke Saito*, Lingyu Wei*, Koki Nagano, Jaewoo Seo, Iman Sadeghi, Jens Fursund, Yen-Chun Chen, Hao Li (* - equal contribution)

garment transfer from
human to animals!

Quasi-Developable Garment Transfer for Animals
SIGGRAPH ASIA 2017 Technical Brief  
Fumiya Narita, Shunsuke Saito, Tsukasa Fukusato, Shigeo Morishima

Outside-in Monocular IR Camera based HMD Pose Estimation via Geometric Optimization
VRST 2017    
Pavel A. Savkin, Shunsuke Saito, Jarich Vansteenberge, Tsukasa Fukusato, Lochlainn Wilson, Shigeo Morishima

Learning Dense Facial Correspondences in Unconstrained Images
ICCV 2017    
Ronald Yu, Shunsuke Saito, Haoxiang Li, Duygu Ceylan, Hao Li

Realistic Dynamic Facial Textures From a Single Image Using GANs
ICCV 2017    
Kyle Olszewski*, Zimo Li*, Chao Yang*, Yi Zhou, Ronald Yu, Zeng Huang, Sitao Xiang, Shunsuke Saito, Pushmeet Kohli, Hao Li (* - equal contribution)

Production-Level Facial Performance Capture Using Deep Convolutional Neural Networks
SCA 2017    
Samuli Laine, Tero Karras, Timo Aila, Antti Herva, Shunsuke Saito, Ronald Yu, Hao Li, Jaakko Lehtinen

Photorealistic Facial Texture Inference Using Deep Neural Networks
CVPR 2017 (Spotlight Presentation)    
Shunsuke Saito*, Lingyu Wei*, Liwen Hu, Koki Nagano, Hao Li (* - equal contribution)
featured in fx guide, Gizmodo, Daily Mail, Seamless, 3ders, and Wired

Multi-View Stereo on Consistent Face Topology
Eurographics 2017    
Graham Fyffe, Koki Nagano, Loc Huynh, Shunsuke Saito, Jay Bush, Andrew Jones, Hao Li, Paul Debevec

High-Fidelity Facial and Speech Animation for VR HMDs
ACM Transaction on Graphics (SIGGRAPH Asia 2016)      
Kyle Olszewski, Joseph J. Lim, Shunsuke Saito, Hao Li

Real-Time Facial Segmentation and Performance Capture from RGB Input
ECCV 2016        [Data]
Shunsuke Saito, Tianye Li, Hao Li

Garment Transfer for Quadruped Characters
Eurographics 2016, short paper      
Fumiya Narita, Shunsuke Saito, Takuya Kato, Tsukasa Fukusato, Shigeo Morishima

Computational Bodybuilding: Anatomically-based Modeling of Human Bodies
ACM Transaction on Graphics (SIGGRAPH 2015)      
Shunsuke Saito, Zi-Ye Zhou, Ladislav Kavan

PatchMove: Patch-based Fast Image Interpolation with Greedy Bidirectional Correspondence
Pacific Graphics 2014, short paper      
Shunsuke Saito, Ryuuki Sakamoto, Shigeo Morishima

Macroscopic and Microscopic Deformation Coupling in Up-sampled Cloth Simulation
Computer Animation and Virtual Worlds Journal (CASA 2014)    
Shunsuke Saito, Nobuyuki Umetani, Shigeo Morishima

Character Transfer: Example-based individuality retargeting for facial animations
International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision 2014 (WSCG 2014)    
Takuya Kato, Shunsuke Saito, Masahide Kawai, Tomoyori Iwao, Akinobu Maejima, Shigeo Morishima

Estimating material parameters of cloth from 4D scans!

Estimating Cloth Simulation Parameter from 3D Cloth Motion Separating Dynamic from Static Motion
ACM SIGGRAPH EuroGraphics Symposium on Computer Animation (SCA 2012), poster  
Shunsuke Saito, Ryusuke Sagawa, Naoya Iwamoto, Shigeo Morishima

Shunsuke Saito

E-mail: shunsuke.saito16 (at)

Technical Paper Reviewer:



MPI Tübingen(2020), FAIR, Adobe Research (2019), Pinscreen (2018), FRL (2017), FOVE (2015), Yahoo! Japan (2012-13)


MPI Saarbruecken/Tuebingen, Industrial Light & Magic, Dagstuhl Seminar (2019), University of Tokyo, CMU Graphics Group, VC Symposium (2018), VC/GCAD Symposium, Autodesk Research (2015)

Scholarship and Awards:

Outstanding Reviewer (3DV2020, ACCV2020),
CVPR Best Paper Finalist (2019, 2021),
Super Creator, IPA Exploratory Software (2015),
ACM Student Research Competition Semi-Finalist (2014),
MIRU Interactive Presentation Award, Okayama, Japan (2014),
UCLA CSST Scholarship Program $9,036 (2013),
Okuma Memorial Special Scholarship $5,000 (2011)