nv-tlabs.github.io
Open in
urlscan Pro
2606:50c0:8003::153
Public Scan
Submitted URL: http://nv-tlabs.github.io/
Effective URL: https://nv-tlabs.github.io/
Submission: On June 20 via api from GB — Scanned from GB
Effective URL: https://nv-tlabs.github.io/
Submission: On June 20 via api from GB — Scanned from GB
Form analysis 0 forms found in the DOM
Text Content
SEARCH * Home * News * Members * Research * Projects * Publications * Contact * * Light Dark Automatic TORONTO AI LAB INTRODUCTION Welcome to the homepage of the NVIDIA Toronto Artificial Intelligence Lab led by Professor Sanja Fidler. Our research group was founded in 2018, and is primarily based in Toronto. The research interests of our lab lie at the intersection of computer vision, machine learning and computer graphics. Our group members are also part of or closely collaborate with academic labs such as the University of Toronto and Vector Institute. We invite applications for the following positions: * full time research scientist * full time research engineer * research scientist intern * research engineer intern Graduate and senior undergraduate students interested in doing an internship in the NVIDIA Toronto AI Lab can directly fill this form. See this link for open positions, or contact our members for more details. NEWS April 2021 - Our work was presented at GTC 2021. December 2020 - New version of the website. May 2020 - 40 Years on, PAC-MAN Recreated with AI by NVIDIA Researchers December 2019 - 2D or Not 2D: NVIDIA Researchers Bring Images to Life with AI November 2019 - NVIDIA Makes 3D Deep Learning Research Easy with Kaolin PyTorch Library November 2019 - NVIDIA Research at ICCV: Generating New City Road Layouts with AI October 2019 - NVIDIA Research at ICCV: Meta-Sim: Learning to Generate Synthetic Datasets June 2019 - NVIDIA Research Released at CVPR Helps Developers Create Better Visual Datasets MEMBERS SANJA FIDLER HASSAN ABU ALHAIJA DAVID ACUNA WENZHENG CHEN CLEMENT FUJI TSANG JUN GAO ZAN GOJCIC AMLAN KAR SAMEH KHAMIS SEUNG WOOK KIM KARSTEN KREIS MARC LAW DAIQING LI HUAN LING OR LITANY JAMES LUCAS OR PEREL JONAH PHILION MASHA SHUGRINA TINGWU WANG FRANCIS WILLIAMS KEVIN XIE KANGXUE YIN AI RESIDENTS RAFID MAHMOOD CURRENT INTERNS MATAN ATZMON SOURAV BISWAS ZHIQIN CHEN TIM DOCKHORN MOHAMED HASSAN ALICE LI MICHAEL LI DEREK LIU JASON PENG DAVIS REMPE GOPAL SHARMA FRANK SHEN AO TANG ZIAN WANG JUNLIN YANG XIAOHUI ZENG RESEARCH TOPICS Our research combines machine learning, computer vision and computer graphics for innovative applications such as video games, simulations, film industry, data servers, medical imaging and self-driving cars. A non-exhaustive list of research topics studied in our group include: * Machine Learning for Computer Graphics: Differentiable rendering, 3D deep learning, computational shape analysis, manipulation of color distributions * Generative modeling: Synthetic data generation, dynamic environment simulation, 3D models. * Representation Learning: Semantic segmentation, road layout modeling, complex continuous data structures to represent hierarchies and graphs, motion planning, planner-centric metrics, numerical Optimization, federated simulation * Machine Learning with limited supervision: active learning, self-supervised learning, few-shot learning, domain adaptation, learning with noisy labels Here are some of our projects: HIGHLIGHTED PROJECTS Quickly discover relevant content by filtering publications. SCORE-BASED GENERATIVE MODELING WITH CRITICALLY-DAMPED LANGEVIN DIFFUSION ICLR 2022 (Spotlight) Score-based generative models (SGMs) have demonstrated remarkable synthesis quality. SGMs rely on a diffusion process that gradually … Tim Dockhorn, Arash Vahdat, Karsten Kreis PDF Code Project TACKLING THE GENERATIVE LEARNING TRILEMMA WITH DENOISING DIFFUSION GANS ICLR 2022 (Spotlight) A wide variety of deep generative models has been developed in the past decade. Yet, these models often struggle with simultaneously … Zhisheng Xiao, Karsten Kreis, Arash Vahdat PDF Code Project ATISS: AUTOREGRESSIVE TRANSFORMERS FOR INDOOR SCENE SYNTHESIS NeurIPS 2021 The ability to synthesize realistic and diverse indoor furniture layouts automatically or based on partial input, unlocks many … Despoina Paschalidou, Amlan Kar, Maria Shugrina, Karsten Kreis, Andreas Geiger, Sanja Fidler PDF Code Project Poster Video DEEP MARCHING TETRAHEDRA: A HYBRID REPRESENTATION FOR HIGH-RESOLUTION 3D SHAPE SYNTHESIS NeurIPS 2021 We introduce DMTet, a deep 3D conditional generative model that can synthesize high-resolution 3D shapes using simple user guides such … Tianchang Shen, Jun Gao, Kangxue Yin, Ming-Yu Liu, Sanja Fidler PDF Project Video SCORE-BASED GENERATIVE MODELING IN LATENT SPACE NeurIPS 2021 Score-based generative models (SGMs) have recently demonstrated impressive results in terms of both sample quality and distribution … Arash Vahdat, Karsten Kreis, Jan Kautz PDF Code Project See all publications PUBLICATIONS Quickly discover relevant content by filtering publications. * GENERATING USEFUL ACCIDENT-PRONE DRIVING SCENARIOS VIA A LEARNED TRAFFIC PRIOR CVPR 2022 Evaluating and improving planning for autonomous vehicles requires scalable generation of long-tail traffic scenarios. To be useful, … Davis Rempe, Jonah Philion, Leonidas Guibas, Sanja Fidler, Or Litany PDF Project LEARNING SMOOTH NEURAL FUNCTIONS VIA LIPSCHITZ REGULARIZATION SIGGRAPH 2022 Neural implicit fields have recently emerged as a useful representation for 3D shapes. These fields are commonly represented as neural … Derek Liu, Francis Williams, Alec Jacobson, Sanja Fidler, Or Litany PDF Project ASE: LARGE-SCALE REUSABLE ADVERSARIAL SKILL EMBEDDINGS FOR PHYSICALLY SIMULATED CHARACTERS SIGGRAPH 2022 The incredible feats of athleticism demonstrated by humans are made possible in part by a vast repertoire of general-purpose motor … Xue Bin Peng, Yunrong Guo, Lina Halper, Sergey Levine, Sanja Fidler PDF Project SCORE-BASED GENERATIVE MODELING WITH CRITICALLY-DAMPED LANGEVIN DIFFUSION ICLR 2022 (Spotlight) Score-based generative models (SGMs) have demonstrated remarkable synthesis quality. SGMs rely on a diffusion process that gradually … Tim Dockhorn, Arash Vahdat, Karsten Kreis PDF Code Project TACKLING THE GENERATIVE LEARNING TRILEMMA WITH DENOISING DIFFUSION GANS ICLR 2022 (Spotlight) A wide variety of deep generative models has been developed in the past decade. Yet, these models often struggle with simultaneously … Zhisheng Xiao, Karsten Kreis, Arash Vahdat PDF Code Project DOMAIN ADVERSARIAL TRAINING: A GAME PERSPECTIVE ICLR 2022 The dominant line of work in domain adaptation has focused on learning invariant representations using domain-adversarial training. In … David Acuna, Marc T. Law, Guojun Zhang, Sanja Fidler PDF LOW-BUDGET ACTIVE LEARNING VIA WASSERSTEIN DISTANCE: AN INTEGER PROGRAMMING APPROACH ICLR 2022 Active learning is the process of training a model with limited labeled data by selecting a core subset of an unlabeled data pool to … Rafid Mahmood, Sanja Fidler, Marc T. Law PDF SPECTRAL UNIONS OF PARTIAL DEFORMABLE 3D SHAPES Eurographics 2022 Spectral geometric methods have brought revolutionary changes to the field of geometry processing – however, when the data to be … Luca Moschella, Simone Melzi, Luca Cosmo, Filippo Maggioli, Or Litany, Maks Ovsjanikov, Leonidas Guibas, Emanuele Rodolà PDF CONTRAST TO DIVIDE: SELF-SUPERVISED PRE-TRAINING FOR LEARNING WITH NOISY LABELS WACV 2022 The success of learning with noisy labels (LNL) methods relies heavily on the success of a warm-up stage where standard supervised … Evgenii Zheltonozhskii, Chaim Baskin, Avi Mendelson, Alex M. Bronstein, Or Litany PDF Code ATISS: AUTOREGRESSIVE TRANSFORMERS FOR INDOOR SCENE SYNTHESIS NeurIPS 2021 The ability to synthesize realistic and diverse indoor furniture layouts automatically or based on partial input, unlocks many … Despoina Paschalidou, Amlan Kar, Maria Shugrina, Karsten Kreis, Andreas Geiger, Sanja Fidler PDF Code Project Poster Video DEEP MARCHING TETRAHEDRA: A HYBRID REPRESENTATION FOR HIGH-RESOLUTION 3D SHAPE SYNTHESIS NeurIPS 2021 We introduce DMTet, a deep 3D conditional generative model that can synthesize high-resolution 3D shapes using simple user guides such … Tianchang Shen, Jun Gao, Kangxue Yin, Ming-Yu Liu, Sanja Fidler PDF Project Video SCORE-BASED GENERATIVE MODELING IN LATENT SPACE NeurIPS 2021 Score-based generative models (SGMs) have recently demonstrated impressive results in terms of both sample quality and distribution … Arash Vahdat, Karsten Kreis, Jan Kautz PDF Code Project TOWARDS OPTIMAL STRATEGIES FOR TRAINING SELF-DRIVING PERCEPTION MODELS IN SIMULATION NeurIPS 2021 Autonomous driving relies on a huge volume of real-world data to be labeled to high precision. Alternative solutions seek to exploit … David Acuna, Jonah Philion, Sanja Fidler PDF Project ULTRAHYPERBOLIC NEURAL NETWORKS NeurIPS 2021 (Spotlight) Riemannian space forms, such as the Euclidean space, sphere and hyperbolic space, are popular and powerful representation spaces in … Marc T. Law PDF DIB-R++: LEARNING TO PREDICT LIGHTING AND MATERIAL WITH A HYBRID DIFFERENTIABLE RENDERER NeurIPS 2021 We consider the challenging problem of predicting intrinsic object properties from a single image by exploiting differentiable … Wenzheng Chen, Joey Litalien, Jun Gao, Zian Wang, Clement Fuji Tsang, Sameh Khamis, Or Litany, Sanja Fidler PDF Project DON'T GENERATE ME: TRAINING DIFFERENTIALLY PRIVATE GENERATIVE MODELS WITH SINKHORN DIVERGENCE NeurIPS 2021 Although machine learning models trained on massive data have led to breakthroughs in several areas, their deployment in … Tianshi Cao, Alex Bie, Arash Vahdat, Sanja Fidler, Karsten Kreis PDF Project EDITGAN: HIGH-PRECISION SEMANTIC IMAGE EDITING NeurIPS 2021 Generative adversarial networks (GANs) have recently found applications in image editing. However, most GAN based image editing methods … Huan Ling, Karsten Kreis, Daiqing Li, Seung Wook Kim, Antonio Torralba, Sanja Fidler PDF Project MIX3D: OUT-OF-CONTEXT DATA AUGMENTATION FOR 3D SCENES 3DV 2021 (Oral) We present Mix3D, a data augmentation technique for segmenting large-scale 3D scenes. Since scene context helps reasoning about object … Alexey Nekrasov, Jonas Schult, Or Litany, Bastian Leibe, Francis Engelmann PDF Code Project LEARNING INDOOR INVERSE RENDERING WITH 3D SPATIALLY-VARYING LIGHTING ICCV 2021 (Oral) In this work, we address the problem of jointly estimating albedo, normals, depth and 3D spatially-varying lighting from a single … Zian Wang, Jonah Philion, Sanja Fidler, Jan Kautz PDF Project 3DSTYLENET: CREATING 3D SHAPES WITH GEOMETRIC AND TEXTURE STYLE VARIATIONS ICCV 2021 (Oral) We propose a method to create plausible geometric and texture style variations of 3D objects in the quest to democratize 3D content … Kangxue Yin, Jun Gao, Maria Shugrina, Sameh Khamis, Sanja Fidler PDF Project VECTOR NEURONS: A GENERAL FRAMEWORK FOR SO(3)-EQUIVARIANT NETWORKS ICCV 2021 (Oral) Invariance and equivariance to the rotation group have been widely discussed in the 3D deep learning community for pointclouds. Yet … Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, Leonidas Guibas PDF Code Project PHYSICS-BASED HUMAN MOTION ESTIMATION AND SYNTHESIS FROM VIDEOS ICCV 2021 Human motion synthesis is an important problem with applications in graphics, gaming and simulation environments for robotics. Existing … Kevin Xie, Tingwu Wang, Umar Iqbal, Yunrong Guo, Sanja Fidler, Florian Shkurti PDF Project SELF-SUPERVISED REAL-TO-SIM SCENE GENERATION ICCV 2021 Synthetic data is emerging as a promising solution to the scalability issue of supervised deep learning, especially when real data are … Aayush Prakash, Shoubhik Debnath, Jean-Francois Lafleche, Eric Cameracci, Gavriel State, Stan Birchfield, Marc T. Law PDF Project F-DOMAIN-ADVERSARIAL LEARNING: THEORY AND ALGORITHMS ICML 2021 Unsupervised domain adaptation is used in many machine learning applications where, during training, a model has access to unlabeled … David Acuna, Guojun Zhang, Marc T. Law, Sanja Fidler PDF Code Project IMAGE-LEVEL OR OBJECT-LEVEL? A TALE OF TWO RESAMPLING STRATEGIES FOR LONG-TAILED DETECTION ICML 2021 Training on datasets with long-tailed distributions has been challenging for major recognition tasks such as classification and … Nadine Chang, Zhiding Yu, Yu-Xiong Wang, Anima Anandkumar, Sanja Fidler, Jose M. Alvarez PDF DATASETGAN: EFFICIENT LABELED DATA FACTORY WITH MINIMAL HUMAN EFFORT CVPR 2021 (Oral) We introduce DatasetGAN: an automatic procedure to generate massive datasets of high-quality semantically segmented images requiring … Yuxuan Zhang, Huan Ling, Jun Gao, Kangxue Yin, Jean-Francois Lafleche, Adela Barriuso, Antonio Torralba, Sanja Fidler Project DRIVEGAN: TOWARDS A CONTROLLABLE HIGH-QUALITY NEURAL SIMULATION CVPR 2021 (Oral) Realistic simulators are critical for training and verifying robotics systems. While most of the contemporary simulators are … Seung Wook Kim, Jonah Philion, Antonio Torralba, Sanja Fidler PDF Project NEURAL GEOMETRIC LEVEL OF DETAIL: REAL-TIME RENDERING WITH IMPLICIT 3D SURFACES CVPR 2021 (Oral) Neural signed distance functions (SDFs) are emerging as an effective representation for 3D shapes. State-of-the-art methods typically … Towaki Takikawa, Joey Litalien, Kangxue Yin, Karsten Kreis, Charles Loop, Derek Nowrouzezahrai, Alec Jacobson, Morgan Mcguire, Sanja Fidler PDF Code Project Video WEAKLY SUPERVISED LEARNING OF RIGID 3D SCENE FLOW CVPR 2021 (Oral) We propose a data-driven scene flow estimation algorithm exploiting the observation that many 3D scenes can be explained by a … Zan Gojcic, Or Litany, Andreas Wieser, Leonidas J Guibas, Tolga Birdal PDF Project SEMANTIC SEGMENTATION WITH GENERATIVE MODELS: SEMI-SUPERVISED LEARNING AND STRONG OUT-OF-DOMAIN GENERALIZATION CVPR 2021 Training deep networks with limited labeled data while achieving a strong generalization ability is key in the quest to reduce human … Daiqing Li, Junlin Yang, Karsten Kreis, Antonio Torralba, Sanja Fidler PDF Project 3DIOUMATCH: LEVERAGING IOU PREDICTION FOR SEMI-SUPERVISED 3D OBJECT DETECTION CVPR 2021 3D object detection is an important yet demanding task that heavily relies on difficult to obtain 3D annotations. To reduce the … He Wang, Yezhen Cong, Or Litany, Yue Gao, Leonidas J. Guibas PDF Project IMAGE GANS MEET DIFFERENTIABLE RENDERING FOR INVERSE GRAPHICS AND INTERPRETABLE 3D NEURAL RENDERING ICLR 2021 (Oral) Differentiable rendering has paved the way to training neural networks to perform ‘inverse graphics’ tasks such as … Yuxuan Zhang, Wenzheng Chen, Huan Ling, Jun Gao, Yinan Zhang, Antonio Torralba, Sanja Fidler PDF Project VAEBM: A SYMBIOSIS BETWEEN VARIATIONAL AUTOENCODERS AND ENERGY-BASED MODELS ICLR 2021 (Spotlight) Energy-based models (EBMs) have recently been successful in representing complex distributions of small images. However, sampling from … Zhisheng Xiao, Karsten Kreis, Jan Kautz, Arash Vahdat PDF Code GRADSIM: DIFFERENTIABLE SIMULATION FOR SYSTEM IDENTIFICATION AND VISUOMOTOR CONTROL ICLR 2021 We consider the problem of estimating an object’s physical properties such as mass, friction, and elasticity directly from video … Krishna Murthy Jatavallabhula, Miles Macklin, Florian Golemo, Vikram Voleti, Linda Petrini, Martin Weiss, Breandan Considine, Jérôme Parent-Lévesque, Kevin Xie, Kenny Erleben, Liam Paull, Florian Shkurti, Derek Nowrouzezahrai, Sanja Fidler PDF Code Project Video RELMOGEN: INTEGRATING REINFORCEMENT LEARNING AND MOTION GENERATION FOR INTERACTIVE NAVIGATION ICRA 2021 Many Reinforcement Learning (RL) approaches use joint control signals (positions, velocities, torques) as action space for continuous … Fei Xia, Chengshu Li, Roberto Martin-Martin, Alexander Toshev, Or Litany, Silvio Savarese PDF Project UNICON: UNIVERSAL NEURAL CONTROLLER FOR PHYSICS-BASED CHARACTER MOTION UniCon The field of physics-based animation is gaining importance due to the increasing demand for realism in video games and films, and has … Tingwu Wang, Yunrong Guo, Maria Shugrina, Sanja Fidler PDF Project Video KAOLIN: A PYTORCH LIBRARY FOR ACCELERATING 3D DEEP LEARNING RESEARCH Pytorch Library Kaolin is a PyTorch library aiming to accelerate 3D deep learning research. Kaolin provides efficient implementations of differentiable … Krishna Murthy Jatavallabhula, Edward Smith, Jean-Francois Lafleche, Clement Fuji Tsang, Artem Rozantsev, Wenzheng Chen, Tommy Xiang, Rev Lebaredian, Sanja Fidler PDF Code Project Source Document VARIATIONAL AMODAL OBJECT COMPLETION NeurIPS 2020 Huan Ling, David Acuna, Karsten Kreis, Seung Wook Kim, Sanja Fidler PDF LEARNING DEFORMABLE TETRAHEDRAL MESHES FOR 3D RECONSTRUCTION NeurIPS 2020 3D shape representations that accommodate learning-based 3D reconstruction are an open problem in machine learning and computer … Jun Gao, Wenzheng Chen, Tommy Xiang, Alec Jacobson, Morgan Mcguire, Sanja Fidler PDF Code Project Video ULTRAHYPERBOLIC REPRESENTATION LEARNING NeurIPS 2020 In machine learning, data is usually represented in a (flat) Euclidean space where distances between points are along straight lines. … Marc T. Law, Jos Stam PDF Code FEDERATED SIMULATION OR MEDICAL IMAGING MICCAI 2020 (Nominated for the Young Scientist Award) Labelling data is expensive and time consuming especially for domains such as medical imaging that contain volumetric imaging data and … Daiqing Li, Amlan Kar, Nishant Ravikumar, Alejandro Frangi, Sanja Fidler PDF Project META-SIM2: UNSUPERVISED LEARNING OF SCENE STRUCTURE FOR SYNTHETIC DATA GENERATION ECCV 2020 Procedural models are being widely used to synthesize scenes for graphics, gaming, and to create (labeled) synthetic datasets for ML. … Jeevan Devaranjan*, Amlan Kar, Sanja Fidler PDF Project LIFT, SPLAT, SHOOT: ENCODING IMAGES FROM ARBITRARY CAMERA RIGS BY IMPLICITLY UNPROJECTING TO 3D ECCV 2020 The goal of perception for autonomous vehicles is to extract semantic representations from multiple sensors and fuse these … Jonah Philion, Sanja Fidler PDF Code Project Video LEARNING TO SIMULATE DYNAMIC ENVIRONMENTS WITH GAMEGAN CVPR 2020 Simulation is a crucial component of any robotic system. In order to simulate correctly, we need to write complex rules of the … Seung Wook Kim, Yuhao Zhou, Jonah Philion, Antonio Torralba, Sanja Fidler PDF Project Video LEARNING TO PREDICT 3D OBJECTS WITH AN INTERPOLATION-BASED DIFFERENTIABLE RENDERER NeurIPS 2019 Many machine learning models operate on images, but ignore the fact that images are 2D projections formed by 3D geometry interacting … Wenzheng Chen, Jun Gao, Huan Ling, Edward J. Smith, Jaakko Lehtinen, Alec Jacobson, Sanja Fidler PDF Code Project Video NEURAL TURTLE GRAPHICS FOR MODELING CITY ROAD LAYOUTS ICCV 2019 (Oral) We propose Neural Turtle Graphics (NTG), a novel gen- erative model for spatial graphs, and demonstrate its ap- plications in modeling … Hang Chu, Daiqing Li, David Acuna, Amlan Kar, Maria Shugrina, Xinkai Wei, Ming-Yu Liu, Antonio Torralba, Sanja Fidler PDF Project Video META SIM: LEARNING TO GENERATE SYNTHETIC DATASETS ICCV 2019 (Oral) Training models to high-end performance requires availability of large labeled datasets, which are expensive to get. The goal of our … Amlan Kar, Aayush Prakash, Ming-Yu Liu, Eric Cameracci, Justin Yuan, Matt Rusiniak, David Acuna, Antonio Torralba, Sanja Fidler PDF Code Project GATED-SCNN GATED SHAPE CNNS FOR SEMANTIC SEGMENTATION ICCV 2019 Current state-of-the-art methods for image segmentation form a dense image representation where the color, shape and texture … Towaki Takikawa, David Acuna, Varun Jampani, Sanja Fidler PDF Code Project DEVIL IS IN THE EDGES: LEARNING SEMANTIC BOUNDARIES FROM NOISY ANNOTATIONS CVPR 2019 (Oral) We tackle the problem of semantic boundary prediction, which aims to identify pixels that belong to object(class) boundaries. We notice … David Acuna, Amlan Kar, Sanja Fidler PDF Code Project CONTACT Our lab is located in Downtown Toronto (20 minutes away from the St-George campus of the University of Toronto) and hosts many students for their co-op programs. Motivated candidates can contact our members to apply for internships and research positions. Unauthorized visitors are not permitted in the Toronto office. * 431 King St W, 6th floor, Toronto, ON M5V 3M4 * Monday-Friday 9:00 to 18:00 +− Leaflet | © OpenStreetMap Published with Wowchemy — the free, open source website builder that empowers creators. CITE × Copy Download