tomaszbednarz / visual-computing.matters.today

SIGGRAPH Asia 2023 Visual Computing TPC

LOCATION	EG02 in E Block	UNSW Art & Design	Oxford St & Greens Rd, Paddington NSW 2021	Australia
TIME	NAME	TALK TITLE	ABSTRACT	SHORT BIO	Session
9:15am - 9:30am	June Kim Ming C. Lin	Welcoming		Conference Chair Technical Papers Program Chair	Opening
	AM Session	Chaired by Baoquan Chen
9:30am - 9:50am	Libin Liu	Representation & Generation of Character Animation	Generating realistic behaviors is a fundamental challenge in computer animation and a critical technique in emerging fields such as digital humans and humanoid robots. Over the past years, there has been tremendous progress in this area, driven in part by advancements in deep learning and reinforcement learning. In this talk, I will introduce our recent work on developing multi-skilled characters capable of executing natural movements. I will provide a brief overview of early efforts using state machines and control snippet scheduling but will focus more on the utilization of more powerful generative models. Particularly, I will detail how variational autoencoders and their quantized versions serve as efficient and compact representations for motion. Additionally, I will explain how we manipulate these generative models to achieve fine-grained control over the produced motion. We utilize multi-modal inputs, such as text, imagery, video, and motion demonstration, to achieve precise control of motion details. This approach not only offers an intuitive interface for users to manipulate motion but also enables seamless integration with advanced AI systems like ChatGPT	Libin Liu is an assistant professor in the School of Artificial Intelligence at Peking University. Before joining Peking University, he served as the Chief Scientist at DeepMotion Inc. He was also a postdoctoral research fellow at Disney Research and the University of British Columbia, after earning his Ph.D. in computer science from Tsinghua University. His research interests include character animation, physics-based simulation, motion control, and related fields such as reinforcement learning, deep learning, and robotics. He has dedicated significant effort to achieving various agile human motions in simulated characters and robots. His research has received the ACM SIGGRAPH Asia 2022 Best Paper Award and an Honorable Mention Award at SIGGRAPH 2023.	Animation & Simulation
9:50am - 10:10am	Yuting Ye	Animating Avatars in AR/VR	Consumer-grade AR/VR devices, such as the Meta Quest, offer new opportunities for character animation research, as they are 3D mediums for both content creation and consumption, as opposed to traditional 3D video games on a 2D screen. This new 3D platform also presents unique challenges in how to animate 3D avatars and present 3D animation. In this talk, I would like to provide an overview of current problems in character animation in the context of AR/VR, and present some explorations we have done at Meta. For example, while motion retargeting has always been an open problem, we now have an opportunity to tackle it end-to-end with concrete user applications in mind. My goal is to spread learnings from our experiments, and hope to interest more research labs to work in the space.	Yuting Ye is a research scientist in Reality Labs Research at Meta. Her research focuses on character animation techniques that enable a user to be present in and to interact with a 3D digital world, such as an AR/VR/MR environment. She is also interested in exploring content creation opportunities within this new 3D medium. Her previous focus at Meta was to enable hand tracking and hand-object interaction on the Meta Quest VR devices. Before joining Meta, Yuting worked in the R&D department of Industrial Light & Magic (ILM), and contributed to many award-winning software systems and blockbuster movies. Prior to ILM, she obtained a PhD in computer science from Georgia Tech. She is currently an associate editor for TOG and has served in the technical papers committee for SIGGRAPH, SIGGRAPH Asia, SCA and CASA. She is also the program co-chair for SCA 2023.	Animation & Simulation
10:10am - 10:30am	Jernej Barbic	Anatomically Based Hand Simulation	I will present our recent work on modeling and animating human hands. Hands are important in many applications, such as computer games, film, ergonomic design, tracking, and medical treatment. I will discuss how to acquire complete human hand anatomy in multiple poses using magnetic resonance imaging (MRI). Acquiring human hand anatomy in multiple poses was previously difficult because MRI scans must be long for high-precision results (over 10 minutes), and because humans cannot hold their hands perfectly still in non-trivial and badly supported poses. We invented a manufacturing process whereby lifecasting materials commonly employed in film special effects industry are used to stabilize the hand during scanning. We demonstrate how to efficiently segment the MRI scans into individual bone, muscle, tendon, ligament, fat and skin meshes in all poses, and how to correspond each organ’s mesh to the same mesh connectivity across all scanned poses. Next, we give a method to simulate the volumetric shape of the organs to any pose in the hand’s range of motion, producing both external skin shapes and internal organ shapes that match ground truth optical scans and medical images (MRI) in multiple scanned poses. We achieve this by combining MRI images in multiple hand poses with FEM multibody nonlinear elasto-plastic simulation. This enables us to start with an arbitrary animation of the hand joint hierarchy, and produce a matching high-quality skin and internal organ animation of the hand. Our system models bones, muscles, tendons, ligaments and fat as separate volumetric organs that mechanically interact through contact and attachments, and whose shape matches medical images (MRI) in the MRI-scanned hand poses. We use our method to produce volumetric renders of the internal anatomy of the human hand in motion, and to compute and render highly realistic hand surface shapes.	Jernej Barbic is a Full Professor of Computer Science at USC. His interests include computer graphics, animation, interactive physics, haptic rendering, visual effects for film, medical simulation and imaging, deformable objects, biomechanics, sound simulation, model reduction, intellectual property law and startup companies. He has published over 50 publications in computer graphics and related fields. He was also a co-founder and CTO of a successful computer animation startup company "Ziva Dynamics" (acquired by Unity Technologies), whereby he contributed technical and business leadership on real-time character deformation, anatomically based modeling, nonlinear elasticity and digital humans. In 2014, he was named a Sloan Research Fellow. In 2011, MIT Technology Review named him one of the Top 35 Innovators under the age of 35 in the world (TR35). Jernej is also the author of Vega FEM, a free C/C++ software physics library for deformable object simulation. He received his Ph.D. from CMU, followed by Postdoctoral Research at MIT.	Animation & Simulation
10:30am - 10:50am	Julien Pettré	From Batman to Metallica: crowd modeling and simulation, in Computer Graphics and beyond.	Computer graphics are an essential contributor to the field of crowd simulation. More than 40 years ago, Craig Reynolds proposed the Boid method for simulating the behaviour of flocks of starlings, producing visually fascinating animations. In this presentation, I will show that the principles guiding the design of crowd simulators have long remained intact, but that profound changes are at work. What is the new place of computer graphics in crowd simulation at the dawn of this transformation?	Julien Pettré is a computer scientist. He is senior researcher at Inria, the French National Institute for Research in Computer Science and Control. He is leading the VirtUs team at the Inria Center of Rennes. He received PhD from the University of Toulouse III in 2003, and Habilitation from the University of Rennes I in 2015. From 2004 to 2006, he was postdoctoral fellow at EPFL in Switzerland. He joined Inria in 2006. Julien Pettré is coordinator of the European H2020 Fet Open CrowdDNA project (2020-2024), dedicated to future emergent technologies for crowd management in public spaces. He previously coordinated the European H2020 Crowdbot project (2018-21) dedicated to the design of robot navigation techniques for crowded environments, as well as the national ANR JCJC Percolation project(2013-17) dedicated to the design of new microscopic crowd simulation algorithms, and the national ANR CONTINT Chrome project, dedicated to efficient and designer-friendly techniques for crowd animation. His research interests are crowd modeling and simulation, computer animation, virtual reality, robot navigation and motion planning.	Animation & Simulation
10:50am - 11:10am	Tao Du	Learning Preconditioners for Conjugate Gradient Solvers	Efficient numerical solvers for partial differential equations empower science and engineering. One commonly employed numerical solver is the preconditioned conjugate gradient (PCG) algorithm, whose performance is largely affected by the preconditioner quality. However, designing high-performing preconditioner with traditional numerical methods is highly non-trivial, often requiring problem-specific knowledge and meticulous matrix operations. We present a new method that leverages learning-based approach to obtain an approximate matrix factorization to the system matrix to be used as a preconditioner in the context of PCG solvers. Our high-level intuition comes from the shared property between preconditioners and network-based PDE solvers that excels at obtaining approximate solutions at a low computational cost. Such observation motivates us to represent preconditioners as graph neural networks (GNNs). In addition, we propose a new loss function that rewrites traditional preconditioner metrics to incorporate inductive bias from PDE data distributions, enabling effective training of high-performing preconditioners. We conduct extensive experiments to demonstrate the efficacy and generalizability of our proposed approach on solving various 2D and 3D linear second-order PDEs.	Tao Du is an Assistant Professor at the Institute for Interdisciplinary Information Sciences (IIIS) at Tsinghua University. His research aims at developing computational methods to help people understand and design physical systems. Before joining Tsinghua University, he completed his Ph.D. at MIT in 2021 and continued as a Postdoctoral Associate from 2021 to 2022. At MIT, he developed differentiable deformable-solid and fluid simulators and explored their downstream applications in machine learning and robotics. His research work has been featured by technical media outlets including WIRED, IEEE Spectrum, TechCrunch, Engadget, and so on. In addition, he was recognized by NeurIPS (2021 and 2022) and ICML (2022) as an outstanding reviewer.	Animation & Simulation
11:10am - 11:30am	TEA BREAK
	Panel	Moderated by June Kim
11:30am - 12:30pm	Masa Inakage Jürgen Hagler Volker Helzel	UNSW x SIGGRAPH Asia Education	Collaboration Panels Moderated by June Kim	As the concluding UNSW Art & Design Education Forum, three distinguished international researchers and academics are featured to provide exclusive insights into their ongoing research projects and educational initiatives at their respective institutions. This event also offers future opportunities for engaging in discussions about potential collaborations and opportunities and student exchanges.	Masa Inakage Dean and Professor Director of Keio Media Design Research Institute, Keio University Graduate School of Media Design https://www.kmd.keio.ac.jp/faculty/masa-inakage Founding Dean and Professor at Keio University Graduate School of Media Design. Active as strategist and futurist, creative director and producer, as well as media artist/ designer. Supervises experience design projects that illustrate future society based on Dream Driven Design method. Story, playfulness and imagination are at the core of the project method to prototype content, service, things and space for creative innovation. Inakage sits on advisory board and executive board of numerous corporations for corporate innovation strategy, as well as has served as members of government committees and international organisations including the World Economic Forum.  Jüergen Hagler Director Ars Electronica Animation Festival Dr. Juergen Hagler is an academic researcher and curator working at the interface of animation, game, and media art. He studied art education, experimental visual design and cultural studies at the University for Art and Design Linz, Austria. Currently, he is a Professor for Computer Animation and Media Studies and the head of studies of the degree programme Digital Arts at the University of Applied Sciences Upper Austria, Hagenberg Campus. Since 2014 he has been the co-head of the research group Playful Interactive Environments, with a focus on the investigation of new and natural forms of interaction and the use of playful mechanisms to encourage specific behavioural patterns. He has been involved in the activities of Ars Electronica since 1997 in a series of different functions. Since 2017 he is the director of the Ars Electronica Animation Festival and initiator and organiser of the Expanded Animation Symposium. Vokler Helzle Head of R&D Senior Lecturer Filmakademie, Baden-württemberg https://animationsinstitut.de/en/studies/lecturers/prof-volker-helzle  Volker Helzle is in charge of Research & Development at Animationsinstitut of Filmakademie Baden-Württemberg. After graduating from Stuttgart Media University in 2000 he moved to California and worked at Eyematic Interfaces on the eve of marker-less facial tracking tools and content. In 2003 he joined Filmakademie to supervise Research & Development. He is the senior lecturer for the Technical Director course and FMX program consultant. In 2013, Volker received the honorary professorship of Filmakademie.
12:30pm - 1:30pm	LUNCH
	PM Session 1	Chaired by Bernd Bickel
1:30pm - 1:50pm	Baoquan Chen	Content Generation without Deep Learning	I will revisit patch-based methods and introduce its applications in 3D scene generation and motion synthesis, contrast to existing deep learning-based methods, which typically require long offline training time, and are prone to visual artifacts. First, we advocate for synthesizing 3D scenes at the patch level, given a single example. At the core of this work lies algorithmic designs wrt the scene representation and generative patch nearest-neighbor module, that address unique challenges arising from lifting classical 2D patch-based framework to 3D generation. These design choices, on a collective level, contribute to a robust, effective, and efficient model that can generate high-quality natural scenes with both realistic geometric structure and visual appearance, in large quantities and varieties.  Then, we introduce motion synthesis using Generative Motion Matching (GenMM), which "mines" as many diverse motions as possible from a single or few example sequences. The generative motion matching module utilizes the bidirectional visual similarity as a generative cost function to motion matching, and operates in a multi-stage framework to progressively refine a random guess using exemplar motion matches. GenMM can synthesize a high-quality motion of highly complex skeletal structures within a fraction of a second, as well, extend to a number of scenarios, including motion completion, key frame-guided generation, infinite looping, and motion reassembly.	Baoquan Chen is a Professor of Peking University, where he is the Associate Dean of the School of Artificial Intelligence. His research interests generally lie in computer graphics, computer vision, and visualization. He published 40+ papers in SIGGRAPH/SIGGRAPH Asia, and won best paper awards in conferences including SIGGRAPH Asia 2022, SIGGRAPH 2022 (honorary mention), IEEE Visualization 2005. He has served as associate editor of ACM TOG and IEEE Transactions on Visualization and Graphics (TVCG), conference steering committee member for ACM SIGGRAPH Asia and IEEE VIS, conference chair of a number of international conferences, including SIGGRAPH Asia (2014) and IEEE Visualization (2005), program chair of IEEE Visualization (2004), as well as program committee member of almost all conferences in the visualization and computer graphics fields for numerous times. He is an IEEE Fellow, and an inductee of IEEE Visualization Academy.	Generative Models
1:50pm - 2:10pm	Jun-Yan Zhu	Data Ownership in Generative Models	Large-scale generative visual models, such as DALL·E2 and Stable Diffusion, have made content creation as little effort as writing a short text description. However, these models are typically trained on an enormous amount of Internet data, often containing copyrighted material, licensed images, and personal photos. How can we remove these images if creators decide to opt-out? How can we properly compensate them if they choose to opt in?  In this task, I will first describe an efficient method for removing copyrighted materials, artistic styles of living artists, and memorized images from pretrained text-to-image diffusion models. I will then discuss our data attribution algorithm for assessing the influence of each training image for a generated sample. Collectively, we aim to enable creators to retain control over the ownership of training images.	Jun-Yan Zhu is an Assistant Professor at CMU’s School of Computer Science. Prior to joining CMU, he was a Research Scientist at Adobe Research and a postdoc at MIT CSAIL. He obtained his Ph.D. from UC Berkeley and B.E. from Tsinghua University. He studies computer vision, computer graphics, and computational photography. He has received the Packard Fellowship, the NSF CAREER Award, the ACM SIGGRAPH Outstanding Doctoral Dissertation Award, and the UC Berkeley EECS David J. Sakrison Memorial Prize for outstanding doctoral research, among other awards. His current research focuses on generative models for visual content creation.	Generative Models
2:10pm - 2:30pm	Hsueh-Ti Derek Liu	Level of Detail for Geometry Computation	Performing computations on surface geometry in real time remains a major challenge in geometry processing. In this talk, we will investigate Level of Detail (LOD) techniques tailored for geometry computation, including computation-driven mesh simplification and maintaining bijective maps across the mesh hierarchy. LOD has been a key facilitator in enabling real-time rendering. We argue that computation LOD could also be the foundation for real-time simulation, and many other applications including compression and mesh-based learning architectures.	Hsueh-Ti Derek Liu is a Research Scientist at Roblox Research, working on digital geometry processing. Derek's work mainly focuses on developing easy-to-use tools for 3D content creation and scalable numerical methods for processing geometric data. Derek received his MS from Carnegie Mellon University and was an Adobe PhD Fellow at the University of Toronto. He was the recipient of the best PhD dissertation from Eurographics and the Alain Fournier awards. (Website - https://www.dgp.toronto.edu/~hsuehtil/)	Geometry Processing
2:30pm - 2:50pm	Marc Alexa	Fibonacci Sampling - On the 3-Sphere and Beyond	I will first present Super-Fibonacci spirals — an extension of Fibonacci spirals, enabling the fast generation of an arbitrary but fixed number of 3D orientations or, equivalently, points on the 3-sphere in Euclidean 4-space. The algorithm is simple and fast. Compared to other methods the generated sets have low discrepancy, minimal spurious components in the power spectrum, and almost identical Voronoi volumes. I hope this makes them useful for a variety of applications. Time permitting, I want to explain why I am utterly surprised that Fibonacci sampling works at at all, given that we usually use it in floating point arithmetic. I show some experiments on the unit interval suggesting it even works for coarse discretizations. The experiments show a connection to ‘very’ irrational numbers, i.e. numbers that are badly approximable by rationals. Why this property plays a role for discrete sampling is unclear to me — but we may get some inspiration for a geometric interpretation of these numbers in hyperbolic space.	Marc Alexa is a Professor in the Faculty of Electrical Engineering and Computer Science at the Technical University of Berlin and heads the Computer Graphics group. He is interested in the representation, generation, modification, and analysis of shapes in 3 and more dimensions. He has been Technical Papers Chair for SIGGRAPH and Editor-in-Chief of ACM Transactions in Graphics, the leading venues in the field. Throughout his career, he received a variety of awards, distinctions, and grants from the European Research Council, Apple Inc., the German Science Foundation, Google, Walt Disney Animation Studios, the Berlin-Brandenburg Academy of Sciences, and others.	Geometry Processing
2:50pm - 3:10pm	Nicholas Sharp	Adam is a Weird Optimization Algorithm	The Adam optimization algorithm is frustratingly effective for a wide range of problems across visual computing, even on some problems which have nothing to do with machine learning. Several properties of this algorithm are quite different from traditional optimization schemes in geometry processing and physics. This talk will be a lighthearted deep dive into several of those properties, and their practical impacts for visual computing.	Nicholas Sharp's research develops new algorithms and representations for 3D geometry, with applications in computer graphics/vision and geometric machine learning. He is currently a Senior Research Scientist at NVIDIA, based in Seattle. Nick received his PhD from Carnegie Mellon University advised by Prof. Keenan Crane, and was a postdoc at the University of Toronto with Prof. Alec Jacobson. He also developed several open-source software libraries for 3D computing, including Polyscope and geometry-central. www.nmwsharp.com	Geometry Processing
3:10pm - 3:40pm	TEA BREAK
	PM Session 2	Chaired by Bernd Bickel
3:40pm - 4:00pm	Karol Myszkowski	Advancing Display Realism for Photographed and Fabricated Content		Karol Myszkowski is a senior researcher at the MPI Informatik, Saarbruecken, Germany. From 1993 to 2000, he served as an associate professor in the Department of Computer Software at the University of Aizu, Japan. From 1986 to 1992 he worked for Integra, Inc., a Japan-based company specialized in developing rendering and global illumination software. He received his PhD (1991) and habilitation (2001) degrees in computer science from Warsaw University of Technology (Poland). In 2011, the President of Poland awarded him a lifetime professor title. His research interests include global illumination and rendering, perception issues in graphics, high dynamic range imaging, and stereo 3D. He co-authored the book High Dynamic Range Imaging, and participated in various committees and editorial boards. He also co-chaired/chaired Rendering Symposium in 2001, ACM Symposium on Applied Perception in Graphics and Visualization in 2008, Spring Conference on Computer Graphics 2008, ACM Siggraph Asia 2020, and Eurographics 2023.	Imaging & Rendering
4:00pm-4:20pm	Wolfgang Heidrich	Learned Imaging Systems	Computational imaging systems are based on the joint design of optics and associated image reconstruction algorithms. Of particular interest in recent years has been the development of end-to-end learned “Deep Optics” systems that use differentiable optical simulation in combination with backpropagation to simultaneously learn optical design and deep network post-processing for applications such as hyperspectral imaging, HDR, or extended depth of field. In this talk I will in particular focus on new developments that expand the design space of such systems from simple DOE optics to compound refractive optics and mixtures of different types of optical components.	Wolfgang Heidrich is a Professor of Computer Science and Electrical and Computer Engineering in the KAUST Visual Computing Center, for which he also served as director from 2014 to 2021. Prof. Heidrich joined King Abdullah University of Science and Technology (KAUST) in 2014, after 13 years as a faculty member at the University of British Columbia. He received his PhD in from the University of Erlangen in 1999, and then worked as a Research Associate in the Computer Graphics Group of the Max-Planck-Institute for Computer Science in Saarbrucken, Germany, before joining UBC in 2000. Prof. Heidrich’s research interests lie at the intersection of imaging, optics, computer vision, computer graphics, and inverse problems. His more recent interest is in computational imaging, focusing on hardware-software co-design of the next generation of imaging systems, with applications such as High-Dynamic Range imaging, compact computational cameras, hyperspectral cameras, to name just a few. Prof. Heidrich’s work on High Dynamic Range Displays served as the basis for the technology behind Brightside Technologies, which was acquired by Dolby in 2007. Prof. Heidrich is a Fellow of the IEEE, AAIA, and Eurographics, and the recipient of a Humboldt Research Award as well as the ACM SIGGRAPH Computer Graphics Achievement Award.	Imaging & Rendering
4:20pm-4:40pm	Evan Y. Peng	Neural Holographic and Light Field Displays		Dr. Peng is an Assistant Professor in EEE and CS departments at the University of Hong Kong (HKU), working on the intersection of optics and graphics research to eanble novel computational camera and display modalities. Dr. Peng is the recipient of the AsiaGraphics Young Researcher Award (2022), the IEEE VR Tech Significant New Researcher Award (2023), the Young Leader Award at SID International Conference on Display Tech (2022).	Imaging & Rendering
4:40pm - 5:00pm	Lingjie Liu	Reconstruction of 3D Real-world Scenes			Imaging & Rendering
5:00pm-5:20pm	Tomasz Bednarz	Pioneering Graphics for the AI Generation		Tomasz is a Director of Strategic Researcher Engagement at NVIDIA Corporation. His team at NVIDIA build active collaborations with top researchers and premier research institutions that do compelling, computationally intense work to solve some of the world’s most challenging scientific problems. Earlier he was a Director and Head of Visualisation at the Expanded Perception & Interaction Centre (EPICentre) at the UNSW and a Research Team Leader at CSIRO’s Data61. He also led Simulation and Modelling Cross-Cuttin g Capability for CSIRO’s Future Science and Technology. Currently serving as a SIGGRAPH Asia Conference Advisory Group (SACAG) Chair, and is voting director of ACM SIGGRAPH Executive Committee. Earlier, he was Conference Chair of ACM SIGGRAPH Asia 2019.	Future of Visual Computing
5:20pm - 5:30pm			Wrap-Up
5:30pm - 7:00pm	Location TBA		TPC SOCIAL