Dr. Binh-Son Hua

This project aims to make 3D content accessible to everyone via developing new methods for automatic generation and manipulation of 3D content using machine intelligence. We take inspirations from large language models and investigate 3D content creation from deep language understanding. First, we propose to study the generation of photorealistic 3D content from text descriptions. This involves integrating photorealistic rendering techniques in computer graphics with text-to-3D generative models. Second, we propose to study pre-training of neural networks on 3D data, as inspired by the next-word prediction task for self-supervision in language models. This involves representing 3D models as a structured composition of 3D primitives and 3D parts and investigating self-supervised learning on these compositions. Third, we propose to study interactive 3D content generation via text guidance. The output of this project will encourage the creation of 3D-first platforms, where 3D data is accessible to everyone. The proposed techniques can benefit the growth of the metaverse and extended reality applications as well as different industries such as game development, film production, and architecture illustration. Language3D also contribute to future AI, where synthetic data can be created on a large scale from 3D content to train AI models with reduced cost.