Realistic 3D digital models of human heads and faces have the potential to transform current medical practices for pre-treatment diagnostics, treatment planning, patient documentation andquantitative assessment of treatments. In contrast to entertainment where 3D digital faces havebecome stunningly detailed and realistic with the help of artificial intelligence (AI), medical settings entail several challenges due to the small sizes of real patient data and under-representation of different age groups in the existing datasets.In this project, we aim at building high-quality models of the human infant face and head learned from real datasets that derive from the medical treatment of infants. Using these models as priors, we will develop methods to reconstruct individual optimal target morphologies, which can be used as a reference in the treatment planning of infants with craniofacial malformations. 3D morphable models (3DMMs), which are classical and powerful tools for face modeling, have not been explored in depth for infant face and head modeling and the use of models for adults gives suboptimal results on infants. Therefore, our starting point will be developing these models for infants. To increase the quality of modeling and to obtain photo-realistic reconstructions, we will design generative adversarial networks (GANs). Since the knowledge of underlying bone structure is an integral part of treatment planning for craniofacial malformations, we then develop deep learning-based methods to reconstruct the skull using 3D face and head surface scans, as an alternative to computed tomography (CT) scans with the goal of reducing the ionized radiation burden on infants.We focus on three common craniofacial malformations observed in infants: positional cranial deformity, cleft lip and palate and craniosynostosis. For each type of these malformations, we will conduct clinical assessments of the computational methods that we will develop on patient data originating from the specific craniofacial treatments.Our work proceeds according to the following work packages (WPs): We first develop high-quality 3DMMs (WP1) and deep learning methods (WP2) for craniofacial modeling and treatment planning for infant heads and faces. We then design deep learning-based methods for skull reconstruction (WP3). In parallel to this computer graphics research at ETH, we conduct data collection and clinical validations at the University of Basel for each of the infant craniofacial malformations that we address in this project (WP4-WP6). In this collaboration between two major Swiss engineering and medical institutions, we expect to achieve a highly valuable impact in the intersection of computer science and medicine. The developed AI-based solutions will not only forward the state-of-the-art in computer graphics andface modeling, but will also have a high potential to facilitate malformation treatments, decrease the burden on physicians and increase the satisfaction of patients. In the future, the developed methods can be extended for adults and other anatomical malformations. The insights gainedin the project will also be valuable for potential applications such as surgery training, surgery simulation and virtual reality-based surgery.
Last updated:27.01.2023
