Current algorithms for modeling clothed people mainly fall into one of two categories, those that model the external surface of the clothed body without modeling the body itself and those that model the body while ignoring the clothes. The former have been shown to work remarkably well on very clean images in which the person can be precisely outlined but rarely account for poor lighting or occlusions that are frequent in the real world. The latter are more robust but do not fully exploit the information the shape of the clothes provides about that of the body. Furthermore, in neither case are many provisions made for the fact that there may be many people in the scene and that they may occlude each other. As a result, whereas modeling people wearing tight-fitting clothes is fast becoming a mature field, handling subjects wearing looser garments remains an open problem when it is to be done in everyday settings where people may hide each other, precise outlines are hard to estimate, and shadows often complicate matters. In this project, we will handle this more difficult task of modeling people and their potentially loose fitting clothing from video. This will include modeling individuals as well as multiple people interacting with each other. In the process, we will also develop new approaches to representing complex surfaces whose topology can change and to modeling their interactions. We believe that this will have an impact beyond the specific topic of people and their clothes, for example in the field of Computer Aided Design where such models are crucial. Finally, by deploying our algorithms in the real-world setting of drones filming people, we will prove that the potential of our technique goes beyond lab environments, which eventually will lead to technology transfer from Academia to Industry.
Last updated:30.01.2023
