L'objectif de ce projet est de fournir une approche radicalement nouvelle et intégrée où les images sont acquises de manière à ce que les outils de reconstruction automatisés soient les plus efficaces. À cette fin, nous voulons combiner notre expertise commune en matière d’imagerie, de vision par ordinateur, et de représentation des structures pour nous assurer que les algorithmes que nous développerons fourniront les meilleurs résultats possibles. Nous travaillerons sur un large éventail de structures de complexité croissante. Nous avons choisi de travailler sur des structures venant de différents domaines car elles partagent des propriétés topologiques communes, ce qui signifie que les mêmes types de méthodes et d'algorithmes peuvent être utilisés pour toutes. Cependant, elles présentent également des différences: Elles vont de structures purement linéaire, comme les dislocations dans les cristaux, les structures neurales et les fibrilles amyloïdes, aux structures partiellement linéaires et partiellement membraneuses, telles que le réticulum endoplasmique, et enfin aux structures purement membraneuses. Certaines d’entre elles peuvent être imagées pour produire des volumes intrinsèquement 3D, tandis que d’autres seront projetées dans des images 2D, faisant que la récupération de leur structure 3D nécessitera une reconstruction stéréo.
Les approches que proposerons seront suffisamment génériques pour les traiter toutes, ce qui garantira que les résultats du projet auront un impact au-delà de la résolution des problèmes de reconstruction spécifiques que nous ciblons. De plus, transformerons nos algorithmes en outils logiciels fiables et conviviaux que les scientifiques pourront utiliser dans leur travail quotidien pour modéliser rapidement des volumes de données volumineux et complexes et accroître ainsi considérablement leur productivité. À leur tour, ils développeront des protocoles d'imagerie optimisés pour l'analyse d'image. Nous rendrons ensuite ces outils et protocoles publiquement accessibles afin que les chercheurs de nombreux domaines puissent en bénéficier.
As light and electron microscopes become ever more powerful, they can produce high-resolution imagery in such large quantities that the bottleneck becomes processing and interpretation of their output. Unfortunately, current image analysis techniques rely heavily on manual intervention and feedback, which severely limits the amount of data that can be processed and restricts research to anecdotal rather than statistical evidence. Furthermore, since human annotators may have different biases and opinions about what constitutes a significant feature, their involvement introduces an undesirable element of unpredictability in the final interpretation. In the worst case, important structures that cannot easily be discerned by human observers may be overlooked. Consequently, targeted data acquisition taking into account the whole chain of data treatments as well as sample specificities is required to take full advantage of the power of cutting-edge instruments.The goal of this project is to provide a radically new, integrated, approach where images are acquired in such a way that the automated reconstruction tools are most effective. To this end, we want to combine our joint expertise in imaging, computer vision, and structure-representation to ensure the algorithms we will develop deliver the best possible results. We will work on a wide range of scientifically relevant structures of increasing complexity. We chose to work on all these structures from different domains because they share common topological properties, which means the same kinds of methods and algorithms can be used on all of them. However, they also exhibit differences: They range from purely linear, such as dislocations, neural structures, and amyloid fibrils, to structures that are partly linear and partly membranous, such as the endoplasmic reticulum, and finally to purely membranous ones. Some of them can be imaged to produce stacks that are inherently 3D, while others will be seen in multiple 2D images, so that recovering their 3D structure will involve stereo reconstruction. The approaches we propose will therefore have to be generic enough to handle them all, which will guarantee that the output of the project will have an impact beyond solving the specific reconstruction problems we target. We will systematically make sure that the researchers who analyze the images provide continuous feedback to those who acquire them so that they can jointly and continuously improve the imaging protocols to guarantee optimal performance for the algorithms. This is a departure from traditional practice, which involves first imaging the data and then having the algorithms deal with the result, no matter how well or ill-suited it is for automated processing. Both because of this and because we will integrate into our algorithms powerful new Machine Learning and Graph Optimization techniques, they will become far more effective than before and allow us to analyze unprecedented amounts of image data. This will enable us to reconstruct brain-wide axonal arborizations of hundreds of individual neurons, which will constitute a landmark achievement for neuroscience. It will also pave the way towards automated guidance of Material Scientists through the Transmission Electron Microscopy (TEM) image acquisition process and deliver much faster analysis of 3D morphologies. This will unleash the full power of modern TEMs and represent a crucial step toward real-time 3D TEM, a holy grail not only for Material Science but also for Bioengineering and Life Science.Finally, we will turn our algorithms into reliable, user-friendly software tools that material scientists and neuroscientists can use in their daily work to quickly model large and complicated data volumes and thereby tremendously increase their productivity. In turn, they will develop imaging protocols that are optimized for image analysis. We will then make both the tools and protocols publicly available so that researchers in many different fields can benefit from them.
Last updated:12.09.2022
