In this talk, we will present two applications of image synthesis.
First, we will focus on texture synthesis. This task consists in synthesizing a new image from a reference image. The synthesized image must be perceptually similar to the reference image while being different. This is an old problem that has been recently updated with the use of convolutional neural networks (CNNs). Most current methods are based on the use of Gram matrices of feature maps from ImageNet trained CNNs. We have developed a simple multi-resolution strategy to take into account large-scale structures. This strategy can be coupled with long-range constraints, such as imposing the Fourier spectrum of the image, or using of autocorrelation of the feature maps. This multi-resolution strategy allows to obtain excellent high resolution synthesis. Combining it with additional constraints improves the results in the case of regular textures. We compared our methods to alternative methods on various texture examples and corroborated our visual observations with quantitative and perceptual evaluations.
Second, we will describe a new unsupervised and document-specific approach for character recognition from lines of text. Our main idea is to build on unsupervised approaches to object discovery and in particular on recent methods of « analysis-by-synthesis », which reconstruct images from a limited number of visual elements, called sprites. We extend these approaches to learn up to a hundred characters and analyze full lines of text by introducing a relevant architecture and an efficient sprite selection strategy. We illustrate the effectiveness of our model on printed documents and ancient manuscripts.
Bio: Nicolas Gonthier received a Data Science M.Eng. degree from ISAE-Supaéro and an M.Sc. degree in statistics from the University of Toulouse, both in 2017 and a Ph.D. degree in image processing from the University Paris-Saclay in 2021. His PhD was funded by an interdisciplinary grant (IDI IDEX) from the University Paris-Saclay and hosted by Télécom Paris, Institut polytechnique de Paris. Currently, he is a postdoctoral researcher at ENPC Paris-Tech. His research interests include deep learning, image processing and machine learning mainly for cultural heritage (historical documents, artworks, etc.).
Le séminaire aura lieu en mode hybride, à ESIEE Paris dans l’amphithéâtre 160 ; et via meet.google.com/nux-njnv-vud pour celles et ceux qui ne pourront pas être présents.
Localisation
Amphithéâtre 160 (ESIEE Paris)