Conditional Gaussian Patch Models for Texture Synthesis

Resumen: Exemplar-based texture synthesis is defined as the process of generating, from an input texture sample, new texture images that are perceptually equivalent to the input. We model texture self-similarities with conditional Gaussian distributions in the patch space. These models are then embedded in a patch-based framework for texture synthesis. The Gaussian distribution for each patch is inferred from the set of its nearest neighbors in the patch space obtained from the input sample.
A key challenge in patch-based approaches is how to seamlessly assemble the synthesized patches. We compare two strategies: the image quilting method [1], which blends overlapping patches by minimizing boundary discrepancies, and a conditional Gaussian approach [2], where new patches are constrained to match the overlapping region with previously synthesized content.
The results show that it is possible to dispose of the quilting step in the case of periodic and pseudo-periodic textures. However, for more complex textures, the conditional models are less performing and are fast limited by the size of the texture input sample.

[1] Efros, Alexei A., and William T. Freeman. "Image quilting for texture synthesis and transfer." Proceedings of the 28th annual conference on Computer graphics and interactive techniques. 2001.
[2] Raad, Lara, Agnès Desolneux, and Jean-Michel Morel. "A conditional multiscale locally Gaussian texture synthesis algorithm." Journal of Mathematical Imaging and Vision 56.2 (2016): 260-279.