Understanding the Impact of Mathematics on Image Processing

This blog is a continuation of my previous blog on the relationship between mathematics and image processing and aims at more technical virtuosity than conceptual understanding.

What are the most influential works by mathematicians on image processing research in the past three decades? I would say MRF in 1980s; Wavelets and PDEs in 1990s; still-too-early-to-tell in 2000s. Before Geman and Geman’s PAMI paper in 1984, image processing was still an art with little scientific insight. It is Gemans’ paper that showed image processing can also be tackled by tools from statistical mechanics. Even though the analogy between pixels of an image and gas particles of an Ising model is artificial, this work has long-lasting impact. It should be noted that the mathematics in this paper is not entirely new (several results such as Gibbs sampling have been established by other researchers before) but it is the first successful application of statistical physics in image processing. From a historical perspective, it is not surprising to see Ising model, which has dramatic impact on modern physics, is also applicable to other scientific fields (MRF, Hopfield network and Boltzmann machine are all related to Ising model).

If one predicts the future of image processing in 1980s based on the history of theoretical physics before 1980s, he might say: renormalization group should be a cool idea for non-physicists to explore because the decade of 1970s belonged to renormalization group (RG) – a mathematical apparatus that allows one to investigate the changes of a physical system as one views it at different distance scales. Indeed,  the decade of 1990s belonged multiscale modeling of images: wavelet and PDE -based approaches both address the issue of scale though from different perspectives. Wavelets are local schemes whose effectiveness lies in the good localization property of wavelet bases; PDE-based models are global schemes which is often characterized by minimization of certain energy functional (note that they do admit local implementation based on diffusion). It is safe to say that both wavelet-based and PDE-based models have high impact in image processing and their underlying connection has been established for certain specific cases. The unsettling issue is the varying role of locality – in physics, it is a fundamental assumption that interactions are local; but such assumption does not hold for images or more precisely for visual perception of image signals because of nonlocal interaction among neurons.

Since 1998, nonlocal image processing has been studied under many disguises – e.g., bilateral filtering, texture synthesis via nonparametric sampling, nonlocal mean denoising (more sophisticated version is BM3D denoising) and nonlocal TV et al. All of a sudden, lots of experimental findings seem to suggest something beyond the scope of wavelet and PDE. What is it? I have been baffled by this question for many years and I am still groping for the answers – one promising direction is to view images as fixed point of some dynamics system (abstraction of neural systems) characterized by similitude and dissimilitude (abstraction of excitatory and inhibitory neurons). History of theoretic physics cannot help here because as the complexity of system increases, physics become chemistry and chemistry evolves into biology. The new breakthrough, if I can make it, will not come from mathematical virtuosity (I simply am not even close to Daubechies or Mumford) but from physical intuition. I think there exists a universal representation of the physical world and a universal cortical algorithm for neurons to encoder the sensory stimuli. From this perspective, image processing is really paving one possible path toward understanding fundamental phenomenon such as biological memory and its implications into intelligence. Mathematics surely will still play a role of communicating my findings to others but hopefully I might only need  math skills of Shannon or Ashby’s level for this task.

Last, I happened to learn that the first Millennium Prize of Clay Institute was awarded to Perelman for his resolution of Poincare’s conjecture. There is a very-well written report about this famous conjecture which I think even non-mathematicians like myself will enjoy reading. My instinct tells me someone might have already applied this new fancy tool of Ricci flow into image processing. Indeed several Jewish researchers have pursued this direction but from their preliminary findings, I think they won’t go very far unless they can supply the Ricci flow with some physical intuition first. It has been suggested “an issues-directed approach is more fruitful than a method-directed approach to science” which really echoes the point of this blog.

About these ads