My research mainly focuses on computational quasi-conformal geometry
(CQC) and its applications to medical imaging, computer vision and
computer graphics.
This work presents a novel
deep neural network with the incorporation of quasiconformal (QC) theories for
large deformation image registration. Image
registration is a key process in many practical tasks in computer visions,
computer graphics and medical imaging. It is a challenging problem especially
when the deformation between images is large. The problem is often formulated
as an optimization problem, which is computationally intensive and time
consuming. To accelerate the process, training deep neural networks for image
registration has recently attracted much attention and shown satisfactory
results while requiring signficiantly less computational time. Nevertheless,
under these deep learning frameworks, the geometric distortion under the
registration map cannot be controlled. To address this issue, we propose a
novel deep neural network based on quasiconformal (QC) theories. Instead of
learning the vector fields for spatial transformation, our network learns a
geometric quantity, called the Beltrami coefficient, to obtain the
registration map. The Beltrami coefficient effectively measures and controls
the local geometric distortions under the registration map. Thus, our network
can obtain diffeomorphic image registration, even with very large deformation.
Extensive experiments on real images demonstrate the effectiveness of our
method. Gallery
Many important problems in the real world don't have unique
solutions. It is thus important for machine learning models to be
capable of proposing different plausible solutions with meaningful
probability measures.
In this work we introduce such a deep learning framework that learns
the one-to-many relationships between the inputs and outputs. The key
of our approach is the use of a discrete latent space, where each item
represents a latent mode hypothesis for a particular type of
input-output relationship.
The discrete latent representations and the uncertainty associated to
any input are learned jointly during training.
We thus call our framework modal uncertainty estimation.
We extensively validate our framework on both real and synthetic
datasets.Gallery
We develop a framework for shape analysis of partial
surfaces using inconsistent surface mapping technqiue. Traditional
landmark-based geometric morphometrics methods suffer from the limited degrees
of freedom, while most of the more advanced non-rigid surface mapping methods
rely on a strong assumption of the global consistency of two surfaces. From a
practical point of view, given two anatomical surfaces with prominent feature
landmarks, it is more desirable to have a method that automatically detects
the most relevant parts of the two surfaces and finds the optimal
landmark-matching alignment between those parts, without assuming any global
1-1 correspondence between the two partial surfaces. Our method is capable of
solving this problem using an inconsistent surface mapping technique based on
quasi-conformal theory. It further enables us to quantify the dissimilarity of
two shapes using quasi-conformal distortion and differences in mean and
Gaussian curvatures, thereby providing a natural way for shape classification.
Experiments on Platyrrhine molars demonstrate the effectiveness of our method
and shed light on the interplay between function and shape in nature.
We address the problem of registering two surfaces, of which a
natural bijection between them does not exist. More precisely, only a
partial subset of the source domain is assumed to be in correspondence
with a subset of the target domain. We call such a problem an
inconsistent surface registration problem. This problem is challenging
as the corresponding regions on each surfaces and a meaningful
bijection between them have to be simultaneously determined. In this
paper, we propose a variational model to solve the inconsistent surface
registration problem by minimizing mapping distortions. Mapping
distortions are described by the Beltrami coefficient as well as the
differential of the mapping. Registration is then guided by feature
landmarks and/or intensities, such as curvatures, defined on each
surfaces. The key idea of the approach is to control angle and scale
distortions via quasiconformal theory as well as minimizing landmark
and/or intensity mismatch. A splitting method is proposed to
iteratively search for the optimal corresponding regions as well as the
optimal bijection between them. Bijectivity of the mapping is easily
enforced by a thresholding of the Beltrami coefficient. We test the
proposed method on both synthetic and real examples. Experimental
results demonstrate the efficacy of our proposed model.
Detection of abnormal deformations from normal motions is
crucial in image analysis, especially for medical image analysis. For
instance, accurate extraction of abnormal cardiac motions is a
necessary procedure for cardiac disease analysis. The combination of
nomral motion and abnormal deformations bring challenges to extract
abnormalities. In this work, we propose a novel method to extract
abnormal deformation from normal (periodic) motions by using the
Beltrami coefficients (BC). BCs are used to represent a sequence of
deformations over a sequence of images capturing an object of interest
in motion. RPCA is then performed on the BCs to decompose the overall
motion into nomral motion and abnormal deformation. Experiments have
been carried out on both synthetic and real medical image sequence,
which demonstrate the efficacy of our proposed method. Gallery
Image segmentation aims to partition an image into
meaningful
regions and extract important objects therein. It is an important and
yet ambiguous problem in computer visions. Common methods usually
impose various constraints on the extracted regions as regularization
to achieve the image segmentation goal. In practical situations, it is
desirable to incorporate shape prior information about the objects into
the segmentation model. In this work, we present a novel shape prior
image segmentation model based on discrete conformality structure. The
discrete conformality structure captures the angle structures of the
meshes representing the segmented objects. Shape prior information can
be prescribed into the segmentation model by imposing constraints on
the conformality structures. Segmentation results with prescribed local
or global shape priors can be easily obtained. We illustrate our idea
on various shape prior segmentation problems such as the convexity
prior segmentation problem. Experimental results demonstrate the
efficacy of our proposed method.
We propose a novel way of computing surface folding
maps via solving a linear PDE. This framework is a generalization to
the existing quasiconformal methods and allows manipulation of the
geometry of folding. Moreover, the crucial quantity that characterizes
the geometry occurs as the coefficient of the equation, namely the
Beltrami coefficient. This allows us to solve an inverse problem of
parametrizing the folded surface given only partial data but with known
folding topology. Various interesting applications such as fold
sculpting on 3D models and self-occlusion reasoning are demonstrated to
show the effectiveness of our method.
We propose an efficient parallel algorithm for
computing the global conformal parameterization of a simply-connected
open or closed surface embedded in R3. The method is specially designed
for large-scale problems, of which surface meshes contain a large
number of vertices. The main idea is based on the observation that a
global conformal parameterization can be retrieved from local
parameterizations of arbitrary patches using the welding maps. A
numerical algorithm for the global conformal parameterization of a
high-resolution surface mesh can then be developed by conformally
glueing the local conformal parameterizations of various patches. Each
local conformal parameterization can be computed in parallel. Hence,
our proposed algorithm is parallelizable. The procedure conformal
glueing can be efficiently carried out through a partial zipper
algorithm, which involves only the boundary vertices. Extensive
experiments on different simply-connected open and closed surface
meshes demonstrate the efficacy of our proposed method. Gallery
Surface registration between cortical surfaces is crucial in medical imaging for performing systematic comparisons between brains. Landmark-matching registration that matches anatomical features, called the sulcal landmarks, is often required to obtain a meaningful 1-1 correspondence between brain surfaces. This is commonly done by parameterizing the surface onto a simple parameter domain, such as the unit sphere, in which the sulcal landmarks are consistently aligned. Landmark-matching surface registration can then be obtained from the landmark aligned parameterizations. For genus-0 closed brain surfaces, the optimized spherical harmonic parameterization, which aligns landmarks to consistent locations on the sphere, has been widely used. This approach is limited by the loss of bijectivity under large deformations and the slow computation. In this paper, we propose FLASH, a fast algorithm to compute the optimized spherical harmonic parameterization with consistent landmark alignment. This is achieved by formulating the optimization problem to
the extended complex plane and thereby linearizing the problem. Errors introduced near the pole are corrected using quasi-conformal theories. Also, by adjusting the Beltrami differential of the mapping, a diffeomorphic (1-1, onto) spherical parameterization can be effectively obtained. The proposed algorithm has been tested on 38 human brain surfaces. Experimental results demonstrate that the computation of the landmark aligned spherical harmonic parameterization is significantly accelerated using the proposed algorithm.
Gallery
We address the problem of restoring a high-quality
image from an observed image sequence strongly distorted by atmospheric
turbulence. A novel algorithm is proposed in this paper to reduce
geometric distortion as well as space and time-varying blur due to
turbulence. By considering an optimization problem, our algorithm first
obtain a sharp reference image and a sub-sampled image sequence
containing sharp and mildly distorted image frames with respect to the
reference image. The sub-sampled image sequence is then stabilized by
applying the Robust Principal Component Analysis (RPCA) on the
deformation fields between image frames and warping the image frames by
a quasiconformal map associated to the low-rank part of the deformation
matrix. After image frames are registered to the reference image, the
low-rank part of them are deblurred via a blind deconvolution, and the
deblurred frmaes are then fused with the enhanced sparse part.
Experiments have been carried out on both synthetic and real
turbulence-distorted video. Results demonstrate our method is effecitve
in alleviating distortions and blur, restoring image details and
enhancing visual quality.
Image retargeting aims to resize an image to one with
prescribed aspect ratio. Simple scaling inevitably introduces unnatural
geometric distortions on the important contents of the image. In this
paper, we propose a simple and yet effective method to resize an image,
which preserves the geometry of the important content, using
quasiconformal theories. Our algorithm allows users to interactively
identify content regions as well as line structures. Image resizing can
then be acheived by warping the image by an orientation-preserving
homeomorphism with controlled distortion. The warping map is
represented by its Beltrami representation, which captures the local
geometric distortion of the map. By carefully setting the values of the
Beltrami representation, we propose three methods to resize an image
for different situations. Our method is simple and does not require
solving any optimization problems throughtout the process. This results
in a simple and efficient algorithm to solve the image retargeting
problem. Experiment results demonstrate the efficacy of our proposed
method.
