SlideShare une entreprise Scribd logo
1  sur  114
Télécharger pour lire hors ligne
Image Segmentation
(deformable segmentation)
Xianghua Xie
Computer Science Department, Swansea University, UK
http://csvision.swan.ac.uk
n  high level understanding of images
n  May involve segmentation
n  mid level image understanding
n  From low level representations, such as pixels and edges, to provide
representation that is compact and expressive
n  Segmentation often involves grouping, perceptual organisation and
fitting.
n  Image partition is taken place in the image spatial domain, but grouping,
fitting and so on can be in other domain, e.g. frequency or spatio-
frequency
Why Segmentation
Image Segmentation
Original image Bottom up segmentation Object level segmentation
Berkeley Segmentation
Dataset is a good starting
point (evaluation metrics)
Image Segmentation
n  Segmentation with little constraint
n  Thresholding
n  Region growing, split and merge
n  Watershed
n  With weak constraint
n  Graph cut
n  Deformable models, such as active contour
n  Interactive segmentation, such as intelligent scissors, grab
cut
n  With strong constraint
n  Active shape model, active appearance model based
segmentation
n  Atlas model based segmentation
n  Registration populating segmentation
n  Two key questions: prior generalisation and model
adaptation
Bottomup
Modeldriven
Outline
n  Deformable Segmentation
n  Edge based 2D segmentation
n  Extension to 3D
n  Direct sequential segmentation of temporal volumetric data
n  Incorporating statistical shape prior
n  3D + T (4D) constrained segmentation
n  Tracking using implicit representation
n  Implicit representation using RBF (region based)
n  Hybrid approach
n  Level set intrinsic regularisation (initialisation invariance)
n  Integrated Reconstruction & Segmentation
n  Combinatorial Optimisation
n  Minimum path: lymphatic membrane segmentation
n  Optimal surface (minimum cut): coronary segmentation
Deformable Segmentation
Deformable Segmentation
Deformable Segmentation
n  Design issues
n  Representation & numerical method
n  Explicit Vs Implicit
n  FEM, FDM, Spectral methods
n  RBF-Level Set (Xie & Mirmehdi 07, Xie 11)
n  Boundary description & stopping function
n  Gradient based (Caselles et al. 97, Xie & Mirmehdi 08, Xie 10
& 11, Yeo et al. 11 )
n  Region based (Paragios 02, Chan & Vese 01, Xie 09 & 11)
n  Hybrid approach (Wang & Vemuri 04, Xie & Mirmehdi 04)
n  Initialisation and convergence
n  Initialisation independency (Xie 10, Xie 11, Yeo et al. 11)
n  Complex topology & shape (Xie & Mirmehdi 07, 08, Paiement
14)
n  These issues are often interdependent
Deformable model
n  Active contour:
n  Dynamic curves within image domain to recover object shapes.
n  Deformable surface:
n  Its extension to 3D.
n  Applications:
n  Object localisation
n  Motion tracking
n  Segmentation (e.g. colour/texture)
n  Two general types: explicit and implicit models
n  Kass et al. 1988, Caselles et al.1993, and many more…
Explicit model
n  Parametric snake
n  Represented explicitly as parameterized curves, spline, polynormial
function
n  Example classic parametric snake
n  Snake evolves to minimize the internal and external forces (Let C(q)
be a parameterized planar curve);
n  Initialisation problem;
n  Concavity convergence problem;
( ) ( ) ( ) ( )( ) .
22
∫∫∫ ∇−ʹ′ʹ′+ʹ′= dqqCIdqqCdqqCCE λβα
Internal forces External force
Explicit model
n  Point based on tracking
n  Resolution problem
n  Addition and deletion
n  And …
n  Topological problems
n  Non-intrinsic, parameterisation dependent;
n  Hard to detect multiple objects simultaneously.
n  Example:
?
Explicit model
n  Advantages
n  Explicit control
n  Point correspondence
n  Probably easier to impose shape regularisation
n  Computational efficiency
n  Should be considered when
n  Known topology
n  No (or predicated) topological changes
n  Open curves
n  …
n  Numerical method
n  Finite element method (FEM)
n  Discretise into sub-domain
n  Cohen & Cohen, IEEE T-PAMI, 1993
Implicit model
n  Popularly based on the Level Set technique
n  Implicit snake models
n  Introduced by Caselles et al. and Malladi et al. (1993);
n  Based on the theory of curve evolution
n  Numerically implemented via level set methods;
n  Snake evolves to minimize the weighted length in a Riemannian space
with a metric derived from the image content;
n  Weighted length minimisation example:
A
B
A
B
Curve evolution
n  Curvature flow
n  k is the curvature, N denotes the inward normal
n  The curvature measures how fast each point moves along its normal
direction;
n  A simple closed curve will evolve toward a circular shape and
disappear;
n  It smoothes the curve.
NCt
!
κ=
Curve evolution
n  Constant flow
n  c is a constant, N denotes the inward normal
n  Each point moves at a constant speed along its normal direction;
n  It can cause a smooth curve to become a singular one;
n  A.k.a. the balloon force.
NcCt
!
=
Level set method
n  A computational technique for tracking propagating interface
n  Embed the curve into a surface, 2D scalar field
n  Zero level set corresponds to the embedded curve
n  Deforming the surface, instead of explicitly deforming the curve
Level set method
n  A computational technique for tracking propagating interface
n  Embed the curve into a surface, 2D scalar field
n  Zero level set corresponds to the embedded curve
n  Deforming the surface, instead of explicitly deforming the curve
Level set method
n  Key ideas introduced by Dervieux & Thomasset
n  Lecture Notes in Physics 1980
n  Well-known after the seminal work by Osher & Sethian
n  Osher & Sethian, J. Computational Physics 1988
n  Fluid dynamics, computational geometry, material science, computer vision,
…
n  Introduced to snake methods by Casselles et al. & Malladi et al.
n  Casselles et al., Nemuer. Math. 1993
n  Malladi et al., IEEE T-PAMI 1995
n  Advantages:
n  Implicit, intrinsic, non-parametric
n  Accurate modelling front propagation
n  Capable of handling topological changes (almost!)
Level set method
n  General curve evolution
n  is the level set function, F is the speed function
n  The curvature flow can be re-formulated as:
n  The constant flow can be re-formulated as:
0|| =Φ∇+Φ Ft
Φ
||
||
Φ∇⎟⎟
⎠
⎞
⎜⎜
⎝
⎛
Φ∇
Φ∇
⋅∇=Φt
|| Φ∇=Φ ct
Level set method
n  Numerical method
n  Finite difference method (FDM)
n  A local method to estimate the partial derivatives
n  Upwind scheme
n  Reinitialisation
n  Reshaping the level set surface to retain smoothness
n  Periodically performed
n  Fast marching method, or
n  By solving the following PDE:
|)|1)((sign Φ∇−Φ=Φt
Level set method
n  Challenges:
n  Computational complexity
n  FDM – a local method to estimate the partial derivatives
n  Dense computational grid
n  More expensive in 3D
n  Fast marching, narrow band, additive operator splitting (AOS)
n  More sophisticated topological changes
n  Signed distance function
n  Reinitialisation is necessary to eliminate accumulated numerical error
n  Prevent the level set developing new components
n  Do not allow perturbations away from zero level set
n  Can not create new contours
n  E.g. fail to localise internal object boundaries
Level set method
n  Challenges:
n  Computational complexity
n  FDM – a local method to estimate the partial derivatives
n  Dense computational grid
n  More expensive in 3D
n  Fast marching, narrow band, additive operator splitting (AOS)
n  More sophisticated topological changes
n  Signed distance function
n  Reinitialisation is necessary to eliminate accumulated numerical error
n  Prevent the level set developing new components
n  Do not allow perturbations away from zero level set
n  Can not create new contours
n  E.g. fail to localise internal object boundaries
Numerical solution
n  The level set time derivative is approximated by forward
difference:
n  Force fields can be classified into three types
n  Curvature flow
n  Constant flow
n  Advection force field
n  Each requires different differencing scheme
Numerical solution
n  Weighted curvature flow:
Central differencing
Numerical solution
n  Weighted constant flow:
Upwinding differencing
n
ji
n
ji Vcg ,0, |)|(|)|(.)( Φ∇=Φ∇
Numerical solution
n  Advection flow:
n  Let denote the external velocity force field
n  Check the sign of each component
n  Construct one-sided upwind differences
n  E.g. GVF,
n  Edge Based 2D Segmentation
Xie & Mirmehdi, MAC, IEEE Trans. Pattern Analysis & Machine Intelligence 2008.
Motivation
n  Convergence study – 4 disc problem
GeodesicDVF GGVF GeoGGVF CVF MAC
DVF: Cohen & Cohen, IEEE T-PAMI, 1993
Geodesic: Caselles et al., IJCV, 1997
GGVF: Xu & Prince, Signal Processing, 1998
GeoGGVF: Paragios et al., IEEE T-PAMI, 2004
CVF: Gil & Radeva, EMMCVPR 2003
CPM: Jalba et al., IEEE T-PAMI 2004
Xie & Mirmehdi, MAC, IEEE Trans. Pattern Analysis & Machine Intelligence 2008.
Motivation
n  Convergence study – 4 disc problem
GeodesicDVF GGVF GeoGGVF CVF MAC
DVF: Cohen & Cohen, IEEE T-PAMI, 1993
Geodesic: Caselles et al., IJCV, 1997
GGVF: Xu & Prince, Signal Processing, 1998
GeoGGVF: Paragios et al., IEEE T-PAMI, 2004
CVF: Gil & Radeva, EMMCVPR 2003
CPM: Jalba et al., IEEE T-PAMI 2004
Xie & Mirmehdi, MAC, IEEE Trans. Pattern Analysis & Machine Intelligence 2008.
Motivation
n  Objectives
n  Long range force interaction
n  Dynamic force field, instead of static
n  Bidirectional – allow cross boundary initialisation
n  Efficiency
n  Region based or Edge based
n  Prior knowledge
n  Boundary assumptions
n  Discontinuity in regional statistics
n  Discontinuity in image intensity
n  Application dependent
n  Goal: improving edge based performance
n  Comparable to region based approaches
n  Benefit from less prior knowledge, simpler assumption, and efficiency
n  There are scenarios boundary description does not need region support
MAC model
n  Proposed method
n  Novel external force field
n  Based on hypothesised magnetic interactions between object boundary
and snake
n  Significant improvements upon initialisation invariancy &
convergence ability
n  Yet, a very simple model
n  Magnetostatics
MAC model
n  Edge orientation
n  Analogy to current orientation
n  Rotating image gradient vectors
= 1: anti-clockwise rotation; = 2: clockwise rotation.
: normalised image gradient vectors.
n  (actually, these are 3D vectors)
n  Current orientation on snake
n  Similar to edge current orientation estimation
n  Rotating level set gradient vectors
MAC model
n  Magnetic force on snake
n  Derive the force on snake exerted from image gradients
: electric current unit vector on snake
: current magnitude on snake, constant
: electric current vector on edges
: current magnitude on edges
: unit vector between two point, x and s
: permeability constant
n  Uniqueness
n  The force on snake is dynamic
n  Relies on both spatial position and evolving contour
n  Always perpendicular to the snake
n  Global force interaction
MAC model
n  Snake formulation
: curvature
: snake inward normal
n  Level set representation
n  Force field extension
n  Snake is extended in a 2D scalar function
n  Accordingly its forces upon it
n  Fast marching
n  In this case, simply compute forces for each level set
MAC model
n  An example of dynamic force field
MAC model
n  Edge preserving force diffusion
n  Minimise noise interference
n  Nonlinear diffusion of magnetic flux density
n  Similar to GGVF, but…
n  Add edge weighting term in diffusion control
n  As little diffusion as possible at strong edges
n  Homogeneous and noisy area which lack consistent support
from edges will have larger diffusion
MAC model
n  Edge preserving force diffusion
n  Fast implementation
n  Decompose the magnetic flux term
n  Fast computation in the Fourier domain
Experimental results
n  Comparative analysis on synthetic images
GeodesicDVF GGVF GeoGGVF CVF MAC
Experimental results
n  Arbitrary initialisation
Experimental results
n  Noise sensitivity
20% noise 30% noise 40% noise 50% noise
Experimental results
n  Weak edges
Geodesic GGVF CPM MAC
Experimental results
n  Weak edges
n  Brief comparison to Region Based
Region based (MoG) MAC
Geodesic GGVF CPM MAC
Experimental results
n  On real images
GeodesicDVF GGVF
GeoGGVF CPM MAC
Experimental results
n  On real images
GeodesicDVF GGVF
GeoGGVF CPM MAC
Experimental results
n  On different types of images
Planar X-ray
CT
Ultrasound MRI
Experimental results
n  Dual level set
Xie & Mirmehdi, MAC, IEEE Trans. Pattern Analysis & Machine Intelligence 2008.
n  Extension to 3D
Yeo, Xie, Sazonov, Nithiarasu, GPF, IEEE Trans. Image Processing 2011.
GPF model
n  Geometrical Potential Force
n  Suitable for 3D data
n  Based on hypothesised geometrically induced force field between
deformable model and object boundary
n  Generalisation of the MAC model
n  Unique bi-directionality
n  Dynamic force interaction
n  Global view of object boundary representation
Yeo, Xie, Sazonov, Nithiarasu, GPF, IEEE Trans. Image Processing 2011.
GPF model
n  Interaction force acting on due to is given as
q  – corresponding geometrically induced potential created by
GPF model
n  Comparative Results
Target objects Initialisations Geodesic GGVF Proposed GPF
Yeo, Xie, Sazonov, Nithiarasu, GPF, IEEE Trans. Image Processing 2011.
GPF model
n  Medical 3D data segmentation
Aneurysm Aorta
GPF model
n  Comparative analysis
Ground-truth GGVFGeodesic Proposed GPF
GPF model
n  Further examples
GPF model
n  Further examples
Human aorta (CT) Human carotid (CT)
GPF model
n  Direct sequential segmentation of temporal volumetric data
GPF model
n  Direct sequential segmentation of temporal volumetric data
GPF model
n  Direct sequential segmentation of temporal volumetric data
GPF model
n  Direct sequential segmentation of temporal volumetric data
GPF model
n  Direct sequential segmentation of temporal volumetric data
GPF model
n  Direct sequential segmentation of temporal volumetric data
GPF model
n  Direct sequential segmentation of temporal volumetric data
GPF model
n  Direct sequential segmentation of temporal volumetric data
GPF model
n  Direct sequential segmentation of temporal volumetric data
GPF model
n  Direct sequential segmentation of temporal volumetric data
GPF model
n  Direct sequential segmentation of temporal volumetric data
GPF model
n  Segmentation using statistical shape prior
GPF model
n  Segmentation using statistical shape prior
Segmentation of corpus callosum from MRI
Image based energy Image and shape based energy
n  3D+T (4D) constrained SPECT segmentation
Yang, Mirmehdi, Xie, Hall, CI2BM, MICCAI workshop 2009. (MIA under-review)
n  Segmentation of LV borders allows quantitative analysis of
perfusion defects and cardiac function.
4D SPECT Segmentation
SPECT slice of the LV A DoughnutCardiac motion (mid-slice)
Yang, Mirmehdi, Xie, Hall, CI2BM, MICCAI workshop 2009. (MIA under-review)
Frontal view of
opaque surface
Top view of
opaque surface
Short-axis view
Frontal view
Correspondence between short-axis
slice and 3D frontal view
Frontal view of opaque surface overlaid on
orthogonal slice planes
Frontal view of
transparent surface
4D SPECT Segmentation
Automatic
Initialization
Snapshots
Input
Slice
CPM
Geodesic
Snake
Geodesic
GVF
CACE
Ground
truth
Example 1
Example 2
Example 3
Problem
Cases
4D SPECT Segmentation
Training Image and
Shape Sequences
Gaussian
Analysis
PCA
Gaussian
Priors
Spatiotemporal
Priors
Segmentation
Unseen
Sequence
CACE
Evolution
Constraint (based on
Gaussian and
Spatiotemporal Priors)‫‏‬
Update Level Sets
and Spatiotemporal
Parameters
Convergence
End
No
Yes
Training
GtL trans.
4D SPECT Segmentation
The image cannot be displayed.
Your computer may not have
enough memory to open the
image, or the image may have
been corrupted. Restart your
computer, and then open the file
again. If the red x still appears, you
may have to delete the image and
then insert it again.
The image cannot be displayed.
Your computer may not have
enough memory to open the
image, or the image may have
been corrupted. Restart your
computer, and then open the file
again. If the red x still appears, you
may have to delete the image and
then insert it again.
The image cannot be displayed.
Your computer may not have
enough memory to open the
image, or the image may have
been corrupted. Restart your
computer, and then open the file
again. If the red x still appears,
you may have to delete the image
and then insert it again.
The image cannot be displayed.
Your computer may not have
enough memory to open the
image, or the image may have
been corrupted. Restart your
computer, and then open the file
again. If the red x still appears, you
may have to delete the image and
then insert it again.
The image cannot be displayed.
Your computer may not have
enough memory to open the
image, or the image may have
been corrupted. Restart your
computer, and then open the file
again. If the red x still appears, you
may have to delete the image and
then insert it again.
The image cannot be displayed.
Your computer may not have
enough memory to open the
image, or the image may have
been corrupted. Restart your
computer, and then open the file
again. If the red x still appears, you
may have to delete the image and
then insert it again.
The image cannot be displayed.
Your computer may not have
enough memory to open the
image, or the image may have
been corrupted. Restart your
computer, and then open the file
again. If the red x still appears, you
may have to delete the image and
then insert it again.
The image cannot be displayed.
Your computer may not have
enough memory to open the
image, or the image may have
been corrupted. Restart your
computer, and then open the file
again. If the red x still appears, you
may have to delete the image and
then insert it again.
The image cannot be
displayed. Your computer
may not have enough
memory to open the image,
or the image may have been
corrupted. Restart your
computer, and then open the
file again. If the red x still
appears, you may have to
delete the image and then
insert it again.
The image cannot be
displayed. Your computer
may not have enough
memory to open the image,
or the image may have been
corrupted. Restart your
computer, and then open the
file again. If the red x still
appears, you may have to
delete the image and then
insert it again.
The image cannot be
displayed. Your computer
may not have enough
memory to open the image,
or the image may have been
corrupted. Restart your
computer, and then open the
file again. If the red x still
appears, you may have to
delete the image and then
insert it again.
The image cannot be
displayed. Your computer
may not have enough
memory to open the image,
or the image may have been
corrupted. Restart your
computer, and then open the
file again. If the red x still
appears, you may have to
delete the image and then
insert it again.
The image cannot be
displayed. Your computer
may not have enough
memory to open the image,
or the image may have been
corrupted. Restart your
computer, and then open the
file again. If the red x still
appears, you may have to
delete the image and then
insert it again.
The image cannot be
displayed. Your computer
may not have enough
memory to open the image,
or the image may have been
corrupted. Restart your
computer, and then open the
file again. If the red x still
appears, you may have to
delete the image and then
insert it again.
The image cannot be
displayed. Your computer
may not have enough
memory to open the image,
or the image may have been
corrupted. Restart your
computer, and then open the
file again. If the red x still
appears, you may have to
delete the image and then
insert it again.
The image cannot be
displayed. Your computer
may not have enough
memory to open the image,
or the image may have been
corrupted. Restart your
computer, and then open the
file again. If the red x still
appears, you may have to
delete the image and then
insert it again.
CCACE: 85.8%
SCMS: 63.1%
Results
on input
image
Results
against
ground
truth
Results
on input
image
Results
against
ground
truth
4D SPECT Segmentation
Slice 24
Slice 25
Slice 26
Slice 27
Slice 28
4D SPECT Segmentation – defect detection
n  Tracking using implicit representation
Chiverton, Xie & Mirmehdi, BMVC 2008 & 2009, IEEE TIP 2012
Tracking & Online Shape Learning
n  Prior independent snake tracking
n  Contour based tracking
n  Probably more difficult than box based tracking
n  No prior knowledge
n  Online shape learning and dynamic updating
60% random noise
Tracking & Online Shape Learning
n  Prior independent snake tracking
n  Contour based tracking
n  Probably more difficult than box based tracking
n  No prior knowledge
n  Online shape learning and dynamic updating
Tracking & Online Shape Learning
n  Contour based object tracking
n  Online shape learning
n  Self-imposed shape
regularisation
Without online shape self-regularisation
Proposed method
Chiverton, Xie & Mirmehdi, BMVC 2008 & 2009.
Automatic Bootstrapping & Tracking
Chiverton, Xie & Mirmehdi, IEEE T-IP 2012.
n  Online shape learning coupled with automatic bootstrapping
n  Finite size shape memory
n  Statistical shape modelling
n  Level set based tracking – similar to previous approach
n  RBF-Level Set based Active Contouring
Xie & Mirmehdi, Image and Vision Computing 2011 & BMVC07.
Conventional level set technique
n  Problems
n  Computational complexity
n  Dense computation grid, particularly expensive in 3D
n  Some solutions: fast marching, narrow band, AoS schemes, …
n  Can’t handle more sophisticated topological changes
n  Usually requires re-initialisation to maintain a smooth surface to prevent
numerical artefacts contaminating the solution
n  Perturbations away from the zero level set are missed
Conventional level set:
The hole is missed!
RBF-Level Set
n  RBF-Level Set
n  Use radial basis function to interpolate level set
n  Updating expansion coefficients to deform level set
n  Transfer PDE to ODE: efficient
n  Much coarser computational grid, even irregular
n  More complex topological changes readily achievable
Conventional level set Proposed method
RBF-Level Set
n  RBF-Level Set
n  Use radial basis function to interpolate level set
n  Updating expansion coefficients to deform level set
n  Transfer PDE to ODE: efficient
n  Much coarser computational grid, even irregular
n  More complex topological changes readily achievable
Conventional level set Proposed method
RBF-Level Set
n  RBF interpolation
n  Level set function, : a scalar function, usually obtained from the signed
distance transform
n  Interpolate using a linear combination of a radial basis function,
where p(x) is a first degree polynomial and are the expansion coefficients
n  The interpolation can be expressed as:
where
6
RBF-Level Set
n  Updating RBF level set
n  Original level set evolution:
where F is the speed function in the normal direction
n  Transferred evolution:
n  The spatial derivatives can be solved analytically
n  First order Euler’s method
n  Iteratively updating the expansion coefficients to evolve level set
n  Benefits:
n  Coarse computational grid, could be irregular
n  No need for re-initialisation
n  More complex topological changes achievable
RBF-Level Set snake
n  Prevent self-flattening
Non-normalised Normalised
RBF-Level Set snake
n  Prevent self-flattening
Non-normalised Normalised
RBF-Level Set snake
n  Prevent self-flattening
Non-normalised Normalised
RBF-Level Set
n  Active modelling using RBF level set
n  A region based approach
n  Texem based modelling
n  Active contour formulation:
n  m is the number of classes
n  1/m is the average expectation of a class
n  u is the posterior of the class of interest
n  Level set representation:
I
RBF-Level Set
n  Texems are image representations at various sizes that
retain the texture or visual primitives of a given image.
n  A two-layer generative model
n  Each texem represented by mean and variance: m={µ,ω}
n  A bottom-up learning procedure
learning
{ }…M
Z
Xie-Mirmehdi, IEEE T-PAMI, 29(8), 2007.
RBF-Level Set
n  Example learnt texems (7x7)
n  Multiscale branch based texems
n  Texem grouping for multi-modal regions
Xie-Mirmehdi, IEEE T-PAMI, 29(8), 2007.
RBF-Level Set
ConventionalProposed
n  On real images
RBF-Level Set
Proposed method
Conventional level set Proposed methodConventional level set Proposed method
RBF-Level Set
n  Deformable modelling in 3D
Recover a hollow sphere
Initialised outside the target object Complex geometry
Xie & Mirmehdi, Image & Vision Computing 2011 & BMVC 2007.
n  Hybrid Approach
Xie & Mirmehdi, IEEE Trans. Image Processing, 2004.
RAGS model
n  Region-aided (RAGS) model
n  Bridge boundary and region-based techniques
n  Fusing global information to local boundary description
n  Improvements towards weak edges
n  More resilient to noise interference
Geodesic snake Proposed methodGGVF snake
Xie & Mirmehdi, IEEE Trans. Image Processing, 2004.
n  Integrated Reconstruction, Registration and Segmentation
A. Paiement et al., IEEE Transactions on Image Processing, January 2014.
Motivation
n  Modelling from 3D/4D imaging data raises two intertwined issues:
n  segmentation
n  interpolation
n  Segmentation
n  partition 3D space containing the object and to distinguish data points belonging to the
object from background points
n  e.g. 2D slices, ranging from simple stacks of parallel slices to more
complicated spatial configurations
Motivation
n  Segmentation
n  2D independent segmentation is often not desirable
n  all the slices are better segmented simultaneously in 3D/4D
n  However, interpolation is thus necessary
n  since data often does not span the whole 3D space
n  only partial support from data
n  Imaging conditions
n  some modalities require integration over a thick slice to improve signal to noise ratio
n  e.g. 1.5T cine cardiac MRI typical slice thickness 7mm; hence spacing is 7mm or bigger
n  large spacing is also desirable in order to reduce acquisition time (patient discomfort,
motion artefact)
n  in the 4D case, data must also be interpolated between available time frames
Motivation
n  Argument
n  “the success of one stage (segmentation or interpolation) depends on the accuracy of the
other”
n  Approaches
n  two sequential approaches: perform these two stages in opposing order
n  some first segment slices independently then interpolate from 2D contours
n  shape interpolation, notable work: Liu et al., surface reconstruction from Non-parallel
curve network, CGF 27(2) 2008.
n  combining registration and segmentation
n  segment sparse volumes made up of 2D slices by registering and deforming a model on
images (e.g. ASM): prior is often necessary
n  level set based method: foundation (earlier work) for what presented here
n  integrate segmentation and interpolation into a new RBF interpolated level set framework
n  simplicity and flexibility of level set
n  stability of RBF
n  inherent interpolation provided by RBF
Proposed Method
n  Interpolate level set function using Strictly Positive Definitive (SPD) RBF
n  combining registration and segmentation
n  segment sparse volumes made up of 2D slices by registering and deforming a model
on images (e.g. ASM): prior is often necessary
n  level set based method: foundation (earlier work) for what presented here
n  integrate segmentation and interpolation into a new RBF interpolated level set framework
n  simplicity and flexibility of level set
n  stability of RBF
n  inherent interpolation provided by RBF
Proposed Method
n  Instead of evolve phi through expansion coefficients (which involves
inverting a large matrix), evolve alpha by minimising an energy functional E:
n  F may be any functional and is defined by the chosen segmentation method.
n  Conventional variational level set method:
n  using chain rule, a gradient descent method yields:
Proposed Method
n  Rename as .
n  S is the speed of the moving front and is generally defined on the contour C
only.
n  The alpha evolution function can thus be simplified as
n  is an approximation of the Dirac function
n  this imposes a restriction of S to the contour C
n  practical choice of regularised Dirac function:
§  epsilon = 1 for sharp RBF; epsilon = 3 for flatter RBF.
Proposed Method
n  RBF based interpolation methods usually define one control point per data
point
n  we define one control point per voxel of a discrete space,
n  thus allow rewrite as a
convolution:
n  The initial expansion coefficients can be computed in the Fourier domain
Results
Chan-Veseinitialisation narrow-band PC
proposed PC initialisation proposed PC
Paiement et al., IEEE Trans. Image Processing 2014.
Results
initial slices (top left quadrant
removed for visualisation)
central T1
weighted slice
modelled shape
central T2
weighted slice
Paiement et al., IEEE Trans. Image Processing 2014.
Results
initial slices
central short
axis slice
modelled shape
long axis
slice
Paiement et al., IEEE Trans. Image Processing 2014.
Example result
Example result
Combinatorial Optimisation
n  Shortest path problem: segmentation
Combinatorial Optimisation
n  Optimal surface (minimum cut)
n  Hard constraint
n  Statistical shape constraint
n  Temporal constraint: Kalman filter & HMM
Combinatorial Optimisation
IVUS s-t cut optimal surface texture RBF star graph proposed
n  Longitudinal cross section
n  Yellow: frame-by-frame; red: proposed; green: ground truth
Combinatorial Optimisation
n  Team
n  Dr. Feng Zhao, Mr. Ehab Essa, Mr. Jingjing Deng, Mr. Mike Edwards,
Mr. Robert Palmer, Mr. Yaxi Ye, Mr. David James, Mr. Jonathan
Jones
n  Alumni
n  Dr. Ben Daubney, Dr. Huazizhong Zhang, Dr. Dongbin Chen, Dr. Si
Yong Yeo, Dr. Cyril Charron, Dr. John Chiverton, Dr. Ronghua Yang,
Mr. Liu Ren, Mr. Arron Lacey
n  Clinical collaborator
n  Swansea Singleton Hospital ABM UHT at Morriston
n  Bristol Royal Infirmary
n  Cardiff Hospital
n  Plymouth Hospital
Acknowledgement
csvision.swan.ac.uk

Contenu connexe

Similaire à Image segmentation

Stixel based real time object detection for ADAS using surface normal
Stixel based real time object detection for ADAS using surface normalStixel based real time object detection for ADAS using surface normal
Stixel based real time object detection for ADAS using surface normalTaeKang Woo
 
Mesh Shape Editing
Mesh Shape EditingMesh Shape Editing
Mesh Shape Editingssuser24ddad
 
study Shading Based Surface Editing
study Shading Based Surface Editingstudy Shading Based Surface Editing
study Shading Based Surface EditingChiamin Hsu
 
A STUDY AND ANALYSIS OF DIFFERENT EDGE DETECTION TECHNIQUES
A STUDY AND ANALYSIS OF DIFFERENT EDGE DETECTION TECHNIQUESA STUDY AND ANALYSIS OF DIFFERENT EDGE DETECTION TECHNIQUES
A STUDY AND ANALYSIS OF DIFFERENT EDGE DETECTION TECHNIQUEScscpconf
 
Física diapositivas-coordinadas tangenciales y normales
Física diapositivas-coordinadas tangenciales y normalesFísica diapositivas-coordinadas tangenciales y normales
Física diapositivas-coordinadas tangenciales y normalesODALISFERNANDAMOPOSI
 
Chapter10 image segmentation
Chapter10 image segmentationChapter10 image segmentation
Chapter10 image segmentationasodariyabhavesh
 
Computer Graphics - Lecture 03 - Virtual Cameras and the Transformation Pipeline
Computer Graphics - Lecture 03 - Virtual Cameras and the Transformation PipelineComputer Graphics - Lecture 03 - Virtual Cameras and the Transformation Pipeline
Computer Graphics - Lecture 03 - Virtual Cameras and the Transformation Pipeline💻 Anton Gerdelan
 
Module-5-1_230523_171754 (1).pdf
Module-5-1_230523_171754 (1).pdfModule-5-1_230523_171754 (1).pdf
Module-5-1_230523_171754 (1).pdfvikasmittal92
 
Point Cloud Segmentation for 3D Reconstruction
Point Cloud Segmentation for 3D ReconstructionPoint Cloud Segmentation for 3D Reconstruction
Point Cloud Segmentation for 3D ReconstructionPirouz Nourian
 
Computer Vision - Alignment and Tracking.pptx
Computer Vision - Alignment and Tracking.pptxComputer Vision - Alignment and Tracking.pptx
Computer Vision - Alignment and Tracking.pptxothersk46
 
Sampling from Massive Graph Streams: A Unifying Framework
Sampling from Massive Graph Streams: A Unifying FrameworkSampling from Massive Graph Streams: A Unifying Framework
Sampling from Massive Graph Streams: A Unifying FrameworkNesreen K. Ahmed
 
Weighted Performance comparison of DWT and LWT with PCA for Face Image Retrie...
Weighted Performance comparison of DWT and LWT with PCA for Face Image Retrie...Weighted Performance comparison of DWT and LWT with PCA for Face Image Retrie...
Weighted Performance comparison of DWT and LWT with PCA for Face Image Retrie...cscpconf
 
[3D勉強会@関東] Deep Reinforcement Learning of Volume-guided Progressive View Inpa...
[3D勉強会@関東] Deep Reinforcement Learning of Volume-guided Progressive View Inpa...[3D勉強会@関東] Deep Reinforcement Learning of Volume-guided Progressive View Inpa...
[3D勉強会@関東] Deep Reinforcement Learning of Volume-guided Progressive View Inpa...Seiya Ito
 
Lecture03 p1
Lecture03 p1Lecture03 p1
Lecture03 p1aa11bb11
 
Lossless image compression via by lifting scheme
Lossless image compression via by lifting schemeLossless image compression via by lifting scheme
Lossless image compression via by lifting schemeSubhashini Subramanian
 
Visualization using tSNE
Visualization using tSNEVisualization using tSNE
Visualization using tSNEYan Xu
 

Similaire à Image segmentation (20)

Stixel based real time object detection for ADAS using surface normal
Stixel based real time object detection for ADAS using surface normalStixel based real time object detection for ADAS using surface normal
Stixel based real time object detection for ADAS using surface normal
 
Mesh Shape Editing
Mesh Shape EditingMesh Shape Editing
Mesh Shape Editing
 
study Shading Based Surface Editing
study Shading Based Surface Editingstudy Shading Based Surface Editing
study Shading Based Surface Editing
 
A STUDY AND ANALYSIS OF DIFFERENT EDGE DETECTION TECHNIQUES
A STUDY AND ANALYSIS OF DIFFERENT EDGE DETECTION TECHNIQUESA STUDY AND ANALYSIS OF DIFFERENT EDGE DETECTION TECHNIQUES
A STUDY AND ANALYSIS OF DIFFERENT EDGE DETECTION TECHNIQUES
 
Física diapositivas-coordinadas tangenciales y normales
Física diapositivas-coordinadas tangenciales y normalesFísica diapositivas-coordinadas tangenciales y normales
Física diapositivas-coordinadas tangenciales y normales
 
Image denoising using curvelet transform
Image denoising using curvelet transformImage denoising using curvelet transform
Image denoising using curvelet transform
 
lecture_16_jiajun.pdf
lecture_16_jiajun.pdflecture_16_jiajun.pdf
lecture_16_jiajun.pdf
 
Chapter10 image segmentation
Chapter10 image segmentationChapter10 image segmentation
Chapter10 image segmentation
 
Computer Graphics - Lecture 03 - Virtual Cameras and the Transformation Pipeline
Computer Graphics - Lecture 03 - Virtual Cameras and the Transformation PipelineComputer Graphics - Lecture 03 - Virtual Cameras and the Transformation Pipeline
Computer Graphics - Lecture 03 - Virtual Cameras and the Transformation Pipeline
 
UNIT-V
UNIT-VUNIT-V
UNIT-V
 
Module-5-1_230523_171754 (1).pdf
Module-5-1_230523_171754 (1).pdfModule-5-1_230523_171754 (1).pdf
Module-5-1_230523_171754 (1).pdf
 
Point Cloud Segmentation for 3D Reconstruction
Point Cloud Segmentation for 3D ReconstructionPoint Cloud Segmentation for 3D Reconstruction
Point Cloud Segmentation for 3D Reconstruction
 
Computer Vision - Alignment and Tracking.pptx
Computer Vision - Alignment and Tracking.pptxComputer Vision - Alignment and Tracking.pptx
Computer Vision - Alignment and Tracking.pptx
 
Sampling from Massive Graph Streams: A Unifying Framework
Sampling from Massive Graph Streams: A Unifying FrameworkSampling from Massive Graph Streams: A Unifying Framework
Sampling from Massive Graph Streams: A Unifying Framework
 
Weighted Performance comparison of DWT and LWT with PCA for Face Image Retrie...
Weighted Performance comparison of DWT and LWT with PCA for Face Image Retrie...Weighted Performance comparison of DWT and LWT with PCA for Face Image Retrie...
Weighted Performance comparison of DWT and LWT with PCA for Face Image Retrie...
 
[3D勉強会@関東] Deep Reinforcement Learning of Volume-guided Progressive View Inpa...
[3D勉強会@関東] Deep Reinforcement Learning of Volume-guided Progressive View Inpa...[3D勉強会@関東] Deep Reinforcement Learning of Volume-guided Progressive View Inpa...
[3D勉強会@関東] Deep Reinforcement Learning of Volume-guided Progressive View Inpa...
 
ECCV WS 2012 (Frank)
ECCV WS 2012 (Frank)ECCV WS 2012 (Frank)
ECCV WS 2012 (Frank)
 
Lecture03 p1
Lecture03 p1Lecture03 p1
Lecture03 p1
 
Lossless image compression via by lifting scheme
Lossless image compression via by lifting schemeLossless image compression via by lifting scheme
Lossless image compression via by lifting scheme
 
Visualization using tSNE
Visualization using tSNEVisualization using tSNE
Visualization using tSNE
 

Plus de potaters

Image formation
Image formationImage formation
Image formationpotaters
 
Ln l.agapito
Ln l.agapitoLn l.agapito
Ln l.agapitopotaters
 
Motion and tracking
Motion and trackingMotion and tracking
Motion and trackingpotaters
 
BMVA summer school MATLAB programming tutorial
BMVA summer school MATLAB programming tutorialBMVA summer school MATLAB programming tutorial
BMVA summer school MATLAB programming tutorialpotaters
 
Statistical models of shape and appearance
Statistical models of shape and appearanceStatistical models of shape and appearance
Statistical models of shape and appearancepotaters
 
Vision Algorithmics
Vision AlgorithmicsVision Algorithmics
Vision Algorithmicspotaters
 
Performance characterization in computer vision
Performance characterization in computer visionPerformance characterization in computer vision
Performance characterization in computer visionpotaters
 
Machine learning for computer vision part 2
Machine learning for computer vision part 2Machine learning for computer vision part 2
Machine learning for computer vision part 2potaters
 
Machine learning fro computer vision - a whirlwind of key concepts for the un...
Machine learning fro computer vision - a whirlwind of key concepts for the un...Machine learning fro computer vision - a whirlwind of key concepts for the un...
Machine learning fro computer vision - a whirlwind of key concepts for the un...potaters
 
Low level vision - A tuturial
Low level vision - A tuturialLow level vision - A tuturial
Low level vision - A tuturialpotaters
 
Local feature descriptors for visual recognition
Local feature descriptors for visual recognitionLocal feature descriptors for visual recognition
Local feature descriptors for visual recognitionpotaters
 
A primer for colour computer vision
A primer for colour computer visionA primer for colour computer vision
A primer for colour computer visionpotaters
 
Cognitive Vision - After the hype
Cognitive Vision - After the hypeCognitive Vision - After the hype
Cognitive Vision - After the hypepotaters
 
Graphical Models for chains, trees and grids
Graphical Models for chains, trees and gridsGraphical Models for chains, trees and grids
Graphical Models for chains, trees and gridspotaters
 
Medical image computing - BMVA summer school 2014
Medical image computing - BMVA summer school 2014Medical image computing - BMVA summer school 2014
Medical image computing - BMVA summer school 2014potaters
 
Decision Forests and discriminant analysis
Decision Forests and discriminant analysisDecision Forests and discriminant analysis
Decision Forests and discriminant analysispotaters
 

Plus de potaters (16)

Image formation
Image formationImage formation
Image formation
 
Ln l.agapito
Ln l.agapitoLn l.agapito
Ln l.agapito
 
Motion and tracking
Motion and trackingMotion and tracking
Motion and tracking
 
BMVA summer school MATLAB programming tutorial
BMVA summer school MATLAB programming tutorialBMVA summer school MATLAB programming tutorial
BMVA summer school MATLAB programming tutorial
 
Statistical models of shape and appearance
Statistical models of shape and appearanceStatistical models of shape and appearance
Statistical models of shape and appearance
 
Vision Algorithmics
Vision AlgorithmicsVision Algorithmics
Vision Algorithmics
 
Performance characterization in computer vision
Performance characterization in computer visionPerformance characterization in computer vision
Performance characterization in computer vision
 
Machine learning for computer vision part 2
Machine learning for computer vision part 2Machine learning for computer vision part 2
Machine learning for computer vision part 2
 
Machine learning fro computer vision - a whirlwind of key concepts for the un...
Machine learning fro computer vision - a whirlwind of key concepts for the un...Machine learning fro computer vision - a whirlwind of key concepts for the un...
Machine learning fro computer vision - a whirlwind of key concepts for the un...
 
Low level vision - A tuturial
Low level vision - A tuturialLow level vision - A tuturial
Low level vision - A tuturial
 
Local feature descriptors for visual recognition
Local feature descriptors for visual recognitionLocal feature descriptors for visual recognition
Local feature descriptors for visual recognition
 
A primer for colour computer vision
A primer for colour computer visionA primer for colour computer vision
A primer for colour computer vision
 
Cognitive Vision - After the hype
Cognitive Vision - After the hypeCognitive Vision - After the hype
Cognitive Vision - After the hype
 
Graphical Models for chains, trees and grids
Graphical Models for chains, trees and gridsGraphical Models for chains, trees and grids
Graphical Models for chains, trees and grids
 
Medical image computing - BMVA summer school 2014
Medical image computing - BMVA summer school 2014Medical image computing - BMVA summer school 2014
Medical image computing - BMVA summer school 2014
 
Decision Forests and discriminant analysis
Decision Forests and discriminant analysisDecision Forests and discriminant analysis
Decision Forests and discriminant analysis
 

Dernier

RHEOLOGY MODIFIERS: ENHANCING PERFORMANCE AND FUNCTIONALITY
RHEOLOGY MODIFIERS: ENHANCING PERFORMANCE AND FUNCTIONALITYRHEOLOGY MODIFIERS: ENHANCING PERFORMANCE AND FUNCTIONALITY
RHEOLOGY MODIFIERS: ENHANCING PERFORMANCE AND FUNCTIONALITYDnyandaBopche
 
Basics Of Computers | The Computer System
Basics Of Computers | The Computer SystemBasics Of Computers | The Computer System
Basics Of Computers | The Computer SystemNehaRohtagi1
 
AI Published & MIT Validated Perpetual Motion Machine Breakthroughs (2 New EV...
AI Published & MIT Validated Perpetual Motion Machine Breakthroughs (2 New EV...AI Published & MIT Validated Perpetual Motion Machine Breakthroughs (2 New EV...
AI Published & MIT Validated Perpetual Motion Machine Breakthroughs (2 New EV...Thane Heins
 
dkNET Webinar "The Multi-Omic Response to Exercise Training Across Rat Tissue...
dkNET Webinar "The Multi-Omic Response to Exercise Training Across Rat Tissue...dkNET Webinar "The Multi-Omic Response to Exercise Training Across Rat Tissue...
dkNET Webinar "The Multi-Omic Response to Exercise Training Across Rat Tissue...dkNET
 
20240315 ACMJ Diagrams Set 2.docx . With light, motor, coloured light, and se...
20240315 ACMJ Diagrams Set 2.docx . With light, motor, coloured light, and se...20240315 ACMJ Diagrams Set 2.docx . With light, motor, coloured light, and se...
20240315 ACMJ Diagrams Set 2.docx . With light, motor, coloured light, and se...Sharon Liu
 
Project report on Fasciola hepatica.docx
Project report on Fasciola hepatica.docxProject report on Fasciola hepatica.docx
Project report on Fasciola hepatica.docxpriyanshimanchanda4
 
Theory of indicators: Ostwald's and Quinonoid theories
Theory of indicators: Ostwald's and Quinonoid theoriesTheory of indicators: Ostwald's and Quinonoid theories
Theory of indicators: Ostwald's and Quinonoid theoriesChimwemweGladysBanda
 
Preparation of enterprise budget for integrated fish farming
Preparation of enterprise budget for integrated fish farmingPreparation of enterprise budget for integrated fish farming
Preparation of enterprise budget for integrated fish farmingbhanilsaa
 
SHAMPOO : OVERVIEW OF SHAMPOO AND IT'S TYPES.
SHAMPOO : OVERVIEW OF SHAMPOO AND IT'S TYPES.SHAMPOO : OVERVIEW OF SHAMPOO AND IT'S TYPES.
SHAMPOO : OVERVIEW OF SHAMPOO AND IT'S TYPES.kapgateprachi@gmail.com
 
Science9 Quarter 3:Latitude and altitude.pptx
Science9 Quarter 3:Latitude and altitude.pptxScience9 Quarter 3:Latitude and altitude.pptx
Science9 Quarter 3:Latitude and altitude.pptxteleganne21
 
Solid waste management_13_409_U1_2024.pptx
Solid waste management_13_409_U1_2024.pptxSolid waste management_13_409_U1_2024.pptx
Solid waste management_13_409_U1_2024.pptxkrishuchavda31032003
 
AI Published & MIT Validated Perpetual Motion Machine Breakthroughs (2 New EV...
AI Published & MIT Validated Perpetual Motion Machine Breakthroughs (2 New EV...AI Published & MIT Validated Perpetual Motion Machine Breakthroughs (2 New EV...
AI Published & MIT Validated Perpetual Motion Machine Breakthroughs (2 New EV...Thane Heins
 
Zoogeographical regions In the World.pptx
Zoogeographical regions In the World.pptxZoogeographical regions In the World.pptx
Zoogeographical regions In the World.pptx2019n04898
 
Advances in Oncolytic Virotherapy - Creative Biolabs
Advances in Oncolytic Virotherapy - Creative BiolabsAdvances in Oncolytic Virotherapy - Creative Biolabs
Advances in Oncolytic Virotherapy - Creative BiolabsCreative-Biolabs
 
Development of a Questionnaire for Identifying Personal Values in Driving
Development of a Questionnaire for Identifying Personal Values in DrivingDevelopment of a Questionnaire for Identifying Personal Values in Driving
Development of a Questionnaire for Identifying Personal Values in Drivingstudiotelon
 
structure of proteins and its type I PPT
structure of proteins and its type I PPTstructure of proteins and its type I PPT
structure of proteins and its type I PPTvishalbhati28
 
Pests of Maize_Dr.UPR_Identification, Binomics, Integrated Pest Management
Pests of Maize_Dr.UPR_Identification, Binomics, Integrated Pest ManagementPests of Maize_Dr.UPR_Identification, Binomics, Integrated Pest Management
Pests of Maize_Dr.UPR_Identification, Binomics, Integrated Pest ManagementPirithiRaju
 
Geometric New Earth, Solarsystem, projection
Geometric New Earth, Solarsystem, projectionGeometric New Earth, Solarsystem, projection
Geometric New Earth, Solarsystem, projectionWim van Es
 
Introduction about protein and General method of analysis of protein
Introduction about protein and General method of analysis of proteinIntroduction about protein and General method of analysis of protein
Introduction about protein and General method of analysis of proteinSowmiya
 
Cultivating various strains of Duckweed Syllabus.pdf
Cultivating various strains of Duckweed Syllabus.pdfCultivating various strains of Duckweed Syllabus.pdf
Cultivating various strains of Duckweed Syllabus.pdfHaim R. Branisteanu
 

Dernier (20)

RHEOLOGY MODIFIERS: ENHANCING PERFORMANCE AND FUNCTIONALITY
RHEOLOGY MODIFIERS: ENHANCING PERFORMANCE AND FUNCTIONALITYRHEOLOGY MODIFIERS: ENHANCING PERFORMANCE AND FUNCTIONALITY
RHEOLOGY MODIFIERS: ENHANCING PERFORMANCE AND FUNCTIONALITY
 
Basics Of Computers | The Computer System
Basics Of Computers | The Computer SystemBasics Of Computers | The Computer System
Basics Of Computers | The Computer System
 
AI Published & MIT Validated Perpetual Motion Machine Breakthroughs (2 New EV...
AI Published & MIT Validated Perpetual Motion Machine Breakthroughs (2 New EV...AI Published & MIT Validated Perpetual Motion Machine Breakthroughs (2 New EV...
AI Published & MIT Validated Perpetual Motion Machine Breakthroughs (2 New EV...
 
dkNET Webinar "The Multi-Omic Response to Exercise Training Across Rat Tissue...
dkNET Webinar "The Multi-Omic Response to Exercise Training Across Rat Tissue...dkNET Webinar "The Multi-Omic Response to Exercise Training Across Rat Tissue...
dkNET Webinar "The Multi-Omic Response to Exercise Training Across Rat Tissue...
 
20240315 ACMJ Diagrams Set 2.docx . With light, motor, coloured light, and se...
20240315 ACMJ Diagrams Set 2.docx . With light, motor, coloured light, and se...20240315 ACMJ Diagrams Set 2.docx . With light, motor, coloured light, and se...
20240315 ACMJ Diagrams Set 2.docx . With light, motor, coloured light, and se...
 
Project report on Fasciola hepatica.docx
Project report on Fasciola hepatica.docxProject report on Fasciola hepatica.docx
Project report on Fasciola hepatica.docx
 
Theory of indicators: Ostwald's and Quinonoid theories
Theory of indicators: Ostwald's and Quinonoid theoriesTheory of indicators: Ostwald's and Quinonoid theories
Theory of indicators: Ostwald's and Quinonoid theories
 
Preparation of enterprise budget for integrated fish farming
Preparation of enterprise budget for integrated fish farmingPreparation of enterprise budget for integrated fish farming
Preparation of enterprise budget for integrated fish farming
 
SHAMPOO : OVERVIEW OF SHAMPOO AND IT'S TYPES.
SHAMPOO : OVERVIEW OF SHAMPOO AND IT'S TYPES.SHAMPOO : OVERVIEW OF SHAMPOO AND IT'S TYPES.
SHAMPOO : OVERVIEW OF SHAMPOO AND IT'S TYPES.
 
Science9 Quarter 3:Latitude and altitude.pptx
Science9 Quarter 3:Latitude and altitude.pptxScience9 Quarter 3:Latitude and altitude.pptx
Science9 Quarter 3:Latitude and altitude.pptx
 
Solid waste management_13_409_U1_2024.pptx
Solid waste management_13_409_U1_2024.pptxSolid waste management_13_409_U1_2024.pptx
Solid waste management_13_409_U1_2024.pptx
 
AI Published & MIT Validated Perpetual Motion Machine Breakthroughs (2 New EV...
AI Published & MIT Validated Perpetual Motion Machine Breakthroughs (2 New EV...AI Published & MIT Validated Perpetual Motion Machine Breakthroughs (2 New EV...
AI Published & MIT Validated Perpetual Motion Machine Breakthroughs (2 New EV...
 
Zoogeographical regions In the World.pptx
Zoogeographical regions In the World.pptxZoogeographical regions In the World.pptx
Zoogeographical regions In the World.pptx
 
Advances in Oncolytic Virotherapy - Creative Biolabs
Advances in Oncolytic Virotherapy - Creative BiolabsAdvances in Oncolytic Virotherapy - Creative Biolabs
Advances in Oncolytic Virotherapy - Creative Biolabs
 
Development of a Questionnaire for Identifying Personal Values in Driving
Development of a Questionnaire for Identifying Personal Values in DrivingDevelopment of a Questionnaire for Identifying Personal Values in Driving
Development of a Questionnaire for Identifying Personal Values in Driving
 
structure of proteins and its type I PPT
structure of proteins and its type I PPTstructure of proteins and its type I PPT
structure of proteins and its type I PPT
 
Pests of Maize_Dr.UPR_Identification, Binomics, Integrated Pest Management
Pests of Maize_Dr.UPR_Identification, Binomics, Integrated Pest ManagementPests of Maize_Dr.UPR_Identification, Binomics, Integrated Pest Management
Pests of Maize_Dr.UPR_Identification, Binomics, Integrated Pest Management
 
Geometric New Earth, Solarsystem, projection
Geometric New Earth, Solarsystem, projectionGeometric New Earth, Solarsystem, projection
Geometric New Earth, Solarsystem, projection
 
Introduction about protein and General method of analysis of protein
Introduction about protein and General method of analysis of proteinIntroduction about protein and General method of analysis of protein
Introduction about protein and General method of analysis of protein
 
Cultivating various strains of Duckweed Syllabus.pdf
Cultivating various strains of Duckweed Syllabus.pdfCultivating various strains of Duckweed Syllabus.pdf
Cultivating various strains of Duckweed Syllabus.pdf
 

Image segmentation

  • 1. Image Segmentation (deformable segmentation) Xianghua Xie Computer Science Department, Swansea University, UK http://csvision.swan.ac.uk
  • 2. n  high level understanding of images n  May involve segmentation n  mid level image understanding n  From low level representations, such as pixels and edges, to provide representation that is compact and expressive n  Segmentation often involves grouping, perceptual organisation and fitting. n  Image partition is taken place in the image spatial domain, but grouping, fitting and so on can be in other domain, e.g. frequency or spatio- frequency Why Segmentation
  • 3. Image Segmentation Original image Bottom up segmentation Object level segmentation Berkeley Segmentation Dataset is a good starting point (evaluation metrics)
  • 4. Image Segmentation n  Segmentation with little constraint n  Thresholding n  Region growing, split and merge n  Watershed n  With weak constraint n  Graph cut n  Deformable models, such as active contour n  Interactive segmentation, such as intelligent scissors, grab cut n  With strong constraint n  Active shape model, active appearance model based segmentation n  Atlas model based segmentation n  Registration populating segmentation n  Two key questions: prior generalisation and model adaptation Bottomup Modeldriven
  • 5. Outline n  Deformable Segmentation n  Edge based 2D segmentation n  Extension to 3D n  Direct sequential segmentation of temporal volumetric data n  Incorporating statistical shape prior n  3D + T (4D) constrained segmentation n  Tracking using implicit representation n  Implicit representation using RBF (region based) n  Hybrid approach n  Level set intrinsic regularisation (initialisation invariance) n  Integrated Reconstruction & Segmentation n  Combinatorial Optimisation n  Minimum path: lymphatic membrane segmentation n  Optimal surface (minimum cut): coronary segmentation
  • 8. Deformable Segmentation n  Design issues n  Representation & numerical method n  Explicit Vs Implicit n  FEM, FDM, Spectral methods n  RBF-Level Set (Xie & Mirmehdi 07, Xie 11) n  Boundary description & stopping function n  Gradient based (Caselles et al. 97, Xie & Mirmehdi 08, Xie 10 & 11, Yeo et al. 11 ) n  Region based (Paragios 02, Chan & Vese 01, Xie 09 & 11) n  Hybrid approach (Wang & Vemuri 04, Xie & Mirmehdi 04) n  Initialisation and convergence n  Initialisation independency (Xie 10, Xie 11, Yeo et al. 11) n  Complex topology & shape (Xie & Mirmehdi 07, 08, Paiement 14) n  These issues are often interdependent
  • 9. Deformable model n  Active contour: n  Dynamic curves within image domain to recover object shapes. n  Deformable surface: n  Its extension to 3D. n  Applications: n  Object localisation n  Motion tracking n  Segmentation (e.g. colour/texture) n  Two general types: explicit and implicit models n  Kass et al. 1988, Caselles et al.1993, and many more…
  • 10. Explicit model n  Parametric snake n  Represented explicitly as parameterized curves, spline, polynormial function n  Example classic parametric snake n  Snake evolves to minimize the internal and external forces (Let C(q) be a parameterized planar curve); n  Initialisation problem; n  Concavity convergence problem; ( ) ( ) ( ) ( )( ) . 22 ∫∫∫ ∇−ʹ′ʹ′+ʹ′= dqqCIdqqCdqqCCE λβα Internal forces External force
  • 11. Explicit model n  Point based on tracking n  Resolution problem n  Addition and deletion n  And … n  Topological problems n  Non-intrinsic, parameterisation dependent; n  Hard to detect multiple objects simultaneously. n  Example: ?
  • 12. Explicit model n  Advantages n  Explicit control n  Point correspondence n  Probably easier to impose shape regularisation n  Computational efficiency n  Should be considered when n  Known topology n  No (or predicated) topological changes n  Open curves n  … n  Numerical method n  Finite element method (FEM) n  Discretise into sub-domain n  Cohen & Cohen, IEEE T-PAMI, 1993
  • 13. Implicit model n  Popularly based on the Level Set technique n  Implicit snake models n  Introduced by Caselles et al. and Malladi et al. (1993); n  Based on the theory of curve evolution n  Numerically implemented via level set methods; n  Snake evolves to minimize the weighted length in a Riemannian space with a metric derived from the image content; n  Weighted length minimisation example: A B A B
  • 14. Curve evolution n  Curvature flow n  k is the curvature, N denotes the inward normal n  The curvature measures how fast each point moves along its normal direction; n  A simple closed curve will evolve toward a circular shape and disappear; n  It smoothes the curve. NCt ! κ=
  • 15. Curve evolution n  Constant flow n  c is a constant, N denotes the inward normal n  Each point moves at a constant speed along its normal direction; n  It can cause a smooth curve to become a singular one; n  A.k.a. the balloon force. NcCt ! =
  • 16. Level set method n  A computational technique for tracking propagating interface n  Embed the curve into a surface, 2D scalar field n  Zero level set corresponds to the embedded curve n  Deforming the surface, instead of explicitly deforming the curve
  • 17. Level set method n  A computational technique for tracking propagating interface n  Embed the curve into a surface, 2D scalar field n  Zero level set corresponds to the embedded curve n  Deforming the surface, instead of explicitly deforming the curve
  • 18. Level set method n  Key ideas introduced by Dervieux & Thomasset n  Lecture Notes in Physics 1980 n  Well-known after the seminal work by Osher & Sethian n  Osher & Sethian, J. Computational Physics 1988 n  Fluid dynamics, computational geometry, material science, computer vision, … n  Introduced to snake methods by Casselles et al. & Malladi et al. n  Casselles et al., Nemuer. Math. 1993 n  Malladi et al., IEEE T-PAMI 1995 n  Advantages: n  Implicit, intrinsic, non-parametric n  Accurate modelling front propagation n  Capable of handling topological changes (almost!)
  • 19. Level set method n  General curve evolution n  is the level set function, F is the speed function n  The curvature flow can be re-formulated as: n  The constant flow can be re-formulated as: 0|| =Φ∇+Φ Ft Φ || || Φ∇⎟⎟ ⎠ ⎞ ⎜⎜ ⎝ ⎛ Φ∇ Φ∇ ⋅∇=Φt || Φ∇=Φ ct
  • 20. Level set method n  Numerical method n  Finite difference method (FDM) n  A local method to estimate the partial derivatives n  Upwind scheme n  Reinitialisation n  Reshaping the level set surface to retain smoothness n  Periodically performed n  Fast marching method, or n  By solving the following PDE: |)|1)((sign Φ∇−Φ=Φt
  • 21. Level set method n  Challenges: n  Computational complexity n  FDM – a local method to estimate the partial derivatives n  Dense computational grid n  More expensive in 3D n  Fast marching, narrow band, additive operator splitting (AOS) n  More sophisticated topological changes n  Signed distance function n  Reinitialisation is necessary to eliminate accumulated numerical error n  Prevent the level set developing new components n  Do not allow perturbations away from zero level set n  Can not create new contours n  E.g. fail to localise internal object boundaries
  • 22. Level set method n  Challenges: n  Computational complexity n  FDM – a local method to estimate the partial derivatives n  Dense computational grid n  More expensive in 3D n  Fast marching, narrow band, additive operator splitting (AOS) n  More sophisticated topological changes n  Signed distance function n  Reinitialisation is necessary to eliminate accumulated numerical error n  Prevent the level set developing new components n  Do not allow perturbations away from zero level set n  Can not create new contours n  E.g. fail to localise internal object boundaries
  • 23. Numerical solution n  The level set time derivative is approximated by forward difference: n  Force fields can be classified into three types n  Curvature flow n  Constant flow n  Advection force field n  Each requires different differencing scheme
  • 24. Numerical solution n  Weighted curvature flow: Central differencing
  • 25. Numerical solution n  Weighted constant flow: Upwinding differencing n ji n ji Vcg ,0, |)|(|)|(.)( Φ∇=Φ∇
  • 26. Numerical solution n  Advection flow: n  Let denote the external velocity force field n  Check the sign of each component n  Construct one-sided upwind differences n  E.g. GVF,
  • 27. n  Edge Based 2D Segmentation Xie & Mirmehdi, MAC, IEEE Trans. Pattern Analysis & Machine Intelligence 2008.
  • 28. Motivation n  Convergence study – 4 disc problem GeodesicDVF GGVF GeoGGVF CVF MAC DVF: Cohen & Cohen, IEEE T-PAMI, 1993 Geodesic: Caselles et al., IJCV, 1997 GGVF: Xu & Prince, Signal Processing, 1998 GeoGGVF: Paragios et al., IEEE T-PAMI, 2004 CVF: Gil & Radeva, EMMCVPR 2003 CPM: Jalba et al., IEEE T-PAMI 2004 Xie & Mirmehdi, MAC, IEEE Trans. Pattern Analysis & Machine Intelligence 2008.
  • 29. Motivation n  Convergence study – 4 disc problem GeodesicDVF GGVF GeoGGVF CVF MAC DVF: Cohen & Cohen, IEEE T-PAMI, 1993 Geodesic: Caselles et al., IJCV, 1997 GGVF: Xu & Prince, Signal Processing, 1998 GeoGGVF: Paragios et al., IEEE T-PAMI, 2004 CVF: Gil & Radeva, EMMCVPR 2003 CPM: Jalba et al., IEEE T-PAMI 2004 Xie & Mirmehdi, MAC, IEEE Trans. Pattern Analysis & Machine Intelligence 2008.
  • 30. Motivation n  Objectives n  Long range force interaction n  Dynamic force field, instead of static n  Bidirectional – allow cross boundary initialisation n  Efficiency n  Region based or Edge based n  Prior knowledge n  Boundary assumptions n  Discontinuity in regional statistics n  Discontinuity in image intensity n  Application dependent n  Goal: improving edge based performance n  Comparable to region based approaches n  Benefit from less prior knowledge, simpler assumption, and efficiency n  There are scenarios boundary description does not need region support
  • 31. MAC model n  Proposed method n  Novel external force field n  Based on hypothesised magnetic interactions between object boundary and snake n  Significant improvements upon initialisation invariancy & convergence ability n  Yet, a very simple model n  Magnetostatics
  • 32. MAC model n  Edge orientation n  Analogy to current orientation n  Rotating image gradient vectors = 1: anti-clockwise rotation; = 2: clockwise rotation. : normalised image gradient vectors. n  (actually, these are 3D vectors) n  Current orientation on snake n  Similar to edge current orientation estimation n  Rotating level set gradient vectors
  • 33. MAC model n  Magnetic force on snake n  Derive the force on snake exerted from image gradients : electric current unit vector on snake : current magnitude on snake, constant : electric current vector on edges : current magnitude on edges : unit vector between two point, x and s : permeability constant n  Uniqueness n  The force on snake is dynamic n  Relies on both spatial position and evolving contour n  Always perpendicular to the snake n  Global force interaction
  • 34. MAC model n  Snake formulation : curvature : snake inward normal n  Level set representation n  Force field extension n  Snake is extended in a 2D scalar function n  Accordingly its forces upon it n  Fast marching n  In this case, simply compute forces for each level set
  • 35. MAC model n  An example of dynamic force field
  • 36. MAC model n  Edge preserving force diffusion n  Minimise noise interference n  Nonlinear diffusion of magnetic flux density n  Similar to GGVF, but… n  Add edge weighting term in diffusion control n  As little diffusion as possible at strong edges n  Homogeneous and noisy area which lack consistent support from edges will have larger diffusion
  • 37. MAC model n  Edge preserving force diffusion n  Fast implementation n  Decompose the magnetic flux term n  Fast computation in the Fourier domain
  • 38. Experimental results n  Comparative analysis on synthetic images GeodesicDVF GGVF GeoGGVF CVF MAC
  • 40. Experimental results n  Noise sensitivity 20% noise 30% noise 40% noise 50% noise
  • 41. Experimental results n  Weak edges Geodesic GGVF CPM MAC
  • 42. Experimental results n  Weak edges n  Brief comparison to Region Based Region based (MoG) MAC Geodesic GGVF CPM MAC
  • 43. Experimental results n  On real images GeodesicDVF GGVF GeoGGVF CPM MAC
  • 44. Experimental results n  On real images GeodesicDVF GGVF GeoGGVF CPM MAC
  • 45. Experimental results n  On different types of images Planar X-ray CT Ultrasound MRI
  • 46. Experimental results n  Dual level set Xie & Mirmehdi, MAC, IEEE Trans. Pattern Analysis & Machine Intelligence 2008.
  • 47. n  Extension to 3D Yeo, Xie, Sazonov, Nithiarasu, GPF, IEEE Trans. Image Processing 2011.
  • 48. GPF model n  Geometrical Potential Force n  Suitable for 3D data n  Based on hypothesised geometrically induced force field between deformable model and object boundary n  Generalisation of the MAC model n  Unique bi-directionality n  Dynamic force interaction n  Global view of object boundary representation Yeo, Xie, Sazonov, Nithiarasu, GPF, IEEE Trans. Image Processing 2011.
  • 49. GPF model n  Interaction force acting on due to is given as q  – corresponding geometrically induced potential created by
  • 50. GPF model n  Comparative Results Target objects Initialisations Geodesic GGVF Proposed GPF Yeo, Xie, Sazonov, Nithiarasu, GPF, IEEE Trans. Image Processing 2011.
  • 51. GPF model n  Medical 3D data segmentation Aneurysm Aorta
  • 52. GPF model n  Comparative analysis Ground-truth GGVFGeodesic Proposed GPF
  • 54. GPF model n  Further examples Human aorta (CT) Human carotid (CT)
  • 55. GPF model n  Direct sequential segmentation of temporal volumetric data
  • 56. GPF model n  Direct sequential segmentation of temporal volumetric data
  • 57. GPF model n  Direct sequential segmentation of temporal volumetric data
  • 58. GPF model n  Direct sequential segmentation of temporal volumetric data
  • 59. GPF model n  Direct sequential segmentation of temporal volumetric data
  • 60. GPF model n  Direct sequential segmentation of temporal volumetric data
  • 61. GPF model n  Direct sequential segmentation of temporal volumetric data
  • 62. GPF model n  Direct sequential segmentation of temporal volumetric data
  • 63. GPF model n  Direct sequential segmentation of temporal volumetric data
  • 64. GPF model n  Direct sequential segmentation of temporal volumetric data
  • 65. GPF model n  Direct sequential segmentation of temporal volumetric data
  • 66. GPF model n  Segmentation using statistical shape prior
  • 67. GPF model n  Segmentation using statistical shape prior Segmentation of corpus callosum from MRI Image based energy Image and shape based energy
  • 68. n  3D+T (4D) constrained SPECT segmentation Yang, Mirmehdi, Xie, Hall, CI2BM, MICCAI workshop 2009. (MIA under-review)
  • 69. n  Segmentation of LV borders allows quantitative analysis of perfusion defects and cardiac function. 4D SPECT Segmentation SPECT slice of the LV A DoughnutCardiac motion (mid-slice) Yang, Mirmehdi, Xie, Hall, CI2BM, MICCAI workshop 2009. (MIA under-review)
  • 70. Frontal view of opaque surface Top view of opaque surface Short-axis view Frontal view Correspondence between short-axis slice and 3D frontal view Frontal view of opaque surface overlaid on orthogonal slice planes Frontal view of transparent surface 4D SPECT Segmentation
  • 72. Training Image and Shape Sequences Gaussian Analysis PCA Gaussian Priors Spatiotemporal Priors Segmentation Unseen Sequence CACE Evolution Constraint (based on Gaussian and Spatiotemporal Priors)‫‏‬ Update Level Sets and Spatiotemporal Parameters Convergence End No Yes Training GtL trans. 4D SPECT Segmentation
  • 73. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. CCACE: 85.8% SCMS: 63.1% Results on input image Results against ground truth Results on input image Results against ground truth 4D SPECT Segmentation
  • 74. Slice 24 Slice 25 Slice 26 Slice 27 Slice 28 4D SPECT Segmentation – defect detection
  • 75. n  Tracking using implicit representation Chiverton, Xie & Mirmehdi, BMVC 2008 & 2009, IEEE TIP 2012
  • 76. Tracking & Online Shape Learning n  Prior independent snake tracking n  Contour based tracking n  Probably more difficult than box based tracking n  No prior knowledge n  Online shape learning and dynamic updating 60% random noise
  • 77. Tracking & Online Shape Learning n  Prior independent snake tracking n  Contour based tracking n  Probably more difficult than box based tracking n  No prior knowledge n  Online shape learning and dynamic updating
  • 78. Tracking & Online Shape Learning n  Contour based object tracking n  Online shape learning n  Self-imposed shape regularisation Without online shape self-regularisation Proposed method Chiverton, Xie & Mirmehdi, BMVC 2008 & 2009.
  • 79. Automatic Bootstrapping & Tracking Chiverton, Xie & Mirmehdi, IEEE T-IP 2012. n  Online shape learning coupled with automatic bootstrapping n  Finite size shape memory n  Statistical shape modelling n  Level set based tracking – similar to previous approach
  • 80. n  RBF-Level Set based Active Contouring Xie & Mirmehdi, Image and Vision Computing 2011 & BMVC07.
  • 81. Conventional level set technique n  Problems n  Computational complexity n  Dense computation grid, particularly expensive in 3D n  Some solutions: fast marching, narrow band, AoS schemes, … n  Can’t handle more sophisticated topological changes n  Usually requires re-initialisation to maintain a smooth surface to prevent numerical artefacts contaminating the solution n  Perturbations away from the zero level set are missed Conventional level set: The hole is missed!
  • 82. RBF-Level Set n  RBF-Level Set n  Use radial basis function to interpolate level set n  Updating expansion coefficients to deform level set n  Transfer PDE to ODE: efficient n  Much coarser computational grid, even irregular n  More complex topological changes readily achievable Conventional level set Proposed method
  • 83. RBF-Level Set n  RBF-Level Set n  Use radial basis function to interpolate level set n  Updating expansion coefficients to deform level set n  Transfer PDE to ODE: efficient n  Much coarser computational grid, even irregular n  More complex topological changes readily achievable Conventional level set Proposed method
  • 84. RBF-Level Set n  RBF interpolation n  Level set function, : a scalar function, usually obtained from the signed distance transform n  Interpolate using a linear combination of a radial basis function, where p(x) is a first degree polynomial and are the expansion coefficients n  The interpolation can be expressed as: where 6
  • 85. RBF-Level Set n  Updating RBF level set n  Original level set evolution: where F is the speed function in the normal direction n  Transferred evolution: n  The spatial derivatives can be solved analytically n  First order Euler’s method n  Iteratively updating the expansion coefficients to evolve level set n  Benefits: n  Coarse computational grid, could be irregular n  No need for re-initialisation n  More complex topological changes achievable
  • 86. RBF-Level Set snake n  Prevent self-flattening Non-normalised Normalised
  • 87. RBF-Level Set snake n  Prevent self-flattening Non-normalised Normalised
  • 88. RBF-Level Set snake n  Prevent self-flattening Non-normalised Normalised
  • 89. RBF-Level Set n  Active modelling using RBF level set n  A region based approach n  Texem based modelling n  Active contour formulation: n  m is the number of classes n  1/m is the average expectation of a class n  u is the posterior of the class of interest n  Level set representation:
  • 90. I RBF-Level Set n  Texems are image representations at various sizes that retain the texture or visual primitives of a given image. n  A two-layer generative model n  Each texem represented by mean and variance: m={µ,ω} n  A bottom-up learning procedure learning { }…M Z Xie-Mirmehdi, IEEE T-PAMI, 29(8), 2007.
  • 91. RBF-Level Set n  Example learnt texems (7x7) n  Multiscale branch based texems n  Texem grouping for multi-modal regions Xie-Mirmehdi, IEEE T-PAMI, 29(8), 2007.
  • 93. n  On real images RBF-Level Set Proposed method Conventional level set Proposed methodConventional level set Proposed method
  • 94. RBF-Level Set n  Deformable modelling in 3D Recover a hollow sphere Initialised outside the target object Complex geometry Xie & Mirmehdi, Image & Vision Computing 2011 & BMVC 2007.
  • 95. n  Hybrid Approach Xie & Mirmehdi, IEEE Trans. Image Processing, 2004.
  • 96. RAGS model n  Region-aided (RAGS) model n  Bridge boundary and region-based techniques n  Fusing global information to local boundary description n  Improvements towards weak edges n  More resilient to noise interference Geodesic snake Proposed methodGGVF snake Xie & Mirmehdi, IEEE Trans. Image Processing, 2004.
  • 97. n  Integrated Reconstruction, Registration and Segmentation A. Paiement et al., IEEE Transactions on Image Processing, January 2014.
  • 98. Motivation n  Modelling from 3D/4D imaging data raises two intertwined issues: n  segmentation n  interpolation n  Segmentation n  partition 3D space containing the object and to distinguish data points belonging to the object from background points n  e.g. 2D slices, ranging from simple stacks of parallel slices to more complicated spatial configurations
  • 99. Motivation n  Segmentation n  2D independent segmentation is often not desirable n  all the slices are better segmented simultaneously in 3D/4D n  However, interpolation is thus necessary n  since data often does not span the whole 3D space n  only partial support from data n  Imaging conditions n  some modalities require integration over a thick slice to improve signal to noise ratio n  e.g. 1.5T cine cardiac MRI typical slice thickness 7mm; hence spacing is 7mm or bigger n  large spacing is also desirable in order to reduce acquisition time (patient discomfort, motion artefact) n  in the 4D case, data must also be interpolated between available time frames
  • 100. Motivation n  Argument n  “the success of one stage (segmentation or interpolation) depends on the accuracy of the other” n  Approaches n  two sequential approaches: perform these two stages in opposing order n  some first segment slices independently then interpolate from 2D contours n  shape interpolation, notable work: Liu et al., surface reconstruction from Non-parallel curve network, CGF 27(2) 2008. n  combining registration and segmentation n  segment sparse volumes made up of 2D slices by registering and deforming a model on images (e.g. ASM): prior is often necessary n  level set based method: foundation (earlier work) for what presented here n  integrate segmentation and interpolation into a new RBF interpolated level set framework n  simplicity and flexibility of level set n  stability of RBF n  inherent interpolation provided by RBF
  • 101. Proposed Method n  Interpolate level set function using Strictly Positive Definitive (SPD) RBF n  combining registration and segmentation n  segment sparse volumes made up of 2D slices by registering and deforming a model on images (e.g. ASM): prior is often necessary n  level set based method: foundation (earlier work) for what presented here n  integrate segmentation and interpolation into a new RBF interpolated level set framework n  simplicity and flexibility of level set n  stability of RBF n  inherent interpolation provided by RBF
  • 102. Proposed Method n  Instead of evolve phi through expansion coefficients (which involves inverting a large matrix), evolve alpha by minimising an energy functional E: n  F may be any functional and is defined by the chosen segmentation method. n  Conventional variational level set method: n  using chain rule, a gradient descent method yields:
  • 103. Proposed Method n  Rename as . n  S is the speed of the moving front and is generally defined on the contour C only. n  The alpha evolution function can thus be simplified as n  is an approximation of the Dirac function n  this imposes a restriction of S to the contour C n  practical choice of regularised Dirac function: §  epsilon = 1 for sharp RBF; epsilon = 3 for flatter RBF.
  • 104. Proposed Method n  RBF based interpolation methods usually define one control point per data point n  we define one control point per voxel of a discrete space, n  thus allow rewrite as a convolution: n  The initial expansion coefficients can be computed in the Fourier domain
  • 105. Results Chan-Veseinitialisation narrow-band PC proposed PC initialisation proposed PC Paiement et al., IEEE Trans. Image Processing 2014.
  • 106. Results initial slices (top left quadrant removed for visualisation) central T1 weighted slice modelled shape central T2 weighted slice Paiement et al., IEEE Trans. Image Processing 2014.
  • 107. Results initial slices central short axis slice modelled shape long axis slice Paiement et al., IEEE Trans. Image Processing 2014.
  • 110. Combinatorial Optimisation n  Shortest path problem: segmentation
  • 111. Combinatorial Optimisation n  Optimal surface (minimum cut) n  Hard constraint n  Statistical shape constraint n  Temporal constraint: Kalman filter & HMM
  • 112. Combinatorial Optimisation IVUS s-t cut optimal surface texture RBF star graph proposed
  • 113. n  Longitudinal cross section n  Yellow: frame-by-frame; red: proposed; green: ground truth Combinatorial Optimisation
  • 114. n  Team n  Dr. Feng Zhao, Mr. Ehab Essa, Mr. Jingjing Deng, Mr. Mike Edwards, Mr. Robert Palmer, Mr. Yaxi Ye, Mr. David James, Mr. Jonathan Jones n  Alumni n  Dr. Ben Daubney, Dr. Huazizhong Zhang, Dr. Dongbin Chen, Dr. Si Yong Yeo, Dr. Cyril Charron, Dr. John Chiverton, Dr. Ronghua Yang, Mr. Liu Ren, Mr. Arron Lacey n  Clinical collaborator n  Swansea Singleton Hospital ABM UHT at Morriston n  Bristol Royal Infirmary n  Cardiff Hospital n  Plymouth Hospital Acknowledgement csvision.swan.ac.uk