3. Semantic segmentationSemantic segmentation
Each pixel has its own label!
picture source (https://www.quora.com/What-does-the-term-
semantic-segmentation-mean-in-the-context-of-Deep-Learning)
4. Typical wayTypical way
Image Model
Outcome
w x h
Label
w x h
cross
entropy
Loss is calculated for each pixel independently.
5. IssueIssue
How to create dense prediction?
related works:
patchwise training
small model -> small receptive eld
post-processing (e.g. superpixel projection, random eld regularization, ltering
...)
saturating tanh
restricted receptive eld
input shifting and output interlacing
multi-scale pyramid processing
7. IdeaIdea
Semantics and location
Global information resolves what while local information resolves where.
global information -> what (semantics)
local information -> where (location)
8. IdeaIdea
Use train by entire image, instead of patch.
Let receptive eld overlap signi cantly to improve ef ciency.
Transfer learning from classi cation net to fully convolution network.
For pixelwise prediction, connect coarse outputs to pixels.
14. Evaluation methodEvaluation method
is the number of the pixel of class predicted to be class
there are different classes
, total number of pixels of class
pixel accuracy:
mean accuracy:
mean region intersection over union (IU):
nij i j
ncl
=ti ∑
j
nij i
/∑
i
nii ∑
i
ti
(1/ ) /ncl ∑
i
nii ∑
i
ti
1
ncl
∑
i
nii
+ −ti ∑
j
nji nii
21. ImportanceImportance
FCN for pixelwise prediction
arbitrary-sized inputs
learning and inference whole image at a time
leverage supervised pre-train model
upsampling (deconvolution)
22. Take home messageTake home message
more convolution, more coarse
combine coarse and ne feature map (skip architecture)
30. My conclusionMy conclusion
Encoder-decoder architecture
Encoder: extract high-level or abstract meanings (semantics)
Decoder: generate instance from abstract meanings
Discriminative model
Generative model
P (y ∣ x)
P (x, y)
31. Q & AQ & A
ReferenceReference
[1]
[2]
[3]
A brief introduction to recent segmentation methods
(https://www.slideshare.net/mitmul/a-brief-introduction-to-recent-
segmentation-methods)
关于FCN 论⽂中的Shift-and-stitch 的详尽解释
(https://www.jianshu.com/p/e534e2be5d7d)
A 2017 Guide to Semantic Segmentation with Deep Learning
(http://blog.qure.ai/notes/semantic-segmentation-deep-learning-review)