texture - homes.cs.washington.edu
TRANSCRIPT
![Page 1: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/1.jpg)
TextureReadings: Ch 7: all of it plus Carson paper
1
• Structural vs. Statistical Approaches• Edge-Based Measures• Local Binary Patterns• Co-occurence Matrices• Laws Filters & Gabor Filters• Blobworld Texture Features that select scale
![Page 2: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/2.jpg)
Texture
Texture is a description of the spatial arrangement of color orintensities in an image or a selected region of an image.
Structural approach: a set of texels in some regular or repeated pattern
2
![Page 3: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/3.jpg)
Problem with Structural Approach
How do you decide what is a texel?
Ideas?
3
![Page 4: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/4.jpg)
Natural Textures from VisTex
grass leaves
What/Where are the texels? 4
![Page 5: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/5.jpg)
The Case for Statistical Texture
• Segmenting out texels is difficult or impossible in real images.
• Numeric quantities or statistics that describe a texture can becomputed from the gray tones (or colors) alone.
• This approach is less intuitive, but is computationally efficient.
• It can be used for both classification and segmentation.
5
![Page 6: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/6.jpg)
Some Simple Statistical Texture Measures
1. Edge Density and Direction
• Use an edge detector as the first step in texture analysis.
• The number of edge pixels in a fixed-size region tells ushow busy that region is.
• The directions of the edges also help characterize the texture
6
![Page 7: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/7.jpg)
Two Edge-based Texture Measures
1. edgeness per unit area
2. edge magnitude and direction histograms
Fedgeness = |{ p | gradient_magnitude(p) ≥ threshold}| / N
where N is the size of the unit area
Fmagdir = ( Hmagnitude, Hdirection )
where these are the normalized histograms of gradientmagnitudes and gradient directions, respectively.
7
![Page 8: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/8.jpg)
Original Image Frei-Chen ThresholdedEdge Image Edge Image
Example
8
![Page 9: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/9.jpg)
Local Binary Pattern Measure
100 101 10340 50 8050 60 90
• For each pixel p, create an 8-bit number b1 b2 b3 b4 b5 b6 b7 b8,where bi = 0 if neighbor i has value less than or equal to p’svalue and 1 otherwise.
• Represent the texture in the image (or a region) by thehistogram of these numbers.
1 1 1 1 1 1 0 0
1 2 3
4
57 6
8
9
![Page 10: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/10.jpg)
Fids (Flexible Image DatabaseSystem) is retrieving imagessimilar to the query imageusing LBP texture as thetexture measure and comparingtheir LBP histograms
Example
10
![Page 11: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/11.jpg)
Low-levelmeasures don’talways findsemanticallysimilar images.
Example
11
![Page 12: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/12.jpg)
Co-occurrence Matrix Features
A co-occurrence matrix is a 2D array C in which
• Both the rows and columns represent a set of possibleimage values.
• C (i,j) indicates how many times value i co-occurs withvalue j in a particular spatial relationship d.
• The spatial relationship is specified by a vector d = (dr,dc).
d
12
![Page 13: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/13.jpg)
1 1 0 01 1 0 00 0 2 20 0 2 20 0 2 20 0 2 2
j
i
1
3
d = (3,1)
0 1 2
012
1 0 32 0 20 0 1
Cd
gray-toneimage
co-occurrencematrix
From Cd we can compute Nd, the normalized co-occurrence matrix,where each value is divided by the sum of all the values.
Co-occurrence Example
13
![Page 14: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/14.jpg)
Co-occurrence Features
sums.
What do these measure?
Energy measures uniformity of the normalized matrix. 14
![Page 15: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/15.jpg)
But how do you choose d?
• This is actually a critical question with all thestatistical texture methods.
• Are the “texels” tiny, medium, large, all three …?
• Not really a solved problem.
Zucker and Terzopoulos suggested using a χ2 statisticaltest to select the value(s) of d that have the most structurefor a given class of images.
15
![Page 16: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/16.jpg)
Example
16
![Page 17: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/17.jpg)
Laws’ Texture Energy Features
• Signal-processing-based algorithms use texture filtersapplied to the image to create filtered images from whichtexture features are computed.
• The Laws Algorithm• Filter the input image using texture filters.• Compute texture energy by summing the absolute
value of filtering results in local neighborhoods around each pixel.
• Combine features to achieve rotational invariance.17
![Page 18: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/18.jpg)
Law’s texture masks (1)
18
![Page 19: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/19.jpg)
Law’s texture masks (2)Creation of 2D Masks
E5L5
E5L519
![Page 20: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/20.jpg)
9D feature vector for pixel• Subtract mean neighborhood intensity from (center) pixel• Apply 16 5x5 masks to get 16 filtered images Fk , k=1 to 16
• Produce 16 texture energy maps using 15x15 windowsEk[r,c] = ∑ |Fk[i,j]|
• Replace each distinct pair with its average map:• 9 features (9 filtered images) defined as follows:
20
![Page 21: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/21.jpg)
Laws Filters
21
![Page 22: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/22.jpg)
Laws Process
22
![Page 23: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/23.jpg)
water
tiger
fence
flag
grass
small flowers
big flowers
Is there aneighborhoodsize problemwith Laws?
Example: Using Laws Features to Cluster
23
![Page 24: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/24.jpg)
Features from sample images
24
![Page 25: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/25.jpg)
Gabor Filters
• Similar approach to Laws• Wavelets at different frequencies and different
orientations
25
![Page 26: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/26.jpg)
Gabor Filters
26
![Page 27: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/27.jpg)
Gabor Filters
27
![Page 28: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/28.jpg)
Segmentation with Color and Gabor-Filter Texture (Smeulders)
28
![Page 29: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/29.jpg)
Blobworld Texture Features
• Choose the best scale instead of using fixed scale(s)
• Used successfully in color/texture segmentation in Berkeley’s Blobworld project
29
![Page 30: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/30.jpg)
Feature Extraction
• Algorithm: Select an appropriate scale for each pixel and extract features for that pixel at the selected scale
Pixel Features PolarityAnisotropyTexture contrast
featureextraction
Original image
30
![Page 31: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/31.jpg)
Texture Scale• Texture is a local neighborhood property.
• Texture features computed at a wrong scale can lead to confusion.
• Texture features should be computed at a scale which is appropriate to the local structure being described.
The white rectangles show some sample texture scales from the image.
31
![Page 32: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/32.jpg)
Scale Selection Terminology
• Gradient of the L* component (assuming that the image is in the L*a*b* color space) :▼I =
• Gaussian filter : Gσ (x, y) of size σ
• Second moment matrix: Mσ (x, y)= Gσ (x, y) * (▼I)(▼I)T
Note: σ controls the size of the window around each pixel [1 2 5 10 17 26 37 50].
Ix2 IxIy
IxIy Iy2
IxIy
32
original Ix Ix2 G*Ix2
![Page 33: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/33.jpg)
Computing Second Moment Matrix M σ
33
1. First compute 3 separate images for• Ix2
• Iy2
• IxIy
2. Then apply a Gaussian filter to each ofthese images.
3. Then M σ(i,j) is computed from Ix2(i,j),Iy2(i,j), and IxIy(i,j).
![Page 34: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/34.jpg)
Scale Selection (continued)• Make use of polarity (a measure of the extent to which the gradient
vectors in a certain neighborhood all point in the same direction) to select the scale at which Mσ is computed
Edge: polarity is close to 1 for all scales σTexture: polarity varies with σUniform: polarity takes on arbitrary values 34
![Page 35: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/35.jpg)
Scale Selection (continued)
• n is a unit vector perpendicular tothe dominant orientation.
• The notation [x]+ means x if x > 0 else 0
The notation [x]- means x if x < 0 else 0
• We can think of E+ and E- as measuresof how many gradient vectors in thewindow are on the positive side andhow many are on the negative side of the dominant orientation in thewindow.
n=[1 1]
x = [1 .6]
x’ = [-1 -.6]
Example:
polarity pσ
35
![Page 36: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/36.jpg)
Scale Selection (continued)
• Texture scale selection is based on the derivative of the polarity with respect to scale σ.
• Algorithm:
kσkσ
1. Compute polarity at every pixel in the image for σk = k/2, (k = 0,1…7).
2. Convolve each polarity image with a Gaussian with standarddeviation 2k to obtain a smoothed polarity image.
3. For each pixel, the selected scale is the first value of σfor which the difference between values of polarity at successive scales is less than 2 percent.
36
![Page 37: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/37.jpg)
Texture Features Extraction
• Extract the texture features at the selected scale– Polarity (polarity at the selected scale) : p = pσ*
– Anisotropy : a = 1 – λ2 / λ1
λ1 and λ2 denote the eigenvalues of Mσ
λ2 / λ1 measures the degree of orientation: when λ1 is largecompared to λ2 the local neighborhood possesses a dominant orientation. When they are close, no dominant orientation.When they are small, the local neighborhood is constant.
– Local Contrast: C = 2(λ1+λ2)3/2
A pixel is considered homogeneous if λ1+λ2 < a local threshold37
![Page 38: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/38.jpg)
Blobworld Segmentation Using Color and Texture
38
![Page 39: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/39.jpg)
Application to Protein Crystal Images
Original image in PGM (Portable Gray Map ) format
• K-mean clustering result (number of clusters is equal to 10 and similarity measure is Euclidean distance)
• Different colors represent different textures
39
![Page 40: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/40.jpg)
Application to Protein Crystal Images
• K-mean clustering result (number of clusters is equal to 10 and similarity measure is Euclidean distance)
• Different colors represent different textures
Original image in PGM (Portable Gray Map ) format
40
![Page 41: Texture - homes.cs.washington.edu](https://reader033.vdocuments.net/reader033/viewer/2022052017/628667d2c96d41088c73c5ca/html5/thumbnails/41.jpg)
References
• Chad Carson, Serge Belongie, Hayit Greenspan, and Jitendra Malik. "Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying." IEEE Transactions on Pattern Analysis and Machine Intelligence 2002; Vol 24. pp. 1026-38.
• W. Forstner, “A Framework for Low Level Feature Extraction,”Proc. European Conf. Computer Vision, pp. 383-394, 1994.
41