Grayscale mathematical morphology

Transkript

1 Grayscale mathematical morphology Václav Hlaváč Czech Technical University in Prague Center for Machine Perception (bridging groups of the) Czech Institute of Informatics, Robotics and Cybernetics and Faculty of Electrical Engineering, Department of Cybernetics Outline of the talk: Set-function equivalence. Umbra and top of a set. Gray scale dilation, erosion. Top-hat transform.

2 A quick informal explanation Grayscale mathematical morphology is the generalization of binary morphology for images with more gray levels than two or with voxels. The point set A E3. The first two coordinates span in the function (point set) domain and the third coordinate corresponds to the function value. The concepts supremum (also the least upper bound), resp. infimum (also the greatest lower bound) play a key role here. Actually, the related operators max, resp. min, are used in computations with finite sets. Erosion (resp. dilation) of the image (with the flat structuring) element assigns to each pixel the minimal (resp. maximal) value in the chosen neighborhood of the current pixel of the input image. The structuring element is a function of two variables. It influences how pixels in the neighborhood of the current pixel are taken into account. The value of the (non-flat) structuring element is added (while dilating), resp. subtracted (while eroding) when the maximum, resp. minimum is calculated in the neighborhood. 2/42

3 Grayscale mathematical morphology explained via binary morphology 3/42 It is possible to introduce grayscale mathematical morphology using the already explained binary (black and white only) mathematical morphology. R.M. Haralick, S.R. Sternberg, X. Zhuang: Image analysis using mathematical morphology, IEEE Pattern Analysis and Machine Intelligence, Vol. 9, No. 4, 1987, pp We will start with this explanation first and introduce an alternative way using sup, inf later. We have to explain the concepts top of the surface and umbra first.

4 Equivalence between sets and functions 4/42 A function can be viewed as a stack of decreasing sets. Each set X λ is the intersection between the umbra of the function and a horizontal plane (line). f(x) l umbra X λ = {X E, f(x) λ} f(x) = sup{λ: x X λ (f)} A function to sets x It is equivalent to say that f is upper semi-continuous or that X λ -s are closed. f(x) Conversely, given {Xλ} of closed set such that λ µ X λ X µ and X λ = {X µ, µ < λ} then there exist a unique an upper semi-continuous. function f whose sections are {X λ }. Sets to a function x

5 Top of the surface 5/42 y=f(x 1, x 2) top surface T[A] 1D function example set A x 2 x support F 1 An arbitrary set Let A En is a domain F = {x En 1 for some y E, (x, y) A}. The top surface, top of the set A, is denoted T [A] is a mapping F E defined as [A](x) = max{y, (x, y) A}. Top surface

6 Umbra 6/42 y=f(x, x ) 1 2 top surface T[A] 1D function example set A x 1 x 2 umbra U[T[A]] Top surface Let F En 1 and f: F E. The Umbra of a function (set) f, denoted U[f], U[f] F E, U[f] = {(x, y) F E, y f(x)} Umbra

7 Gray scale dilation, erosion by binary dilations, erosions 7/42 The umbra function (set) U and the top surface function (set) T are used. Dilation: Erosion: f k = T [U[f] U[k]] f k = T [U[f] U[k]]

8 Grayscale dilation, 1D example 8/42

9 Grayscale erosion, 1D example 9/42

10 Grayscale dilation/erosion via lattice This is the alternative approach which uses the order structure in T in the lattice of functions T E. It leads to The function g represents a structuring element. Dilation (f g)(x) = sup{f(y) + g(x y)} y Y Erosion (f g)(x) = inf {f(y) g(x y)} y Y 10/42 Image courtesy Petr Matula

11 Dilation/erosion with a flat structuring element Flat structuring elements g are defined to be equal to zero on a compact set K and to the value max(t ) elsewhere We can write 11/42 f g = f g = sup y E, x y K inf y E, x y K f(y) = sup y K x f(y) f(y) = inf y K x f(y) Image courtesy Petr Matula

12 Remarks, flat structuring element 12/42 Erosion (f g)(x) = inf y E, x y K f(y) = inf y K x f(y) 60 f(x) dilated original eroded Positive peaks are shrunk. Valleys are expanded. Dilation provides the dual effect structuring element 40 x (f g)(x) = sup y E, x y K Courtesy J. Serra for the image idea. f(y) = sup y K x f(y)

13 Example: dilation, erosion with the flat structuring element 13/42 Image courtesy Petr Matula

14 Example: grayscale morphological preprocessing 14/42 (a) original (b) eroding dark (c) dilating dark in (b) (d) reconstr. cells

15 Remarks on grayscale dilation/erosion 15/42 Dilations and erosions with a flat structuring element on grayscale images are equivalent to applying max and min filters. It is recommended to work on grayscale images as long as possible and defer thresholding at later times. Dilation/erosion compared with convolution Convolution: (f g)(x) = f(x y) g(y) Dilation: (f g)(x) = sup y Y y Y {f(y) + g(x y)} Convolution Dilation/erosion Remark Summation sup or inf nonlinar Product Summation linear

16 Grayscale hit or miss operation 16/42 Dilations and erosion are more powerful when combined. E.g., introducing grayscale hit or miss operation which serves for template matching. Two structuring elements with a common representative point (origin). The first structuring element is the foreground pattern B fg, the second one is the background pattern B fg. Grayscale hit or miss operator is defined as X B = (X B fg ) (X B bg )

17 Opening, closing 17/42 Recall from from the binary morphology lecture that a filter is a morphological filter if and only if it is increasing and idempotent. Grayscale dilation and erosion are morphological filters. Dummy figure Opening: X B = (X B) B Closing: X B = (X B) B

18 Angl. Top Hat transform. Definition: X \ (X K). Top hat tranform It is used for intensity-based object segmentation in the situation in which the background intensity changes slowly. Parts of image larger than the structuring element K are removed. Only removed parts remain after subtraction, i.e. these are objects on the more uniform background. The objects can be found by thresholding now. 18/42 Gray-scale image Opened image Top hat Threshold

19 Example: Production of glass capillaries for thermometers 19/42 Top-hat transform illustrated on the industrial example. Original Erosion with struct. Opening with the Resulting elem same struct. elem. segmentation

20 Conditional dilation 20/42 Serves as the basis of geodetic transformations and reconstruction. Conditional dilation of a set X (called marker) by a structuring element B using a reference set R (called mask) is defined as: δ (1) R,B = (X B) R It is obvious that δ (1) R,B R. The set B is usually small (often a basic structuring element induced by the underlying grid. Set B is often omitted in subscripts. Open path to geodesic methods.

21 Geodesic method in mathematical morphology 21/42 Geodetické metody změní morfologické operace tak, aby operovaly pouze na části obrázku. Příklad: má-li se například rekonstruovat objekt ze značky, řekněme buněčného jádra, je žádoucí zabránit růstu mimo buňku. We will see later that the structuring element can change in every pixel based on image function values in a local neighborhood.

22 Geodesic distance 22/42 Geodesic distance dx(x, y) is the length of the shortest path between two points x, y under the condition that they are members of the set X. If there is no path connecting x, y then the geodesic distance is defined as d X (x, y) = +. w d X(w,y) = y set X d (x,y) X x

23 Geodesic circle (ball) 23/42 Geodesic circle (ball, hyperball for the space dimension > 3) is a circle constrained to set X. Geodesic circle BX(p, n) with the center p X and the radius n is defined B X (p, n) = {p X, d X (p, p ) n}. We can use dilation/erosion only inside the the subset Y of the image X.

24 Geodesic dilation and erosion 24/42 Geodesic dilation δ (n) X (Y ) of size n of a set Y inside the set X, δ (n) X (Y ) = p Y B X (p, n) = { } p X, p Y, d X (p, p ) n. Geodesic erosion ɛ (n) X (Y ) of size n of a set Y inside the set X, ɛ (n) X (Y ) = {p Y, B X (p, n) Y } = { } p Y, p X \ Y, d X (p, p ) > n. δ X (Y) Y X ε X (Y) X X Y X

25 Geodesic dilation, erosion, implementation 25/42 The simplest geodesic dilation of size 1 (δ (1) X ) of a set Y inside X is obtained as the intersection of the unit-size dilation of Y (with respect to the unit ball B) with the set X δ (1) X = (Y B) X. Larger geodesic dilations are obtained by iteratively composing unit dilations n times ( δ (n) X = δ(1) X δ (1) ( (1) X δ X... (δ(1) X }{{ ))). } n times The fast iterative way to calculate geodesic erosion is similar.

26 Morphological reconstruction 26/42 Předpokládejme dvě bodové množiny X, R, X R. Morfologická rekonstrukce ρr(x) množiny X z množiny R je sjednocením souvislých částí množiny R s neprázdným průnikem s R. Reconstruction by dilation is defined as a geodetic dilation repeated until idempotence: δr (X) = δn R (X), where δr n (X) = δn+1 R (X). Reconstruction by dilation is an opening w.r.t. R and closing w.r.t. X. B A

27 Oblast jako sjednocení max. kruhů 27/42 V binární morfologii lze bodovou množinu X také vyjádřit jako sjednocení maximálních kruhů B. Každému bodu p skeletu S(X) je jednoznačně přiřazen maximální kruh o poloměru q X (p) (angl. quench function). Známe-li pro každý bod skeletu qx(p), potom lze výchozí bodovou množinu X rekonstruovat jako sjednocení maximálních kruhů B X = p S(X) (p + q X (p)b).

28 Tři typy extrémů šedotónové obrazové funkce 28/42 Globálním maximem je pixel s nejvyšší hodnotou (nejvyšší vrchol v krajině). Lokálním maximem je pixel p, právě když pro každý sousední pixel q pixelu p platí I(p) I(q). Regionální maximum M digitálního šedotónového obrazu I je souvislá množina pixelů s odpovídající hodnotou h (plató ve výšce h), kde každý pixel sousedící s množinou M má menší hodnotu než h.

29 Tři typy extrémů, příklad 29/42 okolí bod regionální maximum lokální maximum Každý pixel regionálního maxima M v obrazové funkci I je také lokálním maximem. Opak neplatí, tj. existují lokální maxima, která nejsou regionálními maximy.

30 Myšlenka automatického značkování 30/42 Myšlenka: konvexní oblast v binárním obraze lze reprezentovat značkou uprostřed oblasti. Triviálně splněno pro nedotýkající kruhy. Obecně složitější. Postupné erodování, značkou je místo těsně před úplným zmizením oblasti. Pojem: konečná eroze. Pro nekonvexní oblasti se nejdříve rozdělí na jednodušší konvexní části.

31 Postupné erodování, příklad 31/42 Originál 1. eroze 2. eroze 3. eroze 4. eroze 5. eroze

32 Ultimate erosion U lt(x) 32/42 Nechť X je bodová množina, S(X) je její skelet, q X (p) je (quench) funkce přiřazující každému bodu skeletu kruh vepsaný do X se středem c. Konečná eroze U lt(x) je definována jako množina regionálních maxim funkce q X (p). X q (p) R X R 1 2 S(X) c c c 1 c 2 1 2

33 Ultimate erosion, example cont. 33/42 výchozí obrázek výsledek konečné eroze

34 Vyjádření konečné eroze pomocí rekonstrukce 34/42 N je množina přirozených čísel, která poslouží pro rostoucí poloměry kruhů. Konečná eroze se může vyjádřit Ult(X) = n N ((X nb) \ ρ X nb (X (n + 1)B)). Efektivní výpočet konečné eroze se opírá o vzdálenostní funkci.

35 Rychlý výpočet pomocí vzdálenostní transformace 35/42 Vzdálenostní transformace (funkce) distx(p) přiřazuje každému pixelu p z množiny X velikost první eroze množiny, která už neobsahuje pixel p, tj. p X, dist X (p) = min {n N, p not in (X nb)}. dist X(p) je nejkratší vzdáleností mezi pixelem p a doplňkem množiny XC. Vzdálenostní funkce má dvě přímá použití: Konečná eroze množiny X je tvořena sjednocením regionálních maxim vzdálenostní funkce množiny X. Skelet pomocí maximálních kruhů množiny X je dán množinou lokálních maxim vzdálenostní funkce množiny X.

36 Skeleton by influence zones (SKIZ) 36/42 Let X be composed of n connected components Xi, i = 1,..., n. The influence zone Z(Xi) consists of points which are closer to set Xi than to any other connected component of X. Z(X i ) = { p Z 2, i j, d(p, X i ) d(p, X j ) }. The skeleton by influence zones denoted SKIZ(X) is the set of boundaries of influence zones { Z(X i ) }.

37 SKIZ idea, example 37/42

38 SKIZ(X) Skeleton(X) 38/42

39 SKIZ(X) Skeleton(X) 39/42

40 Morphological reconstruction, motivation Assume that we want to reconstruct objects of a given shape from a binary image that was originally obtained by thresholding. All connected components in the input image constitute the set X. However, only some of the connected components were marked by markers that represent the set Y. The task is to 40/42 Reconstruction of X (shown in light gray) from markers Y (black). The reconstructed result is shown in black on the right side.

41 Morphological reconstruction Successive geodesic dilations of the set Y inside the set X enable the reconstruction of the connected components of X that were initially marked by Y. When dilating from the marker, it is impossible to intersect a connected component of X which did not initially contain a marker Y ; such components disappear. Geodesic dilations terminate when all connected components set X previously marked by Y are reconstructed, i.e., idempotency is reached n > n 0, δ (n) X (Y ) = δ(n 0) X (Y ). This operation is called reconstruction and denoted by ρx(y ). Formally 41/42 ρ X (Y ) = lim n δ(n) X (Y ). (1)

42 Several markers for a region, issues Geodesic influence zone 42/42 In some applications it is desirable that one connected component of X is marked by several markers Y. If the above is not acceptable then the notion of influence zones can be generalized to geodesic influence zones of the connected components of set Y inside X.