Klepnutím lze upravit styl předlohy. nadpisů. Kamil Říha Faculty of Electrical Engineering and Communication Department of Telecommunications

Transkript

1 1 /18 SPLAB, BUT Klepnutím lze upravit styl předlohy Kamil Říha Faculty of Electrical Engineering and Communication Department of Telecommunications Czech Republic European Union

2 2 /18 /52 SPLAB, BUT Klepnutím Group of Biomedical lze upravit Signal styl Processing předlohy Biomedical sensors design and signal processing Signal processing for computer tomography and magnetic resonance 3D data Ultrasound images and video sequences processing Medical hardware designing (specialized microprocessor controlled equipment) Diagnosis of nervous system disorders from speech

3 3 /18 /52 SPLAB, BUT Klepnutím lze Overview upravit styl předlohy common carotid artery (CCA ) in ultrasound scans Ultrasound B-mode image processing Preprocessing (de-noising, stabilization) Tracking objects in a sequence Data post-processing (analysis) Artery localization in longitudinal scan in transversal (perpendicular) scan

4 4 /18 /52 SPLAB, BUT The common carotid artery (CCA) Klepnutím lze upravit styl předlohy Common carotid artery (carotis communis) is an artery that supplies the head and neck with oxygenated blood CCA ascends within the neck and it bifurcates into two branches External carotid artery (ECA) Supplying the exterior of the head and face Internal carotid artery (ICA) Supplying the parts within the cranial and orbital cavities Magnetic Resonance Angiography Arteries of the neck

5 5 /18 /52 SPLAB, BUT Klepnutím The common lze upravit carotid styl artery předlohy (CCA) Source of important information Doctors can use it to evaluate the patients health Can be measured noninvasively by using ultrasound imaging Superficial imaging linear array transducer Transverse x longitudinal sections linear array transducer Transverse section Longitudinal section CCA

6 6 /18 /52 SPLAB, BUT Klepnutím Processing lze upravit Chain styl předlohy A. Pre-processing B. Processing C. Post-processing Noise reduction Noise in US images Noise reduction with adaptive filters Noise reduction with wavelet transformation Motion compensation original method based on Optical flow analysis and vector field processing (pulsative movement suppression)

7 7 /18 /52 SPLAB, BUT Klepnutím Motion lze upravit compensation styl předlohy Why? Doctors want to examine a certain region in video-sequence This region is moving in recorded video-sequence Because of free-hand examination probe movement Unwanted translational movement in sonographic videosequences Effort to eliminate such translation movement Stabilization of examined region = motion compensation 7

8 8 /18 /52 SPLAB, BUT Klepnutím Free-hand lze upravit scanning styl předlohy Main idea: translate image in inverse direction against the unwanted movement Before motion compensation After motion compensation 8

9 9 /18 /52 SPLAB, BUT Klepnutím B. Medical lze upravit image processing styl předlohy A. Pre-processing B. Processing C. Post-processing Artery section area tracking in a video-sequence 9

10 10 /18 /52 SPLAB, BUT Klepnutím Artery section lze upravit area styl tracking předlohy Main goal: to develop a method for the tracking of an artery section area in its perpendicular or longitudinal cut in the ultrasound scan. Typical perpendicular cut of an artery in sonographic image: The area inside the artery depends on blood pressure level. Systolic pressure level = maximal, diastolic pressure level = minimal artery section area. Videosequence contains the cardiac cycle data. 10

11 11 /18 /52 SPLAB, BUT Good features to track Klepnutím lze upravit styl předlohy denoting The main goal of this stage is to denote the features being at the artery edge. Firstly, the Hessian is computed for every pixel in the area of interest: H f x, y = 2 f x 2 2 f yx 2 f xy 2 f then some neighbourhood S(p) of every coordinate in resulting second derivative functions is counted in: M x, y = S p S p 2 f x 2 2 f yx y 2 2 f xy finally the good feature is detected based on eigenvalues of the matrix M. S p S p 2 f y 2 11

12 12 /18 /52 SPLAB, BUT Good features to track Klepnutím lze upravit styl předlohy denoting In practice. initial ring defined by hand 12

13 13 /18 /52 SPLAB, BUT Good features to track Klepnutím lze upravit styl předlohy denoting In practise. detected features 13

14 14 /18 /52 SPLAB, BUT Klepnutím Optical Flow lze upravit tracking styl procedure předlohy Optical flow determination by Lucas-Kanade method: E x u + E y v + E t = 0 where E x, y, t is the video sequence (or more likely the coordinates of features) and E x = E x, E y = E y, E t = E t then the over determined equation E x1 E x2 E xn E y1 E y2 E yn u v = E t1 E t2 E tn dx dy and u =, v = dt dt is solved in the least squares sense 14

15 15 /18 /52 SPLAB, BUT Optical Flow: tracking procedure Klepnutím lze upravit styl předlohy Solving of this equation gives us the motion vector (u,v), which implicitly means the coordinate of the feature in next frame and consequently in the whole videosequence. Now we have to compute the area of the artery as an ellipse fitted to feature points (in the sense of least square method) in every frame: The area of the ellipse is a searched value (in dependency on time) computed simply as: ellipse area = π major axis minor axis 15

16 16 /18 /52 SPLAB, BUT Klepnutím experimental lze upravit results styl předlohy Final output: cardiac cycle curve the inner part of artery cross-section, expressed in terms of the number of pixels located within the artery circle, in dependence on time. The time is derived from the FPS of a given video-sequence (32 FPS). 16

17 17 /18 /52 SPLAB, BUT Klepnutím Experimental lze upravit results styl předlohy.demonstration. 17

18 18 /18 /52 SPLAB, BUT Klepnutím Experimental lze upravit results styl předlohy Different methods of analysation are possible (based on principle of fetures tracking) Point-to-Point measurement Very simple distance measurement Multipoint-to-multipoint measurement More precise distance measurement Measurement of a distance between two splines Measure a distance in many points between two curves perpendicularly to axis General shape measurement Analyze general shape and measure inner size perpendicularly to axis These methods are implemented in application called sonoskop Including the circular artery section area analysis, all the methods can be used for medical examination The methods are not strictly for examination of arteries They can be also used for heart examination, universal methods in industry etc. 18

19 19 /18 /52 SPLAB, BUT Multipoint-to-Multipoint Klepnutím lze upravit styl předlohy measurement 19

20 20 /18 /52 SPLAB, BUT Measurement of a Distance Between Klepnutím lze upravit styl předlohy Two Splines 20

21 21 /18 /52 SPLAB, BUT Klepnutím General lze shape upravit measurement styl předlohy 21

22 22 /18 /52 SPLAB, BUT Klepnutím Post-processing lze upravit styl předlohy A. Pre-processing B. Processing C. Post-processing Normalization of different output curves produced by methods with different initialisation step (processing the same video sequence). There is also a possibility to process the curves as a standard 1D signal (e.g. 1D median filtering). 22

23 23 /18 /52 SPLAB, BUT Klepnutím Cardiac lze cycle upravit computation styl předlohy significant features for are fitted by ellipse and tracked through the video sequence Automatic initialization: Hand initialization: 23

24 24 /18 /52 SPLAB, BUT S [pxl 2 ] Klepnutím Cardiac lze cycle upravit computation styl předlohy the cardiac cycles (ellipse areas in dependence on time) are obtained by each method S 1 S 2 When we want to compare both results, we have to normalize these curves considering the acquisition physical principle t 24

25 25 /18 /52 SPLAB, BUT S - S avg [pxl 2 ] Klepnutím Postprocessing lze upravit styl předlohy after the DC component removal 500 n 300 S avg = 1 n i=1 s i t

26 26 /18 /52 SPLAB, BUT S [pxl 2 ] Klepnutím Postprocessing lze upravit styl předlohy after the window-based DC component removal 400 there are differences because absolute values of S 2 (gray) are smaller than corresponding values of S 1 (black) S avg_w i = 1 w w 2 i= w 2 s i+w t S i = s i S avg_w i 26

27 27 /18 /52 SPLAB, BUT Klepnutím Postprocessing lze upravit styl předlohy the movement of points localised on the artery border is independent on the radius of detected area - the radius change is constant, independent on the circle size area of i-th circle: s i = πr 2 i radius of i-th circle: r i = s i π 2 1 r i = s i π S avg_w π 27

28 28 /18 /52 SPLAB, BUT r [pxl] Klepnutím Postprocessing lze upravit styl předlohy after the calculation of radius change 1,5 1 r i = s i π S avg_w π 0,5 0-0,5-1 -1,5 t [sec] 28

29 29 /18 /52 SPLAB, BUT Klepnutím lze upravit styl předlohy A. Pre-processing Partial Conclusion B. Processing C. Post-processing Basic chain for ultrasound medical video sequences processing methods and results have been presented Mainly for carotis communis analysis Automatic detection is needed for the method initialisation 29

30 30 /18 /52 SPLAB, BUT Artery localization in longitudinal scan Klepnutím lze upravit styl předlohy Input: image of longitudinal section of CCA Output: localized CCA in image 30

31 31 /18 /52 SPLAB, BUT Klepnutím DETECTION lze upravit PROBLEMS styl předlohy Artery localization State of the art in artery localization (longitudinal section, US) Manual Automatic (not robust problems with non-horizontally arteries, curved arteries) non-horizontally arteries curved arteries 31

32 32 /18 /52 SPLAB, BUT Klepnutím PROPOSED METHOD lze upravit FOR ARTERY styl LOCALIZATION předlohy Classification Post-processing Sampling, feature extraction SVM classifier Artery points selection RANSAC method px features [1,1] [1.0,5.3,3.1, ] [5,2] [1.1,3.3,1.1, ] [7,2] [0.0,2.3,7.8, ] [10,1] [2.5,5.85,2.1, ] 32

33 33 /18 /52 SPLAB, BUT SAMPLING, FEATURE EXTRACTION Klepnutím lze upravit styl předlohy Classification Post-processing Sampling, feature extraction SVM classifier Artery points selection RANSAC method Sampling sampling Point 1 Point 2 Point 3 Point 4 33

34 34 /18 /52 SPLAB, BUT Klepnutím Extraction lze of local upravit image styl features předlohy Subsequent classification is based on local image features Local image features = parameters obtained form the pixels and their neighborhood Examples of local image features Mean value Standard deviation Max, median, min values Derivatives (Sobel, Laplacian) Haar features etc. Examples of rectangular neighborhoods 34

35 35 /18 /52 SPLAB, BUT Extraction of local image features Klepnutím lze upravit styl předlohy Local features computed in certain neighborhood (rectangular, circular, etc.) of sampled pixels and formatted into the table Sampled input image Point 1 Point 2 Point 3 Point 4 compute features Local features for particular points Feature 1 Feature 2 Point Point Point Point e.g. mean value e.g. standard deviation 35

36 36 /18 /52 SPLAB, BUT SVM CLASSIFIER Klepnutím lze upravit styl předlohy Classification Post-processing Sampling, feature extraction SVM classifier Artery points selection RANSAC method Classification is performed on the basis of local features Two class classifying of pixels artery pixels (depicted as white pixels) other pixels (depicted as black pixels) SVM classifier Support vector machine classification We test two implementations Classical implementation Extended implementation (radial basis function mapping) 36

37 37 /18 /52 SPLAB, BUT SELECTION OF POINTS Klepnutím lze upravit styl předlohy Classification Post-processing Sampling, feature extraction SVM classifier Artery points selection RANSAC method In further processing only pixels classified as artery pixels are considered Why there are misclassified points? You can see, that there are some misclassified pixels Similarity of tissue, poor accuracy of classifier, bad choice of features 37

38 38 /18 /52 SPLAB, BUT RANSAC ALGORITHM Klepnutím lze upravit styl předlohy Classification Post-processing Sampling, feature extraction SVM classifier Artery points selection RANSAC method RANSAC algorithm suppresses points that were misclassified as artery pixels Misclassified points form small clusters, or they are isolated points RANSAC algorithm find the best linear model of artery 38

39 39 /18 /52 SPLAB, BUT PROPOSED METHOD FOR ARTERY LOCALIZATION Klepnutím lze upravit styl předlohy IMPROVEMENT FOR CURVED ARTERIES Classification Post-processing Sampling, feature extraction SVM classifier Artery points selection Proposed RANSAC based method px features px features label [1,1] [1.0,5.3,3.1, ] [5,2] [1.1,3.3,1.1, ] [7,2] [0.0,2.3,7.8, ] [10,1] [2.5,5.85,2.1, ] [1,1] [1.0,5.3,3.1, ] 0 [5,2] [1.1,3.3,1.1, ] 0 [7,2] [0.0,2.3,7.8, ] 1 [10,1] [2.5,5.85,2.1, ] 1 39

40 40 /18 /52 SPLAB, BUT Klepnutím CLASSIFICATION lze upravit STEP styl předlohy Classification Post-processing Sampling, feature extraction SVM classifier Artery points selection Proposed RANSAC based method Classification + Artery points selection SVM (Support Vector Machine) classifier Two category classification (Artery pixels X Other pixels) Feature set for classifier - local image features In further processing only Artery pixels Select only pixels classified as Artery pixels Highlighted artery pixels Due to the classifier inaccuracy misclassified pixels Post-processing 40

41 41 /18 /52 SPLAB, BUT Klepnutím POST-PROCESSING lze upravit styl STEP předlohy Classification Post-processing Sampling, feature extraction SVM classifier Artery points selection Proposed RANSAC based method Suppress misclassified pixels Problem of data containing outliers RANSAC algorithm 41

42 42 /18 /52 SPLAB, BUT Klepnutím PROPOSED lze RANSAC-BASED upravit styl METHOD předlohy Classification Post-processing Sampling, feature extraction SVM classifier Artery points selection Proposed RANSAC based method RANSAC Standard linear model is inapplicable artery can be curved Proposed modification of standard RANSAC algorithm Non-linear mathematical model - explicit polynomial curve (second order) y = p 2 x 2 + p 1 x + p 0. 42

43 43 /18 /52 SPLAB, BUT Artery Klepnutím detection lze upravit in transversal styl předlohy scan Detection of artery in ultrasound image of perpendicular cut The area inside is correlated with blood pressure level Artery section area can be modelled with ellipse shape Videosequence contains the cardiac cycle data Systolic pressure level = maximal artery section area diastolic pressure level = minimal artery section area Sample ultrasound image Artery section 43

44 44 /18 /52 SPLAB, BUT Artery detection in transversal scan Klepnutím lze upravit styl předlohy Circular pattern localization Exact radius determination The aim: Detection of artery Methods: Optical flow estimation Average image calculation Median filtration, thresholding Hough transform The aim: Precise localization Methods: Analysis of average intensity inside the ring 44

45 45 /18 /52 SPLAB, BUT Circular pattern localization Klepnutím lze upravit styl předlohy 1) Input: source frames in video-sequence 45

46 46 /18 /52 SPLAB, BUT Circular pattern localization Klepnutím lze upravit styl předlohy 2) Scalar field of average optical flow (average value of the movement) in the video sequence 46

47 47 /18 /52 SPLAB, BUT Circular pattern localization Klepnutím lze upravit styl předlohy 3) After the dilation step 47

48 48 /18 /52 SPLAB, BUT Circular pattern localization Klepnutím lze upravit styl předlohy 4) After thresholding 48

49 49 /18 /52 SPLAB, BUT Circular pattern localization Klepnutím lze upravit styl předlohy 5) After the median filtering step 49

50 50 /18 /52 SPLAB, BUT Circular pattern localization Klepnutím lze upravit styl předlohy 6) Hough Transform: detected circle (position of artery) 50

51 51 /18 /52 SPLAB, BUT Circular pattern localization Klepnutím lze upravit styl předlohy Detected circle maximum in accumulation array (Hough space) Multiple circles more peaks in accumulator Bayes classifier is used to choose the most probable circle The feature vector for the classifier consists of statistic values acquired from the pixel intensity localised inside the manually denoted circles from the training data set (average value and standard deviation) and for a circle of smaller radius and a circle from binary image 51

52 52 /18 /52 SPLAB, BUT Klepnutím lze Future upravit work styl předlohy real time and long-time tracking (problems with re-initialization of the tracked object) Using of genetic programming approach for the higher reliability of the algorithm Automatic detection and tracking of more complex objects (heard ventricle) 52

53 53 /18 SPLAB, BUT Klepnutím lze upravit styl předlohy Acknowledgement: This work was also prepared with the support of the MSMT project no. 2B06111: The Research of Algorithms for Processing of Digital Images and Image Sequences and the MSMT project The Research of Algorithms for Processing of Digital Images and Image Sequences, code: ME Some functions from the OpenCV library (Open Source Computer Vision) were used for the implementation of algorithms. Czech Republic European Union