Algebraic methods in Computer Vision Zuzana Kukelova, Tomas Pajdla, Martin Bujnak Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague
Motivation 2/34 Many problems in computer vision, robotics and other fields lead to systems of polynomial equations Camera calibration Relative/Absolute pose estimation
3D reconstruction Motivation SfM 3/34 Recognition Tracking Require to solve camera calibration, relative/absolute pose estimation
Motivation 4/34 Systems of equations are not trivial - many equations, many unknowns, high degree x 1 x 2 Contaminated input measurements => problems have to be solved for many different inputs (RANSAC) High or even real-time performance
Motivation 5/34 General algorithms slow not feasible To solve these systems of polynomial equations special algorithms have to be designed to achieve numerical robustness and computational efficiency Not general Fast
One polynomial equation in one unknown - Companion matrix 6/34 1 equation in 1 variable companion matrix eigenvalues... a simple rule Eigenvalues = solutions
System of linear polynomial equations - Gauss elimination 7/34 System of n linear equations in n unknowns We can represent this system in a matrix form backsubstitution Perform Gauss-elimination reduces the matrix to a triangular form
System of m polynomial equations in n unknowns 8/34 A system of m polynomial equations with coefficients from in n variables Our goal is to solve this system Many different methods Numerical methods Symbolic methods Grőbner basis methods Resultant methods
System of m nonlinear polynomial equations in n unknowns - Grőbner basis method 9/34 System of m equations in n unknowns Idea construct a new system of polynomial equations with the same solutions as the initial one, but with a simpler structure Grőbner bases have the same solutions but are easier to solve Zuzana Kúkelová kukelova@cmp.felk.cvut.cz, Tomáš Pajdla pajdla@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University in Prague
System of m nonlinear polynomial equations in n unknowns - Grőbner basis method 10/34 The reality is much harder Equations Grőbner basis Computation of Grőbner bases no simple way is a NP-complete in m, n (takes very long time to compute, needs huge space to remember intermediate results) Fortunately in some cases, Grőbner bases can be obtained easier
System of m nonlinear polynomial equations in n unknowns - Grőbner basis method 11/34 How to find coefficients? Bucheberger s algorithm ~ generalized Gauss elimination
Grőbner basis method w.r.t. lexicographic ordering ~ generalized G-J elimination 12/34 General approach for constructing the Grőbner basis Add new Perform equations Gauss-Jordan monomial elimination multiples of initial equations
System of m polynomial equations in n unknowns multiplication matrix 13/34 Polynomial division m equations in n unknowns Grőbner basis (grevlex ordering) Generalized companion matrix Solutions Buchberger s algorithm Eigenvectors Eigenvectors ~ solutions
Grőbner basis method 14/34 Computing Grőbner basis (even in grevlex ordering) may be very hard Example: 4 polynomials, 3 variables, degree 6, small integer coeffs Have extremly simple Grőbner basis
Grőbner basis method 15/34 However when computed by the Buchberger s algorithm over the rational numbers w.r.t. the grevlex ordering x > y > z, the following polynomial appears during the computation: ~ 80,000 digits
Grőbner basis method General algorithms can construct all GBs but often generate many complicated polynomials 16/34 C o m p l e x i t y G e n e r a l a l g o r i t h m S p e c i f i c a l g o r i t h m Input Grőbner basis Zuzana Kúkelová kukelova@cmp.felk.cvut.cz, Tomáš Pajdla pajdla@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University in Prague
Specific Grőbner basis algorithm 17/34 All ``non-degenerate'' instances of the problem result in systems of polynomial equations of the same form, i.e. systems consisting of the same polynomials and differing only in coefficients Always the same number of solutions Grőbner basis w.r.t. some fixed monomial ordering is always the same and differs for each instance only in coefficients arising from nondegenerate measurements. The way of obtaining the Grőbner basis is always the same The way of obtaining all necessary polynomials for creating the multiplication matrix is always the same
Specific Grőbner basis algorithm 18/34 Find a short path towards the Grőbner basis/multiplication matrix which is independent from the actual ``non-degenerate coefficients, implement it efficiently run long, coefficient growth numerical problems fast, no coefficient growth, no numerical problems
Specific Grőbner basis algorithm 19/34 returns a very similar basis in the fraction of the second
Specific Grőbner basis algorithm Offline Online 20/34 Pose problem in. Pose problem in. Compute the number of solutions and find the form of all necessary polynomials for multiplication matrix Find the way to these polynomials (systematically generate polynomials from the ideal) Elimination template Generate all necessary polynomials for constructing the multiplication matrix Construct the multiplication matrix and find its eigenvalues and eigenvectors Remove unnecessary polynomials Extract solutions from eigenvectors
Specific Grőbner basis algorithm 21/34 Elimination template hard to understand and implement 15min 25min Requires knowledge from algebraic geometry
Automatic generator of Grőbner basis solvers Input Output 22/34 Pose problem in. Pose problem in. Compute the number of solutions and find the form of all necessary polynomials for multiplication matrix Find the way to these polynomials (systematically generate polynomials from the ideal) Elimination template Generate all necessary polynomials for constructing the multiplication matrix Construct the multiplication matrix and find its eigenvalues and eigenvectors Remove unnecessary polynomials Extract solutions from eigenvectors
Automatic generator of Grőbner basis solvers 23/34
Hidden variable & polynomial eigenvalues 24/34 matrix form consider as a parameter Hidden variable method: If B is square matrix then the system has non-trivial solutions iff - polynomial in
Polynomial eigenvalue method an example 25/34 Rewrite in a matrix form Rewrite in a form ~Polynomial eigenvalue formulation
Relative pose problems 26/34 Estimation of relative position and orientation (rotation, translation) of two cameras from point correspondences 1. Calibrated cameras 5-pt relative pose problem (BMVC 08) 2. Cameras with unknown focal length 6-pt focal length problem (BMVC 08) 3. One calibrated and one up to focal length calibrated camera 6-pt on-off problem (ICCV 09) 4. Cameras with unknown focal length observing a dominant plane plane +parallax + unknown focal length from 5 point correspondences (CVVT 10)
6 point focal length problem 27/34 Given 6 point correspondences in two images, find relative position, orientation and focal length of partially calibrated camera 15 solutions
6 point focal length problem 28/34 Epipolar constraint Parametrize the fundamental as a linear combination of a basis of the space of all compatible fundamental matrices Singularity of F Transfer F to essential matrix E Trace constraint of E 10 3 rd and 5 th order equations in 3 variables and with 30 monomials
6 point focal length problem 29/34 Grőbner basis solution Stewenius et al. - CVPR 2005-3x G-J eliminations of 12x33, 16x33 and 18x33 matrices + eigenvalue computation of 15x15 matrix Kukelova et al. ECCV 2008 automatic generator single G-J elimination of 31x46 matrix + eigenvalue computation of 15x15 matrix Polynomial eigenvalue solution Kukelova et al BMVC 2008 Quadratic polynomial eigenvalue problem (degree 2) - => eigenvalue computation of 20x20 matrix Running time - 182μs
Relative pose problems for cameras with radial distortion 30/34 Three new minimal problems for estimating one-parameter radial distortion model and epipolar geometry from point correspondences 1. Minimal algorithm for uncalibrated cameras with same radial distortion and 8 point correspondences (CVPR 2007) 2. Minimal algorithm for partially calibrated cameras with same radial distortion and 6 point correspondences (CVPR 2008) 3. Minimal algorithm for uncalibrated cameras with different radial distortions in each image and 9 point correspondences (CVPR 2008)
8-point algorithm for uncalibrated cameras with same radial distortions 31/34 Results Before correction After correction Left image Right image
Absolute pose problems 32/34 Estimation of absolute position and orientation of one camera from 2D- 3D point correspondences PnP problems 1. Camera with unknown focal length P4P+f (CVPR 2008) 2. Camera with unknown focal length and radial distortion P4P+f+k (ACCV 2010) 3. Calibrated camera and camera with unknown focal length and radial distortion with known vertical direction (rotation around one axis) up2p, up3p+f+k (ACCV 2010) o x 1 X 1 X 2 f X 4 X 3
Conclusion 33/34 Many problems in computer vision and other fields lead to systems of polynomial equations No general robust and efficient method for solving systems of polynomial equations Special algorithms have to be designed to achieve numerical robustness and computational efficiency Grőbner basis methods / polynomial eigenvalue methods Automatic generator of Grőbner basis solvers => efficient algorithms to many minimal computer vision problems
http://cmp.felk.cvut.cz/minimal 34/34 THANK YOU FOR YOUR ATTENTION