OpenCVSharp：学习最佳匹配矩形检测

前言

今天来学习一下OpenCVSharp中最佳匹配矩形检测的例子。其过程可以分为ORB特征检测、特征匹配、最佳匹配筛选、单应性计算与矩形绘制。

效果：

实践

ORB特征检测

进行ORB特征检测：

using var img1 = new Mat(FirstImagePath, ImreadModes.Color); usingvar img2 = new Mat(SecondImagePath, ImreadModes.Color); usingvar orb = ORB.Create(1000); usingvar descriptors1 = new Mat(); usingvar descriptors2 = new Mat(); orb.DetectAndCompute(img1, null, outvar keyPoints1, descriptors1); orb.DetectAndCompute(img2, null, outvar keyPoints2, descriptors2);

ORB（Oriented FAST and Rotated BRIEF）是一种快速高效的特征检测算法，它结合了FAST关键点检测器和BRIEF描述符，并添加了旋转不变性和尺度不变性，能够在保持较高匹配精度的同时提供极快的计算速度，特别适合实时应用和移动设备上的计算机视觉任务。

查看ORB.Create方法：

public static ORB Create(int nFeatures = 500, float scaleFactor = 1.2f, int nLevels = 8, int edgeThreshold = 31, int firstLevel = 0, int wtaK = 2, ORBScoreType scoreType = ORBScoreType.Harris, int patchSize = 31, int fastThreshold = 20) { NativeMethods.HandleException(NativeMethods.features2d_ORB_create(nFeatures, scaleFactor, nLevels, edgeThreshold, firstLevel, wtaK, (int)scoreType, patchSize, fastThreshold, out var returnValue)); return new ORB(returnValue); }

ORB.Create() 方法是OpenCV中用于创建ORB特征检测器的静态工厂方法，它提供了多个参数来定制ORB检测器的行为：

再来看下DetectAndCompute方法：

public virtual void DetectAndCompute(InputArray image, InputArray? mask, out KeyPoint[] keypoints, OutputArray descriptors, bool useProvidedKeypoints = false) { ThrowIfDisposed(); if (image == null) { thrownew ArgumentNullException("image"); } if (descriptors == null) { thrownew ArgumentNullException("descriptors"); } image.ThrowIfDisposed(); mask?.ThrowIfDisposed(); using VectorOfKeyPoint vectorOfKeyPoint = new VectorOfKeyPoint(); NativeMethods.HandleException(NativeMethods.features2d_Feature2D_detectAndCompute(ptr, image.CvPtr, Cv2.ToPtr(mask), vectorOfKeyPoint.CvPtr, descriptors.CvPtr, useProvidedKeypoints ? 1 : 0)); keypoints = vectorOfKeyPoint.ToArray(); GC.KeepAlive(this); GC.KeepAlive(image); GC.KeepAlive(mask); descriptors.Fix(); GC.KeepAlive(descriptors); }

DetectAndCompute 是OpenCV中特征检测器类的核心方法，用于在图像中检测关键点（特征点）并计算这些点的描述符。这是计算机视觉中特征匹配、物体识别、图像拼接等应用的基础。

查看参数含义：

查看得到的关键点数组：

特征匹配

进行汉明特征匹配：

using var bf = new BFMatcher(NormTypes.Hamming, crossCheck: true); var matches = bf.Match(descriptors1, descriptors2);

汉明匹配是一种基于汉明距离的二进制特征描述符匹配方法，通过计算两个二进制字符串之间不同位的数量来衡量相似度，主要用于ORB、BRISK、FREAK等二进制特征描述符的快速匹配。相比传统的欧几里得距离匹配，汉明匹配具有计算速度快、内存占用小的优势，只需简单的位运算和计数操作，特别适合实时应用和移动设备场景。在OpenCV中，通常使用BFMatcher配合NormTypes.Hamming来实现，通过设置距离阈值（如ORB通常为30-70）来筛选最佳匹配，广泛应用于特征匹配、物体识别和图像拼接等计算机视觉任务中。

查看BFMatcher类的这个构造函数：

public BFMatcher(NormTypes normType = NormTypes.L2, bool crossCheck = false) { NativeMethods.HandleException(NativeMethods.features2d_BFMatcher_new((int)normType, crossCheck ? 1 : 0, out ptr)); detectorPtr = null; }

BFMatcher（Brute-Force Matcher，暴力匹配器）是OpenCV中用于特征描述符匹配的基础类，它通过遍历所有可能的描述符对来找到最佳匹配。

再来看下Match方法：

public DMatch[] Match(Mat queryDescriptors, Mat trainDescriptors, Mat? mask = null) { ThrowIfDisposed(); if (queryDescriptors == null) { thrownew ArgumentNullException("queryDescriptors"); } if (trainDescriptors == null) { thrownew ArgumentNullException("trainDescriptors"); } using VectorOfDMatch vectorOfDMatch = new VectorOfDMatch(); NativeMethods.HandleException(NativeMethods.features2d_DescriptorMatcher_match1(ptr, queryDescriptors.CvPtr, trainDescriptors.CvPtr, vectorOfDMatch.CvPtr, Cv2.ToPtr(mask))); GC.KeepAlive(this); GC.KeepAlive(queryDescriptors); GC.KeepAlive(trainDescriptors); GC.KeepAlive(mask); return vectorOfDMatch.ToArray(); }

Match 是OpenCV中描述符匹配器的核心方法，用于在两组描述符之间找到最佳匹配对。这是特征匹配流程中的关键步骤，将查询描述符与训练描述符进行一对一匹配。

返回的是DMatch结构体数组，查看这个结构体

选取最好的10个匹配：

var goodMatches = matches .OrderBy(x => x.Distance) .Take(10) .ToArray();

提取这些关键点坐标：

var srcPts = goodMatches.Select(m => keyPoints1[m.QueryIdx].Pt).Select(p => new Point2d(p.X, p.Y)); var dstPts = goodMatches.Select(m => keyPoints2[m.TrainIdx].Pt).Select(p => new Point2d(p.X, p.Y));

计算单应性矩阵

using var homography = Cv2.FindHomography(srcPts, dstPts, HomographyMethods.Ransac, 5, null);

查看FindHomography方法：

public static Mat FindHomography(IEnumerable<Point2d> srcPoints, IEnumerable<Point2d> dstPoints, HomographyMethods method = HomographyMethods.None, double ransacReprojThreshold = 3.0, OutputArray? mask = null, int maxIters = 2000, double confidence = 0.995) { if (srcPoints == null) { thrownew ArgumentNullException("srcPoints"); } if (dstPoints == null) { thrownew ArgumentNullException("dstPoints"); } Point2d[] obj = (srcPoints as Point2d[]) ?? srcPoints.ToArray(); Point2d[] array = (dstPoints as Point2d[]) ?? dstPoints.ToArray(); NativeMethods.HandleException(NativeMethods.calib3d_findHomography_vector(obj, obj.Length, array, array.Length, (int)method, ransacReprojThreshold, ToPtr(mask), maxIters, confidence, outvar returnValue)); GC.KeepAlive(mask); mask?.Fix(); returnnew Mat(returnValue); }

FindHomography 是OpenCV中用于计算最佳透视变换矩阵的核心方法，它能够找到将源平面点映射到目标平面点的单应性矩阵。这是计算机视觉中图像配准、拼接和三维重建的基础算法。

HomographyMethods 选项:

int h = img1.Height, w = img1.Width; var img2Bounds = new[] { new Point2d(0, 0), new Point2d(0, h-1), new Point2d(w-1, h-1), new Point2d(w-1, 0), }; var img2BoundsTransformed = Cv2.PerspectiveTransform(img2Bounds, homography);

定义图像边界然后变换图像边界。

public static Point2d[] PerspectiveTransform(IEnumerable<Point2d> src, Mat m) { if (src == null) { thrownew ArgumentNullException("src"); } if (m == null) { thrownew ArgumentNullException("m"); } using Mat<Point2d> mat = Mat.FromArray(src); using Mat<Point2d> mat2 = new Mat<Point2d>(); NativeMethods.HandleException(NativeMethods.core_perspectiveTransform_Mat(mat.CvPtr, mat2.CvPtr, m.CvPtr)); GC.KeepAlive(m); return mat2.ToArray(); }

PerspectiveTransform 是OpenCV中用于执行透视变换的核心方法，它能够对二维或三维点集合应用透视变换矩阵，实现坐标系的转换。这是图像几何变换中的基础操作，广泛应用于图像校正、拼接和增强现实等领域。

矩形绘制

using var view = img2.Clone(); var drawingPoints = img2BoundsTransformed.Select(p => (Point)p).ToArray(); Cv2.Polylines(view, new[] { drawingPoints }, true, Scalar.Red, 3);

查看Polylines方法：

public static void Polylines( Mat img, IEnumerable<IEnumerable<Point>> pts, bool isClosed, Scalar color, int thickness = 1, LineTypes lineType = LineTypes.Link8, int shift = 0) { if (img isnull) thrownew ArgumentNullException(nameof(img)); if (pts isnull) thrownew ArgumentNullException(nameof(pts)); img.ThrowIfDisposed(); var ptsList = new List<Point[]>(); var nptsList = new List<int>(); foreach (var pts1 in pts) { var pts1Arr = pts1.ToArray(); ptsList.Add(pts1Arr); nptsList.Add(pts1Arr.Length); } var ptsArr = ptsList.ToArray(); var npts = nptsList.ToArray(); var ncontours = ptsArr.Length; usingvar ptsPtr = new ArrayAddress2<Point>(ptsArr); NativeMethods.HandleException( NativeMethods.imgproc_polylines_Mat( img.CvPtr, ptsPtr.GetPointer(), npts, ncontours, isClosed ? 1 : 0, color, thickness, (int) lineType, shift)); GC.KeepAlive(img); }

Polylines 是OpenCV中用于绘制一个或多个多边形曲线的绘图函数。它可以在图像上绘制连续的线段，形成闭合或开放的多边形形状，是计算机视觉中可视化检测结果、标注区域和绘制轮廓的重要工具。