如何将opencv cv：：Mat转换为CVPixelBuffer

gab6jxml 于 2022-11-15 发布在其他

关注(0)|答案(3)|浏览(274)

我是一名本科生，我正在使用CoreML制作一些HumanSeg iPhone应用程序。由于我的模型需要在原始视频帧上***调整大小和黑色填充***，我不能依赖Vision（它只提供调整大小，但没有黑色填充），必须自己进行转换。
我有CVPixelBuffer帧，我已经使用以下代码将其转换为cv::Mat：

CVPixelBufferLockBaseAddress(pixelBuffer, 0);
int bufferWidth = (int) CVPixelBufferGetWidth(pixelBuffer);
int bufferHeight = (int) CVPixelBufferGetHeight(pixelBuffer);
int bytePerRow = (int) CVPixelBufferGetBytesPerRow(pixelBuffer);
unsigned char *pixel = (unsigned char *) CVPixelBufferGetBaseAddress(pixelBuffer);
Mat image = Mat(bufferHeight, bufferWidth, CV_8UC4, pixel, bytePerRow);
CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);
/*I'll do my resizing and padding here*/

// How can I implement this function?
convertToCVPixelBuffer(image);

“但是现在，在我完成了预处理工作之后，我必须***将***cv::Mat***转换回***一个CVPixelBuffer，然后将其馈送到CoreML模型中。我该如何实现这一点？（或者Vision可以使用一些特殊的技术来实现黑色填充吗？）”
任何帮助都将不胜感激。

opencv

来源：https://stackoverflow.com/questions/47108778/how-to-convert-opencv-cvmat-to-cvpixelbuffer

3条答案

按热度按时间

wqnecbli1#

请看下面的代码...检查宽度和高度是否可以被64整除是必要的，否则我们会得到奇怪的结果，因为BytesPerRow与cv::Mat和CVPixelBuffer不匹配

CVPixelBufferRef getImageBufferFromMat(cv::Mat matimg) {
    cv::cvtColor(matimg, matimg, CV_BGR2BGRA);
    
    /* Very much required see https://stackoverflow.com/questions/66434552/objective-c-cvmat-to-cvpixelbuffer
     height & width has to be multiple of 64 for better caching
     */
    int widthReminder = matimg.cols % 64, heightReminder = matimg.rows % 64;
    if (widthReminder != 0 || heightReminder != 0) {
        cv::resize(matimg, matimg, cv::Size(matimg.cols + (64 - widthReminder), matimg.rows + (64 - heightReminder)));
    }
    
    NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
                             [NSNumber numberWithBool: YES], kCVPixelBufferCGImageCompatibilityKey,
                             [NSNumber numberWithBool: YES], kCVPixelBufferCGBitmapContextCompatibilityKey,
                             [NSNumber numberWithInt: matimg.cols], kCVPixelBufferWidthKey,
                             [NSNumber numberWithInt: matimg.rows], kCVPixelBufferHeightKey,
                             [NSNumber numberWithInt: matimg.step[0]], kCVPixelBufferBytesPerRowAlignmentKey,
                             nil];
    CVPixelBufferRef imageBuffer;
    CVReturn status = CVPixelBufferCreate(kCFAllocatorMalloc, matimg.cols, matimg.rows, kCVPixelFormatType_32BGRA, (CFDictionaryRef) CFBridgingRetain(options), &imageBuffer) ;
    NSParameterAssert(status == kCVReturnSuccess && imageBuffer != NULL);
    
    CVPixelBufferLockBaseAddress(imageBuffer, 0);
    void *base = CVPixelBufferGetBaseAddress(imageBuffer);
    memcpy(base, matimg.data, matimg.total() * matimg.elemSize());
    CVPixelBufferUnlockBaseAddress(imageBuffer, 0);
    
    return imageBuffer;
}

赞(0）回复(0）举报 2022-11-15

x33g5p2x2#

首先，将mat转换为UIImage（或iOS API中的任何其他类），检查此question。然后，将生成的图像转换为CVPixelBuffer，如this。

赞(0）回复(0）举报 2022-11-15

jhdbpxl93#

对于那些将要使用新的OpenCV Swift Package 器的人来说，这里是@Abhinava翻译成Swift的代码

func matToCVPixelBuffer(mat: Mat)-> CVPixelBuffer? {
    let matrix = Mat()
    Imgproc.cvtColor(src: mat, dst: matrix, code: ColorConversionCodes.COLOR_BGR2BGRA)
    
    let widthRemainder = matrix.cols() % 64
    let heightRemainder = matrix.rows() % 64
    
    if widthRemainder != 0 || heightRemainder != 0 {
        Imgproc.resize(src: matrix, dst: matrix, dsize: Size(width: matrix.cols() + (64 - widthRemainder), height: matrix.rows() + (64 - heightRemainder)))
    }
    
    let attributes = [
        kCVPixelBufferMetalCompatibilityKey: kCFBooleanTrue!,
        kCVPixelBufferCGImageCompatibilityKey: kCFBooleanTrue!,
        kCVPixelBufferCGBitmapContextCompatibilityKey: kCFBooleanTrue!,
        kCVPixelBufferWidthKey: matrix.cols(),
        kCVPixelBufferHeightKey: matrix.rows(),
        kCVPixelBufferBytesPerRowAlignmentKey: matrix.step1(0)
    ] as CFDictionary
    
    var pixelBuffer: CVPixelBuffer?
    
    let status = CVPixelBufferCreate(
        kCFAllocatorDefault, Int(matrix.cols()),
        Int(matrix.rows()),
        kCVPixelFormatType_32BGRA,
        attributes,
        &pixelBuffer)
    
    guard let pixelBuffer = pixelBuffer, (status == kCVReturnSuccess) else {
        return nil
    }
    
    CVPixelBufferLockBaseAddress(pixelBuffer, CVPixelBufferLockFlags(rawValue: 0))
    
    let base = CVPixelBufferGetBaseAddress(pixelBuffer)
    memcpy(base, matrix.dataPointer(), matrix.total()*matrix.elemSize())
    
    CVPixelBufferUnlockBaseAddress(pixelBuffer, CVPixelBufferLockFlags(rawValue: 0))
    
    return pixelBuffer
}

赞(0）回复(0）举报 2022-11-15

我来回答

如何将opencv cv：：Mat转换为CVPixelBuffer

3条答案

相关问题

热门标签

最新问答