在内存中将OpenCV视频转换为Streamlit视频

pod7payv  于 2022-11-15  发布在  其他
关注(0)|答案(1)|浏览(299)

我有一个图像帧frames的列表,我希望能够在Streamlit应用程序中显示:st.video(frames_converted) .

挑战:

  • Streamlit采用HTML5,视频需要H264编码
  • 希望在内存中完成所有处理(而不是更常见的保存到临时文件

当前尝试:

## Convert frames to video for streamlit
height, width, layers = frames[0].shape
codec = cv.VideoWriter_fourcc(*'H264')
fps = 1
video = cv.VideoWriter("temp_video",codec, fps, (width,height)) # Initialize video object
for frame in frames:
    video.write(frame)
cv.destroyAllWindows()
video.release()

st.video(video)

当前阻止程序

RuntimeError: Invalid binary data format: <class 'cv2.VideoWriter'>
gev0vcfq

gev0vcfq1#

我们可以使用PyAV对“内存中”的MP4视频进行编码,如我的following answer中所述-视频存储在BytesIO对象中。
我们可以将BytesIO对象作为输入传递给Streamlit(或者将BytesIO对象转换为字节数组并将该数组用作输入)。
代码示例:

import numpy as np
import cv2  # OpenCV is used only for writing text on image (for testing).
import av
import io
import streamlit as st

n_frmaes = 100  # Select number of frames (for testing).

width, height, fps = 192, 108, 10  # Select video resolution and framerate.

output_memory_file = io.BytesIO()  # Create BytesIO "in memory file".

output = av.open(output_memory_file, 'w', format="mp4")  # Open "in memory file" as MP4 video output
stream = output.add_stream('h264', str(fps))  # Add H.264 video stream to the MP4 container, with framerate = fps.
stream.width = width  # Set frame width
stream.height = height  # Set frame height
#stream.pix_fmt = 'yuv444p'   # Select yuv444p pixel format (better quality than default yuv420p).
stream.pix_fmt = 'yuv420p'   # Select yuv420p pixel format for wider compatibility.
stream.options = {'crf': '17'}  # Select low crf for high quality (the price is larger file size).

def make_sample_image(i):
    """ Build synthetic "raw BGR" image for testing """
    p = width//60
    img = np.full((height, width, 3), 60, np.uint8)
    cv2.putText(img, str(i+1), (width//2-p*10*len(str(i+1)), height//2+p*10), cv2.FONT_HERSHEY_DUPLEX, p, (255, 30, 30), p*2)  # Blue number
    return img

# Iterate the created images, encode and write to MP4 memory file.
for i in range(n_frmaes):
    img = make_sample_image(i)  # Create OpenCV image for testing (resolution 192x108, pixel format BGR).
    frame = av.VideoFrame.from_ndarray(img, format='bgr24')  # Convert image from NumPy Array to frame.
    packet = stream.encode(frame)  # Encode video frame
    output.mux(packet)  # "Mux" the encoded frame (add the encoded frame to MP4 file).

# Flush the encoder
packet = stream.encode(None)
output.mux(packet)
output.close()

output_memory_file.seek(0)  # Seek to the beginning of the BytesIO.
#video_bytes = output_memory_file.read()  # Convert BytesIO to bytes array
#st.video(video_bytes)
st.video(output_memory_file)  # Streamlit supports BytesIO object - we don't have to convert it to bytes array.

# Write BytesIO from RAM to file, for testing:
#with open("output.mp4", "wb") as f:
#    f.write(output_memory_file.getbuffer())
#video_file = open('output.mp4', 'rb')
#video_bytes = video_file.read()
#st.video(video_bytes)

我们不能使用cv.VideoWriter,因为它不支持内存视频编码(cv.VideoWriter需要一个“真实文件”)。

相关问题