我正在从事一个计算机视觉项目,涉及使用OpenCV和Django进行对象检测和实时帧流。为了实现这一目标,我使用了Django通道。然而,我遇到了一个新的挑战,我需要执行视频分类。这需要将一批帧发送到模型。有人能提供如何解决这个问题的指导吗?
我在前端添加了一个按钮。我所期望的是从后端consumers.py获得一个时间戳。我获得了开始帧的时间戳,直到按钮被点击的帧的时间戳。
的数据
在consumers.py文件中,我使用OpenCV库来遍历帧并将数据传输到WebSocket客户端。下面是代码的示例(为了更好地理解,我还提供了注解)。
# Import the 'cv2' module, which is the OpenCV library for computer vision tasks.
import cv2
# Import the 'json' module for working with JSON data.
import json
# Import the 'asyncio' module for asynchronous programming support.
import asyncio
# Import the 'datetime' module for working with date and time.
import datetime
# Import the 'numpy' module for numerical and array operations.
import numpy as np
# Import a custom function 'serialize_frame' from the 'processing' module in the current application.
from .processing import serialize_frame
# Import the 'AsyncWebsocketConsumer' class from Django's 'channels.generic.websocket' module.
from channels.generic.websocket import AsyncWebsocketConsumer
# Define a custom WebSocket consumer class named VideoConsumer that handles WebSocket communication.
class VideoConsumer(AsyncWebsocketConsumer):
# Define the 'connect' method for WebSocket connection handling.
# This method is executed when a WebSocket connection is established.
async def connect(self):
# Accept the WebSocket connection.
await self.accept()
# Retrieve the video file path from the URL route parameters.
video_file_path = self.scope['url_route']['kwargs']['video_path']
# Send processed frames using the video file path.
await self.send_processed_frames(video_file_path)
# Define the 'send_frame' method for sending video frame data and match data to the WebSocket client.
# This method takes 'frame_data' and 'match_data' as parameters and sends them as JSON to the WebSocket client.
async def send_frame(self, frame_data, timestamp_str):
# Convert the frame data and match data into a JSON format and send it to the client.
await self.send(text_data=json.dumps({
'frame_data': frame_data,
'timestamp': timestamp_str
}))
# Define the 'send_processed_frames' method for processing and sending video frames and match data.
# This method takes 'video_file_path' as a parameter and processes video frames and related data.
async def send_processed_frames(self, video_file_path):
# Initialize data structures and variables.
# A dictionary for actions
try:
cap = cv2.VideoCapture(video_file_path)
# Start iterating over the frames
while True:
ret, frame = cap.read()
# Get the timestamp of the frame on which the prediction is being performed
timestamp = cap.get(cv2.CAP_PROP_POS_MSEC)
timestamp = datetime.timedelta(milliseconds=timestamp)
timestamp_str = str(timestamp).split('.', 2)[0]
# Encode the frames to base64 to send to the web
frame_data = serialize_frame(frame) # Implement the serialization function
# Send the frame data and match data to the WebSocket client.
await self.send_frame(frame_data, timestamp_str)
# Adjust the frame rate as needed (e.g., 30 frames per second).
await asyncio.sleep(0.033)
# Release the video capture object.
cap.release()
except Exception as e:
# In case of an error, send an error message to the WebSocket client.
await self.send(text_data=f"Error: {str(e)}")
async def receive(self, text_data):
# Handle messages from WebSocket
message = json.loads(text_data)
print(message)
字符串
我已经在routing.py中指定了路线。
# Import necessary modules for URL routing in Django.
from django.urls import re_path
# Import the 'VideoConsumer' class from the 'consumers' module of the current application.
from . import consumers
# Define WebSocket URL patterns for the application.
# Create a WebSocket URL pattern that captures the 'video_path' parameter from the URL.
# It routes WebSocket requests to the 'VideoConsumer' class using the 'as_asgi()' method.
websocket_urlpatterns = [
re_path(r'ws/video/(?P<video_path>[\w\W]+)/$', consumers.VideoConsumer.as_asgi())
]
型
然后将数据从后端发送到前端。
{% extends 'main.html' %}
{% block content %}
<div class="container text-center">
<h1>Processing Video</h1>
<br>
<!-- <img class="rounded" id="liveImage" width="800" height="600"> -->
<div class="liveImage">
<div class="text-secondary text-center" role="status">
</div>
<img class="rounded" id="liveImage">
</div>
<hr>
<h5>To classify the cover type, specify the start duration to clip the video</h5>
<div class="d-grid gap-2">
<button class="btn btn-primary" id="sendTimestampButton" type="button">Button</button>
</div>
</div>
<br>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.6.4/jquery.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.15.3/js/all.min.js"></script>
<script type="text/javascript">
document.addEventListener("DOMContentLoaded", function () {
let videoPath = "{{ video_path }}"; // Use the video path from your Django template
let liveImage = document.getElementById("liveImage");
// Establish a WebSocket connection to your server
let wsProtocol = window.location.protocol === "https:" ? "wss" : "ws";
let wsURL = `${wsProtocol}://${window.location.host}/ws/video/${videoPath}/`;
let socket = new WebSocket(wsURL);
// document.getElementById('sendTimestampButton').addEventListener('click', function () {
// // Send a message to the backend to request the timestamp
// socket.send(JSON.stringify({'request_timestamp': true}));
// console.log('TimeStamp: ');
// });
socket.onopen = function (event) {
console.log('Websocket Connection established')
}
socket.onmessage = function (event) {
let data = JSON.parse(event.data);
document.getElementById('sendTimestampButton').addEventListener('click', function () {
// // Send a message to the backend to request the timestamp
socket.send(JSON.stringify({'timestamp': data.timestamp}));
console.log('Timestamp: ', data.timestamp);
});
liveImage.src = "data:image/jpeg;base64," + data.frame_data;
}
socket.onclose = function (event) {
console.log("WebSocket connection closed.");
};
socket.onerror = function (error) {
console.error("WebSocket error:", error);
};
});
</script>
{% endblock content %}
型
要查看完整的代码,请查看repo:https://github.com/ShahZebYousafzai/Django-Real-Time-Inference-on-video
1条答案
按热度按时间pxiryf3j1#
我能解决这个问题!我在consumers.py中的
VideoConsumer
类的开头定义了一个初始变量。在while循环中,我将这个变量描述为一个类变量,并存储了帧的当前时间戳。在www.example.com中veiws.py,我创建了一个发送JSON响应的函数类。我在该函数中获取时间戳并将其作为JSON响应发送。在前端,我使用AJAX来获得响应。每当我点击按钮时,我都会得到帧的时间戳。