我有下面的C#客户端和快速API主机通过WebSocket通信。
C#客户端捕获实时网络摄像头帧并将其发送到flask API,然后将其发送回C#客户端以在PictureBox
中显示。在这个例子中,Fast API没有对图像做任何事情,但在真实的代码中,图像正在被处理,因为这只是关注主机和客户端之间的通信。
C#(客户端):
private void CaptureCameraCallback(int camInd, PictureBox pictureBox)
{
// Create a new WebSocket connection
using (var ws = new WebSocketSharp.WebSocket("ws://localhost:8000/ws"))
{
// Connect to the WebSocket server
ws.Connect();
// Create a new video capture object
using (var capture = new VideoCapture(camInd))
{
// Start a loop to capture frames and send them to the Flask API.
while (true)
{
// Create a new Mat object to store the frame
using (var frame = new Mat())
{
// Read the next frame from the camera.
capture.Read(frame);
// Check if the frame is not empty.
if (!frame.Empty())
{
// Encode the frame as a JPEG image
var buffer = new byte[frame.Width * frame.Height * frame.ElemSize()];
Cv2.ImEncode(".jpg", frame, out buffer);
// Send the image to the server
ws.Send(buffer);
// Wait for the response from the server
ws.OnMessage += (sender, e) =>
{
// Convert the response from the server back to an image
var responseBytes = e.RawData;
using (var responseStream = new MemoryStream(responseBytes))
{
var responseImage = Image.FromStream(responseStream);
// Display the response image on the UI thread
pictureBox.Invoke((MethodInvoker)(() => {
pictureBox.Image = responseImage;
}));
}
};
}
else
{
// If the frame is empty, log an error and continue to the next frame.
Debug.WriteLine("Unable to capture Frame");
continue;
}
}
}
}
}
}
FAST API(主机):
from fastapi import FastAPI, WebSocket
from fastapi.responses import StreamingResponse
import io
import cv2
import numpy as np
class WebSocketServer:
def __init__(self):
self.app = FastAPI()
@self.app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
print("Accepting Connection")
await websocket.accept()
print('Accepted')
while True:
try:
image_bytes = await websocket.receive_bytes()
nparr = np.frombuffer(image_bytes, np.uint8)
img = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
# Do something with the image here
# ...
# Send a response back (optional)
_, buffer = cv2.imencode('.jpg', img)
response_bytes = buffer.tobytes()
await websocket.send_bytes(response_bytes)
except:
pass
break
def run(self, host: str = '0.0.0.0', port: int = 8000):
import uvicorn
uvicorn.run(self.app, host=host, port=port)
if __name__ == '__main__':
server = WebSocketServer()
server.run()
#uvicorn fastApi:app --reload
图像似乎成功发送到Fast API并发送回来,但帧率极慢(不到一个FPS)。我能做错什么?
我相信也许我没有在图片框上正确处理或显示图像。
1条答案
按热度按时间0ejtzxu11#
这个问题很容易解决,方法是将来自服务器的响应移动到循环之外。
就像这样: