C# &快速API WebSocket通信滞后

ilmyapht  于 2023-05-07  发布在  C#
关注(0)|答案(1)|浏览(254)

我有下面的C#客户端和快速API主机通过WebSocket通信。
C#客户端捕获实时网络摄像头帧并将其发送到flask API,然后将其发送回C#客户端以在PictureBox中显示。在这个例子中,Fast API没有对图像做任何事情,但在真实的代码中,图像正在被处理,因为这只是关注主机和客户端之间的通信。
C#(客户端):

private void CaptureCameraCallback(int camInd, PictureBox pictureBox)
        {
            // Create a new WebSocket connection
            using (var ws = new WebSocketSharp.WebSocket("ws://localhost:8000/ws"))
            {
                // Connect to the WebSocket server
                ws.Connect();

                // Create a new video capture object
                using (var capture = new VideoCapture(camInd))
                {
                    // Start a loop to capture frames and send them to the Flask API.
                    while (true)
                    {
                        // Create a new Mat object to store the frame
                        using (var frame = new Mat())
                        {
                            // Read the next frame from the camera.
                            capture.Read(frame);

                            // Check if the frame is not empty.
                            if (!frame.Empty())
                            {
                                // Encode the frame as a JPEG image
                                var buffer = new byte[frame.Width * frame.Height * frame.ElemSize()];
                                Cv2.ImEncode(".jpg", frame, out buffer);

                                // Send the image to the server
                                ws.Send(buffer);

                                // Wait for the response from the server
                                ws.OnMessage += (sender, e) =>
                                {
                                    // Convert the response from the server back to an image
                                    var responseBytes = e.RawData;
                                    using (var responseStream = new MemoryStream(responseBytes))
                                    {
                                        var responseImage = Image.FromStream(responseStream);

                                        // Display the response image on the UI thread
                                        pictureBox.Invoke((MethodInvoker)(() => {
                                            pictureBox.Image = responseImage;
                                        }));
                                    }
                                };
                            }
                            else
                            {
                                // If the frame is empty, log an error and continue to the next frame.
                                Debug.WriteLine("Unable to capture Frame");
                                continue;
                            }
                        }
                    }
                }
            }
        }

FAST API(主机):

from fastapi import FastAPI, WebSocket
from fastapi.responses import StreamingResponse
import io
import cv2
import numpy as np

class WebSocketServer:
    def __init__(self):
        self.app = FastAPI()

        @self.app.websocket("/ws")
        async def websocket_endpoint(websocket: WebSocket):
            print("Accepting Connection")
            await websocket.accept()
            print('Accepted')
            while True:
                try:
                    image_bytes = await websocket.receive_bytes()
                    nparr = np.frombuffer(image_bytes, np.uint8)
                    img = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
                    
                    # Do something with the image here
                    # ...
                    # Send a response back (optional)
                    _, buffer = cv2.imencode('.jpg', img)
                    response_bytes = buffer.tobytes()
                    await websocket.send_bytes(response_bytes)
                except:
                    pass
                    break
    
    def run(self, host: str = '0.0.0.0', port: int = 8000):
        import uvicorn
        uvicorn.run(self.app, host=host, port=port)

if __name__ == '__main__':
    server = WebSocketServer()
    server.run()

#uvicorn fastApi:app --reload

图像似乎成功发送到Fast API并发送回来,但帧率极慢(不到一个FPS)。我能做错什么?
我相信也许我没有在图片框上正确处理或显示图像。

0ejtzxu1

0ejtzxu11#

这个问题很容易解决,方法是将来自服务器的响应移动到循环之外。
就像这样:

private void CaptureCameraCallback(int camInd, PictureBox pictureBox)
        {
            // Create a new WebSocket connection
            using (var ws = new WebSocketSharp.WebSocket("ws://localhost:8000/ws"))
            {
                // Connect to the WebSocket server
                ws.Connect();

                // Create a new video capture object
                using (var capture = new VideoCapture(camInd))
                {
                    // Create a concurrent queue to store response images
                    var responseQueue = new ConcurrentQueue<Image>();

                    // Set up the event handler to add response images to the queue
                    ws.OnMessage += (sender, e) =>
                    {
                        var responseBytes = e.RawData;
                        using var responseStream = new MemoryStream(responseBytes);
                        var responseImage = Image.FromStream(responseStream);
                        responseQueue.Enqueue(responseImage);
                    };

                    // Start a loop to capture frames and send them to the Flask API.
                    while (true)
                    {
                        // Create a new Mat object to store the frame
                        using (var frame = new Mat())
                        {
                            // Read the next frame from the camera.
                            capture.Read(frame);

                            // Check if the frame is not empty.
                            if (!frame.Empty())
                            {
                                // Encode the frame as a JPEG image
                                var buffer = new byte[frame.Width * frame.Height * frame.ElemSize()];
                                Cv2.ImEncode(".jpg", frame, out buffer);

                                // Send the image to the server
                                ws.Send(buffer);

                                // Check if there is a response image in the queue
                                if (responseQueue.TryDequeue(out var responseImage))
                                {
                                    // Display the response image on the UI thread
                                    pictureBox.Invoke((MethodInvoker)(() =>
                                    {
                                        pictureBox.Image = responseImage;
                                    }));
                                }
                            }
                            else
                            {
                                // If the frame is empty, log an error and continue to the next frame.
                                Debug.WriteLine("Unable to capture Frame");
                                continue;
                            }
                        }
                    }
                }
            }
        }

相关问题