flutter Gstreamer插件，用于在第一帧嵌入抖动冻结

wydwbb8l  于 2022-12-24  发布在  Flutter
关注(0)|答案(1)|浏览(124)

我正在尝试为flutter上的video_player包制作一个视频播放器插件。这都是在一个嵌入式目标（iMX8M Mini）上。这个插件使用gstreamer来工作。然而，虽然终端上的gstreamer管道按预期工作，但通过插件它会挂在视频播放的第一帧。
您可以在这里找到插件的视频播放器代码：https://github.com/mmerah/flutter-elinux-plugins-imx8/blob/main/packages/video_player/elinux/gst_video_player.cc（这只是https://github.com/sony/flutter-elinux-plugins的一个分支）
代码的一些片段：

GstVideoPlayer::GstVideoPlayer(
    const std::string& uri, std::unique_ptr<VideoPlayerStreamHandler> handler)
    : stream_handler_(std::move(handler)) {
  gst_.pipeline = nullptr;
  gst_.playbin = nullptr;
  gst_.video_convert = nullptr;
  gst_.video_sink = nullptr;
  gst_.output = nullptr;
  gst_.bus = nullptr;
  gst_.buffer = nullptr;

  uri_ = ParseUri(uri);
  if (!CreatePipeline()) {
    std::cerr << "Failed to create a pipeline" << std::endl;
    DestroyPipeline();
    return;
  }

  // Prerolls before getting information from the pipeline.
  Preroll();

  // Sets internal video size and buffier.
  GetVideoSize(width_, height_);
  pixels_.reset(new uint32_t[width_ * height_]);

  stream_handler_->OnNotifyInitialized();
}

bool GstVideoPlayer::Play() {
  if (gst_element_set_state(gst_.pipeline, GST_STATE_PLAYING) ==
      GST_STATE_CHANGE_FAILURE) {
    std::cerr << "Failed to change the state to PLAYING" << std::endl;
    return false;
  }
  return true;
}

bool GstVideoPlayer::Pause() {
  if (gst_element_set_state(gst_.pipeline, GST_STATE_PAUSED) ==
      GST_STATE_CHANGE_FAILURE) {
    std::cerr << "Failed to change the state to PAUSED" << std::endl;
    return false;
  }
  return true;
}

bool GstVideoPlayer::Stop() {
  if (gst_element_set_state(gst_.pipeline, GST_STATE_READY) ==
      GST_STATE_CHANGE_FAILURE) {
    std::cerr << "Failed to change the state to READY" << std::endl;
    return false;
  }
  return true;
}

// (...)

bool GstVideoPlayer::CreatePipeline() {
  gst_.pipeline = gst_pipeline_new("pipeline");
  if (!gst_.pipeline) {
    std::cerr << "Failed to create a pipeline" << std::endl;
    return false;
  }
  gst_.playbin = gst_element_factory_make("playbin", "playbin");
  if (!gst_.playbin) {
    std::cerr << "Failed to create a source" << std::endl;
    return false;
  }
  gst_.video_convert = gst_element_factory_make("videoconvert", "videoconvert");
  if (!gst_.video_convert) {
    std::cerr << "Failed to create a videoconvert" << std::endl;
    return false;
  }
  gst_.video_sink = gst_element_factory_make("fakesink", "videosink");
  if (!gst_.video_sink) {
    std::cerr << "Failed to create a videosink" << std::endl;
    return false;
  }
  gst_.output = gst_bin_new("output");
  if (!gst_.output) {
    std::cerr << "Failed to create an output" << std::endl;
    return false;
  }
  gst_.bus = gst_pipeline_get_bus(GST_PIPELINE(gst_.pipeline));
  if (!gst_.bus) {
    std::cerr << "Failed to create a bus" << std::endl;
    return false;
  }
  gst_bus_set_sync_handler(gst_.bus, HandleGstMessage, this, NULL);

  // Sets properties to fakesink to get the callback of a decoded frame.
  g_object_set(G_OBJECT(gst_.video_sink), "sync", FALSE, "qos", FALSE, NULL);
  g_object_set(G_OBJECT(gst_.video_sink), "signal-handoffs", TRUE, NULL);
  g_signal_connect(G_OBJECT(gst_.video_sink), "handoff",
                   G_CALLBACK(HandoffHandler), this);
  gst_bin_add_many(GST_BIN(gst_.output), gst_.video_convert, gst_.video_sink,
                   NULL);

  // Adds caps to the converter to convert the color format to RGBA.
  auto* caps = gst_caps_from_string("video/x-raw");
  auto link_ok =
      gst_element_link_filtered(gst_.video_convert, gst_.video_sink, caps);
  gst_caps_unref(caps);
  if (!link_ok) {
    std::cerr << "Failed to link elements" << std::endl;
    return false;
  }

  auto* sinkpad = gst_element_get_static_pad(gst_.video_convert, "sink");
  auto* ghost_sinkpad = gst_ghost_pad_new("sink", sinkpad);
  gst_pad_set_active(ghost_sinkpad, TRUE);
  gst_element_add_pad(gst_.output, ghost_sinkpad);

  // Sets properties to playbin.
  g_object_set(gst_.playbin, "uri", uri_.c_str(), NULL);
  g_object_set(gst_.playbin, "video-sink", gst_.output, NULL);
  gst_bin_add_many(GST_BIN(gst_.pipeline), gst_.playbin, NULL);

  return true;
}

void GstVideoPlayer::Preroll() {
  if (!gst_.playbin) {
    return;
  }

  auto result = gst_element_set_state(gst_.pipeline, GST_STATE_PAUSED);
  if (result == GST_STATE_CHANGE_FAILURE) {
    std::cerr << "Failed to change the state to PAUSED" << std::endl;
    return;
  }

  // Waits until the state becomes GST_STATE_PAUSED.
  if (result == GST_STATE_CHANGE_ASYNC) {
    GstState state;
    result =
        gst_element_get_state(gst_.pipeline, &state, NULL, GST_CLOCK_TIME_NONE);
    if (result == GST_STATE_CHANGE_FAILURE) {
      std::cerr << "Failed to get the current state" << std::endl;
    }
  }
}

我相信的是等效管道：gst-launch-1.0 playbin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_cropped_multilingual.webm video-sink="videoconvert ! video/x-raw ! fakesink sync=FALSE。当从终端调用时，此操作有效。在终端上使用autovideosink而不是fakesink时也是如此，视频将正确地流传输到目标上的显示器上。
顺便说一句：

  • sync = TRUE时，当从终端调用管道时，视频挂起在第一帧。
  • 格式= RGBA，相同的内容：当从终端调用流水线时，视频挂起在第一帧。

当使用flutter时，视频似乎被正确缓冲（第一帧出现），但当我点击播放时，除了第一帧之外什么也没有出现。它似乎挂在那里（但我确实在我的插件分支中放置了sync = FALSE并删除了format = RGBA）：

root@verdin-imx8mm-06898530:/# GST_DEBUG=3 flutter-client -f -n -b /usr/share/flutter/demo_flutter_app

====== AIUR: 4.7.1 build on Jul 19 2022 06:15:03. ======
        Core: MKVPARSER_01.08.17  build on Apr 26 2022 07:37:32
 file: /usr/lib/imx-mm/parser/lib_mkv_parser_arm_elinux.so.3.1
0:00:03.282695211   868 0xffff600029e0 WARN               aiurdemux aiurdemux.c:752:gst_aiurdemux_handle_sink_event: need to drop sink event GST_EVENT_CUSTOM_DOWNSTREAM_STICKY
0:00:03.830029344   868 0xffff600029e0 WARN               aiurdemux aiurdemux.c:752:gst_aiurdemux_handle_sink_event: need to drop sink event GST_EVENT_CUSTOM_DOWNSTREAM_STICKY
0:00:04.340769125   868 0xffff600029e0 WARN               aiurdemux aiurdemux.c:752:gst_aiurdemux_handle_sink_event: need to drop sink event GST_EVENT_CUSTOM_DOWNSTREAM_STICKY
------------------------
    Track 00 [audio_0] Enabled
        Duration: 0:00:47.791668000
        Language: por
    Mime:
        audio/x-vorbis, channels=(int)2, rate=(int)44100, bitrate=(int)0, framed=(boolean)true
------------------------
0:00:06.614267007   868 0xffff54298760 WARN                   pulse pulsesink.c:617:gst_pulseringbuffer_open_device:<pulsesink0> error: Failed to connect: Connection refused
0:00:06.614774627   868 0xffff54298760 WARN                 playbin gstplaybin2.c:4775:autoplug_select_cb:<playbin> Could not activate sink pulsesink
0:00:06.623663411   868 0xffff54298760 WARN                    alsa pcm_hw.c:1716:snd_pcm_hw_open: alsalib error: open '/dev/snd/pcmC0D0p' failed (-77): File descriptor in bad state

====== BEEP: 4.7.1 build on Jul 19 2022 06:15:03. ======
        Core: OggVorbis decoder Wrapper  build on Dec  7 2017 18:15:03
 file: /usr/lib/imx-mm/audio-codec/wrap/lib_vorbisd_wrap_arm_elinux.so.3
CODEC: OGGVORBISD_ARM_02.04.00_ARMV8  build on Mar 11 2019 19:56:02.
------------------------
    Track 01 [audio_1] Enabled
        Duration: 0:00:47.791668000
        Language: eng
    Mime:
        audio/x-vorbis, channels=(int)2, rate=(int)48000, bitrate=(int)0, framed=(boolean)true
------------------------

====== BEEP: 4.7.1 build on Jul 19 2022 06:15:03. ======
        Core: OggVorbis decoder Wrapper  build on Dec  7 2017 18:15:03
 file: /usr/lib/imx-mm/audio-codec/wrap/lib_vorbisd_wrap_arm_elinux.so.3
CODEC: OGGVORBISD_ARM_02.04.00_ARMV8  build on Mar 11 2019 19:56:02.
------------------------
    Track 02 [video_0] Enabled
        Duration: 0:00:47.791668000
        Language: eng
    Mime:
        video/x-vp8, width=(int)1024, height=(int)436, framerate=(fraction)30/1
------------------------

====== V4L2DEC: 1.20.0 build on Jul 28 2022 06:57:16. ======
0:00:06.719066809   868 0xffff54298760 WARN                    v4l2 gstv4l2object.c:4638:gst_v4l2_object_probe_caps:<v4l2vp8dec0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
------------------------
    Track 03 [audio_2] Enabled
        Duration: 0:00:47.791668000
        Language: spa
    Mime:
        audio/x-vorbis, channels=(int)2, rate=(int)44100, bitrate=(int)0, framed=(boolean)true
------------------------

====== BEEP: 4.7.1 build on Jul 19 2022 06:15:03. ======
        Core: OggVorbis decoder Wrapper  build on Dec  7 2017 18:15:03
 file: /usr/lib/imx-mm/audio-codec/wrap/lib_vorbisd_wrap_arm_elinux.so.3
CODEC: OGGVORBISD_ARM_02.04.00_ARMV8  build on Mar 11 2019 19:56:02.
0:00:07.069225601   868 0xffff600029e0 WARN               aiurdemux aiurdemux.c:752:gst_aiurdemux_handle_sink_event: need to drop sink event GST_EVENT_CUSTOM_DOWNSTREAM_STICKY
0:00:07.070779210   868 0xffff54298760 WARN               aiurdemux aiurdemux.c:3411:aiurdemux_send_stream_newsegment: Pad audio: Send newseg 0:00:00.000000000 first buffer 0:00:00.000000000
0:00:07.071953948   868 0xffff54298760 WARN               aiurdemux aiurdemux.c:3411:aiurdemux_send_stream_newsegment: Pad audio: Send newseg 0:00:00.000000000 first buffer 0:00:00.000000000
0:00:07.073762929   868 0xffff54298760 WARN               aiurdemux aiurdemux.c:3411:aiurdemux_send_stream_newsegment: Pad video: Send newseg 0:00:00.000000000 first buffer 0:00:00.000000000
0:00:07.079756118   868 0xffff54298760 WARN               aiurdemux aiurdemux.c:3411:aiurdemux_send_stream_newsegment: Pad audio: Send newseg 0:00:00.000000000 first buffer 0:00:00.000000000
0:00:07.080847982   868 0xffff48057700 WARN                    v4l2 gstv4l2object.c:2352:gst_v4l2_object_get_streamparm:<v4l2vp8dec0:src> VIDIOC_G_PARM failed
0:00:07.082493465   868 0xffff48057700 WARN                    v4l2 gstv4l2object.c:4638:gst_v4l2_object_probe_caps:<v4l2vp8dec0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
0:00:07.084351571   868 0xffff48057700 WARN                    v4l2 gstv4l2object.c:3382:gst_v4l2_object_save_format:<v4l2vp8dec0:src> Driver bug detected, stride (512) is too small for the width (1024)
0:00:07.093393103   868 0xffff48057700 FIXME                playbin gstplaybin2.c:3308:_uridecodebin_event_probe:<uridecodebin0:src_0> Consider implementing group-id handling on stream-start event
0:00:07.093672601   868 0xffff48057c60 FIXME                playbin gstplaybin2.c:3308:_uridecodebin_event_probe:<uridecodebin0:src_3> Consider implementing group-id handling on stream-start event
0:00:07.094126221   868 0xffff480576a0 FIXME                playbin gstplaybin2.c:3308:_uridecodebin_event_probe:<uridecodebin0:src_2> Consider implementing group-id handling on stream-start event
0:00:07.094386343   868 0xffff60019640 FIXME                playbin gstplaybin2.c:3308:_uridecodebin_event_probe:<uridecodebin0:src_1> Consider implementing group-id handling on stream-start event
0:00:07.125438650   868 0xffff5001b400 WARN         audio-resampler audio-resampler.c:274:convert_taps_gint16_c: can't find exact taps
0:00:07.126845636   868 0xffff5001b400 WARN                    alsa pcm_hw.c:1360:snd_pcm_hw_get_chmap: alsalib error: Cannot read Channel Map ctl
: No such file or directory
0:00:07.130274101   868 0xffff54298760 WARN               aiurdemux aiurdemux.c:3549:aiurdemux_push_pad_buffer: Pad audio push error type -1
(...)
0:00:07.156427583   868 0xffff54298760 WARN               aiurdemux aiurdemux.c:3549:aiurdemux_push_pad_buffer: Pad audio push error type -1
0:00:07.156507082   868 0xffff48057700 WARN               videometa gstvideometa.c:414:gst_video_meta_validate_alignment: Stride of plane 0 defined in meta (1024) is different from the one computed from the alignment (640)
0:00:07.156604956   868 0xffff54298760 WARN               aiurdemux aiurdemux.c:3549:aiurdemux_push_pad_buffer: Pad audio push error type -1
0:00:07.156630956   868 0xffff48057700 WARN               videometa gstvideometa.c:414:gst_video_meta_validate_alignment: Stride of plane 0 defined in meta (1024) is different from the one computed from the alignment (640)
0:00:07.156701455   868 0xffff48057700 WARN               videometa gstvideometa.c:414:gst_video_meta_validate_alignment: Stride of plane 0 defined in meta (1024) is different from the one computed from the alignment (640)
0:00:07.156774830   868 0xffff48057700 WARN               videometa gstvideometa.c:414:gst_video_meta_validate_alignment: Stride of plane 0 defined in meta (1024) is different from the one computed from the alignment (640)
0:00:07.156879954   868 0xffff48057700 WARN               videometa gstvideometa.c:414:gst_video_meta_validate_alignment: Stride of plane 0 defined in meta (1024) is different from the one computed from the alignment (640)
0:00:07.158322064   868 0xffff54298760 WARN               aiurdemux aiurdemux.c:3549:aiurdemux_push_pad_buffer: Pad audio push error type -1
(...)
0:00:07.165199243   868 0xffff54298760 WARN               aiurdemux aiurdemux.c:3549:aiurdemux_push_pad_buffer: Pad audio push error type -1
0:00:07.168740582   868 0xffff30010ea0 WARN            v4l2videodec gstv4l2videodec.c:785:gst_v4l2_video_dec_loop:<v4l2vp8dec0> Received resolution change
0:00:07.169758322   868 0xffff30010ea0 WARN                    v4l2 gstv4l2object.c:2352:gst_v4l2_object_get_streamparm:<v4l2vp8dec0:src> VIDIOC_G_PARM failed
0:00:07.170134818   868 0xffff30010ea0 WARN                    v4l2 gstv4l2object.c:4638:gst_v4l2_object_probe_caps:<v4l2vp8dec0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
0:00:07.170767437   868 0xffff54298760 WARN               aiurdemux aiurdemux.c:3549:aiurdemux_push_pad_buffer: Pad audio push error type -1
(...)
0:00:09.395853468   868 0xffff54298760 WARN               aiurdemux aiurdemux.c:3549:aiurdemux_push_pad_buffer: Pad audio push error type -1

flutter应用程序在这里：https://github.com/mmerah/flutter-elinux-plugins-imx8/blob/main/packages/video_player/example/lib/main.dart GST_DEBUG = 4时，单击播放后的最后几个日志显示如下：

0:00:23.661699891   912 0xaaab07b22000 INFO              GST_STATES gstbin.c:2928:gst_bin_change_state_func:<pipeline> child 'playbin' changed state to 4(PLAYING) successfully
0:00:23.661749015   912 0xaaab07b22000 INFO              GST_STATES gstelement.c:2806:gst_element_continue_state:<pipeline> completed state change to PLAYING
0:00:23.661793639   912 0xaaab07b22000 INFO              GST_STATES gstelement.c:2706:_priv_gst_element_state_changed:<pipeline> notifying about state-changed PAUSED to PLAYING (VOID_PENDING pending)
0:00:23.664287226   912 0xffff5c298b60 WARN               aiurdemux aiurdemux.c:3549:aiurdemux_push_pad_buffer: Pad audio push error type -1
(...)
0:00:23.668285040   912 0xffff5c298b60 WARN               aiurdemux aiurdemux.c:3549:aiurdemux_push_pad_buffer: Pad audio push error type -1
0:00:23.669715393   912 0xaaab07b22000 INFO               GST_EVENT gstevent.c:1363:gst_event_new_seek: creating seek rate 1.000000, format TIME, flags 1, start_type 1, start 0:00:00.000000000, stop_type 1, stop 99:99:99.999999999
0:00:36.603878098   912 0xaaab07db2000 INFO                 basesrc gstbasesrc.c:2913:gst_base_src_loop:<source> pausing after gst_base_src_get_range() = eos
0:00:36.604063346   912 0xaaab07db2000 INFO                    task gsttask.c:368:gst_task_func:<source:src> Task going to paused
0:00:36.604394091   912 0xffff680029e0 INFO                    task gsttask.c:368:gst_task_func:<queue2-0:src> Task going to paused

完整日志的pastebin：https://pastebin.com/wzsbeC0e
这是怎么回事？
预先感谢你的帮助。

flutter

1条答案

按热度按时间
pkwftd7m

pkwftd7m1#

有几件事是错的：
1.当SYNC=TRUE时，0:00:06.614267007 868 0xffff54298760 WARN pulse pulsesink.c:617:gst_pulseringbuffer_open_device:<pulsesink0> error: Failed to connect: Connection refused阻塞了流水线。使用pulseaudio --start可以解决此问题。

  1. flutter-client嵌入器使用这个repo https://github.com/sony/flutter-embedded-linux来构建一个视频播放器插件。我试图将插件代码添加到flutter应用程序中，并认为它会捆绑在一起，但这是错误的。flutter应用程序没有使用插件，使用它的是flutter-client，这没有任何影响，但是当我认为我在插件中设置了SYNC=FALSE时，我实际上没有（同样是因为flutter-client拥有它）。
赞(0）分享  回复(0）举报  2022-12-24
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com