srs RTMP推流RTC拉流时，黑屏只有声音，视频流全丢包了

wmomyfyw 于 2022-10-27 发布在其他

关注(0)|答案(9)|浏览(951)

1. Description(描述)

在使用原始的rtmp2rtc.conf配置文件启动srs服务时，出现以下现象：

使用obs进行rtmp时，rtmp拉流、rtc拉流均可以正常播放；
第一次使用RTC推流时，rtmp拉流、rtc拉流也可以正常播放；
但是，一旦使用了RTC推流后，关闭RTC推流网页的话，再次使用RTMP推流，RTC拉流就不能正常播放了，只有声音可以正常播放，视频只是黑屏。通过chrome://webrtc-internals查看视频流数据时，发现视频流数据还是可以正常接收，但全都Drop掉了，解码帧数始终为0。
上述场景下，在RTC拉流时出现上述异常，但同时用RTMP拉流的话，可以正常播放；另外，如果重新换回RTC推流的话，RTMP拉流、RTC拉流也都可以正常播放。

采用默认的srt2rtc.conf配置文件时，也会出现上述问题。

2. SRS Version(版本): `4.0release 5.0.28`

3. SRS Log(日志):

[2022-05-14 20:59:15.164][Trace][221869][q8567159] TCP: clear zombies=1 resources, conns=3, removing=0, unsubs=0
[2022-05-14 20:59:15.164][Trace][221869][7173158y] TCP: disposing #0 resource(HttpConn)(0x557db80fab20), conns=3, disposing=1, zombies=0
[2022-05-14 20:59:15.420][Trace][221869][7173158y] RTC: session address init RTC拉流端公网IP地址:49305
[2022-05-14 20:59:15.420][Trace][221869][7173158y] RTC: session STUN done, waiting DTLS handshake.
[2022-05-14 20:59:15.421][Trace][221869][42u1778l] <- RTC RECV #12, udp 937, pps 8/14, schedule 937
[2022-05-14 20:59:15.434][Trace][221869][7173158y] DTLS: State Passive RECV, done=0, arq=0/0, r0=1, r1=0, len=157, cnt=22, size=144, hs=1
[2022-05-14 20:59:15.435][Trace][221869][7173158y] DTLS: State Passive SEND, done=0, arq=0/0, r0=-1, r1=2, len=681, cnt=22, size=82, hs=2
[2022-05-14 20:59:15.450][Trace][221869][7173158y] DTLS: State Passive RECV, done=0, arq=0/0, r0=1, r1=0, len=580, cnt=22, size=301, hs=11
[2022-05-14 20:59:15.451][Trace][221869][7173158y] DTLS: State Passive SEND, done=1, arq=0/0, r0=1, r1=0, len=554, cnt=22, size=466, hs=4
[2022-05-14 20:59:15.451][Trace][221869][7173158y] RTC: DTLS handshake done.
[2022-05-14 20:59:15.451][Trace][221869][7173158y] RTC: session pub=0, sub=1, to=30000ms connection established
[2022-05-14 20:59:15.451][Trace][221869][7173158y] RTC: Subscriber url=/live/rtcbugstream established
[2022-05-14 20:59:15.451][Trace][221869][7173158y] create consumer, no gop cache
[2022-05-14 20:59:15.451][Trace][221869][7173158y] RTC: start play url=/live/rtcbugstream, source_id=0q79mo9j/b6gv2366, realtime=1, mw_msgs=0
[2022-05-14 20:59:15.518][Trace][221869][7173158y] RTC: NACK ARQ seq=3924, ssrc=2241098775, ts=1469880, count=1/1, 1214 bytes
[2022-05-14 20:59:17.207][Trace][221869][k557z4lp] Hybrid cpu=16.98%,28MB, cid=1,0, timer=61,0,0, clock=0,37,9,1,1,0,0,0,0, objs=(pkt:494,raw:130,fua:363,msg:1031,oth:1,buf:100)
[2022-05-14 20:59:17.207][Trace][221869][k557z4lp] RTC: Server conns=1, rpkts=(1,rtp:0,stun:1,rtcp:1), spkts=(23,rtp:22,stun:1,rtcp:2), rtcp=(pli:1,twcc:2,rr:1), snk=(24,a:12,v:12,h:0), rnk=(1,1,h:9,m:0), fid=(id:1,fid:1,ffid:0,addr:1,faddr:1)
[2022-05-14 20:59:17.709][Trace][221869][6939h41e] <- CPB time=105013635, okbps=0,0,0, ikbps=2706,2703,0, mr=0/350, p1stpt=20000, pnt=5000
[2022-05-14 20:59:19.225][Trace][221869][6939h41e] -> HLS time=111532644ms, sno=7, ts=livestream-6.ts, dur=12266ms, dva=0p
[2022-05-14 20:59:22.208][Trace][221869][k557z4lp] Hybrid cpu=13.99%,28MB, cid=1,0, timer=61,0,0, clock=0,37,9,1,1,0,0,0,0, objs=(pkt:494,raw:130,fua:363,msg:1031,oth:1,buf:100)
[2022-05-14 20:59:22.208][Trace][221869][k557z4lp] RTC: Server conns=1, rpkts=(1,rtp:0,stun:1,rtcp:1), spkts=(23,rtp:22,stun:1,rtcp:2), rtcp=(pli:1,twcc:2,rr:1), snk=(24,a:12,v:12,h:0), rnk=(1,1,h:9,m:0), fid=(id:1,fid:1,ffid:0,addr:1,faddr:1)
[2022-05-14 20:59:22.709][Trace][221869][6939h41e] <- CPB time=110019424, okbps=0,0,0, ikbps=2707,2703,0, mr=0/350, p1stpt=20000, pnt=5000
[2022-05-14 20:59:25.439][Trace][221869][42u1778l] <- RTC RECV #12, udp 351, pps 10/35, schedule 351
[2022-05-14 20:59:25.591][Trace][221869][7173158y] RTC: NACK ARQ seq=6379, ssrc=2241098775, ts=2375910, count=1947/1947, 1214 bytes
[2022-05-14 20:59:27.208][Trace][221869][k557z4lp] Hybrid cpu=14.99%,28MB, cid=3,28, timer=61,0,38, clock=0,39,7,1,1,0,0,0,0, free=1, objs=(pkt:711,raw:192,fua:518,msg:1248,oth:1,buf:99)
[2022-05-14 20:59:27.208][Trace][221869][k557z4lp] RTC: Server conns=1, rpkts=(1,rtp:0,stun:1,rtcp:1), spkts=(23,rtp:22,stun:1,rtcp:2), rtcp=(pli:1,twcc:2,rr:1), snk=(24,a:12,v:12,h:0), rnk=(1,1,h:9,m:0), fid=(id:1,fid:1,ffid:0,addr:1,faddr:1
[2022-05-14 20:59:29.250][Trace][221869][6939h41e] -> HLS time=121556624ms, sno=8, ts=livestream-7.ts, dur=5600ms, dva=0p
[2022-05-14 20:59:32.209][Trace][221869][k557z4lp] Hybrid cpu=15.98%,28MB, cid=3,28, timer=61,0,38, clock=0,39,7,1,1,0,0,0,0, free=1, objs=(pkt:711,raw:192,fua:518,msg:1248,oth:1,buf:99)
[2022-05-14 20:59:32.209][Trace][221869][k557z4lp] RTC: Server conns=1, rpkts=(34,rtp:0,stun:1,rtcp:34), spkts=(489,rtp:489,stun:1,rtcp:0), rtcp=(pli:21,twcc:0,rr:0), rnk=(8,8,h:197,m:0), fid=(id:0,fid:34,ffid:0,addr:1,faddr:34)
[2022-05-14 20:59:32.710][Trace][221869][6939h41e] <- CPB time=120016150, okbps=0,0,0, ikbps=2706,2692,0, mr=0/350, p1stpt=20000, pnt=5000
[2022-05-14 20:59:35.439][Trace][221869][42u1778l] <- RTC RECV #12, udp 346, pps 12/34, schedule 346
[2022-05-14 20:59:35.702][Trace][221869][7173158y] RTC: NACK ARQ seq=8885, ssrc=2241098775, ts=3245940, count=4118/4118, 1214 bytes
[2022-05-14 20:59:37.211][Trace][221869][k557z4lp] Hybrid cpu=13.99%,28MB, cid=1,34, timer=61,0,48, clock=0,39,8,1,0,0,0,0,0, objs=(pkt:813,raw:206,fua:606,msg:1344,oth:1,buf:98)
[2022-05-14 20:59:37.211][Trace][221869][k557z4lp] RTC: Server conns=1, rpkts=(34,rtp:0,stun:1,rtcp:34), spkts=(489,rtp:489,stun:1,rtcp:0), rtcp=(pli:21,twcc:0,rr:0), rnk=(8,8,h:197,m:0), fid=(id:0,fid:34,ffid:0,addr:1,faddr:34)
[2022-05-14 20:59:39.273][Trace][221869][6939h41e] -> HLS time=131573119ms, sno=8, ts=livestream-7.ts, dur=15634ms, dva=0p
[2022-05-14 20:59:42.211][Trace][221869][k557z4lp] Hybrid cpu=16.00%,28MB, cid=1,34, timer=61,0,48, clock=0,39,8,1,0,0,0,0,0, objs=(pkt:813,raw:206,fua:606,msg:1344,oth:1,buf:98)
[2022-05-14 20:59:42.212][Trace][221869][k557z4lp] RTC: Server conns=1, rpkts=(34,rtp:0,stun:1,rtcp:33), spkts=(515,rtp:515,stun:1,rtcp:0), rtcp=(pli:20,twcc:0,rr:0), rnk=(8,8,h:211,m:0), fid=(id:0,fid:34,ffid:0,addr:1,faddr:34)
[2022-05-14 20:59:43.081][Trace][221869][0q79mo9j] <- CPB time=39986538, okbps=0,1,0, ikbps=2566,2520,0, mr=0/350, p1stpt=20000, pnt=5000
[2022-05-14 20:59:45.440][Trace][221869][42u1778l] <- RTC RECV #12, udp 343, pps 14/34, schedule 343
[2022-05-14 20:59:45.736][Trace][221869][7173158y] RTC: NACK ARQ seq=11446, ssrc=2241098775, ts=4133880, count=6221/6221, 1214 bytes

4. SRS Config(配置):

listen              1935;
max_connections     1000;
daemon              off;
srs_log_tank        console;

http_server {
    enabled         on;
    listen          8080;
    dir             ./objs/nginx/html;
}

http_api {
    enabled         on;
    listen          1985;
}
stats {
    network         0;
}
rtc_server {
    enabled on;
    listen 8000; # UDP port
    # @see https://github.com/ossrs/srs/wiki/v4_CN_WebRTC#config-candidate
    candidate $CANDIDATE;
}

vhost __defaultVhost__ {
    rtc {
        enabled     on;
        # @see https://github.com/ossrs/srs/wiki/v4_CN_WebRTC#rtmp-to-rtc
        rtmp_to_rtc on;
        # @see https://github.com/ossrs/srs/wiki/v4_CN_WebRTC#rtc-to-rtmp
        rtc_to_rtmp on;
    }
    http_remux {
        enabled     on;
        mount       [vhost]/[app]/[stream].flv;
    }
}

5. Replay(重现)

使用 rtmp2rtc.conf 启动 srs 服务;
使用RTC推流，并使用RTC拉流、RTMP拉流验证可以正常播放；
关闭RTC推流网页；
使用OBS或FFMPEG，采用RTMP协议推流；
再次使用RTC拉流播放，发现只有声音，视频黑屏；
可以再次使用RTMP拉流播放，验证此时可以正常播放。

6. Expect(期望行为)

RTC推流与RTMP推流可以正常根据需要切换。

srs

来源：https://github.com/ossrs/srs/issues/3041

9条答案

按热度按时间

piv4azn71#

确认用FFmpeg推 doc/source.flv ，就可以重现？如果不能重现，会直接关闭这个问题。

赞(0）回复(0）举报 2022-10-27

vybvopom2#

目前，在本地（Ubuntu）及阿里云上分别部署了两套srs服务，在本地电脑上，使用ffmpeg推流（没有用源码中的doc/source.flv，用的自己的其他的flv文件）。
今天按照上述复现步骤操作，不管是本地环境下的srs，还是阿里云环境下的srs，均仍能复现上述问题。
更简化的步骤为：

使用RTC推流；
使用RTC拉流，可以正常播放；
关闭RTC推流网页；
使用ffmpeg推流；
使用RTC拉流，此时只有声音，没有画面（今天测试时，发现webrtc-internals页面中，没有接收到任何视频帧数据，之前测的有时会看到虽然有接收到视频帧数据，但解码帧为0，全都drop掉了）

赞(0）回复(0）举报 2022-10-27

yqlxgs2m3#

这个问题解决了没有，是不是rtc推流跟rtmp推流的streamName相同？我这边现象跟你这个很像。webrtc-internals页面中看到video是不是选的VP9？

赞(0）回复(0）举报 2022-10-27

l3zydbqr4#

这个问题解决了没有，是不是rtc推流跟rtmp推流的streamName相同？我这边现象跟你这个很像。webrtc-internals页面中看到video是不是选的VP9？

问题目前还有，尝试从srs日志中找异常原因，未果。就是RTC和RTMP推流的streamName相同，才会出现这种现象；当RTC和RTMP推不同的streamName的时候，没有异常，各个streamName推流和拉流播放都很正常。webrtc-internals中，正常RTC推流拉流时，协商的协议一直都是H264；异常情况下，RTC通过H264推流，拉流没有解析出协议，好像是没有协商好？但是拉流中的answer中，显示的协议是H264。

赞(0）回复(0）举报 2022-10-27

8wtpewkr5#

srs_error_t SrsRtmpConn::acquire_publish(SrsLiveSource* source)
{
srs_error_t err = srs_success;

SrsRequest* req = info->req;

// Check whether RTMP stream is busy.
if (!source->can_publish(info->edge)) {
    return srs_error_new(ERROR_SYSTEM_STREAM_BUSY, "rtmp: stream %s is busy", req->get_stream_url().c_str());
}

// Check whether RTC stream is busy.

ifdef SRS_RTC

SrsRtcSource *rtc = NULL;
bool rtc_server_enabled = _srs_config->get_rtc_server_enabled();
bool rtc_enabled = _srs_config->get_rtc_enabled(req->vhost);
if (rtc_server_enabled && rtc_enabled && !info->edge) {
if ((err = _srs_rtc_sources->fetch_or_create(req, &rtc)) != srs_success) {
return srs_error_wrap(err, "create source");
}

if (!rtc->can_publish()) {
        return srs_error_new(ERROR_SYSTEM_STREAM_BUSY, "rtc stream %s busy", req->get_stream_url().c_str());
    }
}

endif

// Bridge to RTC streaming.

if defined(SRS_RTC) && defined(SRS_FFMPEG_FIT)

if (rtc) {


**rtc->update_stream_desc();**

    SrsRtcFromRtmpBridger *bridger = new SrsRtcFromRtmpBridger(rtc);
    if ((err = bridger->initialize(req)) != srs_success) {
        srs_freep(bridger);
        return srs_error_wrap(err, "bridger init");
    }

    source->set_bridger(bridger);
}

endif

// Start publisher now.
if (info->edge) {
    return source->on_edge_start_publish();
} else {
    return source->on_publish();
}

}

增加rtc->update_stream_desc()，这个函数里面把rtc推流时申请的stream_desc_释放掉，重新给一个默认的stream_desc_,RTMP推流的时候，RTC播放就没有问题了。
void SrsRtcSource::update_stream_desc()
{
if (stream_desc_)
{
srs_freep(stream_desc_);
}

SrsRtcSourceDescription* stream_desc = new SrsRtcSourceDescription();
SrsAutoFree(SrsRtcSourceDescription, stream_desc);

// audio track description
if (true) {
    SrsRtcTrackDescription* audio_track_desc = new SrsRtcTrackDescription();
    stream_desc->audio_track_desc_ = audio_track_desc;

    audio_track_desc->type_ = "audio";
    audio_track_desc->id_ = "audio-" + srs_random_str(8);

    uint32_t audio_ssrc = SrsRtcSSRCGenerator::instance()->generate_ssrc();
    audio_track_desc->ssrc_ = audio_ssrc;
    audio_track_desc->direction_ = "recvonly";

    audio_track_desc->media_ = new SrsAudioPayload(kAudioPayloadType, "opus", kAudioSamplerate, kAudioChannel);
}

// video track description
if (true) {
    SrsRtcTrackDescription* video_track_desc = new SrsRtcTrackDescription();
    stream_desc->video_track_descs_.push_back(video_track_desc);

    video_track_desc->type_ = "video";
    video_track_desc->id_ = "video-" + srs_random_str(8);

    uint32_t video_ssrc = SrsRtcSSRCGenerator::instance()->generate_ssrc();
    video_track_desc->ssrc_ = video_ssrc;
    video_track_desc->direction_ = "recvonly";

    SrsVideoPayload* video_payload = new SrsVideoPayload(kVideoPayloadType, "H264", kVideoSamplerate);
    video_track_desc->media_ = video_payload;

    video_payload->set_h264_param_desc("level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f");
}

set_stream_desc(stream_desc);

}

赞(0）回复(0）举报 2022-10-27

ztigrdn86#

见 #3079

当 rtc 推流后，rtc source对应的video payload type会因sdp协商结果发生变化；

此后如果发生rtmp to rtc或者srt to rtc时，bridge构造rtp packet的video payload type为默认值 102，这与此前存在不匹配的情况，从而导致webrtc拉流端不能正确获取video track

修正办法：把bridge构造rtp packet的video payload type的值设置为rtc source对应的video payload type

赞(0）回复(0）举报 2022-10-27

ekqde3dh7#

使用Docker测试，服务器版本SRS/4.0.253，存在相同RTMP Over WebRTC只有声音没有画面的问题

赞(0）回复(0）举报 2022-10-27

bq3bfh9z8#