FFmpeg SPS/PPS剖析

场景说明
            在解码过程中，需要设置SPS/PPS等解码信息，才能够初始化×××。有两种方式可以设置SPS/PPS，一种是手动指定SPS/PPS内容，指定AVCodecContext
结构体中extradata的值；一种是让FFmpeg通过读取传输数据来分析SPS/PPS信息，一般情况下在每一个I帧之前都会发送一个SPS帧和PPS帧


方法1）一般的摄像机都会在RTSP指令中SDP携带的SPS/PPS内容，例如海康IPC发送的RTSP指令中携带信息如下：sprop-parameter-sets=Z0IAKpY1QPAET8s3AQEBQAABwgAAV+QB,aM48gA==
分号前面是SPS,后面是PPS,这是base64编码的结果，需要解码，才能够传递给extradata,并且还需要在SPS/PPS字符串前面添加起始码（0x00 0x00 0x00 0x01）。补充一点SPS的类型是
0x67, PPS的类型是0x68

char szBase64SPSBuffer[] = { "Z0IAKpY1QPAET8s3AQEBQAABwgAAV+QB" };
uint8_t szDecodedSPS[128] = { 0 };
int nSPSLen = av_base64_decode(szDecodedSPS, szBase64SPSBuffer, sizeof(szBase64SPSBuffer));//nSPSLen=24

char szBase64PPSBuffer[] = { "aM48gA==" };
uint8_t szDecodedPPS[128] = { 0 };
int nPPSLen = av_base64_decode(szDecodedPPS, szBase64PPSBuffer, sizeof(szBase64PPSBuffer));//nPPSLen=4

将SPS/PPS拼凑的结果如下：
unsigned char szSPSPPS[] = { 0x00 ,0x00 ,0x01,0x67,0x42,0x00 ,0x2a ,0x96 ,0x35 ,0x40 ,0xf0 ,0x04 ,0x4f ,0xcb ,0x37 ,0x01 ,0x01 ,0x01 ,0x40 ,0x00 ,0x01 ,0xc2 ,0x00 ,0x00 ,0x57 ,0xe4 ,0x01 ,0x00 ,0x00 ,0x00 ,0x01 ,0x68 ,0xce ,0x3c ,0x80, 0x00};


然后拷贝SPS/PPS数据到AVFormatContext的extradata
unsigned char szSPSPPS[] = { 0x00 ,0x00 ,0x01,0x67,0x42,0x00 ,0x2a ,0x96 ,0x35 ,0x40 ,0xf0 ,0x04 ,0x4f ,0xcb ,0x37 ,0x01 ,0x01 ,0x01 ,0x40 ,0x00 ,0x01 ,0xc2 ,0x00 ,0x00 ,0x57 ,0xe4 ,0x01 ,0x00 ,0x00 ,0x00 ,0x01 ,0x68 ,0xce ,0x3c ,0x80, 0x00};
pFormatContext->streams[0]->codecpar->extradata_size = sizeof(szSPSPPS);
pFormatContext->streams[0]->codecpar->extradata = (uint8_t*)av_mallocz(pFormatContext->streams[0]->codecpar->extradata_size + AV_INPUT_BUFFER_PADDING_SIZE);
memcpy(pFormatContext->streams[0]->codecpar->extradata, szSPSPPS, sizeof(szSPSPPS));

通过avcodec_parameters_to_context将信息从pFormatContext->streams[0]->codecpar拷贝到m_pAVCodecContext
avcodec_open2函数在调用的时候，会解析extradata数据的内容


出错异常

I:2018-01-08 14:23:00 ms:221:nal_unit_type: 7, nal_ref_idc: 3
I:2018-01-08 14:23:00 ms:221:nal_unit_type: 8, nal_ref_idc: 3
I:2018-01-08 14:23:00 ms:221:sps:0 profile:66/42 poc:0 ref:1 120x68 FRM 8B8 crop:0/0/0/8 VUI 420 1800/90000 b8 reo:-1
I:2018-01-08 14:23:00 ms:221:pps:0 sps:0 CAVLC slice_groups:1 ref:1/1  qp:26/26/0/0 LPAR   
I:2018-01-08 14:23:00 ms:222:deprecated pixel format used, make sure you did set range correctly
I:2018-01-08 14:23:00 ms:250:non-existing PPS 0 referenced

错误分析
        从av_log日志可以看出已经正确的解析出来PPS,但是在进行avcodec_read_frame的时候 还是打印出错non-existing PPS 0 referenced,这是什么问题？
这主要是没有在缓冲区中读取到包含SPS/PPS信息的帧,而不是说×××没有正确初始化SPS/PPS参数，这一点需要注意

问题
SDP中sprop-parameter-sets=Z0IAKpY1QPAET8s3AQEBQAABwgAAV+QB,aM48gA==
SPS=Z0IAKpY1QPAET8s3AQEBQAABwgAAV+QB
PPS=aM48gA==
十六进制表示：
5a 30 49 41 4b 70 59 31 51 50 41 45 54 38 73 33 41 51 45 42 51 41 41 42 77 67 41 41 56 2b 51 42
码流中的SPS和PPS
unsigned char sps_pps[] = { 0x00 ,0x00 ,0x01,0x67,0x42,0x00 ,0x2a ,0x96 ,0x35 ,0x40 ,0xf0 ,0x04 ,0x4f ,0xcb ,0x37 ,0x01 ,0x01 ,0x01 ,0x40 ,0x00 ,0x01
,0xc2 ,0x00 ,0x00 ,0x57 ,0xe4 ,0x01 ,0x00 ,0x00 ,0x00 ,0x01 ,0x68 ,0xce ,0x3c ,0x80, 0x00};
两者完全不一致，暂时没有头绪

展望

        从这里就可以看出，实际上并没有多大的必要手动设置SPS/PPS，因为FFmpeg会从缓冲中分析出SPS/PPS，然后解析出码流参数，估计唯一的一种极端情况是I帧之前没有任何的SPS/PPS信息，或者需要很长时间才会发送一帧包含SPS/PPS的信息

相关代码定义
/**
 * This struct describes the properties of an encoded stream.
 
*
 * sizeof(AVCodecParameters) is not a part of the public ABI, this struct must
 
* be allocated with avcodec_parameters_alloc() and freed with
 
* avcodec_parameters_free().
 
*/
typedef struct AVCodecParameters {
/**
     * Extra binary data needed for initializing the decoder, codec-dependent.
     *
     
* Must be allocated with av_malloc() and will be freed by
     
* avcodec_parameters_free(). The allocated size of extradata must be at
     
* least extradata_size + AV_INPUT_BUFFER_PADDING_SIZE, with the padding
     * bytes zeroed.
     */

    uint8_t *extradata;
    
/**
     * Size of the extradata content in bytes.  
*/
    int      extradata_size;
}

描述说明
额外的二进制数据用来初始化×××，主要是初始化SPS/PPS，必须通过av_malloc进行内存分配，然后通过avcodec_parameters_free进行释放。内存分配的大小必须是extradata_size的大小加上AV_INPUT_BUFFER_PADDING_SIZE二进制数据的真实长度大小保存在extradata_size