[Research][Report]淺讀NVIDIA Video Codec SDK

0. 前言:

1. 簡介 NVIDIA Video Codec SDK

2. SDK支援度以及Setup

https://developer.nvidia.com/cuda-gpus#compute
https://docs.nvidia.com/deploy/cuda-compatibility/index.html
https://docs.nvidia.com/deploy/cuda-compatibility/index.html
CPU: i9–9900k
GPU: RTX2060 (Turning架構)
OS: Windows 10
  1. 安裝對應的顯卡驅動: https://www.nvidia.com.tw/Download/index.aspx
  2. 安裝對應CUDA Toolkit(連結為CUDA 10.2): https://developer.nvidia.com/cuda-10.2-download-archive
  3. 註冊Nvidia developer帳號並且下載NVIDIA Video Codec SDK: https://developer.nvidia.com/nvidia-video-codec-sdk/download

3. NVDECODE — Decoder

Video_Codec_SDK_10.0.26
1. Create a CUDA context.  
2. Query the decode capabilities of the hardware decoder.
3. Create the decoder instance(s).
4. De-Mux the content (like .mp4). This can be done using third party software like FFMPEG.
5. Parse the video bitstream using parser provided by NVDECODE API or third-party parser such as FFmpeg.
6. Kick off the Decoding using NVDECODE API.
7. Obtain the decoded YUV for further processing.
8. Query the status of the decoded frame.
9. Depending on the decoding status, use the decoded output for further processing like rendering, inferencing, postprocessing etc. 10. If the application needs to display the output,
‣ Convert decoded YUV surface to RGBA.
‣ Map RGBA surface to DirectX or OpenGL texture.
‣ Draw texture to screen.
11. Destroy the decoder instance(s) after the completion of decoding process.
12. Destroy the CUDA context.

3. NVDECODE — API & Sample Code

AppDec.cpp
int main(int argc, char **argv) 
{
// line 217
DecodeMediaFile(cuContext, szInFilePath, szOutFilePath, bOutPlanar, cropRect, resizeDim);
}
void DecodeMediaFile(CUcontext cuContext, const char *szInFilePath, const char *szOutFilePath, bool bOutPlanar,
const Rect &cropRect, const Dim &resizeDim)
{
//line 64
FFmpegDemuxer demuxer(szInFilePath);
NvDecoder dec(cuContext, false, FFmpeg2NvCodecId(demuxer.GetVideoCodec()), false, false, &cropRect, &resizeDim);
//line 70
do {
demuxer.Demux(&pVideo, &nVideoBytes);
nFrameReturned = dec.Decode(pVideo, nVideoBytes);
// do some else
}while (nVideoBytes);
}
// line 84
class NvDecoder {
//line 206
private:
//line 226
int HandleVideoSequence(CUVIDEOFORMAT *pVideoFormat);
int HandlePictureDecode(CUVIDPICPARAMS *pPicParams);
int HandlePictureDisplay(CUVIDPARSERDISPINFO *pDispInfo);
int ReconfigureDecoder(CUVIDEOFORMAT *pVideoFormat);
}
HandleVideoSequence
獲取video format
HandlePictureDecode
獲取欲解碼的圖像
HandlePictureDisplay
解碼後的圖像處理
ReconfigureDecoder
重新決定decoder
// line 28 nvcuvid.h/********************************************************************************************************************/
//! \file nvcuvid.h
//! NVDECODE API provides video decoding interface to NVIDIA GPU devices.
//! \date 2015-2019
//! This file contains the interface constants, structure definitions and function prototypes.
/********************************************************************************************************************/
// line 28 cuviddec.h/*****************************************************************************************************/
//! \file cuviddec.h
//! NVDECODE API provides video decoding interface to NVIDIA GPU devices.
//! \date 2015-2019
//! This file contains constants, structure definitions and function prototypes used for decoding.
/*****************************************************************************************************/

4. FFmpeg with Video Codec SDK

https://developer.nvidia.com/ffmpeg
  • Hardware-accelerated encoding of H.264 and HEVC
  • Hardware-accelerated decoding of H.264, HEVC, VP9, VP8, MPEG2, and MPEG4
  • Granular control over encoding settings such as encoding preset, rate control and other video quality parameters
  • Create high-performance end-to-end hardware-accelerated video processing, 1:N encoding and 1:N transcoding pipeline using built-in filters in FFmpeg
  • Ability to add your own custom high-performance CUDA filters using the shared CUDA context implementation in FFmpeg
  • Windows/Linux support

5. 補充:關於視訊編解碼器(Video codec)

  1. H.264
  • 多參考影格的運動補償,對某些類型的場景序列,例如快速重複的閃光,反覆的剪下或者背景遮擋的情況,它能很顯著的降低編碼的位元速率。
  • 減少混疊(Aliasing)並得到更銳化的圖像。
  • 加權的運動預測,如淡入、淡出、淡出而後淡入等情況提供相當大的編碼增益。
  • 數據分割區(DP for Data partitioning),能夠將重要程度不同的語法元素分開打包傳輸,增強視訊碼流對抗信道誤碼/丟包的強韌性。
  • HD DVD以及Blu-ray Disc
  • 數位電視
  • 編碼樹單元(Coding Tree Block),可向下分割編碼單元(Coding Unit,CU)、預測單元(Prediction Unit,PU)及轉換單元(Transform Unit,TU)。
  • 影格內編碼(Intra Coding),將預測的方向性增加到33種,並加入平面預測。
  • 去區塊濾波器(Deblocking Filter),從H.264 的一個4×4採樣網格提升至使用一個8×8個採樣網格,並採用有利於平行處理(多執行緒)的濾波方法。
  • 取樣自適應偏移(Sample Adaptive Offset),簡稱SAO濾波器,設計來以提高圖像品質,並減少振盪效應。
  • 4K藍光光碟
  • 4K影片
  • 基於塊的變換編碼格式,相比HEVC更於簡潔。
  • 支援 HTML5
  • royalty-free
  • YouTube
  • 各大瀏覽器:Chromium、Safari、Firefox

6. Reference

--

--

--

HI I’m N2I. Now a SWE in Taiwan. Check out more about me in https://nzi2020.blogspot.com/ or contact via email: nayzi9999@gmail.com

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
N2I

N2I

HI I’m N2I. Now a SWE in Taiwan. Check out more about me in https://nzi2020.blogspot.com/ or contact via email: nayzi9999@gmail.com

More from Medium

Getting Started on Cloud Armor, Protect and Score Protection L

Game Engine Tutorial Series Part 1: Introduction

Memcached Auto Discovery

Installing Apache Nifi on Ubuntu!