GStreamer 基本介紹

. . . .
.

GStreamer 架構

gst-launch-1.0 filesrc location=sintel_trailer-480p.ogv ! \
oggdemux name=demux ! queue ! vorbisdec ! autoaudiosink \
demux. ! queue ! theoradec ! videoconvert ! autovideosink

w:1120px

Element

element 是一個對多媒體流進行處理的 object,也是一個具體的功能模塊,通過 pad 把 element 連接起來構成 pipeline。

element 分類:

  • source (只提供數據源)
  • sink (如播放設備)
  • transform,demuxer,muxer element 的輸入為 sink pad,輸出為 source pad

element 狀態:

  • NULL 默認狀態,該狀態將會回收所有被元件佔用的資源。
  • READY 準備狀態,該狀態會得到所需的全局資源,但數據流並未處理。
  • PAUSED 暫停狀態,元件已經對流開始處理,一旦狀態變為 PLAYING,可以重放數據流,與 PLAYING 狀態的區別是:時鐘是禁止運行的,主要對數據進行 preroll。
  • PLAYING 與 PAUSED 狀態一模一樣,但可以運行時鐘,對數據進行處理。

Pad

pad 相當於 element 的接口,各個 element 即通過 pad 連接進行傳輸數據,同時 pad 會通過 caps 限制特定的數據類型通過,只有當兩個 pad 的 caps 數據類型一致時才可以建立連接。

pad 分類:

  • source pad: 只能接收東西
  • sink pad: 只能發送東西
  • transform pad: 可以接收東西也可以發送東西

pad 時效性:

  • 永久型(pads):一直會存在
  • 隨機型(Dynamic or sometimes pads):某種特定的條件下才存在
  • 請求型(Request pads):應用程序明確發出請求時才出現

pad 屬性:

  • direction: 指定 pad 的方向,可以是 source, sink, transform
  • caps: 指定 pad 的數據類型
  • name: 指定 pad 的名稱
  • presence: 指定 pad 的存在狀態

Cap

cap是指該元素可以接收什麼樣的信息,可以把 cap 看成是電源插座上其可以接受什麼範圍電壓的規則。

gst-launch-1.0 videotestsrc ! "video/x-raw,width=1280,height=720" ! autovideosink

Bin 和 Pipeline

Bin 是一個容器,用於管理多個 element,改變 bin 的狀態時,bin 會自動去修改所包含的 element 的狀態,也會轉發所收到的訊息。如果沒有 bin,我們需要依次操作我們所使用的 element。通過 bin 降低了應用的複雜度。

Pipeline 繼承自 bin,為程式提供一個 bus 用於傳輸訊息,並且對所有子 element 進行同步。當將 pipeline 的狀態設定為 PLAYING 時,pipeline 會在一個/多個新的執行緒中通過 element 處理資料。

gst-launch-1.0 filesrc location=sintel_trailer-480p.ogv ! \
oggdemux name=demux ! queue ! vorbisdec ! autoaudiosink \
demux. ! queue ! theoradec ! videoconvert ! autovideosink

w:1150px

bg h:90%

GStreamer 基礎操作

安裝:

apt-get install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev \
libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-base gstreamer1.0-plugins-good \
gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav \
gstreamer1.0-doc gstreamer1.0-tools gstreamer1.0-x\
gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3\
gstreamer1.0-qt5 gstreamer1.0-pulseaudio

取得攝影機:

ls -ltrh /dev/video*
# OR
sudo apt-get install v4l-utils
v4l2-ctl --list-devices
# OR
gst-device-monitor-1.0

播放影片:

gst-launch-1.0 playbin uri=file:///home/user/gstreamer/test.ogx \
video-sink="ximagesink"
# OR
gst-launch-1.0 filesrc location=C:\\Users\\user\\Desktop\\trailer.mp4 ! \
decodebin ! autovideosink

顯示鏡頭影像:

gst-launch-1.0 autovideosrc device=/dev/video0 ! autovideosink
# OR
gst-launch-1.0 v4l2src device=/dev/video0 ! autovideosink

顯示手機影像:

gst-launch-1.0 playbin uri=rtsp://admin:[email protected]:8554/live

RTSP Server:

~/gst-rtsp-server/examples$ ./test-launch \
"videotestsrc ! x264enc ! rtph264pay name=pay0 pt=96"

udpsrc and udpsink:

# server
gst-launch-1.0 -v videotestsrc ! x264enc ! \
rtph264pay ! udpsink port=5000 host=192.168.30.133

# client
gst-launch-1.0 -v udpsrc port=5000 ! \
"application/x-rtp, media=(string)video, 
clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! \
rtph264depay ! h264parse ! tee name=t t. ! queue ! mp4mux ! \
filesink location=xyz.mp4 -e t. ! queue leaky=1 ! decodebin ! \
videoconvert ! autovideosink sync=false

UDP on OBS Studio:

# server
gst-launch-1.0 -v videotestsrc ! x264enc ! rtph264pay ! \
udpsink port=5001 host=127.0.0.1

# OBS studio (client)
udpsrc port=5001 ! application/x-rtp, encoding-name=H264, width=320 ! \
rtph264depay ! h264parse ! avdec_h264 ! video.

邊緣偵測:

gst-launch-1.0 autovideosrc device=/dev/video0 ! decodebin ! \
videoconvert ! edgetv ! videoconvert ! autovideosink

水波紋:

gst-launch-1.0 autovideosrc device=/dev/video0 ! decodebin ! \
videoconvert ! rippletv ! videoconvert ! autovideosink

Debug level:

export GST_DEBUG="*:0"
# OR
--gst-debug-level=0

GStreamer Debugging

GStreamer X UI

# Demo
~/gst-concept$ python3 main_custom.py

OpenCV faceblur

# Demo
gst-launch-1.0 autovideosrc ! videoconvert ! faceblur ! \
videoconvert ! autovideosink

https://thiblahute.github.io/GStreamer-doc/plugins.html?gi-language=c

YouTube-dl X GStreamer

# install YouTube-dl
sudo pip install --upgrade youtube_dl

# YouTube-dl example
youtube-dl --format mp4 --get-url https://www.youtube.com/watch?v=Ggf4fvmFL3M

# use GStreamer
gst-launch-1.0 souphttpsrc is-live=true \
location="$(youtube-dl --format "best[ext=mp4][protocol=https]" 
--get-url https://www.youtube.com/watch?v=Ggf4fvmFL3M)" ! \
decodebin decodebin ! videoconvert ! autovideosink

Building from source using Meson

# Gstreamer
~/gstreamer$ meson builddir
~/gstreamer$ ninja -C builddir
~/gstreamer$ ninja -C builddir devenv
# Template
~/$ git clone https://gitlab.freedesktop.org/gstreamer/gst-template.git
~/gst-template/gst-plugin/src$ ../tools/make_element myfilter gsttransform

GStreamer Element 簡介

  • playbin : 播放影片 video-sink : the video output element to use (NULL = default sink)

  • filesrc : 播放檔案 location : 檔案路徑

  • autovideosink : 顯示影片 sync : Sync on the clock (boolean)

  • autovideosrc : 載入影片 device : 攝影機路徑

  • v4l2src : v4l2src can be used to capture video from v4l2 devices, like webcams and tv cards. device : 攝影機路徑

  • x264enc : x264enc is a video encoder based on the x264 library.

  • rtph264pay : rtph264pay is a GStreamer element that encapsulates H264 video into RTP packets.

  • rtph264depay : rtph264depay is a GStreamer element that decapsulates RTP packets into H264 video.

  • h264parse : h264parse is a GStreamer element that parses H264 bitstreams.

  • tee : tee is a GStreamer element that splits a pipeline into multiple output pads.

  • queue : queue is a GStreamer element that buffers data. leaky: no (0) – Not Leaky    upstream (1) – Leaky on upstream (new buffers)    downstream (2) – Leaky on downstream (old buffers)

  • videoconvert : videoconvert is a GStreamer element that converts video formats.

  • udpsink : udpsink is a GStreamer element that sends data over the network. port : The port to send data to. (default: 5004) host : The host to send data to.

  • udpsrc : udpsrc is a GStreamer element that receives data over the network. port : The port to receive data from. (default: 5004)

遇到問題

1. 虛擬機鏡頭無法正常啟動

調成 USB3.1 即可

2. ERROR: Program 'flex win_flex' not found or not executable

sudo apt-get install flex bison

參考資料

GStreamer 系列 - 基本介紹

Gstreamer Plugin 編寫 之 入門

GStreamer 教學(3) 什麼條件下你會用到 Dynamic pad, Request pad 以及 Ghost pad?



Tags: None