morgan/UxPlay
1
0
Fork 0
mirror of https://github.com/morgan9e/UxPlay synced 2026-04-14 00:04:13 +09:00
Code Issues Packages Projects Releases Wiki Activity

add option for modifying h264parse in video pipeline

Browse Source
This commit is contained in:
fduncanh
2022-02-28 12:21:18 -05:00
parent 0e6e06def0
commit d6f3599d3e
7 changed files with 60 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
README.html
View File

@@ -18,7 +18,9 @@
<p>UxPlay uses <a href="https://gstreamer.freedesktop.org">GStreamer</a> Plugins for rendering audio and video, This means that video and audio are supported “out of the box”, using a choice of plugins. AirPlay streams video in h264 format: gstreamer decoding is plugin agnostic, and uses accelerated GPU hardware h264 decoders if available; if not, software decoding is used.</p>
<p>For systems with Intel integrated graphics, hardware GPU decoding with the gstreamer VAAPI plugin is preferable. VAAPI is open-source, and in addition to Intel, can support some AMD GPU’s (the open-source “Nouveau” drivers for NVIDIA graphics are also in principle supported when VAAPI is supplemented with firmware extracted from the proprietary NVIDIA drivers).</p>
<p>For NVIDIA graphics with the proprietary drivers, the nvdec plugin (recently renamed nvh264dec) can be used for accelerated video decoding on the NVIDIA GPU with CUDA. The nvdec plugin is part of gstreamer1.0-plugins-bad, but is generally not included in binary packages, as NVIDIA’s proprietary <a href="https://docs.nvidia.com/video-technologies/video-codec-sdk/nvdec-video-decoder-api-prog-guide/">Video Codec SDK</a> must be downloaded, and three header files from it must be added to the gstreamer source before the plugin can be compiled. Users must do this themselves: see <a href="https://gist.github.com/corenel/a615b6f7eb5b5425aa49343a7b409200">these instructions</a>, and adapt them as necessary for your GStreamer installation. This plugin should be used with the <code>-vd nvdec</code> (or nvh264dec) and <code>-vs glimagesink</code> uxplay options.</p>
<p>The Video4Linux2 decoder v4l2h264dec (from gstreamer1.0-plugins-good) is the appropriate choice for the Broadcom GPU in the Raspberry Pi 4, (UxPlay does not run well on the Raspberry Pi if GPU hardware h264 video decoding is not used, as its CPU is not powerful enough for satisfactory software-based decoding without large latency); uxplay options can be used to create GStreamer video pipelines to use non-VAAPI hardware decoders, and a special option <code>-rpi</code> creates a pipeline appropriate for the Raspberry Pi. <em>(This is “Work In Progess”: it does not yet consistently work.)</em></p>
<ul>
<li><strong>support for Raspberry Pi</strong> (<em>work in progess</em>): the R Pi platform has long been supported by <a href="http://github.com/FD-/RPiPlay">RPiPlay</a> using the deprecated 32-bit-only omx (OpenMAX) decoder, now removed from latest Raspberry Pi OS (Bullseye), leading to user interest in getting UxPlay to work on the R Pi (at least on model 4). UxPlay can work with software h264 video decoding (option <code>-avdec</code>), but without hardware decoding has unacceptable latency on the Pi. The Video4Linux2 GStreamer plugin v4l2h264dec (from gstreamer1.0-plugins-good) is now the appropriate choice for accelerated hardware video decoding by the Broadcom GPU on the Pi. UxPlay options can be used to create GStreamer video pipelines to use non-VAAPI hardware decoders, and a new UxPlay option <code>-rpi</code> creates a pipeline that should be appropriate for the Raspberry Pi, and when it has worked, it has worked well. However, it usually does not work because of a “caps” mismatch between source and sink, which could be fixable with GStreamer “know-how”. The GStreamer video pipeline is “<code>(decrypted h264 stream from iOS)! queue ! h264parse ! decodebin ! videoconvert ! autovideosink sync=false</code>” and the uxplay options <code>-vp , -vd, -vc ,-vs</code> can be used to replace <code>h264parse, decodebin, videoconvert, autovideosink</code> by the user’s choices. (<code>-rpi</code> is equivalent to ’-vd v4l2h264dec -vc v4l2convert`). The first frame in the h264 video stream sends the “caps” details (SPS, PPS) to h264parse. UxPlay users with the R Pi (model 4 recommended) are invited to experiment and report any successes.</li>
</ul>
<h3 id="note-to-packagers-openssl-3.0.0-solves-gpl-v3-license-issues.">Note to packagers: OpenSSL-3.0.0 solves GPL v3 license issues.</h3>
<p>Some Linux distributions such as Debian do not allow distribution of compiled GPL code linked to OpenSSL-1.1.1 because its “dual OpenSSL/SSLeay” license has some incompatibilities with GPL, unless all code authors have explicitly given an “exception” to allow such linking (the historical origins of UxPlay make this impossible to obtain). Other distributions treat OpenSSL as a “System Library” which the GPL allows linking to.</p>
<p>For “GPL-strict” distributions, UxPlay can be built using OpenSSL- 3.0.0, which has a new <a href="https://www.openssl.org/blog/blog/2021/09/07/OpenSSL3.Final/">GPLv3-compatible license</a>.</p>

21
README.md
View File

@@ -76,11 +76,22 @@ Users must do this themselves: see [these instructions](https://gist.github.com/
adapt them as necessary for your GStreamer installation.
This plugin should be used with the `-vd nvdec` (or nvh264dec) and `-vs glimagesink` uxplay options.
The Video4Linux2 decoder v4l2h264dec (from gstreamer1.0-plugins-good) is the appropriate choice for the Broadcom GPU in the
Raspberry Pi 4, (UxPlay does not run well on the Raspberry Pi if GPU hardware h264 video decoding is not used,
as its CPU is not powerful enough for satisfactory software-based decoding without large latency); uxplay options can be used
to create GStreamer video pipelines to use non-VAAPI hardware decoders, and a special option `-rpi` creates a pipeline
appropriate for the Raspberry Pi. _(This is "Work In Progess": it does not yet consistently work.)_
* **support for Raspberry Pi** (_work in progess_): the R Pi platform has long been
supported by [RPiPlay](http://github.com/FD-/RPiPlay) using the deprecated 32-bit-only omx (OpenMAX) decoder, now
removed from latest Raspberry Pi OS (Bullseye), leading to user interest in getting UxPlay to work
on the R Pi (at least on model 4). UxPlay can work with software h264 video decoding (option `-avdec`), but without hardware decoding
has unacceptable latency on the Pi. The Video4Linux2 GStreamer plugin v4l2h264dec (from gstreamer1.0-plugins-good) is now the
appropriate choice for accelerated
hardware video decoding by the Broadcom GPU on the Pi. UxPlay options can be used to create GStreamer video pipelines to use
non-VAAPI hardware decoders, and a new UxPlay option `-rpi` creates a pipeline that should be appropriate for the Raspberry Pi,
and when it has worked, it has worked well. However, it usually does not work because of a "caps" mismatch between source and
sink, which could be fixable with GStreamer "know-how". The GStreamer
video pipeline is "` (decrypted h264 stream from iOS)! queue ! h264parse ! decodebin ! videoconvert ! autovideosink sync=false`"
and the uxplay options `-vp , -vd, -vc ,-vs` can be used to
replace `h264parse, decodebin, videoconvert, autovideosink` by the user's
choices. (`-rpi` is equivalent to '-vd v4l2h264dec -vc v4l2convert`). The first frame in the h264 video stream sends the "caps"
details (SPS, PPS) to h264parse. UxPlay users with the R Pi (model 4 recommended) are invited to experiment and report any successes.
### Note to packagers: OpenSSL-3.0.0 solves GPL v3 license issues.

34
README.txt
View File

@@ -100,15 +100,31 @@ and adapt them as necessary for your GStreamer installation. This plugin
should be used with the `-vd nvdec` (or nvh264dec) and `-vs glimagesink`
uxplay options.
The Video4Linux2 decoder v4l2h264dec (from gstreamer1.0-plugins-good) is
the appropriate choice for the Broadcom GPU in the Raspberry Pi 4,
(UxPlay does not run well on the Raspberry Pi if GPU hardware h264 video
decoding is not used, as its CPU is not powerful enough for satisfactory
software-based decoding without large latency); uxplay options can be
used to create GStreamer video pipelines to use non-VAAPI hardware
decoders, and a special option `-rpi` creates a pipeline appropriate for
the Raspberry Pi. *(This is "Work In Progess": it does not yet
consistently work.)*
- **support for Raspberry Pi** (*work in progess*): the R Pi platform
has long been supported by [RPiPlay](http://github.com/FD-/RPiPlay)
using the deprecated 32-bit-only omx (OpenMAX) decoder, now removed
from latest Raspberry Pi OS (Bullseye), leading to user interest in
getting UxPlay to work on the R Pi (at least on model 4). UxPlay can
work with software h264 video decoding (option `-avdec`), but
without hardware decoding has unacceptable latency on the Pi. The
Video4Linux2 GStreamer plugin v4l2h264dec (from
gstreamer1.0-plugins-good) is now the appropriate choice for
accelerated hardware video decoding by the Broadcom GPU on the Pi.
UxPlay options can be used to create GStreamer video pipelines to
use non-VAAPI hardware decoders, and a new UxPlay option `-rpi`
creates a pipeline that should be appropriate for the Raspberry Pi,
and when it has worked, it has worked well. However, it usually does
not work because of a "caps" mismatch between source and sink, which
could be fixable with GStreamer "know-how". The GStreamer video
pipeline is
"`(decrypted h264 stream from iOS)! queue ! h264parse ! decodebin ! videoconvert ! autovideosink sync=false`"
and the uxplay options `-vp , -vd, -vc ,-vs` can be used to replace
`h264parse, decodebin, videoconvert, autovideosink` by the user's
choices. (`-rpi` is equivalent to '-vd v4l2h264dec -vc
v4l2convert\`). The first frame in the h264 video stream sends the
"caps" details (SPS, PPS) to h264parse. UxPlay users with the R Pi
(model 4 recommended) are invited to experiment and report any
successes.
### Note to packagers: OpenSSL-3.0.0 solves GPL v3 license issues.

4
renderers/video_renderer.h
View File

@@ -45,8 +45,8 @@ typedef enum videoflip_e {
typedef struct video_renderer_s video_renderer_t;
void video_renderer_init (logger_t *logger, const char *server_name, videoflip_t videoflip[2], const char *decoder,
const char *converter, const char *videosink);
void video_renderer_init (logger_t *logger, const char *server_name, videoflip_t videoflip[2], const char *parser,
const char *decoder, const char *converter, const char *videosink);
void video_renderer_start ();
void video_renderer_stop ();
void video_renderer_render_buffer (raop_ntp_t *ntp, unsigned char* data, int data_len, uint64_t pts, int type);

7
renderers/video_renderer_gstreamer.c
View File

@@ -105,8 +105,8 @@ void video_renderer_size(float *f_width_source, float *f_height_source, float *f
logger_log(logger, LOGGER_DEBUG, "begin video stream wxh = %dx%d; source %dx%d", width, height, width_source, height_source);
}
void video_renderer_init(logger_t *render_logger, const char *server_name, videoflip_t videoflip[2], const char *decoder,
const char *converter, const char *videosink) {
void video_renderer_init(logger_t *render_logger, const char *server_name, videoflip_t videoflip[2], const char *parser,
const char *decoder, const char *converter, const char *videosink) {
GError *error = NULL;
logger = render_logger;
@@ -123,7 +123,8 @@ void video_renderer_init(logger_t *render_logger, const char *server_name, vide
gst_init(NULL,NULL);
GString *launch = g_string_new("appsrc name=video_source stream-type=0 format=GST_FORMAT_TIME is-live=true ! ");
g_string_append(launch, "queue ! ");
g_string_append(launch, "h264parse ! ");
g_string_append(launch, parser);
g_string_append(launch, " ! ");
g_string_append(launch, decoder);
g_string_append(launch, " ! ");
g_string_append(launch, converter);

2
uxplay.1
View File

@@ -37,6 +37,8 @@ UxPlay 1.48: An open\-source AirPlay mirroring server based on RPiPlay
.TP
\fB\-t\fR n Relaunch server if no connection existed in last n seconds
.TP
\fB\-vp\fI prs \fR Choose GStreamer h264 parser; default "h264parse"
.TP
\fB\-vd\fI dec \fR Choose GStreamer h264 decoder; default "decodebin"
.IP
choices: (software) avdec_h264; (hardware) v4l2h264dec,

10
uxplay.cpp
View File

@@ -77,6 +77,7 @@ static std::string audiosink = "autoaudiosink";
static bool use_audio = true;
static bool new_window_closing_behavior = true;
static bool close_window;
static std::string video_parser = "h264parse";
static std::string video_decoder = "decodebin";
static std::string video_converter = "videoconvert";
static bool show_client_FPS_data = false;
@@ -226,6 +227,7 @@ static void print_info (char *name) {
printf(" \"-p tcp n\" or \"-p udp n\" sets TCP or UDP ports separately\n");
printf("-m Use random MAC address (use for concurrent UxPlay's)\n");
printf("-t n Relaunch server if no connection existed in last n seconds\n");
printf("-vp ... Choose the GSteamer h264 parser: default \"h264parse\"\n");
printf("-vd ... Choose the GStreamer h264 decoder; default \"decodebin\"\n");
printf(" choices: (software) avdec_h264; (hardware) v4l2h264dec,\n");
printf(" nvdec, nvh264dec, vaapih64dec, vtdec,etc.\n");
@@ -450,6 +452,10 @@ int main (int argc, char *argv[]) {
} else if (arg == "-h" || arg == "-v") {
print_info(argv[0]);
exit(0);
} else if (arg == "-vp") {
if (!option_has_value(i, argc, arg, argv[i+1])) exit(1);
video_parser.erase();
video_parser.append(argv[++i]);
} else if (arg == "-vd") {
if (!option_has_value(i, argc, arg, argv[i+1])) exit(1);
video_decoder.erase();
@@ -530,7 +536,7 @@ int main (int argc, char *argv[]) {
}
if (use_video) {
video_renderer_init(render_logger, server_name.c_str(), videoflip,
video_renderer_init(render_logger, server_name.c_str(), videoflip, video_parser.c_str(),
video_decoder.c_str(), video_converter.c_str(), videosink.c_str());
video_renderer_start();
}
@@ -569,7 +575,7 @@ int main (int argc, char *argv[]) {
if (use_audio) audio_renderer_stop();
if (use_video && close_window) {
video_renderer_destroy();
video_renderer_init(render_logger, server_name.c_str(), videoflip,
video_renderer_init(render_logger, server_name.c_str(), videoflip, video_parser.c_str(),
video_decoder.c_str(), video_converter.c_str(), videosink.c_str());
video_renderer_start();
}