morgan/UxPlay
1
0
Fork 0
mirror of https://github.com/morgan9e/UxPlay synced 2026-04-14 00:04:13 +09:00
Code Issues Packages Projects Releases Wiki Activity

support for display of cover art in AirPlay Audio-only (ALAC) mode

Browse Source
This commit is contained in:
fduncanh
2022-06-30 16:02:12 -04:00
parent e710492077
commit f756c77480
5 changed files with 458 additions and 315 deletions
Download Patch File Download Diff File Expand all files Collapse all files

197
README.md
View File

@@ -1,4 +1,4 @@
# UxPlay 1.53: AirPlay/AirPlay-Mirror server for Linux, macOS, and Unix.
# UxPlay 1.54: AirPlay-Mirror and AirPlay-Audio server for Linux, macOS, and Unix.
### Now developed at the GitHub site [https://github.com/FDH2/UxPlay](https://github.com/FDH2/UxPlay) (where all user issues should be posted).
@@ -6,9 +6,9 @@
Highlights:
* GPLv3, open source.
* Support for both AirPlay Mirror and AirPlay Audio-only (Apple Lossless
ALAC) streaming protocols
from current iOS/iPadOS 15.5 clients.
* Originally supported only AirPlay Mirror protocol, now has added support
for AirPlay Audio-only (Apple Lossless ALAC) streaming
from current iOS/iPadOS 15.5 clients. **There is no support for Airplay2 video-streaming protocol, and none is planned.**
* macOS computers (2011 or later, both Intel and "Apple Silicon" M1
systems) can act either as AirPlay clients, or
as the server running UxPlay. Using AirPlay, UxPlay can
@@ -21,8 +21,8 @@ Highlights:
"audiosinks", and a fully-user-configurable video streaming pipeline).
* Support for server behind a firewall.
* **New**: Support for Raspberry Pi, with hardware video acceleration using
Video4Linux2 (which supports both 32- and 64-bit systems, unlike deprecated
OpenMAX (omx), which it replaces). (For GStreamer < 1.22,
Video4Linux2 (vl42), which supports both 32- and 64-bit systems, unlike deprecated
OpenMAX (omx), which is being dropped by RPi distributions in favor of v4l2. (For GStreamer < 1.22,
a [patch](https://github.com/FDH2/UxPlay/wiki/Gstreamer-Video4Linux2-plugin-patches)
to the GStreamer Video4Linux2 plugin, available in the
[UxPlay Wiki](https://github.com/FDH2/UxPlay/wiki), is required, unless
@@ -33,7 +33,7 @@ Highlights:
This project is a GPLv3 open source unix AirPlay2 Mirror server for Linux, macOS, and \*BSD.
It was initially developed by
[antimof](http://github.com/antimof/Uxplay) using code
from [RPiPlay](https://github.com/FD-/RPiPlay), which in turn derives from
from OpenMAX-based [RPiPlay](https://github.com/FD-/RPiPlay), which in turn derives from
[AirplayServer](https://github.com/KqsMea8/AirplayServer),
[shairplay](https://github.com/juhovh/shairplay), and [playfair](https://github.com/EstebanKubata/playfair).
(The antimof site is no longer involved in
@@ -49,9 +49,10 @@ Its main use is to act like an AppleTV for screen-mirroring (with audio) of iOS/
(iPhones, iPads, MacBooks) in a window
on the server display (with the possibility of
sharing that window on screen-sharing applications such as Zoom)
on a host running Linux, macOS, or other unix. UxPlay supports Apple's AirPlay 2
on a host running Linux, macOS, or other unix. UxPlay supports Apple's AirPlay2
protocol using "Legacy Pairing", but some features are missing.
(Details of what is publically known about Apple's AirPlay 2 protocol can be found
[here](https://openairplay.github.io/airplay-spec/),
[here](https://github.com/SteeBono/airplayreceiver/wiki/AirPlay2-Protocol) and
[here](https://emanuelecozzi.net/docs/airplay2)).
@@ -64,21 +65,29 @@ through the avahi-daemon service, and is included in most Linux distributions (
service can also be provided by macOS, iOS or Windows servers).
Connections to the UxPlay server by
iOS/MacOS clients can be initiated both in AirPlay Mirror mode (which streams
iOS/MacOS clients can be initiated both in **AirPlay Mirror** mode (which streams
lossily-compressed AAC audio while mirroring the client screen,
or in the alternative AirPlay Audio mode which streams
Apple Lossless (ALAC) audio without screen mirroring (the accompanying cover art in
this mode is not displayed, but metadata is displayed in the terminal).
_Switching between these two modes during an active connection is
possible: in Mirror mode, close the mirror window and start an Audio mode connection,
switch back by initiating a Mirror mode connection._
or in the alternative **AirPlay Audio** mode which streams
Apple Lossless (ALAC) audio without screen mirroring. In **Audio** mode,
metadata is displayed in the uxplay terminal;
if UxPlay option ``-ca <name>`` is used,
the accompanying cover art is also output
to a periodically-updated file `<name>`, and can be viewed with
a (reloading) graphics viewer of your choice such as `feh`:
run "`uxplay -ca <name> &`" in the background, then run "``feh -R 1 <name>``"
in the foreground; terminate with "`ctrl-C fg ctrl-C`".
_Switching between_ **Mirror** _and_ **Audio** _modes during an active connection is
possible: in_ **Mirror** _mode, stop mirroring (or close the mirror window) and start an_ **Audio** _mode connection,
switch back by initiating a_ **Mirror** _mode connection; cover-art display stops/restarts as you leave/re-enter_ **Audio** _mode._
* **Note that Apple DRM
(as found in Apple TV app content on the client) cannot be decrypted by UxPlay,
* **Note that Apple video-DRM
(as found in AppleTV.app content on the client) cannot be decrypted by UxPlay,
and (unlike a true AppleTV), the UxPlay server does not allow the Apple
client to run a http connection on the server that directly streams
content from the internet to the server, instead of
streaming it to the client, and then re-streaming to the server.**
streaming it to the client, and then re-streaming to the server. Unlike AppleTV.app, DRM-free apps like youtube.app
can be viewed and listened to in Mirror mode, but using the youtube.app icon for AirPlay video will only send
ALAC sound without the accompanying video.**
### Possibility for using hardware-accelerated h264 video-decoding, if available.
@@ -186,20 +195,19 @@ for a distribution, use the cmake option `-DNO_MARCH_NATIVE=ON`.
such as Zoom, see [ZOOMFIX compile-time option](#zoomfix-compile-time-option) below).
**ZOOMFIX is only needed for GStreamer-1.18.x or earlier**.
5. `make`
6. `sudo make install` (you can afterwards uninstall
with `sudo make uninstall` in the same directory in which this was run)
7. Install GStreamer plugins that you
need: `sudo apt-get install gstreamer1.0-<plugin>`; values of
`<plugin>` needed are: "**plugins-base**", "**libav**" (for sound),
"**plugins-good**" (for v4l2 hardware h264 decoding)
and "**plugins-bad**" (for h264 decoding). Also needed may
be "**gl**" for OpenGL support (which may be useful, and should
be used with h264 decoding by the NVIDIA GPU), and "**x**" for
X11 support, although these may already be installed; "**vaapi**"
is needed for hardware-accelerated h264 video decoding by Intel
or AMD graphics (but not for use with NVIDIA using proprietary drivers).
Also install "**tools**" to get the utility gst-inspect-1.0 for
examining the GStreamer installation.
6. `sudo make install` (you can afterwards uninstall with ``sudo make uninstall``
in the same directory in which this was run).
7. Install GStreamer plugins that you need: `sudo apt-get install gstreamer1.0-<plugin>`;
values of `<plugin>` needed are: "**plugins-base**", "**libav**" (for sound),
"**plugins-good**" (for v4l2 hardware h264 decoding)
and "**plugins-bad**" (for h264 decoding). Also needed may
be "**gl**" for OpenGL support (which may be useful, and should
be used with h264 decoding by the NVIDIA GPU), and "**x**" for
X11 support, although these may already be installed; "**vaapi**"
is needed for hardware-accelerated h264 video decoding by Intel
or AMD graphics (but not for use with NVIDIA using proprietary drivers).
Also install "**tools**" to get the utility gst-inspect-1.0 for
examining the GStreamer installation.
_If you intend to modify the code, use a separate "build" directory:
replace_ "`cmake [ ] .`" _by_ "``mkdir build ; cd build ; cmake [ ] ..``"; _you can then clean
@@ -224,11 +232,12 @@ UxPlay from receiving client connection requests unless some network ports
are opened. See [Troubleshooting](#troubleshooting) below for
help with this or other problems.
One common problem involves GStreamer
**One common problem involves GStreamer
attempting to use incorrectly-configured or absent accelerated hardware h264
video decoding (e.g., VAAPI).
Try "`uxplay -avdec`" to force software video decoding; if this works you can
then try to fix accelerated hardware video decoding if you need it.
then try to fix accelerated hardware video decoding if you need it, or just uninstall the GStreamer VAAPI plugin. If
your system uses the Wayland compositor for graphics, use "`uxplay -vs waylandsink`".**
See [Usage](#usage) for more run-time options.
**Raspberry Pi**: GStreamer-1.18.4 or later required for hardware video decoding; for 1.20 or earlier, also see
@@ -262,8 +271,8 @@ gstreamer1-devel gstreamer1-plugins-base-devel gstreamer1-libav gstreamer1-plugi
* **OpenSUSE:**
(sudo zypper install) libopenssl-devel libplist-devel
avahi-compat-mDNSResponder-devel (+ libX11-devel for ZOOMFIX). The required
GStreamer packages are: gstreamer-devel gstreamer-plugins-base-devel gstreamer-plugins-libav gstreamer-plugins-bad (+ gstreamer-plugins-vaapi for Intel graphics);
you may need to use the version of gstreamer-plugins-libav for OpenSUSE from [Packman](https://ftp.gwdg.de/pub/linux/misc/packman/suse/) "Essentials".
GStreamer packages are: gstreamer-devel gstreamer-plugins-base-devel gstreamer-plugins-libav gstreamer-plugins-bad (+ gstreamer-plugins-vaapi for Intel graphics); in some cases,
you may need to use gstreamer packages for OpenSUSE from [Packman](https://ftp.gwdg.de/pub/linux/misc/packman/suse/) "Essentials".
* **Arch Linux**
@@ -350,24 +359,12 @@ as the device is rotated).
Options:
**-p** allows you to select the network ports used by UxPlay (these need
to be opened if the server is behind a firewall). By itself, -p sets
"legacy" ports TCP 7100, 7000, 7001, UDP 6000, 6001, 7011. -p n (e.g. -p
35000) sets TCP and UDP ports n, n+1, n+2. -p n1,n2,n3 (comma-separated
values) sets each port separately; -p n1,n2 sets ports n1,n2,n2+1. -p tcp n
or -p udp n sets just the TCP or UDP ports. Ports must be in the range
[1024-65535].
If the -p option is not used, the ports are chosen dynamically (randomly),
which will not work if a firewall is running.
**-n server_name** (Default: UxPlay); server_name@_hostname_ will be the name that appears offering
AirPlay services to your iPad, iPhone etc, where _hostname_ is the name of the server running uxplay.
This will also now be the name shown above the mirror display (X11) window.
**-nh** Do not append "@_hostname_" at the end of the AirPlay server name.
**-s wxh** (e.g. -s 1920x1080 , which is the default ) sets the display resolution (width and height,
in pixels). (This may be a
request made to the AirPlay client, and perhaps will not
@@ -391,40 +388,20 @@ which will not work if a firewall is running.
**-fs** uses fullscreen mode, but only works with Wayland or VAAPI plugins.
**-fps n** sets a maximum frame rate (in frames per second) for the AirPlay
client to stream video; n must be a whole number less than 256.
(The client may choose to serve video at any frame rate lower
than this; default is 30 fps.) A setting below 30 fps might be useful to
reduce latency if you are running more than one instance of uxplay at the same time.
_This setting is only an advisory to
the client device, so setting a high value will not force a high framerate._
(You can test using "-vs fpsdisplaysink" to see what framerate is being
received, or use the option -FPSdata which displays video-stream performance data
continuously sent by the client during video-streaming.)
**-p** allows you to select the network ports used by UxPlay (these need
to be opened if the server is behind a firewall). By itself, -p sets
"legacy" ports TCP 7100, 7000, 7001, UDP 6000, 6001, 7011. -p n (e.g. -p
35000) sets TCP and UDP ports n, n+1, n+2. -p n1,n2,n3 (comma-separated
values) sets each port separately; -p n1,n2 sets ports n1,n2,n2+1. -p tcp n
or -p udp n sets just the TCP or UDP ports. Ports must be in the range
[1024-65535].
**-FPSdata** Turns on monitoring of regular reports about video streaming performance
that are sent by the client. These will be displayed in the terminal window if this
option is used. The data is updated by the client at 1 second intervals.
**-m** generates a random MAC address to use instead of the true hardware MAC
number of the computer's network card. (Different server_name, MAC
addresses, and network ports are needed for each running uxplay if you
attempt to run two instances of uxplay on the same computer.)
If UxPlay fails to find the true MAC address of a network card, (more
specifically, the MAC address used by the first active network interface detected)
a random MAC address will be used even if option **-m** was not specifed.
(Note that a random MAC address will be different each time UxPlay is started).
Also: image transforms that had been added to RPiPlay have been ported to UxPlay:
**-f {H|V|I}** implements "videoflip" image transforms: H = horizontal flip
(right-left flip, or mirror image); V = vertical flip ; I =
180 degree rotation or inversion (which is the combination of H with V).
**-r {R|L}** 90 degree Right (clockwise) or Left (counter-clockwise)
rotations; these are carried out after any **-f** transforms.
If the -p option is not used, the ports are chosen dynamically (randomly),
which will not work if a firewall is running.
**-avdec** forces use of software h264 decoding using Gstreamer element avdec_h264 (libav h264 decoder). This
option should prevent autovideosink choosing a hardware-accelerated videosink plugin such as vaapisink.
**-vp _parser_** choses the GStreamer pipeline's h264 parser element, default is h264parse. Using
quotes "..." allows options to be added.
@@ -463,9 +440,6 @@ Also: image transforms that had been added to RPiPlay have been ported to UxPlay
Pi "Desktop" systems using the Wayland video compositor (use for
Ubuntu 21.10 for Raspberry Pi 4B).
**-avdec** forces use of software h264 decoding using Gstreamer element avdec_h264 (libav h264 decoder). This
option should prevent autovideosink choosing a hardware-accelerated videosink plugin such as vaapisink.
**-as _audiosink_** chooses the GStreamer audiosink, instead of letting
autoaudiosink pick it for you. Some audiosink choices are: pulsesink, alsasink,
osssink, oss4sink, and osxaudiosink (for macOS). Using quotes
@@ -474,7 +448,15 @@ Also: image transforms that had been added to RPiPlay have been ported to UxPlay
**-as 0** (or just **-a**) suppresses playing of streamed audio, but displays streamed video.
**-reset n** sets a limit of n consective timeout failures of the client to respond to ntp requests
**-ca _filename_** provides a file (where _filename_ can include a full path) used for output of "cover art"
(from Apple Music, _etc._,) in audio-only ALAC mode. This file is overwritten with the latest cover art as
it arrives. Cover art (jpeg format) is discarded if this option is not used. Use with a image viewer that reloads the image
if it changes, or regularly (_e.g._ once per second.). To achieve this, run "`uxplay -ca [path/to/]filename &`" in the background,
then run the the image viewer in the foreground. Example, using `feh` as the viewer: run "``feh -R 1 [path/to/]filename``" (in
the same terminal window in which uxplay was put into the background). To quit, use ```ctrl-C fg ctrl-C``` to terminate
the image viewer, bring ``uxplay`` into the foreground, and terminate it too.
**-reset n** sets a limit of n consecutive timeout failures of the client to respond to ntp requests
from the server (these are sent every 3 seconds to check if the client is still present). After
n failures, the client will be presumed to be offline, and the connection will be reset to allow a new
connection. The default value of n is 5; the value n = 0 means "no limit" on timeouts.
@@ -484,6 +466,37 @@ Also: image transforms that had been added to RPiPlay have been ported to UxPlay
as the window created in macOS by GStreamer does not terminate correctly (it causes a segfault)
if it is still open when the GStreamer pipeline is closed._
**-FPSdata** Turns on monitoring of regular reports about video streaming performance
that are sent by the client. These will be displayed in the terminal window if this
option is used. The data is updated by the client at 1 second intervals.
**-fps n** sets a maximum frame rate (in frames per second) for the AirPlay
client to stream video; n must be a whole number less than 256.
(The client may choose to serve video at any frame rate lower
than this; default is 30 fps.) A setting below 30 fps might be useful to
reduce latency if you are running more than one instance of uxplay at the same time.
_This setting is only an advisory to
the client device, so setting a high value will not force a high framerate._
(You can test using "-vs fpsdisplaysink" to see what framerate is being
received, or use the option -FPSdata which displays video-stream performance data
continuously sent by the client during video-streaming.)
**-f {H|V|I}** implements "videoflip" image transforms: H = horizontal flip
(right-left flip, or mirror image); V = vertical flip ; I =
180 degree rotation or inversion (which is the combination of H with V).
**-r {R|L}** 90 degree Right (clockwise) or Left (counter-clockwise)
rotations; these image transforms are carried out after any **-f** transforms.
**-m** generates a random MAC address to use instead of the true hardware MAC
number of the computer's network card. (Different server_name, MAC
addresses, and network ports are needed for each running uxplay if you
attempt to run two instances of uxplay on the same computer.)
If UxPlay fails to find the true MAC address of a network card, (more
specifically, the MAC address used by the first active network interface detected)
a random MAC address will be used even if option **-m** was not specifed.
(Note that a random MAC address will be different each time UxPlay is started).
**-t _timeout_** will cause the server to relaunch (without stopping uxplay) if no connections
have been present during the previous _timeout_ seconds. You may wish to use this if the Server
is not visible to new Clients that were inactive when the Server was launched, and an idle Bonjour
@@ -631,15 +644,16 @@ default limit is not right for your network, it can be modified using the opti
starts to recover after ntp timeouts, a corrupt video packet from before the timeout may trigger a "connection reset by peer" error, which also causes UxPlay to reset the
connection. When the connection is reset, the "frozen" mirror screen of the previous connection is left in place, and will be taken over by a new client connection when it is made.
### 6. Failure to decrypt ALL video and audio streams from old or non-Apple clients:
### 6. Protocol issues, such as failure to decrypt ALL video and audio streams from old or non-Apple clients:
This triggers an unending stream of error messages, and means that the
A protocol failure may trigger an unending stream of error messages, and means that the
audio decryption key (also used in video decryption)
was not correctly extracted from data sent by the client.
This should not happen for iOS 9.3 or later clients. However, if a client
uses the same older version of the protocol that is used by the Windows-based
AirPlay client emulator _AirMyPC_, the protocol can be switched to the older version
by the setting ```OLD_PROTOCOL_CLIENT_USER_AGENT_LIST``` in lib/global.h.
by the setting ```OLD_PROTOCOL_CLIENT_USER_AGENT_LIST```
in `UxPlay/lib/global.h`.
UxPlay reports the client's "User Agent" string when it connects. If
some other client also fails to decrypt all audio and video, try adding
its "User Agent" string in place of "xxx" in the entry "AirMyPC/2.0;xxx"
@@ -656,10 +670,15 @@ sourceVersion 380.20.1 (an AppleTV 4K 1st gen, introduced 2017, running
tvOS 12.2.1); it seems that the use of "legacy" protocol just requires bit 27 (listed as
"SupportsLegacyPairing") of the
"features" plist code (reported to the client by the AirPlay server) to be set.
The "features" code and other settings are set in `UxPlay/lib/dnssdint.h`.
# ChangeLog
1.53 2022-06-13 Internal changes to audio sync code, revised documentation,
minor bugfix (fix assertion crash when resent audio packets are empty).
1.54 2022-06-25 Add support for "Cover Art" display in Audio-only (ALAC) mode. Reverted a change
that caused VAAPI to crash with AMD POLARIS graphics cards. Minor internal changes to
plist code and uxplay option parsing.
1.53 2022-06-13 Internal changes to audio sync code, revised documentation,
Minor bugfix (fix assertion crash when resent audio packets are empty).
1.52 2022-05-05 Cleaned up initial audio sync code, and reformatted
streaming debug output (readable aligned timestamps with