mirror of
https://github.com/morgan9e/UxPlay
synced 2026-04-14 00:04:13 +09:00
README: Remove FD- 's note on Airplay protocol, replace with link to original.
This commit is contained in:
fduncanh
21
README.html
21
README.html
@@ -196,30 +196,15 @@
|
||||
<h1 id="uxplay-authors">UxPlay authors</h1>
|
||||
<p><em>[adapted from fdraschbacher’s notes on RPiPlay antecedents]</em></p>
|
||||
<p>The code in this repository accumulated from various sources over time. Here is an attempt at listing the various authors and the components they created:</p>
|
||||
<p>UxPlay was initially created by <strong>antimof</strong> from RPiPlay, by replacing its Raspberry-Pi-specific video and audio rendering system with GStreamer rendering for Desktop Linux (antimof’s work on code in <code>renderers/</code> was later backported to RPiPlay). The previous authors of code included in UxPlay by inheritance from RPiPlay include:</p>
|
||||
<p>UxPlay was initially created by <strong>antimof</strong> from RPiPlay, by replacing its Raspberry-Pi-specific video and audio rendering system with GStreamer rendering for Desktop Linux (antimof’s work on code in <code>renderers/</code> was later backported to RPiPlay).</p>
|
||||
<p>The previous authors of code included in UxPlay by inheritance from RPiPlay include:</p>
|
||||
<ul>
|
||||
<li><strong>EstebanKubata</strong>: Created a FairPlay library called <a href="https://github.com/EstebanKubata/playfair">PlayFair</a>. Located in the <code>lib/playfair</code> folder. License: GNU GPL</li>
|
||||
<li><strong>Juho Vähä-Herttua</strong> and contributors: Created an AirPlay audio server called <a href="https://github.com/juhovh/shairplay">ShairPlay</a>, including support for Fairplay based on PlayFair. Most of the code in <code>lib/</code> originally stems from this project. License: GNU LGPLv2.1+</li>
|
||||
<li><strong>dsafa22</strong>: Created an AirPlay 2 mirroring server <a href="https://github.com/dsafa22/AirplayServer">AirplayServer</a> (seems gone now), for Android based on ShairPlay. Code is preserved <a href="https://github.com/jiangban/AirplayServer">here</a>, and <a href="https://github.com/FDH2/UxPlay/wiki/AirPlay2">see here</a> for the description of the analysis of the AirPlay 2 mirror protocol that made RPiPlay possible, by the AirplayServer author. All code in <code>lib/</code> concerning mirroring is dsafa22’s work. License: GNU LGPLv2.1+</li>
|
||||
<li><strong>Florian Draschbacher</strong> and contributors: adapted dsafa22’s Android project for the Raspberry Pi, with extensive cleanups, debugging and improvements. The project <a href="https://github.com/FD-/RPiPlay">RPiPlay</a> is basically a port of dsafa22’s code to the Raspberry Pi, utilizing OpenMAX and OpenSSL for better performance on the Pi. License GPL v3.</li>
|
||||
<li><strong>Florian Draschbacher</strong> (FD-) and contributors: adapted dsafa22’s Android project for the Raspberry Pi, with extensive cleanups, debugging and improvements. The project <a href="https://github.com/FD-/RPiPlay">RPiPlay</a> is basically a port of dsafa22’s code to the Raspberry Pi, utilizing OpenMAX and OpenSSL for better performance on the Pi. License GPL v3. FD- has written an interesting note on the history of <a href="http://github.com/FD-/RPiPlay#airplay-protocol-versions">Airplay protocol versions</a>, available at the RPiPlay github repository.</li>
|
||||
</ul>
|
||||
<p>Independent of UxPlay, but used by it and bundled with it:</p>
|
||||
<ul>
|
||||
<li><strong>Fedor Indutny</strong> (of Node.js, and formerly Joyent, Inc) and contributors: Created an http parsing library called <a href="https://github.com/nodejs/llhttp">llhttp</a>. Located at <code>lib/llhttp/</code>. License: MIT</li>
|
||||
</ul>
|
||||
<h2 id="notes-on-airplay-protocol-versions-by-florian-draschbacher-rpiplay-creator">Notes on AirPlay protocol versions by Florian Draschbacher, RPiPlay creator</h2>
|
||||
<p>(From the https://github.com/FD-/RPiPlay.git repository.)</p>
|
||||
<p>For multiple reasons, it’s very difficult to clearly define the protocol names and versions of the components that make up the AirPlay streaming system. In fact, it seems like the AirPlay version number used for marketing differs from that used in the actual implementation. In order to tidy up this whole mess a bit, I did a little research that I’d like to summarize here:</p>
|
||||
<p>The very origin of the AirPlay protocol suite was launched as AirTunes sometime around 2004. It allowed to stream audio from iTunes to an AirPort Express station. Internally, the name of the protocol that was used was RAOP, or Remote Audio Output Protocol. It seems already back then, the protocol involved AES encryption. A public key was needed for encrypting the audio sent to an AirPort Express, and the private key was needed for receiving the protocol (ie used in the AirPort Express to decrypt the stream). Already in 2004, the public key was reverse-engineered, so that <a href="http://nanocr.eu/2004/08/11/reversing-airtunes/">third-party sender applications</a> were developed.</p>
|
||||
<p>Some time <a href="https://weblog.rogueamoeba.com/2008/01/10/a-tour-of-airfoil-3/">around 2008</a>, the protocol was revised and named AirTunes 2. It seems the changes primarily concerned timing. By 2009, the new protocol was <a href="https://git.zx2c4.com/Airtunes2/about/">reverse-engineered and documented</a>.</p>
|
||||
<p>When the Apple TV 2nd generation was introduced in 2010, it received support for the AirTunes protocol. However, because this device allowed playback of visual content, the protocol was extended and renamed AirPlay. It was now possible to stream photo slideshows and videos. Shortly after the release of the Apple TV 2nd generation, AirPlay support for iOS was included in the iOS 4.2 update. It seems like at that point, the audio stream was still actually using the same AirTunes 2 protocol as described above. The video and photo streams were added as a whole new protocol based on HTTP, pretty much independent from the audio stream. Soon, the first curious developers began to <a href="https://web.archive.org/web/20101211213705/http://www.tuaw.com/2010/12/08/dear-aunt-tuaw-can-i-airplay-to-my-mac/">investigate how it worked</a>. Their conclusion was that visual content is streamed unencrypted.</p>
|
||||
<p>In April 2011, a talented hacker <a href="http://www.macrumors.com/2011/04/11/apple-airplay-private-key-exposed-opening-door-to-airport-express-emulators/">extracted the AirPlay private key</a> from an AirPort Express. This meant that finally, third-party developers were able to also build AirPlay receiver (server) programs.</p>
|
||||
<p>For iOS 5, released in 2011, Apple added a new protocol to the AirPlay suite: AirPlay mirroring. <a href="https://www.aorensoftware.com/blog/2011/08/20/exploring-airplay-mirroring-internals/">Initial investigators</a> found this new protocol used encryption in order to protect the transferred video data.</p>
|
||||
<p>By 2012, most of AirPlay’s protocols had been reverse-engineered and <a href="https://nto.github.io/AirPlay.html">documented</a> (see also <a href="https://openairplay.github.io/airplay-spec">updated version</a>). At this point, audio still used the AirTunes 2 protocol from around 2008, video, photos and mirroring still used their respective protocols in an unmodified form, so you could still speak of AirPlay 1 (building upon AirTunes 2). The Airplay server running on the Apple TV reported as version 130. The setup of AirPlay mirroring used the xml format, in particular a stream.xml file. Additionally, it seems like the actual audio data is using the ALAC codec for audio-only (AirTunes 2) streaming and AAC for mirror audio. At least these different formats were used in <a href="https://github.com/espes/Slave-in-the-Magic-Mirror/issues/12#issuecomment-372380451">later iOS versions</a>.</p>
|
||||
<p>Sometime before iOS 9, the protocol for mirroring was slightly modified: Instead of the “stream.xml” API endpoint, the same information could also be querried in binary plist form, just by changing the API endpoint to “stream”, without any extension. I wasn’t able to figure out which of these was actually used by what specific client / server versions.</p>
|
||||
<p>For iOS 9, Apple made <a href="https://9to5mac.com/2015/09/11/apple-ios-9-airplay-improvements-screen-mirroring/">considerable changes</a> to the AirPlay protocol in 2015, including audio and mirroring. Apparently, the audio protocol was only slightly modified, and a <a href="https://github.com/juhovh/shairplay/issues/43">minor change</a> restored compatibility. For mirroring, an <a href="https://github.com/juhovh/shairplay/issues/43#issuecomment-142115959">additional pairing phase</a> was added to the connection establishment procedure, consisting of pair-setup and pair-verify calls. Seemingly, these were added in order to simplify usage with devices that are connected frequently. Pair-setup is used only the first time an iOS device connects to an AirPlay receiver. The generated cryptographic binding can be used for pair-verify in later sessions. Additionally, the stream / stream.xml endpoint was replaced with the info endpoint (only available as binary plist AFAICT). As of iOS 12, the protocol introduced with iOS 9 was still supported with only slight modifications, albeit as a legacy mode. While iOS 9 used two SETUP calls (one for general connection and mirroring video, and one for audio), iOS 12 legacy mode uses 3 SETUP calls (one for general connection (timing and events), one for mirroring video, one for audio).</p>
|
||||
<p>The release of tvOS 10.2 broke many third-party AirPlay sender (client) programs in 2017. The reason was that it was now mandatory to perform device verification via a pin in order to stream content to an Apple TV. The functionality had been in the protocol before, but was not mandatory. Some discussion about the new scheme can be found <a href="https://github.com/postlund/pyatv/issues/79">here</a>. A full specification of the pairing and authentication protocol was made available on <a href="https://htmlpreview.github.io/?https://github.com/philippe44/RAOP-Player/blob/master/doc/auth_protocol.html">GitHub</a>. At that point, tvOS 10.2 reported as AirTunes/320.20.</p>
|
||||
<p>In tvOS 11, the reported server version was <a href="https://github.com/ejurgensen/forked-daapd/issues/377#issuecomment-309213273">increased to 350.92.4</a>.</p>
|
||||
<p>iOS 11.4 added AirPlay 2 in 2018. Although extensively covered by the media, it’s not entirely clear what changes specifically Apple has made protocol-wise.</p>
|
||||
<p>From captures of the traffic between an iOS device running iOS 12.2 and an AppleTV running tvOS 12.2.1, one can see that the communication on the main mirroring HTTP connection is encrypted after the initial handshake. This could theoretically be part of the new AirPlay 2 protocol. The AppleTV running tvOS 12.2.1 identifies as AirTunes/380.20.1. When connecting from the same iOS device to an AppleTV 3rd generation (reporting as AirTunes/220.68), the communication is still visible in plain. From the log messages that the iOS device produces when connected to an AppleTV 3rd generation, it becomes apparent that the iOS device is treating this plain protocol as the legacy protocol (as originally introduced with iOS 9). Further research showed that at the moment, all available third-party AirPlay mirroring receivers (servers) are using this legacy protocol, including the open source implementation of dsafa22, which is the base for RPiPlay. Given Apple considers this a legacy protocol, it can be expected to be removed entirely in the future. This means that all third-party AirPlay receivers will have to be updated to the new (fully encrypted) protocol at some point.</p>
|
||||
<p>More specifically, the encryption starts after the pair-verify handshake completed, so the fp-setup handshake is already happening encrypted. Judging from the encryption scheme for AirPlay video (aka HLS Relay), likely two AES GCM 128 ciphers are used on the socket communication (one for sending, one for receiving). However, I have no idea how the keys are derived from the handshake data.</p>
|
||||
|
||||
56
README.md
56
README.md
@@ -837,7 +837,11 @@ The code in this repository accumulated from various sources over time. Here
|
||||
is an attempt at listing the various authors and the components they created:
|
||||
|
||||
UxPlay was initially created by **antimof** from RPiPlay, by replacing its Raspberry-Pi-specific video and audio rendering system with GStreamer rendering for
|
||||
Desktop Linux (antimof's work on code in `renderers/` was later backported to RPiPlay). The previous authors of code included in UxPlay by inheritance from RPiPlay include:
|
||||
Desktop Linux (antimof's work on code in `renderers/` was later backported to RPiPlay).
|
||||
|
||||
|
||||
|
||||
The previous authors of code included in UxPlay by inheritance from RPiPlay include:
|
||||
|
||||
* **EstebanKubata**: Created a FairPlay library called [PlayFair](https://github.com/EstebanKubata/playfair). Located in the `lib/playfair` folder. License: GNU GPL
|
||||
* **Juho Vähä-Herttua** and contributors: Created an AirPlay audio server called [ShairPlay](https://github.com/juhovh/shairplay), including support for Fairplay based on PlayFair. Most of the code in `lib/` originally stems from this project. License: GNU LGPLv2.1+
|
||||
@@ -845,56 +849,14 @@ Desktop Linux (antimof's work on code in `renderers/` was later backported to RP
|
||||
preserved [here](https://github.com/jiangban/AirplayServer), and [see here](https://github.com/FDH2/UxPlay/wiki/AirPlay2) for the description
|
||||
of the analysis of the AirPlay 2 mirror protocol that made RPiPlay possible, by the AirplayServer author. All
|
||||
code in `lib/` concerning mirroring is dsafa22's work. License: GNU LGPLv2.1+
|
||||
* **Florian Draschbacher** and contributors: adapted dsafa22's Android project for the Raspberry Pi, with extensive cleanups, debugging and improvements. The
|
||||
* **Florian Draschbacher** (FD-) and contributors: adapted dsafa22's Android project for the Raspberry Pi, with extensive cleanups, debugging and improvements. The
|
||||
project [RPiPlay](https://github.com/FD-/RPiPlay) is basically a port of dsafa22's code to the Raspberry Pi, utilizing OpenMAX and OpenSSL for better performance on the Pi. License GPL v3.
|
||||
FD- has written an interesting note on the history of [Airplay protocol versions](http://github.com/FD-/RPiPlay#airplay-protocol-versions),
|
||||
available at the RPiPlay github repository.
|
||||
|
||||
|
||||
Independent of UxPlay, but used by it and bundled with it:
|
||||
|
||||
* **Fedor Indutny** (of Node.js, and formerly Joyent, Inc) and contributors: Created an http parsing library called [llhttp](https://github.com/nodejs/llhttp). Located at `lib/llhttp/`. License: MIT
|
||||
|
||||
## Notes on AirPlay protocol versions by Florian Draschbacher, RPiPlay creator
|
||||
(From the https://github.com/FD-/RPiPlay.git repository.)
|
||||
|
||||
For multiple reasons, it's very difficult to clearly define the protocol names and versions of the components that make up the AirPlay streaming system. In fact, it seems like the AirPlay version number used for marketing differs from that used in the actual implementation. In order to tidy up this whole mess a bit, I did a little research that I'd like to summarize here:
|
||||
|
||||
|
||||
The very origin of the AirPlay protocol suite was launched as AirTunes sometime around 2004. It allowed to stream audio from iTunes to an AirPort Express station. Internally, the name of the protocol that was used was RAOP, or Remote Audio Output Protocol. It seems already back then, the protocol involved AES encryption. A public key was needed for encrypting the audio sent to an AirPort Express, and the private key was needed for receiving the protocol (ie used in the AirPort Express to decrypt the stream). Already in 2004, the public key was reverse-engineered, so that [third-party sender applications](http://nanocr.eu/2004/08/11/reversing-airtunes/) were developed.
|
||||
|
||||
|
||||
Some time [around 2008](https://weblog.rogueamoeba.com/2008/01/10/a-tour-of-airfoil-3/), the protocol was revised and named AirTunes 2. It seems the changes primarily concerned timing. By 2009, the new protocol was [reverse-engineered and documented](https://git.zx2c4.com/Airtunes2/about/).
|
||||
|
||||
|
||||
When the Apple TV 2nd generation was introduced in 2010, it received support for the AirTunes protocol. However, because this device allowed playback of visual content, the protocol was extended and renamed AirPlay. It was now possible to stream photo slideshows and videos. Shortly after the release of the Apple TV 2nd generation, AirPlay support for iOS was included in the iOS 4.2 update. It seems like at that point, the audio stream was still actually using the same AirTunes 2 protocol as described above. The video and photo streams were added as a whole new protocol based on HTTP, pretty much independent from the audio stream. Soon, the first curious developers began to [investigate how it worked](https://web.archive.org/web/20101211213705/http://www.tuaw.com/2010/12/08/dear-aunt-tuaw-can-i-airplay-to-my-mac/). Their conclusion was that visual content is streamed unencrypted.
|
||||
|
||||
|
||||
In April 2011, a talented hacker [extracted the AirPlay private key](http://www.macrumors.com/2011/04/11/apple-airplay-private-key-exposed-opening-door-to-airport-express-emulators/) from an AirPort Express. This meant that finally, third-party developers were able to also build AirPlay receiver (server) programs.
|
||||
|
||||
|
||||
For iOS 5, released in 2011, Apple added a new protocol to the AirPlay suite: AirPlay mirroring. [Initial investigators](https://www.aorensoftware.com/blog/2011/08/20/exploring-airplay-mirroring-internals/) found this new protocol used encryption in order to protect the transferred video data.
|
||||
|
||||
|
||||
By 2012, most of AirPlay's protocols had been reverse-engineered and [documented](https://nto.github.io/AirPlay.html) (see also [updated version](https://openairplay.github.io/airplay-spec)). At this point, audio still used the AirTunes 2 protocol from around 2008, video, photos and mirroring still used their respective protocols in an unmodified form, so you could still speak of AirPlay 1 (building upon AirTunes 2). The Airplay server running on the Apple TV reported as version 130. The setup of AirPlay mirroring used the xml format, in particular a stream.xml file.
|
||||
Additionally, it seems like the actual audio data is using the ALAC codec for audio-only (AirTunes 2) streaming and AAC for mirror audio. At least these different formats were used in [later iOS versions](https://github.com/espes/Slave-in-the-Magic-Mirror/issues/12#issuecomment-372380451).
|
||||
|
||||
|
||||
Sometime before iOS 9, the protocol for mirroring was slightly modified: Instead of the "stream.xml" API endpoint, the same information could also be querried in binary plist form, just by changing the API endpoint to "stream", without any extension. I wasn't able to figure out which of these was actually used by what specific client / server versions.
|
||||
|
||||
|
||||
For iOS 9, Apple made [considerable changes](https://9to5mac.com/2015/09/11/apple-ios-9-airplay-improvements-screen-mirroring/) to the AirPlay protocol in 2015, including audio and mirroring. Apparently, the audio protocol was only slightly modified, and a [minor change](https://github.com/juhovh/shairplay/issues/43) restored compatibility. For mirroring, an [additional pairing phase](https://github.com/juhovh/shairplay/issues/43#issuecomment-142115959) was added to the connection establishment procedure, consisting of pair-setup and pair-verify calls. Seemingly, these were added in order to simplify usage with devices that are connected frequently. Pair-setup is used only the first time an iOS device connects to an AirPlay receiver. The generated cryptographic binding can be used for pair-verify in later sessions. Additionally, the stream / stream.xml endpoint was replaced with the info endpoint (only available as binary plist AFAICT).
|
||||
As of iOS 12, the protocol introduced with iOS 9 was still supported with only slight modifications, albeit as a legacy mode. While iOS 9 used two SETUP calls (one for general connection and mirroring video, and one for audio), iOS 12 legacy mode uses 3 SETUP calls (one for general connection (timing and events), one for mirroring video, one for audio).
|
||||
|
||||
|
||||
The release of tvOS 10.2 broke many third-party AirPlay sender (client) programs in 2017. The reason was that it was now mandatory to perform device verification via a pin in order to stream content to an Apple TV. The functionality had been in the protocol before, but was not mandatory. Some discussion about the new scheme can be found [here](https://github.com/postlund/pyatv/issues/79). A full specification of the pairing and authentication protocol was made available on [GitHub](https://htmlpreview.github.io/?https://github.com/philippe44/RAOP-Player/blob/master/doc/auth_protocol.html). At that point, tvOS 10.2 reported as AirTunes/320.20.
|
||||
|
||||
|
||||
In tvOS 11, the reported server version was [increased to 350.92.4](https://github.com/ejurgensen/forked-daapd/issues/377#issuecomment-309213273).
|
||||
|
||||
|
||||
iOS 11.4 added AirPlay 2 in 2018. Although extensively covered by the media, it's not entirely clear what changes specifically Apple has made protocol-wise.
|
||||
|
||||
|
||||
From captures of the traffic between an iOS device running iOS 12.2 and an AppleTV running tvOS 12.2.1, one can see that the communication on the main mirroring HTTP connection is encrypted after the initial handshake.
|
||||
This could theoretically be part of the new AirPlay 2 protocol. The AppleTV running tvOS 12.2.1 identifies as AirTunes/380.20.1.
|
||||
When connecting from the same iOS device to an AppleTV 3rd generation (reporting as AirTunes/220.68), the communication is still visible in plain. From the log messages that the iOS device produces when connected to an AppleTV 3rd generation, it becomes apparent that the iOS device is treating this plain protocol as the legacy protocol (as originally introduced with iOS 9). Further research showed that at the moment, all available third-party AirPlay mirroring receivers (servers) are using this legacy protocol, including the open source implementation of dsafa22, which is the base for RPiPlay. Given Apple considers this a legacy protocol, it can be expected to be removed entirely in the future. This means that all third-party AirPlay receivers will have to be updated to the new (fully encrypted) protocol at some point.
|
||||
|
||||
More specifically, the encryption starts after the pair-verify handshake completed, so the fp-setup handshake is already happening encrypted. Judging from the encryption scheme for AirPlay video (aka HLS Relay), likely two AES GCM 128 ciphers are used on the socket communication (one for sending, one for receiving). However, I have no idea how the keys are derived from the handshake data.
|
||||
|
||||
161
README.txt
161
README.txt
@@ -1031,8 +1031,10 @@ they created:
|
||||
UxPlay was initially created by **antimof** from RPiPlay, by replacing
|
||||
its Raspberry-Pi-specific video and audio rendering system with
|
||||
GStreamer rendering for Desktop Linux (antimof's work on code in
|
||||
`renderers/` was later backported to RPiPlay). The previous authors of
|
||||
code included in UxPlay by inheritance from RPiPlay include:
|
||||
`renderers/` was later backported to RPiPlay).
|
||||
|
||||
The previous authors of code included in UxPlay by inheritance from
|
||||
RPiPlay include:
|
||||
|
||||
- **EstebanKubata**: Created a FairPlay library called
|
||||
[PlayFair](https://github.com/EstebanKubata/playfair). Located in
|
||||
@@ -1050,12 +1052,15 @@ code included in UxPlay by inheritance from RPiPlay include:
|
||||
made RPiPlay possible, by the AirplayServer author. All code in
|
||||
`lib/` concerning mirroring is dsafa22's work. License: GNU
|
||||
LGPLv2.1+
|
||||
- **Florian Draschbacher** and contributors: adapted dsafa22's Android
|
||||
project for the Raspberry Pi, with extensive cleanups, debugging and
|
||||
improvements. The project [RPiPlay](https://github.com/FD-/RPiPlay)
|
||||
is basically a port of dsafa22's code to the Raspberry Pi, utilizing
|
||||
OpenMAX and OpenSSL for better performance on the Pi. License GPL
|
||||
v3.
|
||||
- **Florian Draschbacher** (FD-) and contributors: adapted dsafa22's
|
||||
Android project for the Raspberry Pi, with extensive cleanups,
|
||||
debugging and improvements. The project
|
||||
[RPiPlay](https://github.com/FD-/RPiPlay) is basically a port of
|
||||
dsafa22's code to the Raspberry Pi, utilizing OpenMAX and OpenSSL
|
||||
for better performance on the Pi. License GPL v3. FD- has written an
|
||||
interesting note on the history of [Airplay protocol
|
||||
versions](http://github.com/FD-/RPiPlay#airplay-protocol-versions),
|
||||
available at the RPiPlay github repository.
|
||||
|
||||
Independent of UxPlay, but used by it and bundled with it:
|
||||
|
||||
@@ -1063,143 +1068,3 @@ Independent of UxPlay, but used by it and bundled with it:
|
||||
contributors: Created an http parsing library called
|
||||
[llhttp](https://github.com/nodejs/llhttp). Located at
|
||||
`lib/llhttp/`. License: MIT
|
||||
|
||||
Notes on AirPlay protocol versions by Florian Draschbacher, RPiPlay creator
|
||||
---------------------------------------------------------------------------
|
||||
|
||||
(From the https://github.com/FD-/RPiPlay.git repository.)
|
||||
|
||||
For multiple reasons, it's very difficult to clearly define the protocol
|
||||
names and versions of the components that make up the AirPlay streaming
|
||||
system. In fact, it seems like the AirPlay version number used for
|
||||
marketing differs from that used in the actual implementation. In order
|
||||
to tidy up this whole mess a bit, I did a little research that I'd like
|
||||
to summarize here:
|
||||
|
||||
The very origin of the AirPlay protocol suite was launched as AirTunes
|
||||
sometime around 2004. It allowed to stream audio from iTunes to an
|
||||
AirPort Express station. Internally, the name of the protocol that was
|
||||
used was RAOP, or Remote Audio Output Protocol. It seems already back
|
||||
then, the protocol involved AES encryption. A public key was needed for
|
||||
encrypting the audio sent to an AirPort Express, and the private key was
|
||||
needed for receiving the protocol (ie used in the AirPort Express to
|
||||
decrypt the stream). Already in 2004, the public key was
|
||||
reverse-engineered, so that [third-party sender
|
||||
applications](http://nanocr.eu/2004/08/11/reversing-airtunes/) were
|
||||
developed.
|
||||
|
||||
Some time [around
|
||||
2008](https://weblog.rogueamoeba.com/2008/01/10/a-tour-of-airfoil-3/),
|
||||
the protocol was revised and named AirTunes 2. It seems the changes
|
||||
primarily concerned timing. By 2009, the new protocol was
|
||||
[reverse-engineered and
|
||||
documented](https://git.zx2c4.com/Airtunes2/about/).
|
||||
|
||||
When the Apple TV 2nd generation was introduced in 2010, it received
|
||||
support for the AirTunes protocol. However, because this device allowed
|
||||
playback of visual content, the protocol was extended and renamed
|
||||
AirPlay. It was now possible to stream photo slideshows and videos.
|
||||
Shortly after the release of the Apple TV 2nd generation, AirPlay
|
||||
support for iOS was included in the iOS 4.2 update. It seems like at
|
||||
that point, the audio stream was still actually using the same AirTunes
|
||||
2 protocol as described above. The video and photo streams were added as
|
||||
a whole new protocol based on HTTP, pretty much independent from the
|
||||
audio stream. Soon, the first curious developers began to [investigate
|
||||
how it
|
||||
worked](https://web.archive.org/web/20101211213705/http://www.tuaw.com/2010/12/08/dear-aunt-tuaw-can-i-airplay-to-my-mac/).
|
||||
Their conclusion was that visual content is streamed unencrypted.
|
||||
|
||||
In April 2011, a talented hacker [extracted the AirPlay private
|
||||
key](http://www.macrumors.com/2011/04/11/apple-airplay-private-key-exposed-opening-door-to-airport-express-emulators/)
|
||||
from an AirPort Express. This meant that finally, third-party developers
|
||||
were able to also build AirPlay receiver (server) programs.
|
||||
|
||||
For iOS 5, released in 2011, Apple added a new protocol to the AirPlay
|
||||
suite: AirPlay mirroring. [Initial
|
||||
investigators](https://www.aorensoftware.com/blog/2011/08/20/exploring-airplay-mirroring-internals/)
|
||||
found this new protocol used encryption in order to protect the
|
||||
transferred video data.
|
||||
|
||||
By 2012, most of AirPlay's protocols had been reverse-engineered and
|
||||
[documented](https://nto.github.io/AirPlay.html) (see also [updated
|
||||
version](https://openairplay.github.io/airplay-spec)). At this point,
|
||||
audio still used the AirTunes 2 protocol from around 2008, video, photos
|
||||
and mirroring still used their respective protocols in an unmodified
|
||||
form, so you could still speak of AirPlay 1 (building upon AirTunes 2).
|
||||
The Airplay server running on the Apple TV reported as version 130. The
|
||||
setup of AirPlay mirroring used the xml format, in particular a
|
||||
stream.xml file. Additionally, it seems like the actual audio data is
|
||||
using the ALAC codec for audio-only (AirTunes 2) streaming and AAC for
|
||||
mirror audio. At least these different formats were used in [later iOS
|
||||
versions](https://github.com/espes/Slave-in-the-Magic-Mirror/issues/12#issuecomment-372380451).
|
||||
|
||||
Sometime before iOS 9, the protocol for mirroring was slightly modified:
|
||||
Instead of the "stream.xml" API endpoint, the same information could
|
||||
also be querried in binary plist form, just by changing the API endpoint
|
||||
to "stream", without any extension. I wasn't able to figure out which of
|
||||
these was actually used by what specific client / server versions.
|
||||
|
||||
For iOS 9, Apple made [considerable
|
||||
changes](https://9to5mac.com/2015/09/11/apple-ios-9-airplay-improvements-screen-mirroring/)
|
||||
to the AirPlay protocol in 2015, including audio and mirroring.
|
||||
Apparently, the audio protocol was only slightly modified, and a [minor
|
||||
change](https://github.com/juhovh/shairplay/issues/43) restored
|
||||
compatibility. For mirroring, an [additional pairing
|
||||
phase](https://github.com/juhovh/shairplay/issues/43#issuecomment-142115959)
|
||||
was added to the connection establishment procedure, consisting of
|
||||
pair-setup and pair-verify calls. Seemingly, these were added in order
|
||||
to simplify usage with devices that are connected frequently. Pair-setup
|
||||
is used only the first time an iOS device connects to an AirPlay
|
||||
receiver. The generated cryptographic binding can be used for
|
||||
pair-verify in later sessions. Additionally, the stream / stream.xml
|
||||
endpoint was replaced with the info endpoint (only available as binary
|
||||
plist AFAICT). As of iOS 12, the protocol introduced with iOS 9 was
|
||||
still supported with only slight modifications, albeit as a legacy mode.
|
||||
While iOS 9 used two SETUP calls (one for general connection and
|
||||
mirroring video, and one for audio), iOS 12 legacy mode uses 3 SETUP
|
||||
calls (one for general connection (timing and events), one for mirroring
|
||||
video, one for audio).
|
||||
|
||||
The release of tvOS 10.2 broke many third-party AirPlay sender (client)
|
||||
programs in 2017. The reason was that it was now mandatory to perform
|
||||
device verification via a pin in order to stream content to an Apple TV.
|
||||
The functionality had been in the protocol before, but was not
|
||||
mandatory. Some discussion about the new scheme can be found
|
||||
[here](https://github.com/postlund/pyatv/issues/79). A full
|
||||
specification of the pairing and authentication protocol was made
|
||||
available on
|
||||
[GitHub](https://htmlpreview.github.io/?https://github.com/philippe44/RAOP-Player/blob/master/doc/auth_protocol.html).
|
||||
At that point, tvOS 10.2 reported as AirTunes/320.20.
|
||||
|
||||
In tvOS 11, the reported server version was [increased to
|
||||
350.92.4](https://github.com/ejurgensen/forked-daapd/issues/377#issuecomment-309213273).
|
||||
|
||||
iOS 11.4 added AirPlay 2 in 2018. Although extensively covered by the
|
||||
media, it's not entirely clear what changes specifically Apple has made
|
||||
protocol-wise.
|
||||
|
||||
From captures of the traffic between an iOS device running iOS 12.2 and
|
||||
an AppleTV running tvOS 12.2.1, one can see that the communication on
|
||||
the main mirroring HTTP connection is encrypted after the initial
|
||||
handshake. This could theoretically be part of the new AirPlay 2
|
||||
protocol. The AppleTV running tvOS 12.2.1 identifies as
|
||||
AirTunes/380.20.1. When connecting from the same iOS device to an
|
||||
AppleTV 3rd generation (reporting as AirTunes/220.68), the communication
|
||||
is still visible in plain. From the log messages that the iOS device
|
||||
produces when connected to an AppleTV 3rd generation, it becomes
|
||||
apparent that the iOS device is treating this plain protocol as the
|
||||
legacy protocol (as originally introduced with iOS 9). Further research
|
||||
showed that at the moment, all available third-party AirPlay mirroring
|
||||
receivers (servers) are using this legacy protocol, including the open
|
||||
source implementation of dsafa22, which is the base for RPiPlay. Given
|
||||
Apple considers this a legacy protocol, it can be expected to be removed
|
||||
entirely in the future. This means that all third-party AirPlay
|
||||
receivers will have to be updated to the new (fully encrypted) protocol
|
||||
at some point.
|
||||
|
||||
More specifically, the encryption starts after the pair-verify handshake
|
||||
completed, so the fp-setup handshake is already happening encrypted.
|
||||
Judging from the encryption scheme for AirPlay video (aka HLS Relay),
|
||||
likely two AES GCM 128 ciphers are used on the socket communication (one
|
||||
for sending, one for receiving). However, I have no idea how the keys
|
||||
are derived from the handshake data.
|
||||
|
||||
Reference in New Issue
Block a user