Skip to content

Update sideloaded text tracks connector readme #126

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ceyhun-o wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Update sideloaded text tracks connector readme #126

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 19 additions & 19 deletions Code/Sideloaded-TextTracks/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# THEOplayer ❤️ Sideloaded Subtitles

THEOplayer-Connector-SideloadedSubtitle brings sideloaded subtitle support to THEOplayer's iOS SDK.
THEOplayer-Connector-SideloadedSubtitle brings sideloaded subtitle support to iOS SDK.

By using an experimental API from THEOplayer, this library aims to achieve sideloaded TextTrack support across all Apple devices and all media playback scenarios.
By using an experimental API from THEOplayer, this library aims to achieve sideloaded text track support across all Apple devices and all media playback scenarios.

The presented technique is superior to any other similar options as it brings sideloading support to Airplay, Picture-in-Picture and DRM playback (and their combinations) and is out-of-the-box compatible with the TextTrackStyling API from THEOplayer.
The presented technique is superior to any other similar options as it brings sideloading support to AirPlay, Picture-in-Picture and DRM playback (and their combinations) and is out-of-the-box compatible with the `TextTrackStyling` API from THEOplayer.

This connector is intended to be used for subtitle text tracks. For metadata text tracks, such as WebVTT thumbnails, starting from v7.10 the core THEOplayerSDK will handle the sideloading without the help of this connector.
This connector is intended to be used for subtitle text tracks specific with the `kind` `.subtitles`. For `.metadata` text tracks, such as WebVTT thumbnails, starting from v7.10 the iOS SDK handles the sideloading without the help of this connector.

## Installation

Expand All @@ -28,7 +28,7 @@ import THEOplayerConnectorSideloadedSubtitle

After importing the module, a new API will be available on the THEOplayer instance called `setSourceWithSubtitles(source: SourceDescription?)`

If you use this method to set a source, THEOplayer will be able to present sideloaded subtitles (via the `TextTrackDescription` already known from THEOplayer 4.x) configured with your `SourceDescription`, e.g.:
If you use this method to set a source, the player will be able to present sideloaded subtitles via the `TextTrackDescription` configured with your `SourceDescription`, e.g.:

```swift
public static var sourceWithSideloadedTextTrack : SourceDescription {
Expand All @@ -40,13 +40,13 @@ public static var sourceWithSideloadedTextTrack : SourceDescription {
```swift
theoplayer.setSourceWithSubtitles(source: sourceWithSideloadedTextTrack)
```
**NOTE:** Replace every `theopalyer.source = newSource` call with `theoplayer.setSourceWithSubtitles(source: newSource)` in your application if you intend to use (at least once) subtitle sideloading. Even if you don't have sideloaded subtitles on some of your sources. **Combining both approaches could lead to unexpected behavior.**
**NOTE:** Replace every `theoplayer.source = newSource` call with `theoplayer.setSourceWithSubtitles(source: newSource)` in your application if you intend to use (at least once) subtitle sideloading. Even if you don't have sideloaded subtitles on some of your sources. **Combining both approaches could lead to unexpected behavior.**

## Limitations
1. THEOplayer-Connector-SideloadedSubtitle relies on an experimental API from THEOplayer which is subject to change. **Always use matching THEOplayer and Connector versions.**
2. **Supported formats are: iOS-compatible `WebVTT` subtitles and SRT** via this connector today.
3. The presented **label within THEOplayer is auto-generated by the operating system** based on the language code and **the configured label will be disregarded**. (e.g. "eng" --> "English" (if the device language is English; "eng" --> "Engels", if the device language is Dutch). If an invalid language code is specified, the system will use that one as label. (e.g. "MyCustomLanguageCode" --> "MyCustomLanguageCode")
4. The `isDefault: true` setting will be not taken into account on sideloaded TextTracks as it could lead to invalid source (if the stream already contains default subtitles). Enabling the desired sideloaded subtitle can be achieved via `ADD_TRACK` listener.
1. THEOplayer-Connector-SideloadedSubtitle relies on an experimental API from the player which is subject to change. **Always use compatible player and connector versions.** Any major version of the connector is intended to be compatible with any player version within the same major version (e.g. connector 11.0.3 is compatible with player 11.5.0).
2. **Supported formats are: iOS-compatible WebVTT subtitles and SRT** via this connector today.
3. The presented **label within THEOplayer is auto-generated by the operating system** based on the language code and **the configured label will be disregarded**. (e.g. "eng" -> "English" (if the device language is English; "eng" -> "Engels", if the device language is Dutch). If an invalid language code is specified, the system will use that one as label. (e.g. "MyCustomLanguageCode" -> "MyCustomLanguageCode")
4. The `isDefault: true` setting will be not taken into account on sideloaded text tracks as it could lead to invalid source (if the stream already contains default subtitles). Enabling the desired sideloaded subtitle can be achieved via `ADD_TRACK` listener.

```swift
theoplayer.textTracks.addEventListener(type: TextTrackListEventTypes.ADD_TRACK) { ev in
Expand All @@ -63,10 +63,10 @@ theoplayer.textTracks.addEventListener(type: TextTrackListEventTypes.ADD_TRACK)

It is possible to shift the presentation of the cues by using the extended `SSTextTrackDescription` class instead of `TextTrackDescription`. It will provide an additional property `vttTimestamp` which allows to specify a [X-TIMESTAMP-MAP](https://datatracker.ietf.org/doc/html/draft-pantos-http-live-streaming-22#section-3.5).

The X-TIMESTAMP-MAP is optional and can be omitted from the WebVTT file and in general, it is not required. It is **only necessary if you use a WEBVTT file inside an HLS subtitle rendition** and you try to achieve subtitle synchronization.
The `X-TIMESTAMP-MAP` is optional and can be omitted from the WebVTT file and in general, it is not required. It is **only necessary if you use a WebVTT file inside an HLS subtitle rendition** and you try to achieve subtitle synchronization.
And this is exactly the way how this connector operates.

The APIs below provide a way of adding or modifying the timestamp in case it is required by **mapping the PTS** (e.g. from a video rendition) **to a local time within the WEBVTT** file.
The APIs below provide a way of adding or modifying the timestamp in case it is required by **mapping the PTS** (e.g. from a video rendition) **to a local time within the WebVTT** file.

When converting different formats to WebVTT, this module does not add the timestamp by default.

Expand All @@ -84,7 +84,7 @@ public static var sourceWithSideloadedTextTrack : SourceDescription {
theoplayer.setSourceWithSubtitles(source: sourceWithSideloadedTextTrack)
```

If the source WebVTT file specified in the `SSTextTrackDescription` already contains the X-TIMESTAMP-MAP timestamp, then it can be accessed by using the `extractSourceTimestamp` method:
If the source WebVTT file specified in the `SSTextTrackDescription` already contains the `X-TIMESTAMP-MAP` timestamp, then it can be accessed by using the `extractSourceTimestamp` method:

```swift
let textTrack = SSTextTrackDescription(src: "https://sideloadedurl.com/subtitle.vtt", srclang: "language_code", isDefault: false, kind: .subtitles, label:"Label", format: .WebVTT)
Expand Down Expand Up @@ -121,28 +121,28 @@ textTrack.vttTimestamp = .init(pts: "900000", localTime: "00:00:00.000")

#### (Experimental) Shifting the presentation of the cues forward (later) by 5 seconds.

You can achieve this by adding values to the first digit of the PTS timestamp with localTime set to 0:
You can achieve this by adding values to the first digit of the PTS timestamp with `localTime` set to `0`:
```swift
let textTrack = SSTextTrackDescription(src: "https://sideloadedurl.com/subtitle.vtt", srclang: "language_code", isDefault: false, kind: .subtitles, label:"Label", format: .WebVTT)
// we assume here that the PTS in the video rendition is 900000, so 900000 + 500000 = 1400000
textTrack.vttTimestamp = .init(pts: "1400000", localTime: "00:00:00.000")
```

The timestamp is relative to the localTime, so the following settings will cancel each other out:
The timestamp is relative to the `localTime`, so the following settings will cancel each other out:
```swift
let textTrack = SSTextTrackDescription(src: "https://sideloadedurl.com/subtitle.vtt", srclang: "language_code", isDefault: false, kind: .subtitles, label:"Label", format: .WebVTT)
textTrack.vttTimestamp = .init(pts: "1400000", localTime: "00:00:05.000")
```

#### (Experimental) Shifting the presentation of the cues backwards (earlier) by 5 seconds.

You can achieve this by subtracting values to the first digit of the PTS timestamp with localTime set to 0:
You can achieve this by subtracting values to the first digit of the PTS timestamp with `localTime` set to `0`:
```swift
let textTrack = SSTextTrackDescription(src: "https://sideloadedurl.com/subtitle.vtt", srclang: "language_code", isDefault: false, kind: .subtitles, label:"Label", format: .WebVTT)
// we assume here that the PTS in the video rendition is 900000, so 900000 - 500000 = 400000
textTrack.vttTimestamp = .init(pts: "400000", localTime: "00:00:00.000")
```
or set the localTime to 5 seconds while keeping the timestamp value the same:
or set the `localTime` to 5 seconds while keeping the timestamp value the same:
```swift
let textTrack = SSTextTrackDescription(src: "https://sideloadedurl.com/subtitle.vtt", srclang: "language_code", isDefault: false, kind: .subtitles, label:"Label", format: .WebVTT)
// we assume here that the PTS in the video rendition is 900000
Expand All @@ -152,13 +152,13 @@ textTrack.vttTimestamp = .init(pts: "900000", localTime: "00:00:05.000")
**Note:** Every 100000 of PTS timestamp results in roughly 1 second in time mapping. Use this carefully as it is not officially documented.

#### Warnings
1. You should always use HLS subtitle rendition-compliant WEBVTT files which are in sync with your streams and you should not be using the time-shifting functionality.
1. You should always use HLS subtitle rendition-compliant WebVTT files which are in sync with your streams and you should not be using the time-shifting functionality.
2. Shifting cues backwards is only possible until the timestamp of the first cue (e.g. if your first cue has to be rendered at 10 sec, you should not shift the subtitle by 15 sec).
3. Shifting forward is possible, but considered as hack, and it is not a specification-compliant manifest entry. Use it at your own risk!

## Automatic synchronization

In case the presentation timestamp is not known, or it cannot be obtained, the `automaticTimestampSyncEnabled` property can be enabled to estimate the value of the PTS and automatically set the X-TIMESTAMP-MAP in the WEBVTT file.
In case the presentation timestamp is not known, or it cannot be obtained, the `automaticTimestampSyncEnabled` property can be enabled to estimate the value of the PTS and automatically set the `X-TIMESTAMP-MAP` in the WebVTT file.
This property is defined on the class `SSTextTrackDescription`.
Since the value is calculated retrospectively, it will cause a re-toggle of the enabled text track. This might cause a brief flash when a cue is present while the syncing takes effect.

Expand Down
Loading