Coder Social home page Coder Social logo

open-m3u8's Introduction

Open M3U8 (working name)

Description

This is an open source M3U8 playlist parser and writer java library that attempts to conform to this specification:

http://tools.ietf.org/html/draft-pantos-http-live-streaming-16

Currently the functionality is more than sufficient for our needs. However, there is still a lot of work to be done before we have full compliance. Pull requests are welcome!

Rationale

We would like to give back to the open source community surrounding Android that has helped make iHeartRadio a success. By using the MIT license we hope to make this code as usable as possible.

Artifacts

We now have artifacts in Maven Central! Artifacts are typically built with Java 7.

Gradle

dependencies {
  compile 'com.iheartradio.m3u8:open-m3u8:0.2.4'
}

Maven

<dependency>
  <groupId>com.iheartradio.m3u8</groupId>
  <artifactId>open-m3u8</artifactId>
  <version>0.2.4</version>
</dependency>

Getting started

Important: The public API is still volatile. It will remain subject to frequent change until a 1.0.0 release is made.

Getting started with parsing is quite easy: Get a PlaylistParser and specify the format.

InputStream inputStream = ...
PlaylistParser parser = new PlaylistParser(inputStream, Format.EXT_M3U, Encoding.UTF_8);
Playlist playlist = parser.parse();

Creating a new Playlist works via Builders and their fluent with*() methods. On each build() method the provided parameters are validated:

TrackData trackData = new TrackData.Builder()
    .withTrackInfo(new TrackInfo(3.0f, "Example Song"))
    .withPath("example.mp3")
    .build();

List<TrackData> tracks = new ArrayList<TrackData>();
tracks.add(trackData);

MediaPlaylist mediaPlaylist = new MediaPlaylist.Builder()
    .withMediaSequenceNumber(1)
    .withTargetDuration(3)
    .withTracks(tracks)
    .build();

Playlist playlist = new Playlist.Builder()
    .withCompatibilityVersion(5)
    .withMediaPlaylist(mediaPlaylist)
    .build();

The Playlist is similar to a C style union of a MasterPlaylist and MediaPlaylist in that it has one or the other but not both. You can check with Playlist.hasMasterPlaylist() or Playlist.hasMediaPlaylist() which type you got.

Modifying an existing Playlist works similar to creating via the Builders. Also, each data class provides a buildUpon() method to generate a new Builder with all the data from the object itself:

TrackData additionalTrack = new TrackData.Builder()
    .withTrackInfo(new TrackInfo(3.0f, "Additional Song"))
    .withPath("additional.mp3")
    .build();

List<TrackData> updatedTracks = new ArrayList<TrackData>(playlist.getMediaPlaylist().getTracks());
updatedTracks.add(additionalTrack);

MediaPlaylist updatedMediaPlaylist = playlist.getMediaPlaylist()
    .buildUpon()
    .withTracks(updatedTracks)
    .build();

Playlist updatedPlaylist = playlist.buildUpon()
    .withMediaPlaylist(updatedMediaPlaylist)
    .build();

A PlaylistWriter can be obtained directly or via its builder.

OutputStream outputStream = ...

PlaylistWriter writer = new PlaylistWriter(outputStream, Format.EXT_M3U, Encoding.UTF_8);
writer.write(updatedPlaylist);

writer = new PlaylistWriter.Builder()
                 .withOutputStream(outputStream)
                 .withFormat(Format.EXT_M3U)
                 .withEncoding(Encoding.UTF_8)
                 .build();

writer.write(updatedPlaylist);

causing this playlist to be written:

#EXTM3U
#EXT-X-VERSION:5
#EXT-X-TARGETDURATION:3
#EXT-X-MEDIA-SEQUENCE:1
#EXTINF:3.0,Example Song
example.mp3
#EXTINF:3.0,Additional Song
additional.mp3
#EXT-X-ENDLIST

Currently, writing multiple playlists with the same writer is not supported.

Advanced usage

Parsing mode

The parser supports a mode configuration - by default it operats in a strict mode which attemps to adhere to the specification as much as possible.

Providing the parser a ParsingMode you can relax some of the requirements. Two parsing modes are made available, or you can build your own custom mode.

ParsingMode.LENIENT // lenient about everything
ParsingMode.STRICT // strict about everything

Example:

InputStream inputStream = ...
PlaylistParser parser = new PlaylistParser(inputStream, Format.EXT_M3U, Encoding.UTF_8, ParsingMode.LENIENT);
Playlist playlist = parser.parse();
if (playlist.hasMasterPlaylist() && playlist.getMasterPlaylist().hasUnknownTags()) {
    System.err.println(
        playlist.getMasterPlaylist().getUnknownTags());
} else if (playlist.hasMediaPlaylist() && playlist.getMediaPlaylist().hasUnknownTags()) {
    System.err.println(
        playlist.getMediaPlaylist().getUnknownTags());
} else {
    System.out.println("Parsing without unknown tags successful");
}

=======

Build

This is a Gradle project. Known compatible gradle versions:

  • 2.1
  • 2.4

Build and test via:

gradle build

The output can be found in the generated /build/libs/ dir.

Cobertura is configured to report the line coverage:

gradle cobertura

producing the coverage report at build/reports/cobertura/index.html

open-m3u8's People

Contributors

carlanton avatar ck1125 avatar derjust avatar eventh avatar ketankhairnar avatar raniz85 avatar sunglee413 avatar wopple avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

open-m3u8's Issues

KEYFORMATVERSIONS, KEYFORMAT attribute default handling

An application I made that uses open-m3u8 has a section where it reads a playlist then creates a new one based on edited values in the original list. I noticed the original playlist didn't have the attributes KEYFORMATVERSIONS and KEYFORMAT. But the new one has the attributes with their default values.

I read the specification "HTTP Live Streaming draft-pantos-http-live-streaming-16". It states in there that these two attributes are optional and their omission means the default values will be assumed. So newly created playlists don't need to include these unless their values are non-default.

So I was wondering if this behavior would be desired (new playlists won't include these attributes unless they were specified before or their values are non-default). If so I'll try to work on it in a new fork.

This isn't an important issue nonetheless. The rewritten playlist with these two attributes included worked fine.

Unable to parse tvg-logo

Trying to parse a m3u8 playlist using library, but exception thrown

com.iheartradio.m3u8.ParseException: bad format found for an EXT tag: EXTINF - #EXTINF:-1 tvg-logo="http://some_url.com/logo.png" group-title="INFO",* test IPTV

#EXTINF:-1 tvg-logo="http://some_url.com/logo.png" group-title="INFO",* test IPTV

I am using Linent mode

Missing BOM tag (UTF-8 support incomplete)

Hi,

I created an playlist with this library and enabled the Encoding.UTF8 flag. This playlist works fine in Winamp, but Foobar isn't able to play the UTF8 paths.

After searching on the internet I found a topic talking about the BOM tag (first three bytes of the file: EF BB BF), I tested this with Foobar and it actually works!

Anyway, Am i missing some configuration option or is this missing from the library?

For reference: http://www.hydrogenaud.io/forums/index.php?showtopic=93905

Edit: I temporarily solved it using:

OutputStream outputStream = new FileOutputStream(fullPlayListPath);
PlaylistWriter writer = new PlaylistWriter();
outputStream.write(new byte[] {(byte) 0xEF, (byte) 0xBB, (byte) 0xBF});
writer.write(outputStream, playlist, Format.EXT_M3U, Encoding.UTF_8);

Would ofcourse be nicer if the library would take care of this.

Separate validation from builders

Currently some of the builders perform validation to ensure the data is representative of the specification. The responsibility for validation should be its own component. The builders should only be concerned with building the type.

Problem parsing m3u8 file

Parsing the file with:

PlaylistParser parser = new PlaylistParser();
Playlist playlist = parser.parse(inputStream, Format.EXT_M3U, Extension.M3U8);

causes ParseException with message: "invalid attribute name: EXT-X-STREAM-INF - #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=101485, RESOLUTION=140x80"

m3u8 file content is:

EXTM3U

EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=101485, RESOLUTION=140x80

50k.mp4/chunk.m3u8

EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=151395, RESOLUTION=210x120

100k.mp4/chunk.m3u8

EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=300555, RESOLUTION=308x176

200k.mp4/chunk.m3u8

EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=500025, RESOLUTION=470x268

400k.mp4/chunk.m3u8

EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=1007569, RESOLUTION=756x432

900k.mp4/chunk.m3u8

What should I do then?

Live playlist

How can you define a live playlist ? (no #EXT-X-ENDLIST at the end)

Bug in MasterPlaylist constructor?

Please checkout below 2 lines from the constructor of com.iheartradio.m3u8.data.MasterPlaylist.
mPlaylists = playlists == null ? Collections.emptyList() : Collections.unmodifiableList(playlists);
mIFramePlaylists = playlists == null ? Collections.emptyList() : Collections.unmodifiableList(iFramePlaylists);

If one forgets to add IFrameStreamInfo list while creating MasterPlaylist. It throws runtime NPE

The FORCED attribute present when the TYPE is not SUBTITLES

According to the HLS specification, also linked in the documentation, the FORCED attribute should only be present with subtitles: "The FORCED attribute MUST NOT be present unless the TYPE is SUBTITLES."
https://tools.ietf.org/html/draft-pantos-http-live-streaming-16

However, when writing an m3u8 file, it is present with audio as well:
#EXT-X-MEDIA:LANGUAGE="eng",AUTOSELECT=YES,FORCED=NO,TYPE=AUDIO,URI="...",GROUP-ID="audio_aac",DEFAULT=YES,NAME="eng"

Version: 0.2.6

EXT-X-PROGRAM-DATE-TIME support integration

Hi.
I used this library for a small application I made. The m3u8 files I was downloading had the "ext-x-program-date-time" tag. I added some code so that open-m3u8 supports the tag in strict parsing mode.
I'm not sure how to commit it though. I've never used github. I couldn't find any option to make a new branch or anything. So would anyone help guiding on what I should do next?

parser.parse()

PlaylistParser parser = new PlaylistParser(inputStream , Format.EXT_M3U, Encoding.UTF_8);
Playlist playlist = parser.parse();
parse() method throw EOFExcention,how can i fix it?

Thanks & C# version

Hi, I just wanted to say thank you for making this library available. It's always nice to see people (especially companies) giving back to the open source community.

I also wanted to let you know that I've forked this project and translated it into C#, targeting .NET Standard 2.0. I've tried to provide attribution and references back to this repository. Feel free to take a look and let me know if you'd like to see any more/different attribution.

https://github.com/bzier/open-m3u8

Thanks again,
-Brian

unsupported ext tag detected: EXT-X-INDEPENDENT-SEGMENTS

Hi.
I've run across a new issue. Maybe just evolution of m3u8 playlists, but a new tag that's not supported.

com.iheartradio.m3u8.ParseException: unsupported ext tag detected: EXT-X-INDEPENDENT-SEGMENTS - #EXT-X-INDEPENDENT-SEGMENTS
at com.iheartradio.m3u8.ParseException.create(ParseException.java:35)
at com.iheartradio.m3u8.ExtendedM3uParser.parse(ExtendedM3uParser.java:66)
at com.iheartradio.m3u8.PlaylistParser.parse(PlaylistParser.java:115)

You can get the playlist that has these tags here.
https://ngcstatpack-a.akamaihd.net/Nat_Geo_Channels/896/599/1509829223445/Long_Road_101_HD_Clean_AUTH_movie.m3u8

I will probably work on integrating this is my clone so if you don't have anything by the time I do I'll post my edits for you.

CHANNELS attribute not supported in EXT-X-MEDIA tag

All EXT-X-MEDIA attributes except CHANNELS seems to be supported.

The specification describes the CHANNELS attribute as following:
CHANNELS

  The value is a quoted-string that specifies an ordered,
  "/"-separated list of parameters.  If the TYPE attribute is AUDIO
  then the first parameter is a count of audio channels expressed as
  a decimal-integer, indicating the maximum number of independent,
  simultaneous audio channels present in any Media Segment in the
  Rendition.  For example, an AC-3 5.1 rendition would have a
  CHANNELS="6" attribute.  No other CHANNELS parameters are
  currently defined.

  All audio EXT-X-MEDIA tags SHOULD have a CHANNELS attribute.  If a
  Master Playlist contains two renditions encoded with the same
  codec but a different number of channels, then the CHANNELS
  attribute is REQUIRED; otherwise it is OPTIONAL.

Allow for variable size initialization vectors

Currently only 16-octet and 32-octet initialization vectors are allowed. User code should be able to use arbitrarily strong encryption. To that end, initialization vectors should be of variable size.

Handle end of playlist properly

The specification has an EXT-X-ENDLIST tag which delimits the end of a playlist. The parser should stop reading from the InputStream when this tag is detected and return the parsed playlist.

In both cases of EXT-X-ENDLIST or reaching the end of the InputStream, the parser should not close the InputStream. This is to support reading multiple playlists from a single stream and to not violate the principle of least astonishment.

Bypass unsupported Tag Exceptions through config if implementation pending

I came across situation where one of the m3u8 had tag EXT-X-PROGRAM-DATE-TIME which is part of spec [https://tools.ietf.org/html/draft-pantos-http-live-streaming-16#section-4.3.2.6]

Since library is not complete it ends up throwing exception. Adding parsing for all such tags is going to take a while. Till then is there better alternative than throwing exception?

Can one bypass that [through config maybe]? Can you help with example of LENIENT mode.

Here us sample m3u8 which fails to parse

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-ALLOW-CACHE:NO
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-TARGETDURATION:11
#EXT-X-PROGRAM-DATE-TIME:1970-01-01T00:00:00Z
#EXTINF:10.010010,no-desc
Fragments(video=0,format=m3u8-aapl-v3,audiotrack=aac_UND_2_127)
#EXTINF:10.010010,no-desc
Fragments(video=100100100,format=m3u8-aapl-v3,audiotrack=aac_UND_2_127)
#EXTINF:10.010010,no-desc
Fragments(video=200200200,format=m3u8-aapl-v3,audiotrack=aac_UND_2_127)
#EXTINF:10.010010,no-desc
Fragments(video=300300300,format=m3u8-aapl-v3,audiotrack=aac_UND_2_127)

AES-encryption

I'm having trouble getting encryption part to work.

  • how can I set it without the KEYFORMAT and KEYFORMATVERSION ? Or what should it be set to ?
    currently our EXT-KEY looks like this which works.
#EXT-X-KEY:METHOD=AES-128,URI="https://domain/plist/Cn4/3/d628933e949ed2285c1a8abb1573c66/47406/index.key" 

But when I tried generating same playlist with open-m3u8 I could not get it to play.
I've used #EXT-X-KEY:KEYFORMATVERSIONS="1" and KEYFORMAT="identity".

  • how can you add multiple keys for tracks ? (I've done a loop and I've set key for every track/chunk but only one was at the top.

Thanks.

Ignore wrong durations

I faced the case when a duration in EXTINF tag was given as negative integer (-1), because it was a list of infinite audio streams.
(Here the link to the M3U list of the internet radio: http://promodj.com/radio/channels-hq.m3u)

#EXTM3U
#EXTINF:-1,TOP 100
http://radio.promodj.com:8000/top100-192
#EXTINF:-1,Channel 5
http://radio.promodj.com:8000/channel5-192
#EXTINF:-1,Klubb
http://radio.promodj.com:8000/klubb-192

So I've got "com.iheartradio.m3u8.ParseException: bad format found for an EXT tag".

Specification talks about positive integers and floating points only, but real life dictates own rules. Maybe it's better just to ignore wrong duration?

Version 0.1.2

Support for custom EXT-X-*, and EXT-X-DATERANGE

Hi,
I'm very interested in your library, and was wondering if there are plans for supporting custom EXT-X tags. I'm specifically interested in the EXT-X-CUE tags for scte-35 markers, which are not in the pantos spec, but which the industry widely supports. Alternatively, Pantos just introduced the EXT-X-DATERANGE tag with their newest version: https://tools.ietf.org/html/draft-pantos-http-live-streaming-19#section-4.3.2.7.1

which essentially adds this functionality. If this is on your roadmap, I'd love to know!

Spaces in TARGETDURATION and MEDIA-SEQUENCE tags

I've observed spaces in generated m3u8 from CDNs for these tags. Note the space after colon. Do we need to handle this as part of library? Spec doesn't say whether space is allowed.

should be #EXT-X-TARGETDURATION:<s>  but seen as #EXT-X-TARGETDURATION: <s>
should be #EXT-X-MEDIA-SEQUENCE:<number> but seen as #EXT-X-MEDIA-SEQUENCE: <number>

For now I've handled with below change in Constants.java.

public static final Pattern EXT_X_TARGETDURATION_PATTERN = Pattern.compile("^#" + EXT_X_TARGETDURATION_TAG + EXT_TAG_END + OPTIONAL_WHITESPACE_REGEX +"(" + INTEGER_REGEX + ")"+OPTIONAL_WHITESPACE_REGEX+"$");
public static final Pattern EXT_X_MEDIA_SEQUENCE_PATTERN = Pattern.compile("^#" + EXT_X_MEDIA_SEQUENCE_TAG + EXT_TAG_END + OPTIONAL_WHITESPACE_REGEX +"(" + INTEGER_REGEX + ")"+OPTIONAL_WHITESPACE_REGEX+"$");

Where regex for optional white space is as below

private static final String OPTIONAL_WHITESPACE_REGEX = "\\s*?";

Let me know if this makes sense. There is no harm if no spaces exist. regex takes care of that case too.

Lenient mode

Create a way to specify lenient mode which ignores unsupported tags rather than throwing an exception.

IV in EXT-X-KEY is corrupted

Because Integer.toHexString is used negative bytes will be prefixed with ffffff and bytes between 0 and 15 will only be written as one digit, this causes the IV in the playlist to be corrupt.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.