eikek / meth

Commandline client for mediathekview

meth

The name “meth” is short for “mediathek”, a german online media portal. Meth is a command line tool that allows to query, play and download media from various state channels. It uses the files build by ZDF Mediathek. It cannot work without it.

Install

Clone the repo and build it using sbt:

sbt assembly

Then link the resulting executable target/scala-2.12/meth-* somewhere in your $PATH. For example:

ln -snf target/scala-2.12/meth-* ~/bin/meth

Usage

There are 3 main commands: query, play and download. The query command accepts a search string and prints the results. Play and download use external tools mpv and curl, respectively.

Query

The search syntax is quite simple. There are few basic forms:

1. key:value
2. value
3. dow:[sun|mon|tue|wed|thu|fri|sat]
4. >X or <X

The first matches when value is contained in the field named key. Available keys are

• station
• subject
• title
• description
• date
• time
• new

The second variant matches when value is contained in either subject or title. Then it is possible to find shows aired on a specific weekday. Form 4 can be used to match shows with a duration greater or smaller X minutes.

These forms can be combined with or, and and not:

• and: (& ...)
• or: (| ...)
• not: -...

The - in front of any term negates it. If many forms are separated by space they are combined via and.

Example: meth refugee '>10' dow:fri

Download

Instead of printing the results of a query, you can download these shows using the download command. By default this calls out to the curl program. This can be configured in the config file. See the reference.conf file in src/main/resources.

The command allows to specify a filename using a pattern. The pattern is simply a string with placeholders for data that is obtained from a show:

• station
• title
• subject
• year
• month
• day
• hour
• min
• date
• time

A placeholder is specified by putting it inside a %[]. Example: %[subject]-%[title]. Note that any placeholder can evaluate to the empty string if there is no value (i.e. if it is mispelled).

Quite often it is necessary to append or prepend text only if a placeholder evaluates to non-empty. This can be achieved by this syntax: %[subject|prepend-%[subject]-append]. This means “check if there is a subject value and if so evaluate the pattern after |”.

The default pattern is: %[subject|%[subject]-%]%[title]-%[date].

1. thetvdb.com

   Note: this is a very experimental feature.

   It is possible to extend the set of placeholder variables by searching the site http://thetvdb.com. This allows to use placeholders in the output file pattern with the information from thetvdb.com. This can be useful to get e.g. the episode number to generate file names that can be recognized by certain players like kodi.

   For this to work, you must register at thetvdb.com and add the apikey to the config file. Also the series id of the series to search must be supplied to the download command. At last there is only one way to search: using the date of a show as the firstAired date. But this must be selected explicitely using the --tvdb-firstaired option.

   All placeholders are prefixed by tvdb..

   Example: s%[year]e%[tvdb.airedEpisodeNumber]_%[tvdb.episodeName]

Configure

You can configure it via a file ~/.config/meth/meth.conf. Every setting there can be overriden by setting a system property. Use the environment variable METH_JAVA_OPTS for this.

Example: METH_JAVA_OPTS=-Dmeth.directory=/tmp/meth meth

The Config library is used. Please see its documentation fo find out more. Please see the default config at src/main/resources for all options.

Example

meth's Issues

[feature request]: download log for downloaded media

It would be great to have a download log similar to the one youtube-dl provides which "records" downloaded media and does not download it again, even if the media file is deleted/moved away. Currently meth re-downloads media which has been deleted or moved, but it would be very handy if it looked up already-downloaded files in a file and does not get them again.

errors while searching

Almost every time while searching, there are errors, the search crashes and I only get a stack trace. Example:

meth report station:orf

generates

[...]
[ORF] Ex-Kriegsreporterin über Kriegsverbrechen <00:08:08, Monday, April 18, 2022 at 9:51:29 PM Central European Summer Time>Exception in thread "main" javax.xml.stream.XMLStreamException: ParseError at [row,col]:[13,136]
Message: The reference to entity "support" must end with the ';' delimiter.
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.next(XMLStreamReaderImpl.java:652)
        at meth.settings$Settings.$anonfun$listUrl$1(settings.scala:67)
        at fs2.Task$.$anonfun$delay$1(Task.scala:191)
        at fs2.Task$.$anonfun$suspend$2(Task.scala:199)
        at fs2.util.Attempt$.apply(Attempt.scala:12)
        at fs2.Task$.$anonfun$suspend$1(Task.scala:199)
        at fs2.internal.Future.step(Future.scala:53)
        at fs2.internal.Future.listen(Future.scala:30)
        at fs2.internal.Future.runAsync(Future.scala:69)
        at fs2.internal.Future.run(Future.scala:79)
        at fs2.TaskPlatform$JvmSyntax.unsafeRun(TaskPlatform.scala:14)
        at meth.movielist$.listUrl$lzycompute(movielist.scala:17)
        at meth.movielist$.listUrl(movielist.scala:17)
        at meth.movielist$.$anonfun$isListCurrent$1(movielist.scala:81)
        at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
        at fs2.Task$.$anonfun$delay$1(Task.scala:191)
        at fs2.Task$.$anonfun$suspend$2(Task.scala:199)
        at fs2.util.Attempt$.apply(Attempt.scala:12)
        at fs2.Task$.$anonfun$suspend$1(Task.scala:199)
        at fs2.internal.Future.step(Future.scala:54)
        at fs2.internal.Future.listen(Future.scala:30)
        at fs2.internal.Future.runAsync(Future.scala:69)
        at fs2.internal.Future.run(Future.scala:79)
        at fs2.TaskPlatform$JvmSyntax.unsafeRun(TaskPlatform.scala:14)
        at meth.main$.main(main.scala:34)
        at meth.main.main(main.scala)

OS: Debian 11.3
meth: latest (meth-0.0.3)

increase log level for troubleshooting?

Some downloads seem to go okay, but download only files with approximately 600 bytes in size. Is there a possibility to increase the log level or have a log with more detailed errors for these cases?

[feature request] Media title as filenames

Is it possible to rename downloaded files according to their title as shown during searches, e.g.

title
Tatort: Wie einst Lilly--> Tatort: Wie einst Lilly.mp4

instead of

http://hrardmediathek-a.akamaihd.net/video/as/tatort/2017_11/171130201733_2017-11-29_20-55-02_Tatort__Wie_einst_Lilly_1280x720-50p-5000kbit.mp4 ?

Thanks in advance!

Updating movie list at beginning of bash script does not seem to work

I created a short bash script in order to execute meth periodically via a cron job and set auto-download = false in order to not update the file with every new line of the script.

Example lines in the script:

meth download --normalize false --pattern "%[date]_%[subject]_%[title]" 'Universum >40' station:orf
meth download --normalize false --pattern "%[date]_%[subject]_%[title]" 'Am Schauplatz' station:orf

The first line in the script is

meth update and I expected a movie list update and then queries/downloads with the new and updated file but instead there are these info messages before every download saying

Info: There is a newer filmlist version available.

I do not understand why I get those messages, surely the movie list file should up to date when the other meth queries start? It says so at the beginning:

Downloading movie list… ok
Unpacking movie list file… ok

Am I doing this wrong?

updating movie list crashes

A normal "meth update" does not work anymore and crashes meth as a whole.

Strack trace:

Downloading movie list… ok
Unpacking movie list file… Exception in thread "main" org.tukaani.xz.XZFormatException: Input is not in the XZ format
    at org.tukaani.xz.common.DecoderUtil.decodeStreamHeader(Unknown Source)
    at org.tukaani.xz.SingleXZInputStream.<init>(Unknown Source)
    at org.tukaani.xz.SingleXZInputStream.<init>(Unknown Source)
    at org.tukaani.xz.XZInputStream.<init>(Unknown Source)
    at org.tukaani.xz.XZInputStream.<init>(Unknown Source)
    at org.tukaani.xz.XZInputStream.<init>(Unknown Source)
    at org.tukaani.xz.XZInputStream.<init>(Unknown Source)
    at meth.movielist$.$anonfun$unpackFile$1(movielist.scala:103)
    at fs2.Task$.$anonfun$delay$1(Task.scala:191)
    at fs2.Task$.$anonfun$suspend$2(Task.scala:199)
    at fs2.util.Attempt$.apply(Attempt.scala:12)
    at fs2.Task$.$anonfun$suspend$1(Task.scala:199)
    at fs2.internal.Future.step(Future.scala:54)
    at fs2.internal.Future.listen(Future.scala:30)
    at fs2.internal.Future.runAsync(Future.scala:69)
    at fs2.internal.Future.run(Future.scala:79)
    at fs2.TaskPlatform$JvmSyntax.unsafeRun(TaskPlatform.scala:14)
    at meth.main$.main(main.scala:34)
    at meth.main.main(main.scala)

No changes have been made in the config, has been working for months.

Option for modifying the download command?

I was wondering how I could modify the download command in order to include an user agent. Simply adding the "--user-agent" directive in the meth.conf did not work unfortunately.

ORF titles are not being handled properly

Titles of ORF videos are not being handled properly, as seen in this mediathek issue. Over there, it apparently has been fixed, but meth does not show the correct titles unfortunately.

Examples:

meth 'universum >40 station:orf'
[ORF] Universum: Das könnte Sie auch interessieren: <00:45:54, Sunday, December 8, 2019 at 3:23:47 PM Central European Standard Time>
[ORF] Universum History: Das könnte Sie auch interessieren: <00:53:43, Friday, December 6, 2019 at 10:35:00 PM Central European Standard Time>

parse error in filmlists

While updating the filmlist with meth update I always get parsing errors:

Downloading movie list… Exception in thread "main" javax.xml.stream.XMLStreamException: ParseError at [row,col]:[13,136]
Message: The reference to entity "support" must end with the ';' delimiter.
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.next(XMLStreamReaderImpl.java:652)
        at meth.settings$Settings.$anonfun$listUrl$1(settings.scala:67)
        at fs2.Task$.$anonfun$delay$1(Task.scala:191)
        at fs2.Task$.$anonfun$suspend$2(Task.scala:199)
        at fs2.util.Attempt$.apply(Attempt.scala:12)
        at fs2.Task$.$anonfun$suspend$1(Task.scala:199)
        at fs2.internal.Future.step(Future.scala:53)
        at fs2.internal.Future.listen(Future.scala:30)
        at fs2.internal.Future.runAsync(Future.scala:69)
        at fs2.internal.Future.run(Future.scala:79)
        at fs2.TaskPlatform$JvmSyntax.unsafeRun(TaskPlatform.scala:14)
        at meth.movielist$.listUrl$lzycompute(movielist.scala:17)
        at meth.movielist$.listUrl(movielist.scala:17)
        at meth.movielist$.$anonfun$downloadList$1(movielist.scala:95)
        at fs2.Task$.$anonfun$delay$1(Task.scala:191)
        at fs2.Task$.$anonfun$suspend$2(Task.scala:199)
        at fs2.util.Attempt$.apply(Attempt.scala:12)
        at fs2.Task$.$anonfun$suspend$1(Task.scala:199)
        at fs2.internal.Future.step(Future.scala:54)
        at fs2.internal.Future.listen(Future.scala:30)
        at fs2.internal.Future.runAsync(Future.scala:69)
        at fs2.internal.Future.run(Future.scala:79)
        at fs2.TaskPlatform$JvmSyntax.unsafeRun(TaskPlatform.scala:14)
        at meth.main$.main(main.scala:34)
        at meth.main.main(main.scala)

It does not matter, which filmlist (I tried the one from a fallback url as well), it always crashes like this.

cannot download anything from ORF

meth does not download correctly from ORF, all it does is create files which are approximately 700 bytes in size.

Is there any kind of log which could give me a hint what's going on here or is similar to the known problem with MediathekView ?