bailli / opus2tonie

Commandline tool to create Tonie compatible file from Ogg opus file(s)

opus2tonie.py

---

Current state

At the time of writing the script can produce files that can be played by the Tonie box.

Usage

usage: opus2tonie.py [-h] [--ts TIMESTAMP] [--ffmpeg FFMPEG]
                     [--opusenc OPUSENC] [--bitrate BITRATE] [--cbr]
                     [--append-tonie-filename] [--no-tonie-header] [--info]
                     [--split]
                     SOURCE [TARGET]

Create Tonie compatible file from Ogg opus file(s).

positional arguments:
  SOURCE                input file or directory or a file list (.lst)
  TARGET                the output file name (default: 500304E0)

optional arguments:
  -h, --help            show this help message and exit
  --ts TIMESTAMP        set custom timestamp / bitstream serial
  --ffmpeg FFMPEG       specify location of ffmpeg
  --opusenc OPUSENC     specify location of opusenc
  --bitrate BITRATE     set encoding bitrate in kbps (default: 96)
  --cbr                 encode in cbr mode
  --append-tonie-filename
                        append [500304E0] to filename
  --no-tonie-header     do not write Tonie header
  --info                Check and display info about Tonie file
  --split               Split Tonie file into opus tracks

Firmware problems

The firmware of the Tonie box has drawbacks when it can connect to the internet via WiFi:

• Reading custom NFC tags will result in the deletion of the associated audio data.

• Reading an official Tonie that has custom content stored on the box will delete the custom content and download the original files again, unless the internal timestamps are identical

• On startup, or when idle, it will set the "hidden" FAT filesystem attribute for custom audio data files with incongruous internal timestamps. This enables "live" mode for these files and the NFC tag will always trigger playback from the beginning of the file (no matter whether another NFC tag was used in-between). You can only enter the offline mode of the box after startup, so to avoid this effect you need to block the box on your wireless network before connecting it to the batteries again.
  You may also plug an "extension cord" into the internal SD card slot to make it reachable without disconnecting the battery. Then you can eject the card and add new files or fix the FAT fs attribute while the box is in sleep mode.

When you use original Tonie figurines (or their NFC tag) the latter two issues can be circumvented by using the timestamp that is officially associated with the Tonie for the custom content. You can identify this timestamp through ./opus2tonie.py --info and generate your custom files with the optional --ts <timestamp> parameter.

Tonie header

The header format is roughly described here. However, a proto file is not provided.

To generate the python output run:

protoc --python_out=. tonie_header.proto

Input files

If you have ffmpeg and opusenc in your path (or specify their location) you can use any input files which ffmpeg can read. Otherwise you are limited to stereo 48 kHz opus files.

A list file (the extension must be .lst) can contain either relative or absolute files. Additionally, you can specify (short) text strings which will be synthesized with Google Cloud text2speech (see below)

text2speech

Lines starting with text: in a list file input will be sent to Google Cloud Text-to-Speech. You will need to have the librecaptcha package installed. Solving the captcha seems to take a while but it should be only necessary once (for a "session").

You can change the default settings at the top of the script (T2S_xxx variables). Possible values can be taken from here: https://cloud.google.com/text-to-speech

Some useful resources

• https://en.wikipedia.org/wiki/Ogg_page
• https://github.com/toniebox-reverse-engineering/toniebox/wiki/Audio-file-format
• https://www.xiph.org/ogg/doc/rfc3533.txt
• https://tools.ietf.org/html/rfc7845#section-4
• https://tools.ietf.org/html/rfc6716#section-3.1

opus2tonie's Issues

Can't generate acceptable opus files through this script

Thanks you for working on this! This is so much better and more useful than the full-blown teddy system. 🏅

Sadly I can't get it to generate files that my box accepts yet. 😿
If I split an existing file and create it from the individual opus files again, it seems to work, but I can't get opusenc (0.2 (using libopus 1.3.1-fuzzing)) to produce files your tool accepts.

If I feed in an arbitrary audio file, I see the exception

RuntimeError: Please add frame size for config value 15

If I naively add 15 here, the resulting files only turn the light of the box red.

Looking somewhat deeper into online resources I can't find a direct explanation of this config_value you extract from the files, so my questions at the moment are

• Should oggenc generate 15 or is there a way to force a different value? Did you miss to add a parameter to the command line and the defaults changed at some point?
• What should be the correct frame size for 15?
• What does the config_value even mean and where is it documented?

Inconsistent check for T2S?

I was trying to add some T2S to the conversion, but failed to find librecaptcha for HomeBrew (which is what I'm successfully using with opus2tonie, Python 3.10).
The import therefore fails (and requests is not even considered any longer it seems), but addition of a text: line to the .lst file results in an error

$ ./opus2tonie.py text.lst x
[01/37] text:Here it begins
Traceback (most recent call last):
  File "opus2tonie.py", line 1051, in <module>
    create_tonie_file(out_filename, files, args.no_tonie_header, args.user_timestamp,
  File "opus2tonie.py", line 669, in create_tonie_file
    handle = get_t2s_tempfile(ffmpeg, opusenc, fname[5:], bitrate, not cbr)
  File "opus2tonie.py", line 881, in get_t2s_tempfile
    t2s_base64 = get_t2s_base64_data(text)
  File "opus2tonie.py", line 865, in get_t2s_base64_data
    response = requests.post(T2S_ENDPOINT + token,
NameError: name 'requests' is not defined

It seems that get_t2s_base64_data() is called without get_t2s_token() having had the opportunity to fail the assertion...

Despite of this minor issue, opus2tonie has eventually resolved my problem of converting files back and forth on Apple Silicon, without having to go for mono or VMs. Many thanks for that :)

PS.: Is there a simple means to append tracks to an existing file?

Update _pb2.py to a more recent version

TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:

1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

ValueError: Precision not allowed in integer format specifier

Hello and thank u for your script again!
In the new version ive got an error.

python opus2tonie.py SOURCE
Traceback (most recent call last):
File "opus2tonie.py", line 1052, in
args.bitrate, args.cbr, args.ffmpeg, args.opusenc)
File "opus2tonie.py", line 658, in create_tonie_file
format_string = "[{{:0{}d}}/{:0{}d}] {{}}".format(pad_len, len(input_files), pad_len)
ValueError: Precision not allowed in integer format specifier

Please add some setup functionality like setup.py or similar

I would love to package this for Linux distros, but without some info about dependencies I can't.

Merge existing split .opus files instead of re-encoding them?

May we have a --merge option (the opposite of --split which doesn't touch the stream data) that just merges existing .opus files (after checking their encoding parameters are OK, of course), to avoid extra loss?

Deduct audio id / timestamp by fixed value to mark them as custom content

Within TeddyBench and TeddyCloud we detect custom encoded content if the audioid is below 0x50000000.
When encoding custom content we always deduct the audioid by 0x50000000.

Why?

We need a mark to detect custom content, as we are also maintaining the tonies.json to detect tonies by audioid. If people send us new SHA/audioids (report function in TeddyBench) we are otherwise unable to see it was official content or content encoded with a third party tool.
Creative Tonies are also a problem, but there is nothing we can change.

What is your opinion? Maybe you already have some marker we could detect?

Provide install script or dependencies

A dependency needs to be installed via
pip3 install protobuf
apt install ffmpeg opus-tools

Otherwise, the user will get an error:
ModuleNotFoundError: No module named 'google'`

Too many segments: 256 - max 255 allowed

Hello and thank you so much for your script!
I always get the error with one file too many segments , what does this mean exactly?
If i encode this file alone it works, even if the file is encoded first but if the file comes in after another one, then the error occurs

THANKS!