MPV OSD parses fontconfig incorrectly and loads Emojione Color by default about emojione-color-font

[13rac1]

What is your default font?

$ fc-match -s sans | head -n3
Vera.ttf: "Bitstream Vera Sans" "Roman"
EmojiOneColor-SVGinOT.ttf: "Emoji One Color" "Regular"
DejaVuSans.ttf: "DejaVu Sans" "Book"

I've re-enabled the 0-9 glyphs required to make the numpad emoji function, which will be a problem if you are using the original fontconfig.

[DistantThunder]

This is my output:

fc-match -s sans | head -n3                                                                                                          
Vera.ttf: "Bitstream Vera Sans" "Roman"
EmojiOneColor-SVGinOT.ttf: "Emoji One Color" "Regular"
DejaVuSans.ttf: "DejaVu Sans" "Book"

[13rac1]

Vera has the same number glyphs as DejaVu, so it's odd MPV would use the second font on the list. The problem is similar to what happened with VLC using the original fontconfig in #5. Firefox has an issue caching the the fontlist causing similar issues #31. Does this problem happen anywhere else?

[DistantThunder]

Yes, I just noticed this on Skype:

http://i.imgur.com/urmPD37.png

$ cat ~/.config/fontconfig/fonts.conf

<?xml version='1.0'?>
<!DOCTYPE fontconfig SYSTEM 'fonts.dtd'>
<fontconfig>
 <match target="font">
  <edit mode="assign" name="rgba">
   <const>none</const>
  </edit>
 </match>
 <match target="font">
  <edit mode="assign" name="hinting">
   <bool>true</bool>
  </edit>
 </match>
 <match target="font">
  <edit mode="assign" name="hintstyle">
   <const>hintfull</const>
  </edit>
 </match>
 <match target="font">
  <edit mode="assign" name="antialias">
   <bool>true</bool>
  </edit>
 </match>
</fontconfig>

$ tree ~/.config/fontconfig/

/home/$USER/.config/fontconfig/
├── conf.d
│   └── 56-emojione-color.conf
├── fonts.conf
├── fonts.conf.bak
└── fonts.conf.xz

[13rac1]

Quick solution(workaround): Subset the font to remove the 0-9 glyphs using nototools/subset.py.

Skype: Skype is using the wrong fonts? Those characters aren't in this font. Looks like yet another caching issue.

MPV: What command line or options are you using to display the FPS? There aren't any problems on my machine displaying the timestamp in mpv:

[supgaf]

How do i use nototools?
I'm not good with this kind of thing, sorry.

[13rac1]

@supgaf Are you having this same issue? Are you on Arch? Do you have Bitstream Vera installed?

[supgaf]

@eosrei Yes, is the exactly same issue.
I'm on OpenSuse Tumbleweed and yeah Bitstream Vera is installed.

[13rac1]

Can you tell me the command line (or GUI?) you are using? I want to reproduce this.

[supgaf]

I'm just running MPV without any option.

[13rac1]

I've installed the current Launchpad repo version from: https://launchpad.net/~mc3man/+archive/ubuntu/mpv-tests

mpv git-449b948 (C) 2000-2016 mpv/MPlayer/mplayer2 projects
 built on Thu May 19 11:12:22 UTC 2016

But I'm not having this issue:

Are you compiling mpv? Do you have any custom font configs? This issue is clearly caused by mpv loading the incorrect font.

Edit: I doubt this has anything to do with the specific Linux distribution

[supgaf]

Are you compiling mpv?

No, I'm using the package from the Opesuse repositories.(0.17.0-1.2)

Do you have any custom font configs?

Yes, infinality-ultimate.

[13rac1]

Yes, infinality-ultimate.

IDK for sure, but this may be the issue. I don't have infinality on any of my test machines. Is it possible to test w/o it? I'm not able to support all these advanced fontconfigs some people have, but I'm happy to add additional details to the readme if there is anything to fix this. Hopefully we can work out a infinality-ultimate configuration to put in the example fontconfig directory!

Quick Temporary Workaround: Install the font manually without the provided fontconfig. It'll be overridden in a number of places, but shouldn't be the primary number font for mpv any longer.

[supgaf]

I've tested without infinality, but it not solved the problem.

[13rac1]

I recommend going with the "Quick temporary workaround" for now then. I'll try OpenSUSE in a VM as as soon as I can, but I'm not sure what other solutions there are for you at this point since I cannot reproduce the problem.

[DistantThunder]

Skype: Skype is using the wrong fonts? Those characters aren't in this font. Looks like yet another caching issue.

Maybe, I always run the fc-cache -v -f command after installing or removing a font. The Skype problem disappeared anyway so it must have been what you said.

MPV problem persist though.

MPV: What command line or options are you using to display the FPS? There aren't any problems on my machine displaying the timestamp in mpv:

osd-msg1='FPS: ${estimated-vf-fps}\nHWDEC_ACTIVE: ${hwdec-detected}'
osd-font='Fantasque Sans Mono:antialias=true:autohint=true'
osd-font-size=16

I tried some more advanced options and verbose with MPV and I may be getting close:
mpv -v --osd-font "Ubuntu Regular" Vidéos/DOOM\ –\ Guns,\ Demons,\ Speed.mp4

global] config path: 'fonts' -/-> '/home/$USER/.config/mpv/fonts'
[global] config path: 'fonts' -/-> '/home/$USER/.mpv/fonts'
[global] config path: 'fonts' -/-> '/etc/mpv/fonts'
[osd/libass] Shaper: FriBidi 0.19.7 (SIMPLE) HarfBuzz-ng 1.2.7 (COMPLEX)
[global] config path: 'subfont.ttf' -/-> '/home/$USER/.config/mpv/subfont.ttf'
[global] config path: 'subfont.ttf' -/-> '/home/$USER/.mpv/subfont.ttf'
[global] config path: 'subfont.ttf' -/-> '/etc/mpv/subfont.ttf'
[global] config path: 'fonts.conf' -/-> '/home/$USER/.config/mpv/fonts.conf'
[global] config path: 'fonts.conf' -/-> '/home/$USER/.mpv/fonts.conf'
[global] config path: 'fonts.conf' -/-> '/etc/mpv/fonts.conf'
[osd/libass] Setting up fonts...
[osd/libass] Using font provider fontconfig
[osd/libass] Done.
[osd/libass] fontselect: (Ubuntu Regular, 400, 0) -> /home/$USER/.local/share/fonts/EmojiOneColor-SVGinOT.ttf, 0, EmojiOneColorSVGinOT
[osd/libass] Glyph 0x46 not found, selecting one more font for (Ubuntu Regular, 400, 0)
[osd/libass] fontselect: (Ubuntu Regular, 400, 0) -> /usr/share/fonts/TTF/Vera.ttf, 0, BitstreamVeraSans-Roman
[global] config path: 'fonts' -/-> '/home/$USER/.config/mpv/fonts'
[global] config path: 'fonts' -/-> '/home/$USER/.mpv/fonts'
[global] config path: 'fonts' -/-> '/etc/mpv/fonts'
[osd/libass] Shaper: FriBidi 0.19.7 (SIMPLE) HarfBuzz-ng 1.2.7 (COMPLEX)
[global] config path: 'subfont.ttf' -/-> '/home/$USER/.config/mpv/subfont.ttf'
[global] config path: 'subfont.ttf' -/-> '/home/$USER/.mpv/subfont.ttf'
[global] config path: 'subfont.ttf' -/-> '/etc/mpv/subfont.ttf'
[global] config path: 'fonts.conf' -/-> '/home/$USER/.config/mpv/fonts.conf'
[global] config path: 'fonts.conf' -/-> '/home/$USER/.mpv/fonts.conf'
[global] config path: 'fonts.conf' -/-> '/etc/mpv/fonts.conf'
[osd/libass] Setting up fonts...
[osd/libass] Using font provider fontconfig
[osd/libass] Done.
[osd/libass] fontselect: (Ubuntu Regular, 400, 0) -> /home/$USER/.local/share/fonts/EmojiOneColor-SVGinOT.ttf, 0, EmojiOneColorSVGinOT

Somehow, it looks like the way to specify fonts have changed in MPV. It ends up selecting EmojiOneColor when it shouldn't?

Note: MPV will behave exactly the same way no matter what font name I give it, whether it actually exists or not doesn't seem to be relevant.

[13rac1]

I'm quite surprised how many programs incorrectly use the Linux fontconfig system.

Running mpv -v on my machine shows it selecting EmojiOne first also, although it's still rendering correctly somehow. I may just remove the 0-9 digits for right now since they are causing far more issues than the keypad numbers ligatures are worth.

Per the debug log, this is an issue with: https://github.com/libass/libass It isn't loading fonts in the order specified in the fontconfig.

[DistantThunder]

Running mpv -v on my machine shows it selecting EmojiOne first also, although it's still rendering correctly somehow. I may just remove the 0-9 digits for the timebeing since they are causing far more issues than the keypad numbers are worth.

I'm certain I only have the release version but could you please confirm that we do have the same font file?
$sha256sum ~/.local/share/fonts/EmojiOneColor-SVGinOT.ttf

2caeedf5fc7b3e79c4c38c3f06c0a565f3578a4753da6312df10b07eedab61e4 /home/$USER/.local/share/fonts/EmojiOneColor-SVGinOT.ttf

[13rac1]

Ah ha! I was using the PPA version, which hasn't been updated with the 0-9 digits yet. Oops! No problem though, I believed the problem existed. Either way, this is an issue with libass. It is not sorting the fonts correctly according to the fontconfig settings.

The exact issue is caused by this section of 56-emojione-color.conf:

  <match>
    <!-- If the requested font is serif -->
    <test qual="any" name="family">
        <string>serif</string>
    </test>
    <!-- Make Bitstream Vera Serif the first result -->
    <edit name="family" mode="prepend_first">
      <string>Bitstream Vera Serif</string>
    </edit>
    <!-- Followed by Emoji One Color -->
    <edit name="family" mode="prepend_first">
      <string>Emoji One Color</string>
    </edit>
  </match>

I tested and retested the provided fontconfig for hours when I first wrote it. Fontconfig documentation is here: https://www.freedesktop.org/software/fontconfig/fontconfig-user.html

If I flip them, mpv works correctly, but fc-match and therefore everything else, does not:

  <match>
    <!-- If the requested font is serif -->
    <test qual="any" name="family">
        <string>serif</string>
    </test>
    <!-- Followed by Emoji One Color -->
    <edit name="family" mode="prepend_first">
      <string>Emoji One Color</string>
    </edit>
    <!-- Make Bitstream Vera Serif the first result -->
    <edit name="family" mode="prepend_first">
      <string>Bitstream Vera Serif</string>
    </edit>
  </match>

$ fc-match sans -s | head -n 3
Vera.ttf: "Bitstream Vera Sans" "Roman"
DejaVuSans.ttf: "DejaVu Sans" "Book"
DejaVuSans-Bold.ttf: "DejaVu Sans" "Bold"

Any ideas? Seems like a libass bug at this point... although definitely arguable that the fontconfig config is unclear about how it should work. You'd expect the second "prepend_first" to override the first, but it doesn't work that way. This why I wrote the "unit tests" to confirm the configuration.

I really would rather not disable the 0-9 ligatures to workaround a bug in libass. Any other ideas?

[DistantThunder]

I think there's no way but disabling the digits glyphs as a hot patch for people that have this problem and submitting a bug against libass.

I tried using nototools's subset script with this CLI but it failed it seems:

$python2.7 subset.py ~/.local/share/fonts/EmojiOneColor-SVGinOT.ttf EmojiOneColor-SVGinOT_stripped.ttf exclude=0123456789

But the resulting font looked like it included those characters instead... ?

[supgaf]

So, on Opesuse i can set the font on YaST, but i hadn't time to check in how way this influence the other fonts, but if i do so, mpv stop using the Emoji font. I need to mark the never use other fonts box too.
The Emojis still working.

[DistantThunder]

@supgaf

I need to mark the never use other fonts box too.

I think that's what does the trick.

Can you try an $fc-match -f -v command and report?

[supgaf]

@DistantThunder
I've tried $fc-match -f -v, but it doesn't return anything.
So, i tried just $fc-match.
That's the result before making the change on YaST:
Vera.ttf: "Bitstream Vera Sans" "Roman"
And that's after:
OpenSans-Regular.ttf: "Open Sans" "Regular"

[13rac1]

@DistantThunder and @supgaf The "unit tests" I created for this can check to make sure the config works as designed. May be of use in testing options.

The command to test the font list for "sans-serif":

fc-match -s sans

Clear the cache manually after changing any files in /etc/fonts/conf.d using:

fc-cache -f -v

I think the issue is the mpv font system is interpreting "append_first" incorrectly. mpv seems to work like you expect "append_first" to work, but that's not how fontconfig uses it.

[DistantThunder]

@supgaf
Thanks. I indeed confused fc-match with fc-cache.

What gives @eosrei test though? (fc-cache -f -v after making the change, and then fc-match -s sans)

[13rac1]

@DistantThunder Sorry, I don't understand your question. Can you rephrase?

[supgaf]

@DistantThunder
Here you go.
https://ghostbin.com/paste/oeg92

[DistantThunder]

@eosrei Oh, Sorry, I was talking to @supgaf. But I haven't find a way to remove the numeric glyphs from EmojiOneColor. Like I wrote earlier, the Nototools script doesn't work as intended...

@supgaf Thanks! I'll try that font and see what gives, although I suspect like @eosrei said this just modify fontconfig's file to give priority the prefered font in all situation without considering any of the prepend-first stuff.

However I suspect that maybe MPV's --osd-font remains broken... It will use Open Sans digits over EmojiOne's but you may still be unable to select any other font as long as libass is misinterpreting things.

[13rac1]

@DistantThunder If you just want it to work, use v1.0-beta3. It doesn't have the 0-9 ligatures.

I wouldn't be surprised if there is a better solution for the provided 56-emojione-color.conf config, but at this point, I've already put at least 10hrs in that one. I think I've now officially spent more time on font configuration bugs than making the fonts in the first place 😞

[DistantThunder]

@eosrei Haha, yeah, I'll just use the pre-release font, they worked without a hitch before. Are you going to file a bug against libass? I could try but I don't have much knowledge in fonts and font-config.

[13rac1]

@DistantThunder I don't have time right now to make a detailed bug report. I'll leave this open until there is a solution. Hopefully someone else can do it in the meantime.