xuanwo / go-locale

Cross platform locale detection for Golang

go-locale

go-locale is a Golang lib for cross-platform locale detection.

OS Support

Support all OS that Golang supported, except android:

• aix: IBM AIX operating system
• android (have no idea on this)
• darwin: Drawin, Mac OS X
• dragonfly: DragonFly BSD
• freebsd: FreeBSD
• hurd: GNU Hurd
• illumos
• js: JavaScript runtime, WebAssembly
• Linux: Ubuntu, CentOS, RHEL, Archlinux...
• nacl: Native Client
• netbsd: NetBSD
• openbsd: OpenBSD
• plan9: Plan 9 from Bell Labs
• solaris: Solaris
• windows: Windows
• zos: z/OS

POSIX Compatible Systems

• Lookup env LANGUAGE
• Lookup env LC_ALL
• Lookup env LC_MESSAGES
• Lookup env LANG
• Read file $XDG_CONFIG_HOME/locale.conf
• Read file $HOME/.config/locale.conf
• Read file /etc/locale.conf

Js

• Lookup env LANGUAGE
• Lookup env LC_ALL

Windows

• Lookup env LANGUAGE
• Lookup env LC_ALL
• Lookup env LC_MESSAGES
• Lookup env LANG
• Windows Registry

macOS X (darwin)

• Lookup env LANGUAGE
• Lookup env LC_ALL
• Lookup env LC_MESSAGES
• Lookup env LANG
• macOS X User Defaults System
  • Lookup user AppleLocale
  • Lookup user AppleLanguages
  • Lookup global AppleLocale
  • Lookup global AppleLanguages

Usage

import (
    "github.com/Xuanwo/go-locale"
)

func main() {
    tag, err := locale.Detect()
    if err != nil {
        log.Fatal(err)
    }
    // Have fun with language.Tag!

    tags, err := locale.DetectAll()
    if err != nil {
        log.Fatal(err)
    }
    // Get all available tags
}

Acknowledgments

Inspired by jibber_jabber

go-locale's Issues

LC_MESSAGES=C should overwrite $LANGUAGE env

References:

https://www.gnu.org/software/gettext/manual/gettext.html#Locale-Environment-Variables

Not all programs have translations for all languages. By default, an English message is shown in place of a nonexistent translation. If you understand other languages, you can set up a priority list of languages. This is done through a different environment variable, called LANGUAGE. GNU gettext gives preference to LANGUAGE over LC_ALL and LANG for the purpose of message handling, but you still need to have LANG (or LC_ALL) set to the primary language; this is required by other parts of the system libraries. For example, some Swedish users who would rather read translations in German than English for when Swedish is not available, set LANGUAGE to ‘sv:de’ while leaving LANG to ‘sv_SE’.

Monkeypatching for tests?

https://github.com/Xuanwo/go-locale/blob/master/locale_linux_test.go#L16

Why?

Just. Why?

Add test for go 1.18

Roadmap to v1

• Bump version to v1
• Add env support for windows
• Cover all platform that go supports

Support all GOOS

• android
• darwin
• dragonfly
• freebsd
• linux
  - nacl
• netbsd
• openbsd
• plan9
• solaris
• windows
• zos

locale is not valid for locale detect

#!/bin/bash

set -e
set -o pipefail

unset LANGUAGE
unset LANG
unset LC_ALL

locale

LANG=
LANGUAGE=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

locale needs envs to genreate correct data.

Add benchmarks

Windowns OLE detection seems inacurate

My current system setting for Internationalization in the registry key HKCU/ControlPanel/International/LocalName states pl_PL, but the library querying the system language via OLE is getting back en-US as the language.

The code below seems simpler and gets me the right locale every time:

k, err := registry.OpenKey(registry.CURRENT_USER, `Control Panel\International`, registry.QUERY_VALUE)
if err != nil {
	panic(err)
}
defer k.Close()

locale, _, err := k.GetStringValue("LocaleName")
if err != nil {
	panic(err)
}
log.printf("Detected locale: %s", locale)

LocaleName will return a full locale string, while Locale will match with the current windows_generated.go map. Tested on Windows XP/7/8/10. Sorry, I do not have Vista anywhere to test it out.

Alternatively using PowerShell is as reliable:

  cmd := exec.Command("powershell", "Get-Culture | select -exp Name")
  output, err := cmd.Output()
  if err == nil {
    return strings.Trim(string(output), "\r\n"), nil
  }

I just believe that querying the registry seems like a cleaner solution.

---

Code below is a live one I tested with, output at the end.

import gloc "github.com/Xuanwo/go-locale"

#[...]
	if locTag, err := gloc.Detect(); err ==nil {
		log.Printf("Detedcted language: %s\n", locTag.String())
		locale = display.Tags(locTag)
	}

	k, err := registry.OpenKey(registry.CURRENT_USER, `Control Panel\International`, registry.QUERY_VALUE)
	if err != nil {
		panic(err)
	}
	defer k.Close()

	print, _, err := k.GetStringValue("LocaleName")
	if err != nil {
		panic(err)
	}
	log.Printf("Language from registry: %s", print)

2020/08/02 14:52:55 Detedcted language: en-US
2020/08/02 14:52:55 Language from registry: pl-PL

My locale is and should be pl-PL

Bump golang.org/x/sys to latest to addressing potential bugs

Windows locale detection

Looking at the windows detector, there is an map used to convert windows uint32 language code to the string representation.

go-locale/locale_windows.go

Lines 14 to 28 in 36d835c

	// osLanguageCode is a mapping from Microsoft Windows language code to language.Tag
	//
	// ref:
	// - https://docs.microsoft.com/en-us/windows/win32/cimwin32prov/win32-operatingsystem
	// - https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry
	var osLanguageCode = map[uint32]string{
	0x4: "zh-Hans-CN", // Chinese (Simplified)– China
	0x9: "en", // English
	0x404: "zh-Hant-TW", // Chinese (Traditional) – Taiwan
	0x409: "en-US", // English – United States
	0x411: "ja", // Japanese
	0x412: "ko", // Korean
	0x804: "zh-Hans-CN", // Chinese (Simplified) – PRC
	0x809: "en-US", // English – United Kingdom
	}

This map isn't aligned with the content of https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry

What happen if I use a system configured from fr-latin-FR ?
Does this modules works other language than the one defined in this map ?

I'm looking for an alternative to the fork: https://github.com/cubiest/jibberjabber (the uses of some dependency is questionable)

Remove out-of-date comments

• // FIXME: LC_ALL=C should overwrite $LANGUAGE env
• // FIXME: LC_MESSAGES=C should overwrite $LANGUAGE env

LC_ALL=C should overwrite $LANGUAGE env