wout / rosetta

A blazing fast internationalization (i18n) library for Crystal with compile-time key lookup.

Rosetta

A blazing fast internationalization (i18n) library for Crystal with compile-time key lookup. You'll never have a missing translation in your app, ever again.

Why use Rosetta?

You'll never have a missing translation

Rosetta is different from other internationalization libraries because it handles key lookup at compile-time rather than runtime. The significant advantage is that you'll be able to find missing translations - or typos in your locale keys - during development rather than after you've deployed your app. This is also true for translation keys in all additional locales.

You'll never have a missing interpolation

In Rosetta, interpolation keys are arguments to the translation method. So if you're missing an argument, the compiler will complain. The parser will also compare interpolation keys in additional locales to the ones found in the default locale, and complain if some are missing.

Rosetta is more than 12x faster than similar libraries

Benchmarking against other libraries which also use YAML or JSON backends, Rosetta is 12x to 700x faster than any other one.

For simple translations:

crimson-knight/i18n.cr translation 303.57k (  3.29µs) (± 4.62%)  801B/op  702.21× slower
     crystal-i18n/i18n translation  18.07M ( 55.35ns) (± 7.28%)  48.0B/op  12.39× slower
         syeopite/lens translation   5.09M (196.47ns) (± 4.60%)   176B/op  43.98× slower
          wout/rosetta translation 223.86M (  4.47ns) (± 2.20%)   0.0B/op        fastest

For translations with interpolations:

crimson-knight/i18n.cr interpolation 318.12k (  3.14µs) (± 0.85%)    801B/op  108.51× slower
     crystal-i18n/i18n interpolation  65.55k ( 15.26µs) (± 1.01%)  28.2kB/op  664.37× slower
         syeopite/lens interpolation   2.04M (490.17ns) (± 1.35%)    565B/op   21.35× slower
          wout/rosetta interpolation  43.55M ( 22.96ns) (± 4.81%)   80.0B/op         fastest

Rosetta is that much faster because a lot of the hard work happens at compile-time. And because the majority of the data is stored on the stack rather than the heap, out of the scope of garbage collector.

Read more on the official docs page.

Installation

1. Add the dependency to your shard.yml:

dependencies:
  rosetta:
    github: wout/rosetta

2. Run shards install

3. Run bin/rosetta --init

4. Require the generated config file:

# src/app_name.cr
require "../config/rosetta"

5. Include the Rosetta::Translatable mixin:

# src/pages/main_layout.cr
include Rosetta::Translatable

6. Localize your app

Rosetta.locale = :es

class Hello::ShowPage < MainLayout
  def content
    h1 r("welcome_message").t(name: "Brian") # => "¡Hola Brian!"
  end
end

Read more on the official docs page.

Development

Make sure you have Guardian.cr installed. Then run:

$ guardian

This will automatically:

• run ameba for src and spec files
• run the relevant spec for any file in src
• run spec file whenever they are saved
• install shards whenever you save shard.yml

Documentation

• Reference
• API Docs

Contributing

To the lib

1. Fork it (https://github.com/wout/rosetta/fork)
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

To the docs

Check out the docs branch and run the following command to launch the docs locally:

docker run --rm -it -p 8000:8000 -v ${PWD}:/docs squidfunk/mkdocs-material

Contributors

• wout - creator and maintainer

Acknowledgements

This shard pulls inspiration from the following projects:

• crimson-knight/i18n.cr
• crystal-i18n/i18n
• syeopite/lens
• Rails

rosetta's Issues

Create a shim for the development binary

The post-install hook currently builds the development binary every time the shard is installed. Building the binary takes a lot of time, and for production environments, it's completely unnecessary.

A solution would be to create a shim that builds the binary if and when the binary is invoked. A lazy build if you will. The shim could still be installed in the bin directory of the project by the post-install hook, and could still be called the same (bin/rosetta).

On invocation, it should check if the compiled binary is present and if not, compile it there and then. The newly built file could be called bin/.rosetta, or just completely replace the shim.

Add variants

Allow keys with multiple variants, for example:

en:
  colors:
    variants:
      pink: "pink"
      teal: "teal"
      yellow: "yellow"

If a key has one child and it's called variants, then this is a multi-variant translation.

Better compile-time messages for not calling `.t`

Use method overload in Lucky (and other frameworks) to provide helpful error messages at compile-time if .t isn't called for locales without interpolations.

Implement fallbacks

Allow regional locales (e.g. en-US and en-GB) to provide a partial keyset and pull the missing keys from the base locale (en).

Bug in v0.11.0: undefined macro method 'NilLiteral#args'

I discovered this bug while trying to use Rosetta v0.11.0 on a fresh shards install and a simple sample Kemal app:

Crystal 1.8.2 (2023-05-09)

Using rosetta (0.11.0)
In src/app.cr:18

 18 | require "../config/rosetta"
      ^
Error: while requiring "../config/rosetta"

In config/rosetta.cr:8:18

 8 | Rosetta::Backend.load("./config/rosetta")
                      ^---
Error: expanding macro

In lib/rosetta/src/rosetta/backend.cr:70:60

 70 | rules = Rosetta.annotation(Rosetta::FallbackRules).args.first ||
                                                         ^---
Error: undefined macro method 'NilLiteral#args'

Note that when downgrading to v0.10.1 it works just fine.

Implement key nesting

Allow re-use of existing translations with nested keys. For example:

en:
  example:
    hello: "Hello %r{example.subject}!"
    subject: "world"

Those values will be pre-filled by the parser, so any interpolation keys in the nested keys will be transferred automatically.

Add support for attribute name translations

As discussed in luckyframework/lucky#1773, attribute names are not translatable. Prepare Rosetta for the proposed solution.

Windows support

I have no idea if it works on Windows; it needs to be checked and fixed if needed.

Question: recommendation how to migrate from `yml` to `json` format

Hello @wout, it's me again — a grateful user. 😉

Do you recommend any tooling / workflow if a project is currently using yml format and wants to migrate to json?

PS: Maybe you could activate the discussions tab for questions like that?
PPS: thank you for your work!

Add better integration for Lucky

In some situations, the t method needs to be called explicitly even if no interpolations are present. This is the case with:

• flash messages
• the raw tag in pages and components
• form submit
• custom validation messages in operations

All except the last one can be fixed with the following overloads:

class Lucky::FlashStore
  {% for shortcut in [:failure, :info, :success] %}
    def {{ shortcut.id }}=(message : Rosetta::Translation)
      set(:{{ shortcut.id }}, message.t)
    end
  {% end %}

  def set(key : Key, value : Rosetta::Translation) : String
    set(key, value.t)
  end
end

module Lucky::SpecialtyTags
  def raw(string : Rosetta::Translation) : Nil
    view << string.t
  end
end

module Lucky::FormHelpers
  def submit(text : Rosetta::Translation, **html_options) : Nil
    submit(text.t, **html_options)
  end
end

Use annotations for configuration

Currently, configuration of the library is done using constants, which feels a bit clunky. A more elegant solution would be to use annotations. Something along the lines of:

@[Rosetta::DefaultLocale(:nl)]
@[Rosetta::AvailableLocales(:en, :nl)]
@[Rosetta::PluralizationRules({
  en: MyRule,
  nl: MyOtherRule
})]
module Rosetta
end

Rosetta::Backend.load("config/rosetta")

Configuration using constants would then raise compiler errors from version 0.7.0 and be removed in version 1.0.0.

Union with Rosetta::Translation compile error message

Having a union type with Rosetta::Translation results in vague error messages.

Example:

class Shared::LayoutHead < BaseComponent
  needs page_title : String | Rosetta::Translation

  def render
    head do
      utf8_charset
      title "My App - #{@page_title}"
      css_link asset("css/app.css"), data_turbolinks_track: "reload"
      css_link "https://rsms.me/inter/inter.css"
      js_link asset("js/app.js"), defer: "true", data_turbolinks_track: "reload"
      meta name: "turbolinks-cache-control", content: "no-cache"
      csrf_meta_tags
      responsive_meta_tag
    end
  end
end

Results in the following compile error:

Error: Rosetta.find("rosetta_localization.time.formats.short") expected to receive t(time : Time | Tuple(Int32, Int32, Int32)) but to_s was called instead

Implement Avram::I18nBackend

Add an Avram::I18nBackend to the Lucky integration macro. Something like:

struct Rosetta::AvramBackend < Avram::I18nBackend
  def get(key : String | Symbol) : String
    {% begin %}
      case key
      {% for val in %w[
                      exact_size_of
                      max_size_of
                      min_size_of
                      numeric_max
                      numeric_min
                    ] %}
      when :validate_{{val.id}}
        Rosetta.find("avram.validate_{{val.id}}").t(size: "%d")
      {% end %}
      {% for val in %w[
                      acceptance_of
                      at_most_one_filled
                      confirmation_of
                      exactly_one_filled
                      format_of
                      inclusion_of
                      numeric_nil
                      required
                      uniqueness_of
                    ] %}
      when :validate_{{val.id}}
        Rosetta.find("avram.validate_{{val.id}}").t
      {% end %}
      else
        raise "Avram translation missing for '#{key}'"
      end
    {% end %}
  end
end

Implement a CLDR converter

• https://github.com/unicode-org/cldr/releases

Implement pluralization

Pluralization needs:

• included rules
• configurable rules
• multiple variations within one locale key