putting strings that I don't want to translate into 1 single json file might also have huge manual workload.
If we have mark strings in go source code, this tool can exclude them when scanning the source files.

it returns 404 page not found

Since Go is also used for developing web applications and generating HTML, I think this will be a worth while addition to workflows using this package.

• One option is to support in extractor so it looks for things like this:
  {{T "Sample string"}}
• Other is to parse HTML files and wrap text nodes with T. This IMO is unnecessarily complex, since text/template is not only for generating HTML so what constitutes as a string will be different for different templates.

If it is already present this issue can be closed, I couldn't find any information on it in README.

There is an issue with replacing the FULL_IMPORT_PATH when simulating the rewrite-package command. This is leading to the following errors:

Total files parsed: 1
Total rewritten strings: 1
Total time: 422.804µs
• Failure [0.011 seconds]
rewrite-package [...] --init-code-snippet-filename some-file
...i18n4go/integration/rewrite_package/init_code_snippet_filename_test.go:14
  invokes rewrite-package command and uses the specified --init-code-snippet-filename
  ...i18n4go/integration/rewrite_package/init_code_snippet_filename_test.go:83
    rewrites the source go file wrapping strings with T() and generates a i18n_init.go using teamplate file [It]
...integration/rewrite_package/init_code_snippet_filename_test.go:108

    Expected
        <string>: package input_files

        import (
                "fmt"
                "path/filepath"

                "github.com/maximilien/i18n4go/i18n"
                goi18n "github.com/nicksnyder/go-i18n/i18n"
        )

        var T goi18n.TranslateFunc

        func init() {
                fmt.Println("DEBUG: this is a test i18n_init.go file")
                T = i18n.Init(filepath.Join(""), i18n.GetResourcesPath())
        }

    to equal
        <string>: package input_files

        import (
                "fmt"
                "path/filepath"

                "github.com/maximilien/i18n4go/i18n"
                goi18n "github.com/nicksnyder/go-i18n/i18n"
        )

        var T goi18n.TranslateFunc

        func init() {
                fmt.Println("DEBUG: this is a test i18n_init.go file")
                T = i18n.Init(filepath.Join("test_fixtures", "rewrite_package", "init_code_snippet_filename", "input_files"), i18n.GetResourcesPath())
        }

Create-translation does not create a nested folder structure. It only creates the last file if it is able.

Just wanted to see if this is supposed to happen, or if this was a bug in the test suite.

git status
On branch master
Your branch is up-to-date with 'origin/master'.

nothing to commit, working directory clean
  master  ~/go/src/github.com/maximilien/i18n4go 
» ./bin/test

 Cleaning build artifacts...

 Formatting packages...

 Integration Testing packages:
ok      github.com/maximilien/i18n4go/integration/checkup   1.707s
ok      github.com/maximilien/i18n4go/integration/create_translations   1.717s
ok      github.com/maximilien/i18n4go/integration/extract_strings   1.913s
ok      github.com/maximilien/i18n4go/integration/fixup 1.882s
ok      github.com/maximilien/i18n4go/integration/merge_strings 1.629s
ok      github.com/maximilien/i18n4go/integration/rewrite_package   1.989s
ok      github.com/maximilien/i18n4go/integration/show_missing_strings  1.570s
?       github.com/maximilien/i18n4go/integration/test_helpers  [no test files]
ok      github.com/maximilien/i18n4go/integration/verify_strings    1.752s

 Vetting packages for potential issues...

SWEET SUITE SUCCESS
  master ●  ~/go/src/github.com/maximilien/i18n4go 
» git status
On branch master
Your branch is up-to-date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   test_fixtures/rewrite_package/d_option/expected_output/doption/en.all.json
    modified:   test_fixtures/rewrite_package/d_option/expected_output/doption/test.go.en.json
    modified:   test_fixtures/rewrite_package/d_option/expected_output/doption/test2.go.en.json

The project is named i18n4go but the binary is gi18n which can be confused with existing goi18n project binary.

@simonleung8, @jberkhahn, and @goehmen,

Please note this coming changed. Been doing lots of clean ups on the project. This particular change will require you to change the CF CLI code so that it uses i18n4go instead of gi18n as the expected binary.

I am also planning to rename the package gi18n to main since that's really what it is.

Please reply to this issue if you have any comments.

Best,

max

Really cool project!

If I understand correctly, the workflow is to write your Go code as usual, process it with this command line tool which rewrites it to insert T() calls and extract messages, and then build the rewritten files?

Just a suggestion: it would be helpful to have the high level workflow in the documentation overview, since it's not a usual sort of thing

Context

The fixup command supports removing and adding missing translation but there are situations when a user may only want to remove the translations.

github.com/maximilien/i18n4go/i18n/init.go:94: undefined: resources in resources.Asset

When running this module binary command (i18n4go [options]) with go 1.17.7, it throws panic: flag "-help" begins with -

Not a big issue, but found that flag -p is not working as per the instrution

section: https://github.com/maximilien/i18n4go#extract-strings

command:$ gi18n -c extract-strings -v -p -f ./tmp/cli/cf/app/app.go -o ./tmp/cli/i18n -output-match-package

error: flag provided but not defined: -p

instead, flag --po works

Current README.md only explains how to use the various commands individually. What we need here is a an example usage for all the commands together on an example project.

Some of the current integration tests input files use the CLI and this creates a dependency with CF cli code base. It would be better if this tool is independent of the CF CLI.

Currently for each i18n4go command the user needs to use -c before the command name. With COBRA support we can remove this, so for instance:

$i18n4go -c extract-strings -f i18n4go/i18n4go.go -v

should now be

$i18n4go extract-strings -f i18n4go/i18n4go.go -v

Also, since we will be deprecating -c let's add a warning when user still have -c like:

WARNING: the -c before <command-name> is now DEPRECATED and no longer needed. Future versions of i18n4go will remove -c alltogether

or something similar.

Currently, the func init() that is added with the rewrite-package command depends on the CF CLI code since it uses an Init() function there. It would be much better if i18n4go would have its own Init() code and thus remove any dependencies from CF CLI and be self-sufficient. The func Init() code in question is the following in rewrite_package.go:

const (
    INIT_CODE_SNIPPET = `package __PACKAGE__NAME__

import (
    "path/filepath"

    "github.com/cloudfoundry/cli/cf/i18n"
    goi18n "github.com/nicksnyder/go-i18n/i18n"
)

var T goi18n.TranslateFunc

func init() {
    T = i18n.Init(__FULL_IMPORT_PATH__, i18n.GetResourcesPath())
}`
)

Hi,

It's not clear for me how a modified flag in a language resource JSON file is set / used?

My guess is that it is set to true when an original (en_US) text is modified but the translated text is not updated. Am I right?

And thus it should be set to true by the tool, not by human, and reset to false by human. Is it correct?

In the documentation, -rewrite-package has an optional -o flag. It says that the original file will be overwritten if not specified, but the current behavior writes it to the current directory.

This should allow developers to modify an existing string (Key and Value) from a source file (or multiple) and also update all translation JSON files. A warning or error should be expressed if the new modified string would conflict with the existing one.

Something completely unexpected happened. This is a bug in i18n4go.

When running this command:

i18n4go -c fixup

this error occurred:

assignment to entry in nil map

and this stack trace:

/src/github.com/maximilien/i18n4go/i18n4go/i18n4go.go:398 (0x53c2)
    displayCrashDialog: stackTrace := "\t" + strings.Replace(string(debug.Stack()), "\n", "\n\t", -1)
/github.com/maximilien/i18n4go/i18n4go/i18n4go.go:370 (0x5344)
    handlePanic: displayCrashDialog(err)
/usr/local/Cellar/go/1.4.2/libexec/src/runtime/asm_amd64.s:401 (0x3faf5)
    call16: CALLFN(·call16, 16)
/usr/local/Cellar/go/1.4.2/libexec/src/runtime/panic.go:387 (0x17a48)
    gopanic: reflectcall(unsafe.Pointer(d.fn), deferArgs(d), uint32(d.siz), uint32(d.siz))
/usr/local/Cellar/go/1.4.2/libexec/src/runtime/hashmap.go:385 (0xc334)
    mapassign1: panic("assignment to entry in nil map")
/github.com/maximilien/i18n4go/cmds/fixup.go:326 (0x96d71)
    addTranslations: localeMap[id] = common.I18nStringInfo{ID: id, Translation: id}
/github.com/maximilien/i18n4go/cmds/fixup.go:90 (0x95335)
    (*Fixup).Run: addTranslations(foreignStringInfos, i18nFile[0], foreignMissingTranslations)
/github.com/maximilien/i18n4go/i18n4go/i18n4go.go:201 (0x4490)
    fixupCmd: err := fixup.Run()
/github.com/maximilien/i18n4go/i18n4go/i18n4go.go:44 (0x21fb)
    main: fixupCmd()
/usr/local/Cellar/go/1.4.2/libexec/src/runtime/proc.go:63 (0x19593)
    main: main_main()
/usr/local/Cellar/go/1.4.2/libexec/src/runtime/asm_amd64.s:2232 (0x41c11)
    goexit:

@yuzhangcmu, @fordaz

If T is in the form gi18n.T("...") the string will not be recognized by either of these commands.

If you issue the command -merge-strings before -rewrite-package, only the first occurrence of a duplicate string that should be in a templated format under the package will be rewritten with a T(). This is because the all.json file will be converted into a templated format.

1. Add scripts:

installers/homebrew/i18n4go-bin.rb
installers/homebrew/i18n4go-src.rb

1. submit to the Homebrew project
2. update README.md with new Homebrew

and it should be cleaned up after each test.

Rewrite-package does not rewrite a string, if it is a parameter multiple time. Problem with the map only storing the value once.

When a file has strings in var (...) then rewrite-package will use the normal T(...), however, this fails at runtime since the T function is only set when init() for the package has executed.

The solution we used in the CF cli is to initialize a local t() and use use instead. So:

var(
 t = Init()
 someVar = t("some string")
)

I occasionally find duplicate string id can exist in the same json resource file for one language. This is found when rebasing some code in CLI project for some reason. Perhaps we need a check if it is unique for each string id and reported an error or warning message. Here is an example:

https://github.com/xingzhou/cli/blob/68b153cb9ef90a42e9def2d359d38b83a595e7bb/cf/i18n/resources/fr_FR.all.json#L1473

   {
      "id": "Creating service key {{.ServiceKeyName}} for service instance {{.ServiceInstanceName}} as {{.CurrentUser}}...",
      "translation": "Creating service key {{.ServiceKeyName}} for service instance {{.ServiceInstanceName}} as {{.CurrentUser}}...",
      "modified": false
   },

https://github.com/xingzhou/cli/blob/68b153cb9ef90a42e9def2d359d38b83a595e7bb/cf/i18n/resources/fr_FR.all.json#L5238

   {
      "id": "Creating service key {{.ServiceKeyName}} for service instance {{.ServiceInstanceName}} as {{.CurrentUser}}...",
      "translation": "Creating service key {{.ServiceKeyName}} for service instance {{.ServiceInstanceName}} as {{.CurrentUser}}...",
      "modified": false
   },

Current demo1 is a basic hello world in Golang. We need another one that includes or uses the following Golang features:

1. more than main package
2. strings with more than one argument
3. strings with at least one argument that is a function call
4. simple local variable strings
5. date formatting and maybe numbers as arguments

Context

Similar to checkup, a user may use a different qualifier to import the translation function such as

import (
  translation "github.ibm.com/arf/cli-dev-plugin/plugin/i18n"
)

Adding the -q to the command will fix the issue

Hello, I am encountering this error:

go: downloading github.com/nicksnyder/go-i18n v2.0.2
../../../go/pkg/mod/github.com/maximilien/[email protected]/i18n/init.go:13:2: zip for github.com/nicksnyder/[email protected] has unexpected file github.com/nicksnyder/[email protected]+incompatible/.codecov.yml
internal/cmd/newer/i18n_init.go:7:2: zip for github.com/nicksnyder/[email protected] has unexpected file github.com/nicksnyder/[email protected]+incompatible/.codecov.yml

It seems like the go-i18n package has some change?

String that are in a return statement do not become wrapped in a T().

Completely support all commands in new Cobra fashion and make old fashion optional but "deprecated" with a message

Based on Knative / Kubernetes headers:

// Copyright © 2015-2023 The Knative Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

and

# Copyright © 2015-2023 The Knative Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

When we are trying to run extract string on files that have already been extracted/rewritten we are running into a problem where the keys in these maps are being extracted into the en.json files. We have to add all these keys to the exclude file to work around this; but this causes issues when the keys are equal to strings that do need to be translated.

When looking at the greater scheme of things, we think that string keys in maps almost never need to be translated, and we can't think of a case where we would want to translate them.

@XenoPhex && @pivotal-devin-fallak

It seems that when a return statement calls a function which contains a string as a parameter, and that string should be rewritten, it appears to not rewrite that string.
Ex: The following code: "Reading config %s" should be rewritten, however it is not.

...
    bytes, err := fs.ReadFile(path)
    if err != nil {
        return config, bosherr.WrapError(err, "Reading config %s", path)
    }
...

When running fixup on translations, I would like to have foreign translations be updated with English translations if the ID and Translation are identical in the translation json file. This is because the English string has been updated, but the translation is still in English.

1. Move to Cobra
   1. Use Cobra (old fashion but using Cobra)
   2. Compatible (new and old fashions)
   3. Remove old fashion — only support new
2. Cleaning up code base
   1. Fixing existing bugs
   2. Adding more tests (integration, …)
   3. CI/CD — automate releases
   4. GH actions — stale, running tests on PRs, etc
   5. Standard commit notations
   6. Automate changelogs — for new releases
3. Use i184go on i18n4go
   1. Translate to French
   2. Make it easy for others to submit other languages
4. Exposing a library for i18n4go

As per discussion with @Aerex the 4 milestones above (some with listed sub-items) will be what we work on for 2023 to reach v2.0. Each of these will be converted into issues and milestones to track.

If you are reading this and have items or milestones you want to add please reply to this issue.

Closing #51 since it was positive and will open new issues for milestone 1.

Some duplicate strings that should be in a templated format in the same file do not become wrapped in a T() when issuing the command gi18n -rewrite-package .... Only the the first instance does. This is because the translation JSON files change to a templated format and the string can no longer be found.

Since this tool is generic and can be used for any Go language program it should be named appropriately i18n4go.

This would make it easier to pass files in a dir.

Might be solved if we move this code to go-flags

Looks like extract-strings is not picking up some cases in the CF CLI codebase:

mkdir lang_temp
i18n4go -c extract-strings -e excluded.json -s specialStrings.json -o lang_temp -f cf/commands/plugin/install_plugin.go
i18n4go -c extract-strings -e excluded.json -s specialStrings.json -o lang_temp -f cf/commands/plugin/uninstall_plugin.go

I went through a few dozen cases of missing strings, and it looks like the second case is where most of our strings are missing.

Context

The rewrite-strings and verify-strings commands have optional flags (ignore-regexp, outputDir) used but not present in the usage text.

Build on #65 and now allow all commands to be called without -c (making old fashion optional)

Add Cobra support keep all commands the same using old style `-c

Unsure how hard this is to fix, but I noticed that > would be translated to its unicode/ascii value. Similar odd things are happening for newlines and other assorted characters. It might be hard to address this in all cases, so for now I propose we work around it.

Give the following directory structure:

.
├── a
│   └── a.go
├── b
│   └── a.go
└── c
    └── a.go

If we call i18n4go -c extract-strings -o out -d . -r the result is 1 json file in the out directory, which includes only strings from c/a.go and nothing from a/a.go or b/b.go.

@luan && Difan