maximilien / i18n4go Goto Github PK
View Code? Open in Web Editor NEWi18n tooling for Golang
License: Apache License 2.0
i18n tooling for Golang
License: Apache License 2.0
putting strings that I don't want to translate into 1 single json file might also have huge manual workload.
If we have mark strings in go source code, this tool can exclude them when scanning the source files.
it returns 404 page not found
Since Go is also used for developing web applications and generating HTML, I think this will be a worth while addition to workflows using this package.
{{T "Sample string"}}
text/template
is not only for generating HTML so what constitutes as a string will be different for different templates.If it is already present this issue can be closed, I couldn't find any information on it in README.
There is an issue with replacing the FULL_IMPORT_PATH when simulating the rewrite-package
command. This is leading to the following errors:
Total files parsed: 1
Total rewritten strings: 1
Total time: 422.804µs
• Failure [0.011 seconds]
rewrite-package [...] --init-code-snippet-filename some-file
...i18n4go/integration/rewrite_package/init_code_snippet_filename_test.go:14
invokes rewrite-package command and uses the specified --init-code-snippet-filename
...i18n4go/integration/rewrite_package/init_code_snippet_filename_test.go:83
rewrites the source go file wrapping strings with T() and generates a i18n_init.go using teamplate file [It]
...integration/rewrite_package/init_code_snippet_filename_test.go:108
Expected
<string>: package input_files
import (
"fmt"
"path/filepath"
"github.com/maximilien/i18n4go/i18n"
goi18n "github.com/nicksnyder/go-i18n/i18n"
)
var T goi18n.TranslateFunc
func init() {
fmt.Println("DEBUG: this is a test i18n_init.go file")
T = i18n.Init(filepath.Join(""), i18n.GetResourcesPath())
}
to equal
<string>: package input_files
import (
"fmt"
"path/filepath"
"github.com/maximilien/i18n4go/i18n"
goi18n "github.com/nicksnyder/go-i18n/i18n"
)
var T goi18n.TranslateFunc
func init() {
fmt.Println("DEBUG: this is a test i18n_init.go file")
T = i18n.Init(filepath.Join("test_fixtures", "rewrite_package", "init_code_snippet_filename", "input_files"), i18n.GetResourcesPath())
}
Create-translation does not create a nested folder structure. It only creates the last file if it is able.
Just wanted to see if this is supposed to happen, or if this was a bug in the test suite.
git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean
master ~/go/src/github.com/maximilien/i18n4go
» ./bin/test
Cleaning build artifacts...
Formatting packages...
Integration Testing packages:
ok github.com/maximilien/i18n4go/integration/checkup 1.707s
ok github.com/maximilien/i18n4go/integration/create_translations 1.717s
ok github.com/maximilien/i18n4go/integration/extract_strings 1.913s
ok github.com/maximilien/i18n4go/integration/fixup 1.882s
ok github.com/maximilien/i18n4go/integration/merge_strings 1.629s
ok github.com/maximilien/i18n4go/integration/rewrite_package 1.989s
ok github.com/maximilien/i18n4go/integration/show_missing_strings 1.570s
? github.com/maximilien/i18n4go/integration/test_helpers [no test files]
ok github.com/maximilien/i18n4go/integration/verify_strings 1.752s
Vetting packages for potential issues...
SWEET SUITE SUCCESS
master ● ~/go/src/github.com/maximilien/i18n4go
» git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)
modified: test_fixtures/rewrite_package/d_option/expected_output/doption/en.all.json
modified: test_fixtures/rewrite_package/d_option/expected_output/doption/test.go.en.json
modified: test_fixtures/rewrite_package/d_option/expected_output/doption/test2.go.en.json
The project is named i18n4go but the binary is gi18n
which can be confused with existing goi18n
project binary.
@simonleung8, @jberkhahn, and @goehmen,
Please note this coming changed. Been doing lots of clean ups on the project. This particular change will require you to change the CF CLI code so that it uses i18n4go
instead of gi18n
as the expected binary.
I am also planning to rename the package gi18n
to main
since that's really what it is.
Please reply to this issue if you have any comments.
Best,
max
Really cool project!
If I understand correctly, the workflow is to write your Go code as usual, process it with this command line tool which rewrites it to insert T() calls and extract messages, and then build the rewritten files?
Just a suggestion: it would be helpful to have the high level workflow in the documentation overview, since it's not a usual sort of thing
The fixup
command supports removing and adding missing translation but there are situations when a user may only want to remove the translations.
github.com/maximilien/i18n4go/i18n/init.go:94: undefined: resources in resources.Asset
When running this module binary command (i18n4go [options]) with go 1.17.7, it throws panic: flag "-help" begins with -
Not a big issue, but found that flag -p is not working as per the instrution
section: https://github.com/maximilien/i18n4go#extract-strings
command:$ gi18n -c extract-strings -v -p -f ./tmp/cli/cf/app/app.go -o ./tmp/cli/i18n -output-match-package
error: flag provided but not defined: -p
instead, flag --po works
Current README.md
only explains how to use the various commands individually. What we need here is a an example usage for all the commands together on an example project.
Some of the current integration tests input files use the CLI and this creates a dependency with CF cli code base. It would be better if this tool is independent of the CF CLI.
Currently for each i18n4go
command the user needs to use -c
before the command name. With COBRA support we can remove this, so for instance:
$i18n4go -c extract-strings -f i18n4go/i18n4go.go -v
should now be
$i18n4go extract-strings -f i18n4go/i18n4go.go -v
Also, since we will be deprecating -c
let's add a warning when user still have -c
like:
WARNING: the -c before <command-name> is now DEPRECATED and no longer needed. Future versions of i18n4go will remove -c alltogether
or something similar.
Currently, the func init()
that is added with the rewrite-package
command depends on the CF CLI code since it uses an Init()
function there. It would be much better if i18n4go would have its own Init()
code and thus remove any dependencies from CF CLI and be self-sufficient. The func Init()
code in question is the following in rewrite_package.go
:
const (
INIT_CODE_SNIPPET = `package __PACKAGE__NAME__
import (
"path/filepath"
"github.com/cloudfoundry/cli/cf/i18n"
goi18n "github.com/nicksnyder/go-i18n/i18n"
)
var T goi18n.TranslateFunc
func init() {
T = i18n.Init(__FULL_IMPORT_PATH__, i18n.GetResourcesPath())
}`
)
Hi,
It's not clear for me how a modified
flag in a language resource JSON file is set / used?
My guess is that it is set to true
when an original (en_US) text is modified but the translated text is not updated. Am I right?
And thus it should be set to true
by the tool, not by human, and reset to false
by human. Is it correct?
In the documentation, -rewrite-package
has an optional -o
flag. It says that the original file will be overwritten if not specified, but the current behavior writes it to the current directory.
This should allow developers to modify an existing string (Key and Value) from a source file (or multiple) and also update all translation JSON files. A warning or error should be expressed if the new modified string would conflict with the existing one.
Something completely unexpected happened. This is a bug in i18n4go.
When running this command:
i18n4go -c fixup
this error occurred:
assignment to entry in nil map
and this stack trace:
/src/github.com/maximilien/i18n4go/i18n4go/i18n4go.go:398 (0x53c2)
displayCrashDialog: stackTrace := "\t" + strings.Replace(string(debug.Stack()), "\n", "\n\t", -1)
/github.com/maximilien/i18n4go/i18n4go/i18n4go.go:370 (0x5344)
handlePanic: displayCrashDialog(err)
/usr/local/Cellar/go/1.4.2/libexec/src/runtime/asm_amd64.s:401 (0x3faf5)
call16: CALLFN(·call16, 16)
/usr/local/Cellar/go/1.4.2/libexec/src/runtime/panic.go:387 (0x17a48)
gopanic: reflectcall(unsafe.Pointer(d.fn), deferArgs(d), uint32(d.siz), uint32(d.siz))
/usr/local/Cellar/go/1.4.2/libexec/src/runtime/hashmap.go:385 (0xc334)
mapassign1: panic("assignment to entry in nil map")
/github.com/maximilien/i18n4go/cmds/fixup.go:326 (0x96d71)
addTranslations: localeMap[id] = common.I18nStringInfo{ID: id, Translation: id}
/github.com/maximilien/i18n4go/cmds/fixup.go:90 (0x95335)
(*Fixup).Run: addTranslations(foreignStringInfos, i18nFile[0], foreignMissingTranslations)
/github.com/maximilien/i18n4go/i18n4go/i18n4go.go:201 (0x4490)
fixupCmd: err := fixup.Run()
/github.com/maximilien/i18n4go/i18n4go/i18n4go.go:44 (0x21fb)
main: fixupCmd()
/usr/local/Cellar/go/1.4.2/libexec/src/runtime/proc.go:63 (0x19593)
main: main_main()
/usr/local/Cellar/go/1.4.2/libexec/src/runtime/asm_amd64.s:2232 (0x41c11)
goexit:
If T is in the form gi18n.T("...")
the string will not be recognized by either of these commands.
If you issue the command -merge-strings
before -rewrite-package
, only the first occurrence of a duplicate string that should be in a templated format under the package will be rewritten with a T(). This is because the all.json
file will be converted into a templated format.
installers/homebrew/i18n4go-bin.rb
installers/homebrew/i18n4go-src.rb
and it should be cleaned up after each test.
Rewrite-package does not rewrite a string, if it is a parameter multiple time. Problem with the map only storing the value once.
When a file has strings in var (...)
then rewrite-package will use the normal T(...)
, however, this fails at runtime since the T
function is only set when init()
for the package has executed.
The solution we used in the CF cli
is to initialize a local t()
and use use instead. So:
var(
t = Init()
someVar = t("some string")
)
I occasionally find duplicate string id can exist in the same json resource file for one language. This is found when rebasing some code in CLI project for some reason. Perhaps we need a check if it is unique for each string id and reported an error or warning message. Here is an example:
{
"id": "Creating service key {{.ServiceKeyName}} for service instance {{.ServiceInstanceName}} as {{.CurrentUser}}...",
"translation": "Creating service key {{.ServiceKeyName}} for service instance {{.ServiceInstanceName}} as {{.CurrentUser}}...",
"modified": false
},
{
"id": "Creating service key {{.ServiceKeyName}} for service instance {{.ServiceInstanceName}} as {{.CurrentUser}}...",
"translation": "Creating service key {{.ServiceKeyName}} for service instance {{.ServiceInstanceName}} as {{.CurrentUser}}...",
"modified": false
},
Current demo1
is a basic hello world in Golang. We need another one that includes or uses the following Golang features:
main
packageSimilar to checkup, a user may use a different qualifier to import the translation function such as
import (
translation "github.ibm.com/arf/cli-dev-plugin/plugin/i18n"
)
Adding the -q
to the command will fix the issue
Hello, I am encountering this error:
go: downloading github.com/nicksnyder/go-i18n v2.0.2
../../../go/pkg/mod/github.com/maximilien/[email protected]/i18n/init.go:13:2: zip for github.com/nicksnyder/[email protected] has unexpected file github.com/nicksnyder/[email protected]+incompatible/.codecov.yml
internal/cmd/newer/i18n_init.go:7:2: zip for github.com/nicksnyder/[email protected] has unexpected file github.com/nicksnyder/[email protected]+incompatible/.codecov.yml
It seems like the go-i18n
package has some change?
String that are in a return statement do not become wrapped in a T().
Completely support all commands in new Cobra fashion and make old fashion optional but "deprecated" with a message
Based on Knative / Kubernetes headers:
// Copyright © 2015-2023 The Knative Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
and
# Copyright © 2015-2023 The Knative Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
When we are trying to run extract string on files that have already been extracted/rewritten we are running into a problem where the keys in these maps are being extracted into the en.json
files. We have to add all these keys to the exclude file to work around this; but this causes issues when the keys are equal to strings that do need to be translated.
When looking at the greater scheme of things, we think that string keys in maps almost never need to be translated, and we can't think of a case where we would want to translate them.
@XenoPhex && @pivotal-devin-fallak
It seems that when a return statement calls a function which contains a string as a parameter, and that string should be rewritten, it appears to not rewrite that string.
Ex: The following code: "Reading config %s"
should be rewritten, however it is not.
...
bytes, err := fs.ReadFile(path)
if err != nil {
return config, bosherr.WrapError(err, "Reading config %s", path)
}
...
When running fixup on translations, I would like to have foreign translations be updated with English translations if the ID and Translation are identical in the translation json file. This is because the English string has been updated, but the translation is still in English.
As per discussion with @Aerex the 4 milestones above (some with listed sub-items) will be what we work on for 2023 to reach v2.0. Each of these will be converted into issues and milestones to track.
If you are reading this and have items or milestones you want to add please reply to this issue.
Closing #51 since it was positive and will open new issues for milestone 1.
Some duplicate strings that should be in a templated format in the same file do not become wrapped in a T() when issuing the command gi18n -rewrite-package ...
. Only the the first instance does. This is because the translation JSON files change to a templated format and the string can no longer be found.
Since this tool is generic and can be used for any Go language program it should be named appropriately i18n4go
.
This would make it easier to pass files in a dir.
Might be solved if we move this code to go-flags
Looks like extract-strings
is not picking up some cases in the CF CLI codebase:
mkdir lang_temp
i18n4go -c extract-strings -e excluded.json -s specialStrings.json -o lang_temp -f cf/commands/plugin/install_plugin.go
i18n4go -c extract-strings -e excluded.json -s specialStrings.json -o lang_temp -f cf/commands/plugin/uninstall_plugin.go
Filename | Category | Line:Example |
---|---|---|
cf/commands/plugin/install_plugin.go | string embedded inside a func , inside a func in a return |
206:Plugin name {{.PluginName}} is already taken |
cf/commands/plugin/uninstall_plugin.go | string embedded inside a func , inside a func |
100:Plugin {{.PluginName}} successfully uninstalled. |
I went through a few dozen cases of missing strings, and it looks like the second case is where most of our strings are missing.
The rewrite-strings
and verify-strings
commands have optional flags (ignore-regexp
, outputDir
) used but not present in the usage text.
Build on #65 and now allow all commands to be called without -c
(making old fashion optional)
Add Cobra support keep all commands the same using old style `-c
Unsure how hard this is to fix, but I noticed that >
would be translated to its unicode/ascii value. Similar odd things are happening for newlines and other assorted characters. It might be hard to address this in all cases, so for now I propose we work around it.
Give the following directory structure:
.
├── a
│ └── a.go
├── b
│ └── a.go
└── c
└── a.go
If we call i18n4go -c extract-strings -o out -d . -r
the result is 1 json file in the out directory, which includes only strings from c/a.go
and nothing from a/a.go
or b/b.go
.
@luan && Difan
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.