microsoft / vscode-nls-dev Goto Github PK

The tools automates the extraction of strings to be externalized from TS and JS code. It therefore help localizing VSCode extensions and language servers written in TS and JS

License: Other

TypeScript 93.73% JavaScript 6.27%

vscode-nls-dev's People

Stargazers

Watchers

vscode-nls-dev's Issues

Update strip-ansi dependency to fix security warning

To fix security warning described: https://github.com/microsoft/vscode-remote-ssh/security/dependabot/open-ssh-remote/yarn.lock/ansi-regex/open

Allow createXlfFiles() respect existing <note> on xlf file.

We use createXlfFiles to generate ENU xlf file from 'package.nls.json', 'nls.metadata.header.json', and 'nls.metadata.json' files. Our loc team uses this ENU xlf file to produce translated xlfs.

The issue is that we have to have <note> elements in the trans units in our ENU xlf to pass instructions to the loc team, and createXlfFiles() just overwrites the ENU xlf, losing all the notes.

We had to work it around by getting the notes from the existing ENU xlf file first, and then applying them in our copy of createXlfFiles() as in this PR

I think this is a common scenario to be enabled in vscode-nls-dev

Typescript esModuleInterop incompatibility issue

Hi. Using the exact sample project here: https://github.com/microsoft/vscode-extension-samples/tree/master/i18n-sample, if I add "esModuleInterop": true to the compilerOptions in tsconfig.json, then language files for all typescript files are not generated when running gulp build. Is this expected behaviour? I'd like to use esModuleInterop for importing 3rd party packages.

It's not clear if i should post this issue in https://github.com/microsoft/vscode-nls instead.

Make locale tag resolution lower case based

For all the details see microsoft/vscode#4520

XLF.parse() fails to extract target text if <target> element has "state" attribute

If XLF.parse() is used to parse the following xliff xml, it'll return [Object object] instead of proper translations:

<xliff version="1.2" xmlns="urn:oasis:names:tc:xliff:document:1.2">
  <file original="package" source-language="en" target-language="zh-Hant" datatype="plaintext"><body>
    <trans-unit id="extension.liveshare.start.title">
      <source xml:lang="en">Start Collaboration Session (Share)</source>
      <target state="translated">開始共同作業工作階段 (共用)</target>
    </trans-unit>
...

The reason is that if the <target> element has state attribute, then unit.target.toString() returns '[Object object]' on line 591 in main.ts

Unable to bundle library which uses nls

Hey, I trying to create a plugin on top of vscode-chrome-debug-core, which uses nls to handle translations. When I bundle my plugin with webpack using nls-dev bundler and loader, it does not import translations of vscode-chrome-debug-core dependency. As a result, I see "Failed to load message bundle..." message in my extension.

Is it possible to bundle dependency translations as part of my extension?

const config = {
  target: 'node',

  node: {
    __dirname: false
  },

  entry: './src/extension.ts',

  output: {
    path: path.resolve(__dirname, 'dist'),
    filename: 'extension.js',
    libraryTarget: 'commonjs2',
    devtoolModuleFilenameTemplate: '../[resource-path]',
  },

  devtool: 'source-map',

  externals: [{
    vscode: 'commonjs vscode',
  }],

  resolve: {
    extensions: ['.ts', '.js'],
    plugins: [
      new TsconfigPathsPlugin({
        configFile: path.resolve(__dirname, '..', '..', 'tsconfig.json'),
      }),
    ],
  },

  plugins: [
    new NLSBundlePlugin('reactvr-vscode-plugin'),
  ],

  module: {
    rules: [
      {
        test: /\.ts$/,
        exclude: /node_modules/,
        use: [{
					loader: 'vscode-nls-dev/lib/webpack-loader',
					options: {
						base: path.join(__dirname, 'src')
					}
				}, {
					loader: 'ts-loader',
				}]
      },
    ],
  },
};

All localized strings show up as 'Failed to load message bundle' in extension development host

E.g. for RemoteHub:

createKeyValuePairFile should provide options to join comments into one string

Sourcemaps lose their inline content when going through rewriteLocalizeCalls

We need to do the same dance done here:

Fails on ES6 backtick quotes

We have this code that vscode-nls-dev rewrites:

public static failedToEnumerateFilesInDir(path: string, errCode: string, errMessage: string): string {
    return localize(
        {
            key: 'Failed.To.Enumerate.Files.In.Dir',
            comment:
            [
                'Error message when failed to enumerate files in a directory.'
            ]
        },
        `Failed to enumerate files in directory '{0}'. Error code={1}, Error message={2}`,
        path,
        errCode,
        errMessage
    );
}

When targeting ES5, all is well. But once we target ES6, the backtick as a quote character is preserved:

static failedToEnumerateFilesInDir(path, errCode, errMessage) {
    return localize({
        key: 'Failed.To.Enumerate.Files.In.Dir',
        comment: [
            'Error message when failed to enumerate files in a directory.'
        ]
    }, `Failed to enumerate files in directory '{0}'. Error code={1}, Error message={2}`, path, errCode, errMessage);
}

This causes vscode-nls-dev to fail during build with this error:

libraryResourceStrings.js(661,11): second argument of a localize call must be a string literal.

For now we workaround it by changing our own use of backtick to single quotes. But that requires much more escaping in our English strings because we use single-quotes inside the localizable string.

Please enable use of backticks.

The default branch has been renamed from 'master' to 'main'

⚠️ Call to Action ⚠️

If you have a clone of this repository locally, you'll need to run the following commands to catch-up with the rename

git branch -m master main
git fetch origin
git branch -u origin/main main

Update to xml2js from 4.23 to 0.5.0

This needs to be done for security purposes in projects that use it.

`TypeError: sourceMapConsumer.eachMapping is not a function` with `"source-map": "^0.8.0-beta.0"`

Our CI lab upgraded Node to v18 on our CI VM's. That caused us to hit an issue (discussed here: parcel-bundler/parcel#8005 )

The suggested work-around for that issue helped. (to add "source-map": "^0.8.0-beta.0" to the resolutions section).

Unfortunately, this fixed version of source-map appears to break compatibility with vscode-nls-dev. We're now getting:

ERROR in ./src/main.ts
Module build failed (from ./node_modules/vscode-nls-dev/lib/webpack-loader.js):
TypeError: sourceMapConsumer.eachMapping is not a function
    at new TextModel (Z:\repos\vscode-cpptools\Extension\node_modules\vscode-nls-dev\lib\lib.js:116:31)
    at Object.processFile (Z:\repos\vscode-cpptools\Extension\node_modules\vscode-nls-dev\lib\lib.js:572:23)
    at module.exports (Z:\repos\vscode-cpptools\Extension\node_modules\vscode-nls-dev\lib\webpack-loader.js:17:26)

Need a way to provide a comment/hint in package.nls.json

This is a request to provide support for specifying a comment/hint in package.nls.json, for translators

We'd like to start using the markdownDescription field for some of our extension settings. We'd like to be able to provide a comment/hint to translators that these strings (in package.nls.json) contain markdown, so translators are aware of it and avoid removing or corrupting the markdown syntax.

I don't see any comment/hint field handled for package.nls.json, in the implementation of createXlfFiles(). I see that it supports specifying an object with a message field instead of a string field. But, I don't see anywhere that it also handles a comment/hint field in that object.

rewriteLocalizeCalls is messing up sourcemappings in vscode-node-debug

In nodeDebug.ts, this comment exists at line 431:

/*
this._node.on('beforeCompile', (event: NodeV8Event) => {
	this.outLine(`beforeCompile ${this._scriptToPath(event.body.script)}`);
});

this._node.on('afterCompile', (event: NodeV8Event) => {
	this.outLine(`afterCompile ${this._scriptToPath(event.body.script)}`);
});
*/

After this comment, the sourcemappings are incorrect, making it very hard to debug. You can see this by building and using the sourcemap visualizer: https://sokra.github.io/source-map-visualization. There shouldn't be several mappings on these lines, on individual letters, etc:

It looks like the empty line between those two blocks is removed, and the subsequent mappings are all off by a line. Looking at the code, I have no clue why that would happen.

Fail Compile TypeScript

On compile typescript generate error:

[failed] [1 error]

ERROR in ./src/extension.ts 22:21
Module parse failed: Unexpected token (22:21)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
| const localize = nls.config({ messageFormat: nls.MessageFormat.file })();
|

let pendingValidation: boolean = false;
|
| let projeto: ValidaProjeto;

My webpack.config.js

/*---------------------------------------------------------------------------------------------
 *  Copyright (c) Microsoft Corporation. All rights reserved.
 *  Licensed under the MIT License. See License.txt in the project root for license information.
 *--------------------------------------------------------------------------------------------*/

//@ts-check

'use strict';

const path = require('path');

/**@type {import('webpack').Configuration}*/
const config = {
  target: 'node', // vscode extensions run in a Node.js-context 📖 -> https://webpack.js.org/configuration/node/

  entry: './src/extension.ts', // the entry point of this extension, 📖 -> https://webpack.js.org/configuration/entry-context/
  output: { // the bundle is stored in the 'dist' folder (check package.json), 📖 -> https://webpack.js.org/configuration/output/
    path: path.resolve(__dirname, 'dist'),
    filename: 'extension.js',
    libraryTarget: "commonjs2",
    devtoolModuleFilenameTemplate: "../[resource-path]",
  },
  devtool: 'source-map',
  externals: {
    vscode: "commonjs vscode" // the vscode-module is created on-the-fly and must be excluded. Add other modules that cannot be webpack'ed, 📖 -> https://webpack.js.org/configuration/externals/
  },
  resolve: { // support reading TypeScript and JavaScript files, 📖 -> https://github.com/TypeStrong/ts-loader
    extensions: ['.js', '.ts', '.tsx']
  },
  module: {
    rules: [
      {
        test: /\.\.\/*$/,
        exclude: /node_modules/,
        use: [
          {
            loader: 'vscode-nls-dev/lib/webpack-loader',
            options: {
              base: path.join(__dirname, 'src')
            }
          }
        ]
      }
    ]
  },
}

module.exports = config;
`

File paths should be absolute when emitting errors

From microsoft/vscode#48393

Pre-release pipelines failing with `host.fileExists is not a function`

vscode-nls-dev version 4.0.1

Module build failed (from ./node_modules/vscode-nls-dev/lib/webpack-loader.js):
TypeError: host.fileExists is not a function
    at Object.fileExists (/mnt/vss/_work/1/s/node_modules/typescript/lib/typescript.js:167586:63)
    at Object.host.fileExists (/mnt/vss/_work/1/s/node_modules/typescript/lib/typescript.js:117448:47)
    at tryFileLookup (/mnt/vss/_work/1/s/node_modules/typescript/lib/typescript.js:44293:28)
    at tryFile (/mnt/vss/_work/1/s/node_modules/typescript/lib/typescript.js:44285:20)
    at tryExtension (/mnt/vss/_work/1/s/node_modules/typescript/lib/typescript.js:44277:24)
    at tryAddingExtensions (/mnt/vss/_work/1/s/node_modules/typescript/lib/typescript.js:44255:32)
    at loadModuleFromFile (/mnt/vss/_work/1/s/node_modules/typescript/lib/typescript.js:44188:45)
    at nodeLoadModuleByRelativeName (/mnt/vss/_work/1/s/node_modules/typescript/lib/typescript.js:44112:36)
    at tryResolve (/mnt/vss/_work/1/s/node_modules/typescript/lib/typescript.js:44069:34)
    at /mnt/vss/_work/1/s/node_modules/typescript/lib/typescript.js:44032:69

ref: https://dev.azure.com/monacotools/Monaco/_build/results?buildId=184047&view=logs&j=eb79cae4-0ffb-5516-9c82-40a537d0707b&t=c0e8a551-bf7a-588d-e12e-b60cae8f30b8

When bundling, *.nls.metadata.json files are kept around

These files shouldn't be included in the destination and are intermidiate.

rewriteLocalizeCalls() doesn't work with 'importHelpers = true'

When importHelpers is set to true in tsconfig.json, rewriteLocalizeCalls() doesn't generate any bundle files so all localize() invocations go to fallback.

Did a little debug and found out that, the condition here

vscode-nls-dev/src/lib.ts

Line 472 in 545e01a

    
           if (ts.isCallExpression(parent) && ts.isIdentifier(parent.expression) && parent.expression.text === '__importStar') {

is not met.

Possible JS transpiled when using tslib import helpers:

const nls = tslib_1.__importStar(require("vscode-nls"));

Here the expression turns from a global function call to a member access. This case can be handled by (for example):

if (ts.isCallExpression(parent) && ((ts.isIdentifier(parent.expression) && parent.expression.text === '__importStar') || (ts.isPropertyAccessExpression(parent.expression) && parent.expression.name.text === "__importStar")))

Import syntax matters

From my testing, the import syntax used had to be in the form:

import * as nls from 'vscode-nls';

and the following wasn't picked up:

import { loadMessageBundle } from 'vscode-nls';

Need a way to communicate translation hints

Similar to #20 , we would like to be able to provide notes/hints for translators. Perhaps hints could be passed directly to the call to localize() for inclusion in exported xlf files.

One possible approach might be an alternate localize() function which accepts an additional string for notes/hints. i.e. localize_with_hint(key, hint, localizableString, ...stringArgs)

This would seem to require a change to the format of the exported JSON, or perhaps an additional exported file containing a mapping of hints to string keys, which can be read when generating the xlf for export.

Please provide instructions how to initialize i18n files without Transifex

Hi,

as I understand this tool can take existing vscode extension (where all messages are directly in package.json and in .ts files) and extract the messages to package.nls.json and to files inside i18n, replacing the messages with %placeholders% and locallize().

It can also push/pull messages from the Transifex platform, but it does not need the platform to extract the original messages in the above "init" step.

If that's true, could you please provide instructions how to use this tool to init the i18n files without the Transifex platform?

Thank you.

Suppress unnecessary error messages

In my node-debug extension I have a utilities.ts with this contents:

'use strict';

import * as vscode from 'vscode';
import * as nls from 'vscode-nls';

export const localize = nls.config(process.env.VSCODE_NLS_CONFIG)();

export function log(message: string) {
	vscode.debug.activeDebugConsole.appendLine(message);
}

When building I always get these unhelpful error messages:

[12:10:42] [i18n] Generating localized messages for 'chs' resulted in the following problems:
[12:10:42] [i18n]
[12:10:42] [i18n] Message file chs/out/node/extension/utilities.i18n.json not found. Missing messages: 0
[12:10:42] [i18n]
[12:10:42] [i18n]
[12:10:42] [i18n] Generating localized messages for 'cht' resulted in the following problems:
[12:10:42] [i18n]
[12:10:42] [i18n] Message file cht/out/node/extension/utilities.i18n.json not found. Missing messages: 0
[12:10:42] [i18n]
[12:10:42] [i18n]
[12:10:42] [i18n] Generating localized messages for 'jpn' resulted in the following problems:
[12:10:42] [i18n]
[12:10:42] [i18n] Message file jpn/out/node/extension/utilities.i18n.json not found. Missing messages: 0
[12:10:42] [i18n]
[12:10:42] [i18n]
[12:10:42] [i18n] Generating localized messages for 'kor' resulted in the following problems:
[12:10:42] [i18n]
[12:10:42] [i18n] Message file kor/out/node/extension/utilities.i18n.json not found. Missing messages: 0
[12:10:42] [i18n]
[12:10:42] [i18n]
[12:10:42] [i18n] Generating localized messages for 'deu' resulted in the following problems:
[12:10:42] [i18n]
[12:10:42] [i18n] Message file deu/out/node/extension/utilities.i18n.json not found. Missing messages: 0
[12:10:42] [i18n]
[12:10:42] [i18n]
[12:10:42] [i18n] Generating localized messages for 'fra' resulted in the following problems:
[12:10:42] [i18n]
[12:10:42] [i18n] Message file fra/out/node/extension/utilities.i18n.json not found. Missing messages: 0
[12:10:42] [i18n]
[12:10:42] [i18n]
[12:10:42] [i18n] Generating localized messages for 'esn' resulted in the following problems:
[12:10:42] [i18n]
[12:10:42] [i18n] Message file esn/out/node/extension/utilities.i18n.json not found. Missing messages: 0
[12:10:42] [i18n]
[12:10:42] [i18n]
[12:10:42] [i18n] Generating localized messages for 'rus' resulted in the following problems:
[12:10:42] [i18n]
[12:10:42] [i18n] Message file rus/out/node/extension/utilities.i18n.json not found. Missing messages: 0
[12:10:42] [i18n]
[12:10:42] [i18n]
[12:10:42] [i18n] Generating localized messages for 'ita' resulted in the following problems:
[12:10:42] [i18n]
[12:10:42] [i18n] Message file ita/out/node/extension/utilities.i18n.json not found. Missing messages: 0
[12:10:42] [i18n]
[12:10:42] [i18n]

Yes, there is no "utilities.i18n.json" but there are no missing translated strings either.
It would be great if these unnecessary errors could be suppressed.

localize('example', 'example \
example \
example')

... results in the backslashes being included in the exported string. However, these backslashes are intended to allow the literal to be multi-line, and are not intended to be part of the string.

microsoft / vscode-nls-dev Goto Github PK

vscode-nls-dev's People

Stargazers

Watchers

Forkers

vscode-nls-dev's Issues

Recommend Projects

Recommend Topics

Recommend Org