mozilla / web-ext

A command line tool to help build, run, and test web extensions

Web-ext

This is a command line tool to help build, run, and test WebExtensions.

Ultimately, it aims to support browser extensions in a standard, portable, cross-platform way. Initially, it will provide a streamlined experience for developing Firefox Extensions.

Documentation

• Getting started with web-ext
• Command reference

Here are the commands you can run. Click on each one for detailed documentation or use --help on the command line, such as web-ext build --help.

• run
  • Run the extension
• lint
  • Validate the extension source
• sign
  • Sign the extension so it can be installed in Firefox
• build
  • Create an extension package from source
• docs
  • Open the web-ext documentation in a browser

Installation

Using npm

First, make sure you are running the current LTS (long term support) version of NodeJS.

Global command

You can install this command onto your machine globally with:

npm install --global web-ext

For your project

Alternatively, you can install this command as one of the devDependencies of your project. This method can help you control the version of web-ext as used by your team.

npm install --save-dev web-ext

Next you can use the web-ext command in your project as an npm script. Here is an example where the --source-dir argument specifies where to find the source code for your extension.

package.json

"scripts": {
  "start:firefox": "web-ext run --source-dir ./extension-dist/",
}

You can always pass in additional commands to your npm scripts using the -- suffix. For example, the previous script could specify the Firefox version on the command line with this:

npm run start:firefox -- --firefox=nightly

Using Homebrew (unofficial)

The community maintains a web-ext formula.

brew install web-ext

Installation from source

You'll need:

• Node.js (current LTS)
• npm, 8.0.0 or higher is recommended

Optionally, you may like:

• nvm, which helps manage node versions

If you had already installed web-ext from npm, you may need to uninstall it first:

npm uninstall --global web-ext

Change into the source and install all dependencies:

git clone https://github.com/mozilla/web-ext.git
cd web-ext
npm ci

Build the command:

npm run build

Link it to your node installation:

npm link

You can now run it from any directory:

web-ext --help

To get updates, just pull changes and rebuild the executable. You don't need to relink it.

cd /path/to/web-ext
git pull
npm run build

Using web-ext in NodeJS code

Note: There is limited support for this API.

Aside from using web-ext on the command line, you may wish to execute web-ext in NodeJS code.

As of version 7.0.0, the web-ext npm package exports NodeJS native ES modules only. If you are using CommonJS, you will have to use dynamic imports.

Examples

You are able to execute command functions without any argument validation. If you want to execute web-ext run you would do so like this:

import webExt from 'web-ext';

webExt.cmd
  .run(
    {
      // These are command options derived from their CLI conterpart.
      // In this example, --source-dir is specified as sourceDir.
      firefox: '/path/to/Firefox-executable',
      sourceDir: '/path/to/your/extension/source/',
    },
    {
      // These are non CLI related options for each function.
      // You need to specify this one so that your NodeJS application
      // can continue running after web-ext is finished.
      shouldExitProgram: false,
    },
  )
  .then((extensionRunner) => {
    // The command has finished. Each command resolves its
    // promise with a different value.
    console.log(extensionRunner);
    // You can do a few things like:
    // extensionRunner.reloadAllExtensions();
    // extensionRunner.exit();
  });

If you would like to run an extension on Firefox for Android:

import * as adbUtils from "web-ext/util/adb";

// Path to adb binary (optional parameter, auto-detected if missing)
const adbBin = "/path/to/adb";
// Get an array of device ids (Array<string>)
const deviceIds = await adbUtils.listADBDevices(adbBin);
const adbDevice = ...
// Get an array of Firefox APKs (Array<string>)
const firefoxAPKs = await adbUtils.listADBFirefoxAPKs(
  deviceId, adbBin
);
const firefoxApk = ...

webExt.cmd.run({
  target: 'firefox-android',
  firefoxApk,
  adbDevice,
  sourceDir: ...
}).then((extensionRunner) => {...});

If you would like to control logging, you can access the logger object. Here is an example of turning on verbose logging:

import * as webExtLogger from 'web-ext/util/logger';

webExtLogger.consoleStream.makeVerbose();
webExt.cmd.run({ sourceDir: './src' }, { shouldExitProgram: false });

You can also disable the use of standard input:

webExt.cmd.run({ noInput: true }, { shouldExitProgram: false });

web-ext is designed for WebExtensions but you can try disabling manifest validation to work with legacy extensions. This is not officially supported.

webExt.cmd.run(
  { sourceDir: './src' },
  {
    getValidatedManifest: () => ({
      name: 'some-fake-name',
      version: '1.0.0',
    }),
    shouldExitProgram: false,
  },
);

Should I Use It?

Yes! The web-ext tool enables you to build and ship extensions for Firefox. This platform stabilized in Firefox 48 which was released in April of 2016.

Get Involved

Hi! This tool is under active development. To get involved you can watch the repo, file issues, create pull requests, or contact us to ask a question. Read the contributing section for how to develop new features.

Some Questions and Answers

Why do we need a command line tool?

This is a great question and one that we will ask ourselves for each new web-ext feature. Most WebExtension functionality is baked into the browsers themselves but a complimentary command line tool will still be helpful. Here is a partial list of examples:

• File watching.
  • When you edit a file, you may need to trigger certain commands (tests, installation, etc).
• Integrating with services.
  • Mozilla offers some useful services such as linting and signing extensions.

web-ext's Issues

Remove all the Flow regex-ignore hacks

Flow's slow startup time has been fixed in 0.22 (see facebook/flow#1461 for context). We can now remove the auto-generated ignore rules. Things to remove:

• config fixing tasks
• config check from tests and travis CI
• all mention of it in the docs

We will still need to ignore some modules because of Flow bugs (e.g. facebook/flow#1420)

Set up initial scaffolding to run as an ES6 command line script

Just get ./bin/web-ext to work

Save built Firefox extension to a firefox directory

To prepare for multiple build targets (#119) an easy fix would be to output the .xpi Firefox extension into a directory like web-ext-build/firefox.

Add --ignore-files to ignore a custom list of files in build, run, and lint

#85 added some simple defaults for ignoring files when building an XPI. Let's support --ignore-files that the developer can use to declare custom rules.

See #176 for general config file support which will allow developers to configure this option.

Enforce a minimum percentage of code coverage

We landed code coverage in #89 but it doesn't cause the tests to fail if a patches fails to achieve a set percentage of code coverage (correct me if I'm wrong). We should set something for this, maybe enforce 90% coverage?

Protect commands from using the wrong options

We have command handlers like:

export function build({sourceDir}) {
  // work with sourceDir...
}

which rely on the main program from parsing --source-dir and passing it like build({sourceDir: ...}).

I would like to find a way to guarantee that command handlers are declaring only the available option arguments. One solution is to execute unnecessary code such as the main program code for every unit test of a function. I'd like to avoid that if possible since it could introduce slowness (maybe it's not so slow?).

I thought I could do this with Flow and did so in #39 but as of #105 it is becoming difficult to maintain so I removed it.

Install/run a web extension using custom preferences

A run/install command was added in #6 but it only launches Firefox with default preferences. Allow the caller to specify custom preferences. This would be a more lightweight way of customizing Firefox as opposed to creating a custom profile (#69).

Run greenkeeper to keep libs up to date

http://greenkeeper.io/

Set up initial scaffolding to run tests

Get npm test set up

Cannot build web-ext in node 5.7.0

It works fine in node 4.3.0 and under. I'm not sure what this error is about:

$ npm run build

> [email protected] build /Users/kumar/tmp/web-ext-trash/web-ext
> node -e "require('grunt').cli()" null build

Running "clean:dist" (clean) task
Warning: Cannot delete files outside the current working directory. Use --force to continue.

Aborted due to warnings.

npm ERR! Darwin 14.5.0
npm ERR! argv "/Users/kumar/.nvm/versions/node/v5.7.0/bin/node" "/Users/kumar/.nvm/versions/node/v5.7.0/bin/npm" "run" "build"
npm ERR! node v5.7.0
npm ERR! npm  v3.6.0
npm ERR! code ELIFECYCLE
npm ERR! [email protected] build: `node -e "require('grunt').cli()" null build`
npm ERR! Exit status 6
npm ERR!
npm ERR! Failed at the [email protected] build script 'node -e "require('grunt').cli()" null build'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the web-ext package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node -e "require('grunt').cli()" null build
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs web-ext
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls web-ext
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/kumar/tmp/web-ext-trash/web-ext/npm-debug.log

Do basic manifest validation

This is to check for some very basic manifest validation errors before building a package. This is not to add full integration with addons-linter.

See: https://github.com/mozilla/web-extension-manifest-schema/
Also: mozilla/addons-linter#494

Command to print web-ext version

Print the current package version of web-ext

Add init command to start a web extension

It might be a nice shortcut to provide an init command similar to jpm that starts a web extension from scratch by interactively prompting you for manifest values.

Validate value of extension ID before run/install

Before executing the run/install command (#6), the contents of the manifest ID value should be validated. This is because it will be used to write a file to disk in the profile/extensions directory and bad things might happen if the ID is invalid.

Hook up Travis CI for tests

tests and lint checks

Convert all of web-ext promise chains to coroutines

All code that currently builds chains of promises would be way more concise using co. Let's rewrite it!

Example of rewriting test code:

it('zips a package', () => withTempDir(
  (tmpDir) =>
     co(function*() {
       let zipFile = new ZipFile();

       let {extensionPath} = yield adapter.buildMinimalExt(tmpDir);
       assert.match(extensionPath, /minimal_extension-1\.0\.xpi$/);

       yield zipFile.open(extensionPath);

       let fileNames = [];
       let result = yield zipFile.readEach((entry) => fileNames.push(entry.fileName));

       assert.deepEqual(fileNames, ['manifest.json']);

       return result;
     })
));

Run an extension in a linked profile for persistence

Calling run --firefox-profile=myprofile will copy a custom profile into a temporary profile. It will not affect the original profile. This new feature is to run the extension directly in the custom profile so that any profile changes will be saved.

Example use case: if you're developing an extension that has a splash screen on install, you may not want to see that screen every time you launch Firefox to test it.

We should probably introduce a new option that can be called alongside the existing ones like this:

web-ext run --firefox-profile=myprofile --keep-profile-changes

web-ext --version results in ENOENT: no such file or directory, open 'package.json'

The code that checks for a package.json file is broken. We rely on __dirname which webpack apparently sets to a relative directory which is unhelpful. We need a better way to detect the path and find the package.json file. Maybe there is a way to reach into node internals? For some reason require.main.filename is also undefined.

ideas:

• Look at $PATH and see if we can find the web-ext executable
• Look at some system module and try to figure out where web-ext is

Exit non-zero upon exception

There is just a placeholder right now where a fatal error just gets logged to the console. We should:

• Exit non-zero
• Show some nice output
• Preserve the traceback

Check for updates to self and alert users

Because it's likely to be installed globally and not manually updated (I, and--anecdotally--many other developers, aren't in the habit of updating their globally installed commands on a regular basis), web-ext should check for updates to itself every once in awhile. A day or two is probably fine.

It should simply alert the user to the fact that there's a new version and the npm command to install it.

Protect command functions via static typing

Since we have Flow, I think there is a way to protect all command handlers from signature breakage by annotating them with a custom. This would allow us to unit test them directly without worry since either the test invocation or program invocation would fail when the type changes.

Protect option configuration from breaking handlers

In #39 I protected command handlers from accessing undeclared program options. However, it is still possible to add an option to the program and forget to declare it (in ProgramOptions). This could result in handler unit tests that silently fail because they are testing with stale options. Let's see if we can solve it without resorting to a slow integration test. Perhaps it can be solved with static analysis.

Automatically reload extension in Firefox as you edit code

When editing web extension source code, reload the extension in Firefox so you can test it as you save your source files. There might be a couple ways to do this.

Firstly, the platform supports a special symlink file: If you put a text file in profile/extensions whose name is the addon id and the content is the absolute path to your addon source, you can reload it.

To reload it, you just need to disable it, flush the cache, and re-enable it (see flushJarCache() and flushStartupCache()). We will add a button for this but ideally web-ext could use the debugging API to send a message to Firefox to reload when necessary.

When reloaded, the add-on will see the shutdown and startup methods called, it won't see the uninstall/install called, but you probably don't want that for reload anyway

This reload trick won't work for XPI paths, only for directory paths.

Name built xpi according to manifest

The build command should name the destination XPI accordingly. It might be easy to do #34 first.

Run an extension in a custom profile

#6 will create a temporary profile every time. Add an option to copy an existing profile.

Run web extension tests

This command should execute automated tests for a web extension within Firefox

Set up Flow for better mock testing

Flow lets you do opt-in static type checking in JavaScript. I'd like to add this mostly for its duck-typing features so that we can get more effective mock testing. For example, when you refactor code that uses mocks this should help catch errors for mocks that no longer reflect reality.

Install/run an extension from a directory of source code

#6 builds an XPI then runs the extension. It can be modified to run from a directory of code (i.e. skip the build step).

Initial command to build (zip up) a web extension

This should zip a directory of source into a web extension package

Remove all Flow adapters that are used for testing

Dev dependencies are ignored by Flow because of slow startup time. This meant that it was hard to opt-in test files to Flow. After #76 I think we will be able to opt-in all test files. This also means we can delete all the tests/.../adapter.js files which were a hacky way to get some Flow coverage.

Document how to install / run web-ext from source

needs explicit instructions in the README

Set up initial scaffolding to run lint checks

run lint checks on web-ext code itself

Build web extensions without an ID

In support of https://wiki.mozilla.org/WebExtensions/Applications, this will let you build an extension without an ID.

Add a logger

We need a logger for all web-ext commands, one that can be configured for verbose logging.

Add code coverage

Before we get too deep into web-ext, let's set up code coverage

Error: Cannot find module 'source-map-support'

I tried that tools and I get that errors that I've fixed in this way.
After npm install and launch npm run build I get this error on fx-runner cujojs/when#482
Fixed with this: https://github.com/yahoo/flux-examples/issues/137#issuecomment-93917057

When I try to run web-ext:

mte90:/tmp/web-ext/bin  ⑂ master +  $  ./web-ext                                                                                                                
module.js:341                                                                                                                                                   
    throw err;                                                                                                                                                  
    ^                                                                                                                                                           

Error: Cannot find module 'source-map-support'
    at Function.Module._resolveFilename (module.js:339:15)
    at Function.Module._load (module.js:290:25)
    at Module.require (module.js:367:17)
    at require (internal/module.js:16:19)
    at Object.<anonymous> (/tmp/web-ext/dist/web-ext.js:1:63)
    at Module._compile (module.js:413:34)
    at Object.Module._extensions..js (module.js:422:10)
    at Module.load (module.js:357:32)
    at Function.Module._load (module.js:314:12)
    at Module.require (module.js:367:17)

fixed with npm install source-map-support

After that fix the tools works without problem :-)

Ignore common artifacts when zipping a directory

The build command (#4) currently zips up the entire directory. It should ignore things like .git and other artifact. This is how jpm does it: https://github.com/mozilla-jetpack/jpm/blob/master/lib/ignore.js#L23

Load temporary add-on from `run` command

The install/run command landed in #90 and by default this creates a temporary profile, opens Firefox, and loads an extension. However, we have a button for this same work flow in Firefox. It would be great if you could load a temporary add-on into your current running Firefox from web-ext.

Command to install/run a web extension

This command should install a web extension into Firefox for development. Here is a list of nice things that might be done now or later:

• create a temporary profile
• disable signing by default
• allow custom profiles
• install as a temporary add-on
• open a debugger
• option to keep Firefox running, waiting for changes, so it can reinstall the extension

Add contribution docs

Let's move the development docs to a standard CONTRIBUTING.md file so that it shows up before someone submits a pull request.

Run web-ext tests continuously

We need to watch for file changes and run the test suite on each change

Run extension in a custom Firefox binary

#6 installs/runs the extension in the default Firefox. Add an option to specify a path to a custom Firefox.

Build command should not overwrite existing extension

The build command currently clobbers an existing extension at the destination path. We should probably not do that and maybe provide an option to allow it.

Create official documentation

Document how to use the web-ext command. This will probably live on MDN because we'd want contributors to edit it easily.

Hide logs from test output

Once we have a logger (#33) we can hide log output from the test run. However, I want to see log output when there is a failure. Hopefully we can accomplish that by creating a mocha reporter or something.

Add command to sign a web extension

Similar to jpm sign, one should be able to run web-ext sign to build and sign the extension.

Change --firefox-binary to --firefox

web-ext run --firefox=/path/to/firefox-bin seems more concise. We should also give it a short alias like -f.

Define all commands as yargs modules

I think it would be a nicer layout if all commands were defined as yargs modules.

This would allow us to code to the yargs interface which should make things more robust. It may even address #65 . At the very least it will make it easier to look at the configuration of a single command all in one place.

Upgrade grunt-flow-type-check when released

When isuttell/grunt-flow-type-check#4 is fixed, switch from the git repo to the npm release (in package.json).

Package also for chrome

Is possible add a parameter for build that package the extension for chrome in a zip file?
Basically the web-ext works also for chrome without the applications section in the manifest.json.
Actually to do that I remove that part in the manifest.json and create a zip to upload on the Chrome Web Store with the same files of the firefox version.