rolldown / rolldown

Fast Rust bundler for JavaScript with Rollup-compatible API.

🚧 Work in Progress

Rolldown is currently in active development and not usable for production yet.

Rolldown

Rolldown is a JavaScript bundler written in Rust intended to serve as the future bundler used in Vite. It provides Rollup-compatible APIs and plugin interface, but will be more similar to esbuild in scope.

For more information, please check out the documentation at rolldown.rs.

Contributing

We would love to have more contributors involved!

To get started, please read our Contributing Guide.

Credits

The Rolldown project is heavily inspired by:

• Rollup, created by Rich Harris and maintained by Lukas Taegert-Atkinson.
• esbuild, created by Evan Wallace.

And supported by:

• napi-rs for Node.js add-ons in Rust via Node-API.
• oxc for the underlying parser, resolver, and sourcemap support.

Licenses

This project is licensed under the MIT License.

This project also partially contains code derived or copied from the following projects:

• rollup(MIT)
• esbuild(MIT)

Licenses of these projects are listed in THIRD-PARTY-LICENSE

rolldown's Issues

render module with source mutation instead of ast mutaition

Description

Module render implementation has two ways.

Module Render With Ast Mutation

Replaced the ast nodes then codegen ast.eg Babel transform, esbuild bundling.

Advantages

The chunk render can be implemented by concat the modules ast, and also chunk minify/format can be implemented by one ast parse. The cold start performance is very fast.

Drawbacks

Do ast mutation causes the ast to be dirty at rebuild, we must parse the code again and cause rebuild performance regression.

Other the developer experience is bad because we need to manually create ast nodes, and the code snip is huge and difficult to review.

Module Render With Source Mutation

Replace the original source using the ast span position. eg Rollup using MagicString, webpack using ReplaceSource.

Drawbacks

The chunk render can be implemented by concat the modules mutation source and the minify/format needs to be parse again.

Rolldown main branch implement

• Module render with Ast Mutation

Here has walk ast tree twice, one is scan ast to launch module graph, one is walk ast tress to run ast mutation

• Chunk render with concat modules ast codegen result.

Launch stage

Should be finished at early stage, it will block tree shaking codegen and other things.

Launch Source Mutation Step

Add source Mutation action abstract and scan the mutation actions with walking ast tree once.

runtime module

Runtime Module

Summary

The bundler will attach some runtime code to keep the bundler code running successfully.

Interop Runtime Module

Summary

Because the ecosystem package has lots of commnjs packages. The rollup provider @rollup/plugin-commonjs transforms commonjs to esm, but it is complex becuase commonjs can't static analyzer at some edge cases. So rolldown will treat commonjs as first-class citizens and support it by following esbuild implementation.

The esbuild introduce __toESM __commonJS __toCommonJS __esm for commonjs/esm interop.

__commonJS

If a module is commonjs and it is imported by esm. The module will wrap a __commonJS closure and return a require() function, the require function will return module.exports. See a example

// index.js
import value from './commonjs.js'
value;
// ./commonjs.js
module.exports = 1

It will compiled to

var require_commonjs = __commonJS({
  "./commonjs.js"(exports, module) {
    module.exports = 1;
  }
});

var import_util = __toESM(require_commonjs());
console.log((0, import_util.default)());

__toESM

Converts the module from CommonJS to ESM. The interop esbuild following webpack implementation to compat node.js mode & babel mode.

• In an ".mjs" file or package.json has "type: module", use node.js compat mode.
• detect __esModule to compat babel mode.

__toCommonJS

Converts the module from ESM to CommonJS. It will add __esModule to support babel compat mode.

__esm

This is for lazily-initialized ESM code.

How to inject runtime module

Esbuild

The esbuild will generate a runtime file as normal module, collect the usage symbols and link it at module graph, so unused symbols will be deleted at tree shaking.
But the implementation will generate a runtime chunk at multiple entries and it is smaller.

Webpack

The webpack use lots of RuntimeGlobals and sets their dependency information manual. You can see here for detailed at
web-infra-dev/rspack#1049.

Rolldown

Rolldown will fllowing the esbuild implementation because it is simple and clear for runtime architecture.

Implementation Step

• #16
• generate runtime file at module graph
• link runtime symbols

Execute the build results of rust side tests

This make passed tests more reliable

Pass https://github.com/rolldown-rs/rolldown/tree/main/crates/rolldown/tests/esbuild/splitting/.shared-commonjs-into-es6

Enable rollup test suites

• #45
• #44
• #22
• #18
• #46
• #47
• #48
• #49
• #16
• #52

Pass `test_tests__esbuild__default__export_forms_common_js__test_config_json `

generated code

// entry.js
(init_commonjs(), __toCommonJS(commonjs_ns))
(init_c(), __toCommonJS(c_ns))
(init_d(), __toCommonJS(d_ns))
(init_e(), __toCommonJS(e_ns))
(init_f(), __toCommonJS(f_ns))
(init_g(), __toCommonJS(g_ns))
(init_h(), __toCommonJS(h_ns))

error

(init_d(), __toCommonJS(d_ns))
^

TypeError: (init_c(...) , __toCommonJS(...)) is not a function

reason

Js treat

(init_commonjs(), __toCommonJS(commonjs_ns))
(init_c(), __toCommonJS(c_ns))

as

(init_commonjs(), __toCommonJS(commonjs_ns))(init_c(), __toCommonJS(c_ns))

while we want to emit code like

(init_commonjs(), __toCommonJS(commonjs_ns));
(init_c(), __toCommonJS(c_ns));

[Tracking] Support `output.format: cjs`

• Rust test suite for testing cjs output
• Emit cjs output

Runtime module should be treated as normal module in code splitting

See this for details.

support commonjs

Summary

As description of #35, we will following esbuild bundle commonjs strategy.

Linker

• iter the import records of all modules to identify which module should be wrapped.
• create binding symbol for esm module import cjs module

Module codegen

Wrap module to lazy execute

• commonjs module

module.exports = 1
// compiled
var require_a = __commonJS({
  "a.js"(exports, module) {
    module.exports = 1;
  }
});

• the esm module which imported by commonjs

export const a = 1;
// compiled
var a; 
var init_b = __esm({
  "b.js"() {
    a = 1;
  }
});

Commonjs module codegen

1. wrapped module with __commonJS
2. rewrite require call to init module expr call

• require esm module

require('esm');
// compiled
var require_a = __commonJS({
  "a.js"(exports, module) {
     (init_esm(), __toCommonJS(esm_exports));
  }
});

• require cjs module

require('cjs');
// compiled
var require_a = __commonJS({
 "a.js"(exports, module) {
   require_cjs();
 }
});

Esm module codegen

• rewrite import cjs module

import cjs from './cjs'
console.log(cjs)
// compiled
var import_cjs = __toESM(require_cjs());
console.log(import_cjs.default)

• rewrite import wrapped esm module

import esm from './esm'
console.log(esm)
// compiled
init_esm()
console.log(esm)

Wrapped esm module codegen

• the import declaration is same as esm module
• wrapped module with __esm
• hoisted named declaration function

export function a(){}
// compiled
function a(){}
var b_ns = {
   get a () => a
}
var init_b = __esm({
  "b.js"() {
  }
});

• hoisted default declaration function

export default function a(){}
// compiled
function a(){}
var b_ns = {
   get default () => a
}
var init_b = __esm({
  "b.js"() {
  }
});

• hoisted top-level symbol definition, including namespace definition

export default  1
export const a = 1
// compiled
var a, b_default;
var b_ns = {
   get default () => b_default,
   get a() => a
}
var init_b = __esm({
  "b.js"() {
      b_default = 1
      a = 1
  }
});

Implemention

• scan module kind (cjs/esm)
• scan wrapped module at linker, create symbols for esm module import cjs module, including ast codegen
• runtime symbol deconflict
• runtime symbol mark used at linker
• hoisted function using magic string move api
• #42
• add cjs_compat mode option
• using magic string ident api to ident wrap module function
• #55
• reexport with commonjs
• export * from commonjs at entry
• #74
• #86

remove rolldown_plugin, move plugin code to the rolldown crate

rolldown_plugin will used a lot of data of rolldown crate. If we don't put them together, we need to create a lot proxy structs to forward the data. This is not ideal.

Image that we need put NormalModule into rolldown_common crate.

Auto generated esbuild tests.

1. A tool automatically converts esbuild tests to snapshot test

Merge `render` and `finalize` passes into one pass

This might lead to remove SourceMutation abstraction. We could mutate MagicSting directly, rather than using SourceMutation.

esm & cjs interop support node.js compat

Interop with Esm & commonjs has two mode.

• babel mode will detect __esModule to identify the original commonjs or esm transformed commonjs and do symbol binding
• node mode don't do this, node.js always binds the default export to the value of module.exports. It will enabled at .mjs file and type: module at package.json.

ref: https://babeljs.io/docs/babel-plugin-transform-modules-commonjs#importinterop

support mix esm and cjs module

If esm module mixed cjs require syntax, it can worked becuase treat it as esm.

• Using mix esm import and cjs require
• Using mix esm export and cjs require

So here only has cjs exports syntax will be as cjs module.

If cjs module mixed esm syntax

• Using cjs export and mix esm import , it will give an warning and treat it as cjs.
  https://esbuild.github.io/try/#YgAwLjE5LjMALS1idW5kbGUgLS1zcGxpdHRpbmcgLS1vdXRkaXI9ZGlzdCAtLWZvcm1hdD1lc20AZQBlbnRyeS5tanMAaW1wb3J0ICcuL2EnCm1vZHVsZS5leHBvcnRzID0gMQAAYS5qcwBpbXBvcnQgJy4vYicKbW9kdWxlLmV4cG9ydHMgPSAxAABiLmpzAG1vZHVsZS5leHBvcnRzID0gMQ

• Using cjs export and mix esm export,It will give an warning and treat it as esm(cjs export is ignored).
  https://esbuild.github.io/try/#YgAwLjE5LjMALS1idW5kbGUgLS1zcGxpdHRpbmcgLS1vdXRkaXI9ZGlzdCAtLWZvcm1hdD1lc20AZQBlbnRyeS5qcwBpbXBvcnQgJy4vYScKbW9kdWxlLmV4cG9ydHMgPSAxAABhLmpzAGV4cG9ydCBjb25zdCBhID0gMQptb2R1bGUuZXhwb3J0cyA9IDEAAGIuanMAbW9kdWxlLmV4cG9ydHMgPSAx

• warning for syntax error

• implement mix commnjs and esm.

Ensure chunk execution order are correct

[Tracking]: error/warning mechanism

The issue is to tracking todos about error/warning mechanism

• #331
• #154
• Print errors/warnings in node binding

Enhancements

• #152

Misc

`test_tests__esbuild__default__common_js_from_es6__test_config_json` has different compiled output compared to esbuild

https://github.com/evanw/esbuild/blob/4e11b50fe3178ed0a78c077df78788d66304d379/internal/bundler_tests/bundler_default_test.go#L110-L135

rolldown

import { __esm, __toCommonJS } from "./_rolldown_runtime.mjs";

// bar.js
function bar$1() {
	return 'bar'
}
var bar_ns = {
  get bar() { return bar$1 }
};
var init_bar = __esm({
'bar.js'() {

}
});
// foo.js
function foo$1() {
	return 'foo'
}
var foo_ns = {
  get foo() { return foo$1 }
};
var init_foo = __esm({
'foo.js'() {

}
});
// entry.js
const {foo} = (init_foo(), __toCommonJS(foo_ns))
console.log(foo(), bar())
const {bar} = (init_bar(), __toCommonJS(bar_ns)) // This should not be hoisted

esbuild

https://github.com/evanw/esbuild/blob/4e11b50fe3178ed0a78c077df78788d66304d379/internal/bundler_tests/snapshots/snapshots_default.txt#L802-L834

// foo.js
var foo_exports = {};
__export(foo_exports, {
  foo: () => foo
});
function foo() {
  return "foo";
}
var init_foo = __esm({
  "foo.js"() {
  }
});

// bar.js
var bar_exports = {};
__export(bar_exports, {
  bar: () => bar
});
function bar() {
  return "bar";
}
var init_bar = __esm({
  "bar.js"() {
  }
});

// entry.js
var { foo: foo2 } = (init_foo(), __toCommonJS(foo_exports));
console.log(foo2(), bar2());
var { bar: bar2 } = (init_bar(), __toCommonJS(bar_exports));

problems need to investigate

• different orders
• different output for the same module

Wasm Build

Let's check for wasm build early so we don't get caught up with technical debt when we start shipping wasm.

See https://github.com/web-infra-dev/oxc/blob/3605f7c0fc2421496a869c2b8fb470340903a21f/.github/workflows/ci.yml#L57

Setup warm cache for CI to speed it up

See how it's being configured and used in oxc:

• https://github.com/web-infra-dev/oxc/blob/4b719026bf9ce1fd6f6f82eaf7a3c95d43a6bdfb/.github/workflows/ci.yml#L38
• https://github.com/web-infra-dev/oxc/blob/4b719026bf9ce1fd6f6f82eaf7a3c95d43a6bdfb/.github/actions/rustup/action.yml#L79-L84

The main idea is that the cache is only saved on the main branch, and PR branches only pull the cache but never save them. This setup will always hand us a warm cache everywhere due to the 10G cache constraint. You may find the cache storage limit page helpful https://github.com/web-infra-dev/oxc/actions/caches

More clippy rules

https://github.com/web-infra-dev/oxc/blob/main/.cargo/config.toml#L15

Improve Rustfmt

See https://github.com/web-infra-dev/oxc/blob/main/rustfmt.toml

Let's use use_small_heuristics = "Max", my experience is that it provides the best readability. This also aligns with rustc.

You may also copy more from https://github.com/rust-lang/rust/blob/ddef56d5dfa18f169af9db912dc8e8343797eebb/rustfmt.toml#L3

Support basic plugin hooks

https://github.com/rolldown-rs/rolldown_legacy/blob/main/crates/rolldown_plugin/src/plugin.rs

basic supporting for external module

Description

The external module

Setup rollup test suite

https://github.com/rolldown-rs/rolldown_legacy/tree/main/packages/rollup-tests

[Tracking]: code splitting

The issue is to tracking todos about code splitting for milestone 1

Correctness

• Prevent exports of entry chunk/module being renamed

This won't happens.

• #122
• #110

Enhancements

• #137

Refactor

• #103

Misc

• #111

basic code splitting

Summary

The following will trigger code splitting.

• entries
• shared module
• dynamic import
• manual chunks option

Split Modules to chunk by entries

1. Mark an entry that has a corresponding bit flag.
2. Mark the entry bit flag of the module by iter entries. The entry bits flag of the module may be imported by multiple entries bit flag.
3. Generate chunks for entries, and add the corresponding bit flag to the chunk.
4. Iter modules to compare the entry bit flag of the module with the entry bit flag of already generated chunks.

• If a chunk has an equal entry bit flag of the module and connect the module with the chunk.
• Else generate a new chunk for the entry bit flag of the module and connect the module with the chunk.

Cross chunks symbol imports and exports

Dynamic import module generate a chunk

Scanned the dynamic import module added to entries.

Step implementation

• split Modules to chunk by entries
• cross chunks symbol imports and exports
• dynamic import as entries
• manual chunk object
• manual chunk function

Cargo deny

See https://github.com/web-infra-dev/oxc/blob/main/deny.toml and https://github.com/web-infra-dev/oxc/blob/3605f7c0fc2421496a869c2b8fb470340903a21f/.github/workflows/ci.yml#L85

basic error/warning support

• show code frame
• add warning kind

Diagnostics for errors/warnings

I had a prototype for this.

For unresolved entry error, we would have a Dianostic to show

rather than the current

https://github.com/rolldown-rs/rolldown/blob/0c9ab02bd89da358a0d9fb9719e85cd63d0f4aad/crates/rolldown/tests/fixtures/errors/unresolved_entry/artifacts.snap#L8-L12

Milestone 1: replace esbuild with rolldown in vite

POC 1

• #22
• #18
• #16
• #35
• #36
• #113

POC 2

• support format: cjs
• node binding
  • plugin & plugin driver
• hash support
• complete supporting for  external module
• chunk hash

• module graph
• esm syntax ast scan & linking imports/exports
• module render with ast mutaition and ast codegen (symbol deconflict)
• chunk render with concat module ast codegen result
• #17
• glob import
• ci release build
• simple benchmark
• more esbuild/rollup test migrate
• options

support `output.exports: 'auto'`

Rollup tests rely on this feature https://github.com/rolldown-rs/rolldown_legacy/blob/b2c5660c83a501b091bba8b74e52da4974b55c6f/packages/rollup-tests/test/function/index.js#L76

basic tree shaking

How to apply __esmMin or __esm runtime symbol

Esbuil provide __esmMin which using production and __esm which using development, it is controlled by MinifyIdentifiers option. But it is need discuss how to control at rolldown. Here has two ways to control it.

• Add similar option to do it, maybe MinifyIdentifiers
• Add mode which value be production or development

Support `shimMissingExports`

https://rollupjs.org/configuration-options/#shimmissingexports

The feature block
tests

• test_tests__esbuild__import_star__import_namespace_undefined_property_side_effect_free_file__test_config_json

in #120 being passed.

Should not generate namespace binding for `ExportsKind::CommonJs`

Founded in #126. See https://github.com/rolldown-rs/rolldown/blob/c18181625c3aae63bf2fe7bda0dd66ff33a283d5/crates/rolldown/tests/esbuild/splitting/shared-commonjs-into-es6/artifacts.snap for example.

Some modules try to import the namespace binding of a cjs module, which is not exist. This make test_tests__esbuild__splitting__shared_commonjs_into_es6__test_config_json failed. To make the test works, #126 make the cjs module generate namespace binding.

This is a temp workaround. I will investigate why some modules try to import the namespace binding of a cjs module.

Should render paths in errors in relative based on the `cwd` that users passed

See https://github.com/rolldown-rs/rolldown/blob/0c9ab02bd89da358a0d9fb9719e85cd63d0f4aad/crates/rolldown_error/src/lib.rs#L16-L21 for details.

panic with empty module if wrapped with commonjs

Description

Because magic-string new will panic if the source is empty.

runtime symbol codegen at chunk gen

Use oxc_resolver

https://docs.rs/oxc_resolver/latest/oxc_resolver/

Support `external` option

Fix tests that are executed failed

Those tests are failed to be executed, founded in #119.

• test_tests__esbuild__default__arrow_fn_scope__test_config_json
• #135
• test_tests__esbuild__default__const_with_let__test_config_json
• #136
• test_tests__esbuild__default__forbid_const_assign_when_bundling__test_config_json
• test_tests__esbuild__default__forbid_const_assign_when_lowering_using__test_config_json
• test_tests__esbuild__default__import_missing_common_js__test_config_json
• test_tests__esbuild__default__import_then_catch__test_config_json
• test_tests__esbuild__default__nested_require_without_call__test_config_json
• test_tests__esbuild__default__preserve_key_comment__test_config_json
• test_tests__esbuild__default__require_property_access_common_js__test_config_json
• test_tests__esbuild__default__require_resolve__test_config_json
• test_tests__esbuild__default__require_without_call__test_config_json
• test_tests__esbuild__default__this_inside_function__test_config_json
• test_tests__esbuild__default__var_relocating_bundle__test_config_json
• test_tests__esbuild__import_star__import_namespace_undefined_property_side_effect_free_file__test_config_json
• test_tests__esbuild__import_star__import_of_export_star__test_config_json
• test_tests__esbuild__import_star__import_of_export_star_of_import__test_config_json
• test_tests__esbuild__splitting__shared_commonjs_into_es6__test_config_json

Enhancements

Not bug.

• test_tests__esbuild__default__dynamic_import_with_template_iife__test_config_json

Refactor logic of generating exports of entry chunk

Current solution is a temp workaround not thinking deeply.

[Tracking]: prettify the build output

"prettify the build output" is not the primary goal currently, the issue is only for recording.

• #440
• Empty wrapper should be in a single line
• Merge import specifiers that import from the same module
• remove empty statements
• Remove meaningless lines of modules.

Split runtime module into a single chunk in snapshot test of rust side

node binding

Should also have https://github.com/rolldown-rs/rolldown_legacy/tree/main/packages/core.

Rust v1.73.0

Deterministic chunk name for dynamic chunk and shared chunk

• Shared chunk name should not be number https://github.com/rolldown-rs/rolldown/blob/2401386c6c9d8c48c6f2eec03373b007d8fdd571/crates/rolldown/src/bundler/bundle/bundle.rs#L95

• The name here for dynamic chunk could be duplicated https://github.com/rolldown-rs/rolldown/blob/2401386c6c9d8c48c6f2eec03373b007d8fdd571/crates/rolldown/src/bundler/module_loader/module_loader.rs#L96

Render passed and failed tests status in details

See https://github.com/web-infra-dev/oxc/blob/main/tasks/transform_conformance/babel.snap.md for example.

symbol deconflicting for nested scopes

https://rollupjs.org/repl/?version=4.1.4&shareable=JTdCJTIyZXhhbXBsZSUyMiUzQW51bGwlMkMlMjJtb2R1bGVzJTIyJTNBJTVCJTdCJTIyY29kZSUyMiUzQSUyMmltcG9ydCUyMCU3QiUyMHglMjAlN0QlMjBmcm9tJTIwJy4lMkZkZXAuanMnJTNCJTVDbiU1Q25jb25zdCUyMHZhbHVlJTIwJTIwJTNEJTIwJ21haW4uanMnJTVDbiU1Q24lNUNuYXNzZXJ0LmVxdWFsKHgoJ25hbWUnKSUyQyUyMCduYW1lMScpJTNCJTVDbmNvbnNvbGUubG9nKHZhbHVlKSUyMiUyQyUyMmlzRW50cnklMjIlM0F0cnVlJTJDJTIybmFtZSUyMiUzQSUyMm1haW4uanMlMjIlN0QlMkMlN0IlMjJjb2RlJTIyJTNBJTIydmFyJTIwdmFsdWUlMjAlM0QlMjAnMSclM0IlNUNuJTVDbmV4cG9ydCUyMGZ1bmN0aW9uJTIweCh2YWx1ZSUyNDEpJTIwJTdCJTVDbiU1Q3RyZXR1cm4lMjB2YWx1ZSUyNDElMjAlMkIlMjB2YWx1ZSUzQiU1Q24lN0QlMjIlMkMlMjJpc0VudHJ5JTIyJTNBZmFsc2UlMkMlMjJuYW1lJTIyJTNBJTIyZGVwLmpzJTIyJTdEJTVEJTJDJTIyb3B0aW9ucyUyMiUzQSU3QiUyMm91dHB1dCUyMiUzQSU3QiUyMmZvcm1hdCUyMiUzQSUyMmVzJTIyJTdEJTJDJTIydHJlZXNoYWtlJTIyJTNBdHJ1ZSU3RCU3RA==

Vite PreBundling used esbuild api

Scan Deps

• esbuild.context

context.cancel()
context.rebuild()
context.dispose()

• esbuild plugin

 build.onLoad({filter, namespace}, ({path , importer, pluginData}) => {}) => { loader, contents, pluginData} // using loader ts
 build.onResolve({filter, namespace}, ({path , importer})  => {} ) => { path, namespace, external }

Bundling Deps

• esbuild.context

context.cancel()
context.rebuild(({metafile}) => {}) // metafile.outputs
context.dispose() 

• esbuild plugin

 build.onEnd()
 build.onResolve({filter, namespace}, ({ path , importer, kind}) => {} ) => { path, namespace, external }

• define
• platform 'node' : 'browser'
• banner
• tsconfig
• metafile
• sourcemap
• logLevel
• outdir
• supported dynamic-import. import-meta

Rolldown task

• plugin end
• esbuild plugin adapter or just rewrite it to rollup plugin

Need to discuss should implement the following api and how to do it at rolldown.

• esbuild.context
• banner
• define
• platform
• tsconfig
• metafile
• plugin pluginData
• esbuild loader ts(astro files using)