nodejs / node-eps

Node.js Enhancement Proposals for discussion on future API additions/changes to Node core

Node.js Enhancement Proposals

Overview

This repository contains the Node Enhancement Proposals (EPs) collection. These are documents describing an enhancement proposal for inclusion in Node.

EPs are used when the proposed feature is a substantial new API, is too broad or would modify APIs heavily. Minor changes do not require writing an EP. What is and isn't minor is subjective, so as a general rule, users should discuss the proposal briefly by other means (issue tracker, mailing list or IRC) and write an EP when requested by the Node core team.

Rational

The idea behind the EP process is to keep track of what ideas will be worked on and which ones were discarded, and why. This should help everyone (those closely involved with the project and newcomers) have a clear picture of where Node stands and where it wants to be in the future.

Format

EP documents don't follow a given format (other than being written in MarkDown). It is, however, required that all EPs include the following template and information at the top of the file:

| Title  | Tile of EP      |
|--------|-----------------|
| Author | @gihub_handle   |
| Status | DRAFT           |
| Date   | YYYY-MM-DD      |

The document file name must conform to the format "XXX-title-ish.md" (literally starting with XXX and not a self assigned number). At the time the EP lands it will be assigned a number and added to 000-index.md. There is no need for a PR author to add the file to the index since no number has yet been given.

Files should follow the convention of keeping lines to a maximum of 80 characters. Exceptions can be made in cases like long URLs or when pasting the output of an application. For example a stack trace from gdb.

More information of the "Status" field can be found in Progress of an EP.

Content

EP documents should be as detailed as possible. Any type of media which helps clarify what it tries to describe is more than welcome, be that an ASCII diagram, pseudocode or actual C code.

Licensing

All EP documents must be MIT licensed.

Progress of an EP

All EPs will be committed to the repository regardless of their acceptance. The initial status shall be "DRAFT".

If the document is uncontroversial and agreement is reached quickly it might be committed directly with the "ACCEPTED" status. Likewise, if the proposal is rejected the status shall be "REJECTED". When a document is rejected a member of the core team should append a section describing the reasons for rejection.

A document shall also be committed in "DRAFT" status. This means consensus has not been reached yet.

The author of an EP is expected to actually pursue and implement the proposal.

node-eps's Issues

Archiving this repo

The @nodejs/tsc has decided to archive this repo and move away from the node-eps process. See nodejs/TSC#335 for details.

Importing json

@jdalton invited me to file this issue. I didn't see anything in the proposal about JSON support. Will importing JSON be supported?

import data from './data.json'

Due to the fact that JSON is static, I think it'd be reasonable to use ImportSpecifiers as well. One drawback is it could be confusing. But in scenarios where the JSON file is large, tree shaking could be helpful in this scenario.

Here's an example:

data.json

{  
  "gender":"male",
  "name":{  
    "title":"mr",
    "first":"isaac",
    "last":"sanz"
  },
  "location":{  
    "street":"5286 calle de arturo soria",
    "city":"logroño",
    "state":"castilla y león",
    "postcode":50435
  },
  "email":"[email protected]",
  "login":{  
    "username":"organicduck350",
    "password":"jazzman",
    "salt":"gFH7zlZ9",
    "md5":"b284fbb639b2cb23493ab2ac0263ca01",
    "sha1":"7d7715926ce1f7dff3bef56c4c96a8f8daee5197",
    "sha256":"6e9961e11e17ee33a62545de397af1de4d5fda93d913956d2b0ee8aa75fcd774"
  },
  "registered":924488158,
  "dob":870811843,
  "phone":"994-654-870",
  "cell":"632-172-267",
  "id":{  
    "name":"DNI",
    "value":"21433930-U"
  },
  "picture":{  
    "large":"https://randomuser.me/api/portraits/men/29.jpg",
    "medium":"https://randomuser.me/api/portraits/med/men/29.jpg",
    "thumbnail":"https://randomuser.me/api/portraits/thumb/men/29.jpg"
  },
  "nat":"ES"
}

index.js

import {picture} from './data.json'

I think that this would be reasonable. As cool as it would be to have a destructuring-like syntax to get a specific picture (large for example), I don't think that's really in the cards here.

All of that said, tree-shaking is really a bundler concern and I don't think that Node proper really needs to concern itself with that. So honestly, just getting the entire parsed JS object as a default export which could then be destructured as normal would be fantastic.

WRT PR #3 - ES module detection

This is the place to discuss specifics of how node will determine the mode a given a source (file / source string). Source string examples include stdin or -e, they do not include JS Function or eval calls. The proposal itself can be seen in PR #3.

Discussions here should regard:

• Ways to detect the module target for a specific source
• Impact of detecting the module on other environments
  • How other environments will provide the detection in the case of interoperability
• Potential incompatibilities introduces for existing workflows with module detection possibilities.
• Transition goals for a time when ES modules are the normal target.

Note: CJS will be the default target of node for the foreseeable future, "normal" here means what developers write daily. This is a constraint that can be easily worked around for a single source via a cli flag to note that the source is an ES module. This flag should not be seen as relevant to the discussion in this thread.

Note: We are only discussing choices that do not require parsing of the source file.

It should not discuss import/export conversion, evaluation ordering, or module path resolution.

002- Clarification of import evaluation order (and how it affects REPLs)

Apologies, if this is the wrong place to ask this, I will happily move it somewhere else. I'm one of the developers of Tonic (tonicdev.com) and have been wondering about the effect import would have on REPL behavior for a while now, specifically as it could potentially create a new divergence with evaluation order that doesn't currently exist with require.

If I understand correctly, the ES6 spec states that with code like this:

// code
import x from '../x'

"../x" will be parsed and evaluated BEFORE any code in the parent is run, essentially as if this had been done instead:

import x from '../x'
// code

The issue comes up with REPLs, where if you were to type "import" on the second prompt, it would necessarily have to import afterwards, leading to a divergence in behavior that REPLs didn't use to have with require if you were to take all the REPL inputs and concat them into a file:

5 + 5
< 10
import x from '../x'
< whatever

--> 5 + 5 THEN '../x'
vs.
--> '../x' THEN 5 +5

This would have a devastating effect on Tonic, since it means that to maintain parity with single-file execution, any time someone types an import in a cell, we are forced to re-run the script from the first line (as opposed to the nice "pick up from where you left off" behavior it currently can deterministically/correctly exhibit).

For an analogy, this is equivalent to how in Tonic today, if you write a function that would, after hoisting, affect an earlier cell, we have to rewind the state of the machine and re-run from the earlier cell:

This however is rather explicit and semantic from the user. On the other hand, merely importing a new library on your 10th input of a REPL does not seem to obviously imply "please re-run everything from the first cell, as any file mutation that takes place in this input would technically have to precede all the actions in this code".

So I guess my questions are: 1) are my interpretations of these behaviors correct, and 2) if they are, is it possibly to propose a different evaluation order.

[meta] set status on the file name or README

Browsing those proposal from github does not immediately display which ones are accepted or not, it could be a bit misleading.

fetch()

fetch() runs in a browser.
And sometimes I port my code to node.
But in node, I need to 'polyfill' fetch()

That is an example where I think there should be more parity.

002 - Provide a link to the counter-proposal

In addition to characterizing the counter-proposal (in my opinion, incorrectly), it would be helpful if this document linked directly to the counterproposal (https://github.com/dherman/defense-of-dot-js/blob/master/proposal.md).

I understand that the CTC has decided to go with .mjs, but for posterity it would be helpful if future readers of this decision could easily find the proposal it is criticizing.

redundant statement?

Doesn't

A document shall also be committed in "DRAFT" status""

mean the same thing as

All EPs will be committed to the repository regardless of their acceptance. The initial status shall be "DRAFT".

002 - Would appreciate more detail on HTTP/PUSH issue

In the section referring to the package.json style of solution, this issue is raised:

HTTP/2 PUSH solutions would also need this and would be affected

I would appreciate more information on what issue this is referring to. I'd be happy to write a PR expanding on the issue once I understand it. 😄

002 - web compat of file extension/index searching

In talks with @domenic a concern about using error behavior of a URL derived from step 2 of https://html.spec.whatwg.org/multipage/webappapis.html#resolve-a-module-specifier . In particular, the current plan has been to search for file extension such that:

import './foo';

Would search multiple file extensions .mjs, .js, .json, and .node.

Since ./, ../, and / do not immediately error upon resolution (they would error upon failed fetch) the question is if path searching could be limited to the error behavior for so called "bare" URLs within step 2.

002 - Would appreciate more detail on the "asset pipeline" description

In 002:

Build asset pipelines in particular would be affected such as the Rails asset pipeline

Can you elaborate what scenario you are referring to here? As a Rails core alumni who helped work on the asset pipeline (and whose product uses Rails today), I don't understand it.

Comment

I think this repo is awesome! This is what we have to do! :)
Super fantastic!

Improving support for symlinks in the module system

Continuation of the discussions nodejs/node#10132 and nodejs/node#10107 ... Please use this thread to discuss the high level architectural, merits, and long term impact of the proposed change so that the PR discussion can be focused on the actual code review.

002 - Clarification on consuming CommonJS from ES modules

In the ES modules proposal, there are several examples related to loading CommonJS modules from ES modules, but there is no example covering the use of this in an exported module. For example:

// CommonJS file util.js
module.exports.name = "commonjs";
module.exports.getName = function() {
    return this.name;
};

The getName() method makes use of this to determine the value to return. The proposal states that exports from CommonJS modules can be loaded in ES modules via import using those names:

import { getName } from "./util";

console.log(getName());              // what's the output?

My question is if the this binding remains in getName() or not? I would expect it to, but I don't see that mentioned in the proposal. In effect, that would mean the local binding getName in this example would not be directly equal to getName as exported from util.js but rather equivalent to getName.bind(module.exports).

Is that the way it's intended to work? If so, can that detail be added into the proposal?

WRT PR #3 - CJS <-> ES import/export conversion

This is the place to discuss specifics of how imports and exports are wrapped / converted between the CommonJS (CJS) and EcmaScript (ES) module systems. The proposal itself can be seen in PR #3.

Discussions here should regard:

• how to generate ModuleNamespaces for ES modules to consume from CJS module exports
• how to generate values for CJS to consume from ES module exports

It should not discuss module path resolution, evaluation ordering, or detecting file mode.

WRT PR #3 - Import relative to package root

The current proposal adds support for the following path resolutions:

• import relative to node_modules (import x from 'something')
• import relative to the current file (import x from './something')
• import relative to the filesystem root (import x from '/something')

But what's missing from this list is importing relative to the current project (package) root. This is quite common in other languages, and other languages even prefer absolute imports and discourage relative imports. It's therefore unfortunate that in node we don't have the option or opportunity to decide for ourselves on a per-project basis.

The semantics are pretty straight forward: traverse the directory tree upwards from the current file looking for a folder that contains a file with the name package.json, and use this folder as root. This is pretty similar to the way the node_modules folder is found, so won't add any performance issues.

My suggestion for the syntax is to use the /something rule, and to use file:///something for absolute imports from the filesystem root. A rule of thumb in api design is that common problems should have simple solutions and uncommon problems should have solutions. I believe that importing modules from the filesystem root is uncommon, while importing modules from the package root is common, and therefore the latter should have a simpler solution than the former.

This doesn't make anything that is possible today impossible, and doesn't make it significantly more difficult to transpile or to convert CJS modules to ES modules.

002 ES modules in a `vm`

I'm curious how the proposal for handling ES2015 modules in node will work with the vm module. Specifically I am interested in how to intercept imports during runtime.

Currently I can inject my own implementation of require into a context and then run JavaScript code inside of the context. This allows me to control how to resolve modules.

Are there hooks planned for manipulating/intercepting import/export in a vm?

"tree shaking" on nodejs

Have it / can it / should it be considered?

The only benefit seems to be saving some memory consumption for the module resolution.
But if all references are resolved at runtime, then there is no benefit to be observed?

Or this may be fit into a larger discussion of "when to build"? 🌷

implements fibers in nodejs

like fibjs did,to get better performance。

WRT PR #3 - Module evaluation ordering

This is the place to discuss specifics of the order in which modules will be evaluated while mixing CommonJS (CJS) and EcmaScript (ES) modules. The proposal itself can be seen in PR #3.

Discussions here should regard:

• Order of module evaluation execution
• The dependency graph of a module and possible problems mixing declarative pre-order evaluation and imperative in-order evaluation.

It should not discuss import/export conversion, module path resolution, or detecting file mode.

.mjs extension trade-offs revision

I obviously can't speak for the whole community, but it seems like a lot of people are not happy with .mjs.

One of the main arguments to keep .js is that if we can detect 99% of cases where we CAN tell if is it CJS or ESM (or where we just know what to do), we may just call rest 1% edge cases and deal with it.

We can even come up with some linter rules and/or workarounds to simply teach people to do the right thing.

Notify CTC members of new repository

@nodejs/ctc New repository to discuss in-depth changes to core. My initial thought was the AsyncWrap API, and how it should be made public. Additionally would be @indutny's JSStream API. Could also use this for @nodejs/http's suggestions on how to make the http module plug-able, or @nodejs/api on how to abstract the native API.

Going to leave this issue open for a week or two to give everyone time to notice this repo has been created.

WRT PR #3 - ES module path resolution

This is the place to discuss specifics of how the EcmaScript (ES) module imports will resolve paths. The proposal itself can be seen in PR #3.

Discussions here should regard:

• What differences module resolution in ES module import syntax will have from require's module resolution algorithm.

It should not discuss import/export conversion, evaluation ordering, or detecting file mode.