Coder Social home page Coder Social logo

tomeraberbach / parse-imports Goto Github PK

View Code? Open in Web Editor NEW
50.0 3.0 4.0 602 KB

⚡ A blazing fast ES module imports parser.

Home Page: https://npm.im/parse-imports

License: Apache License 2.0

JavaScript 41.19% TypeScript 58.81%
es6 esm import-export wasm es-module dynamic-import parser javascript nodejs dynamic-imports

parse-imports's Introduction

parse-imports

A blazing fast ES module imports parser.

Features

  • Uses the superb WASM-based es-module-lexer under the hood
  • Identifies module specifier types (e.g. relative file import, package import, builtin import, etc.)
  • Unescapes module specifier escape sequences
  • Collects default, named, and namespace imports
  • Works with dynamic imports
  • Resolves module specifier paths via require.resolve

Install

$ npm i parse-imports

Usage

import { parseImports } from 'parse-imports'

const code = `
  import a from 'b'
  import * as c from './d'
  import { e as f, g as h, i } from '/j'
  import k, { l as m } from 'n'
  import o, * as p from "./q"
  import r, { s as t, u } from "/v"
  import fs from 'fs'

  ;(async () => {
    await import("w")
    await import("x" + "y")
  })()
`

// Lazily iterate over iterable of imports
for (const $import of await parseImports(code)) {
  console.log($import)
}

// Or get as an array of imports
const imports = [...(await parseImports(code))]

console.log(imports[0])
//=>
// {
//   startIndex: 3,
//   endIndex: 20,
//   isDynamicImport: false,
//   moduleSpecifier: {
//     type: 'package',
//     startIndex: 17,
//     endIndex: 20,
//     isConstant: true,
//     code: `'b'`,
//     value: 'b',
//     resolved: undefined
//   },
//   importClause: {
//     default: 'a',
//     named: [],
//     namespace: undefined
//   }
// }

console.log(imports[1])
//=>
// {
//   startIndex: 23,
//   endIndex: 47,
//   isDynamicImport: false,
//   moduleSpecifier: {
//     type: 'relative',
//     startIndex: 42,
//     endIndex: 47,
//     isConstant: true,
//     code: `'./d'`,
//     value: './d',
//     resolved: undefined
//   },
//   importClause: {
//     default: undefined,
//     named: [],
//     namespace: 'c'
//   }
// }

console.log(imports[5])
//=>
// {
//   startIndex: 153,
//   endIndex: 186,
//   isDynamicImport: false,
//   moduleSpecifier: {
//     type: 'absolute',
//     startIndex: 182,
//     endIndex: 186,
//     isConstant: true,
//     code: '"/v"',
//     value: '/v',
//     resolved: undefined
//   },
//   importClause: {
//     default: 'r',
//     named: [
//       { specifier: 's', binding: 't' },
//       { specifier: 'u', binding: 'u' }
//     ],
//     namespace: undefined
//   }
// }

console.log(imports[7])
//=>
// {
//   startIndex: 238,
//   endIndex: 249,
//   isDynamicImport: true,
//   moduleSpecifier: {
//     type: 'package',
//     startIndex: 245,
//     endIndex: 248,
//     isConstant: true,
//     code: '"w"',
//     value: 'w',
//     resolved: undefined
//   },
//   importClause: undefined
// }

console.log(imports[8])
//=>
// {
//   startIndex: 260,
//   endIndex: 277,
//   isDynamicImport: true,
//   moduleSpecifier: {
//     type: 'unknown',
//     startIndex: 267,
//     endIndex: 276,
//     isConstant: false,
//     code: '"x" + "y"',
//     value: undefined,
//     resolved: undefined
//   },
//   importClause: undefined
// }

API

Use parseImports when you're able to await a Promise result and parseImportsSync otherwise.

Important

You can only call parseImportsSync once the WASM has loaded. You can be sure this has happened by awaiting the exported wasmLoadPromise.

See the type definitions for details.

Types

type ModuleSpecifierType =
  | 'invalid'
  | 'absolute'
  | 'relative'
  | 'builtin'
  | 'package'
  | 'unknown'

type Import = {
  startIndex: number
  endIndex: number
  isDynamicImport: boolean
  moduleSpecifier: {
    type: ModuleSpecifierType
    startIndex: number
    endIndex: number
    isConstant: boolean
    code: string
    value?: string
    resolved?: string
  }
  importClause?: {
    default?: string
    named: string[]
    namespace?: string
  }
}

Import

code.substring(startIndex, endIndex) returns the full import statement or expression. code.substring(moduleSpecifier.startIndex, moduleSpecifier.endIndex) returns the module specifier including quotes.

moduleSpecifier.isConstant is true when the import is not a dynamic import (isDynamicImport is false), or when the import is a dynamic import where the specifier is a simple string literal (e.g. import('fs'), import("fs"), import(`fs`)).

If moduleSpecifier.isConstant is false, then moduleSpecifier.type is 'unknown'. Otherwise, it is set according to the following rules:

  • 'invalid' if the module specifier is the empty string
  • 'absolute' if the module specifier is an absolute file path
  • 'relative' if the module specifier is a relative file path
  • 'builtin' if the module specifier is the name of a builtin Node.js package
  • 'package' otherwise

moduleSpecifier.code is the module specifier as it was written in the code. For non-constant dynamic imports it could be a complex expression.

moduleSpecifier.value is moduleSpecifier.code without string literal quotes and unescaped if moduleSpecifier.isConstant is true. Otherwise, it is undefined.

moduleSpecifier.resolved is set if the resolveFrom option is set and moduleSpecifier.value is not undefined.

importClause is only undefined if isDynamicImport is true.

importClause.default is the default import identifier or undefined if the import statement does not have a default import.

importClause.named is the array of objects representing the named imports of the import statement. It is empty if the import statement does not have any named imports. Each object in the array has a specifier field set to the imported identifier and a binding field set to the identifier for accessing the imported value. For example, import { a, x as y } from 'something' would have the following array for importClause.named: [{ specifier: 'a', binding: 'a' }, { specifier: 'x', binding: 'y' }].

importClause.namespace is the namespace import identifier or undefined if the import statement does not have a namespace import.

Contributing

Stars are always welcome!

For bugs and feature requests, please create an issue.

For pull requests, please read the contributing guidelines.

License

Apache 2.0

This is not an official Google product.

parse-imports's People

Contributors

tomeraberbach avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

parse-imports's Issues

Provide CJS build

While it worked for us locally, as we are also using modules, I dumbly forgot to check whether the package supports CJS.

Any chance you'd provide a CJS build?

Any chance to support TypeScript/increase robustness against syntax errors?

At the moment, I'm getting a large number of parse errors from lexer when parsing .ts files because, obviously, TypeScript is a superset of plain JavaScript. Nevertheless, import statements are not very complex - the only difference I'm aware of is TypeScript's import … = statement. But also apart from JavaScript, the import parser breaks completely if there any missing comma or something like this at the end of a file. Would it be in the scope of this package to increase robustness, and maybe even to support TypeScript imports? Thanks in advance!

Using import.meta throws an error

Code:

import parseImports from 'parse-imports';

const code = `
  console.log(import.meta);
`;

for (const $import of await parseImports(code)) {
  console.log($import);
}

Error:

node:internal/process/esm_loader:97
    internalBinding('errors').triggerUncaughtException(
                              ^

AssertionError [ERR_ASSERTION]: false == true
    at parseModuleSpecifier (file:///.../node_modules/parse-imports/src/parse-module-specifier/index.js:27:3)
    at Object.[Symbol.iterator] (file:///.../node_modules/parse-imports/src/index.js:47:33)
    at Generator.next (<anonymous>)
    at file:///.../test.mjs:7:12 {
  generatedMessage: true,
  code: 'ERR_ASSERTION',
  actual: false,
  expected: true,
  operator: '=='
}

Sync method

Hi,

I'm working with eslint-plugin-jsdoc and we have a need for parsing import statements (for use within @import tags). However, ESLint does not allow asynchronous rules.

I know that es-module-lexer mentions using async to be safe, but when I tested it within eslint, it seems enough time has passed for Wasm to load such that I could successfully get results using its parse method synchronously.

I am wondering if you could expose a sync version of parseImports (that presupposes it is called with enough time for Wasm to load)?

Support JSX syntax

Here are a couple of examples. I show the error message on the specific line referenced in the error.

                <FilterFieldClearButton
                  aria-label="Clear search"
                  onClick={() => {
                    setFilterValue('');
                    setSortedColumns(getFilteredColumns(''));
                  }}
                > 
                </FilterFieldClearButton>    ****** Parse error @:287:42
function SandboxApp() {
  return (
    <App>
      <DatagridInlineRowGroup licenseKey={licenseKey} />
    </App>  ****** Parse error @:9:11
  );
}

Feature request: Indicate position in code

For my use case, it would be very helpful to trace every import back to the place in a file where it was defined. This could be achieved by adding a field like Import.moduleSpecifier.position: {row, column} and storing these data during parsing.

Import broken since v1.0.0/6c210101a283d4fe46bdcf14d33bcdd6317a09ff

  • Original failure in my project: LinqLover/downstream-repository-mining#37:

        Cannot find module 'parse-imports' from 'src/references.ts'
    
        Require stack:
          src/references.ts
          test/references.test.ts
    
           6 | import _ from "lodash"
           7 | import tqdm from 'ntqdm'
        >  8 | import parseImports from 'parse-imports'
             | ^
           9 | import path from 'path'
          10 |
          11 | import { getCacheDirectory } from './npm-deps'
    
          at Resolver.resolveModule (node_modules/jest-resolve/build/index.js:306:11)
          at Object.<anonymous> (src/references.ts:8:1)
    
  • Minimal example to reproduce: https://github.com/LinqLover/parse-imports-example/pull/1

    $ tsc && node index.js
    internal/modules/cjs/loader.js:1085
          throw new ERR_REQUIRE_ESM(filename, parentPath, packageJsonPath);
          ^
    
    Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /home/runner/work/parse-imports-example/parse-imports-example/node_modules/parse-imports/src/index.js
    require() of ES modules is not supported.
    require() of /home/runner/work/parse-imports-example/parse-imports-example/node_modules/parse-imports/src/index.js from /home/runner/work/parse-imports-example/parse-imports-example/index.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
    Instead rename /home/runner/work/parse-imports-example/parse-imports-example/node_modules/parse-imports/src/index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /home/runner/work/parse-imports-example/parse-imports-example/node_modules/parse-imports/package.json.
    
        at Object.Module._extensions..js (internal/modules/cjs/loader.js:1085:13)
        at Module.load (internal/modules/cjs/loader.js:933:32)
        at Function.Module._load (internal/modules/cjs/loader.js:774:14)
        at Module.require (internal/modules/cjs/loader.js:957:19)
        at require (internal/modules/cjs/helpers.js:88:18)
        at Object.<anonymous> (/home/runner/work/parse-imports-example/parse-imports-example/index.js:4:49)
        at Module._compile (internal/modules/cjs/loader.js:1068:30)
        at Object.Module._extensions..js (internal/modules/cjs/loader.js:1097:10)
        at Module.load (internal/modules/cjs/loader.js:933:32)
        at Function.Module._load (internal/modules/cjs/loader.js:774:14) {
      code: 'ERR_REQUIRE_ESM'
    }
    

Note that in both cases, the CI passes when I downgrade to v0.0.5 again. What's happening here? How can I keep using your package after you have upgraded it to ESM?

Is that possible to expose the start index and end index of the specifier?

e.g. the case

//=>
// {
//   startIndex: 3,
//   endIndex: 20,
//   isDynamicImport: false,
//   moduleSpecifier: {
//     type: 'package',
//     isConstant: true,
//     code: `'b'`,
//     value: 'b',
//     resolved: undefined
//   },
//   importClause: {
//     default: 'a',
//     named: [],
//     namespace: undefined
//   }
// }

can become

//=>
// {
//   startIndex: 3,
//   endIndex: 20,
//   isDynamicImport: false,
//   moduleSpecifier: {
//     type: 'package',
//     isConstant: true,
//     code: `'b'`,
//     value: 'b',
//     resolved: undefined
//     startIndex: 18, // or 17 which includes the quotation mark
//     endIndex: 19 // or 20 which includes the quotation mark
//   },
//   importClause: {
//     default: 'a',
//     named: [],
//     namespace: undefined
//   }
// }

That would be very helpful for users to replace the specifier further.

Currently, I'm using `moduleSpecifier.code` to do that. However, I feel indices would be much accurate.

Thanks.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.