Coder Social home page Coder Social logo

dprint-swc-ext's Introduction

dprint-swc-ext

CI

Extensions for swc used in dprint-plugin-typescript and Deno.

What does this do?

  1. Adds a SourcePos and SourceRange type to compensate for swc having BytePos(0) as a magical value.
  2. Adds many helper methods.

With the view cargo feature enabled:

  1. Creates a wrapper AST around swc's AST that stores the node parents.
    • This is similar to a "red tree", but it creates it for every node. It's very fast to create these.
    • Most of this code is code generated.
  2. Adds a Node enum type to allow referencing any kind of node.

Helpers

All (SourceRanged trait):

  • .start(&self) -> SourcePos
  • .end(&self) -> SourcePos
  • .text_fast(&self, root_node: &dyn SourceTextInfoProvider) -> &'a str -- Doesn't require going up the tree to the root node
  • .start_line_fast(&self, root_node: &dyn SourceTextInfoProvider) -> usize
  • .end_line_fast(&self, root_node: &dyn SourceTextInfoProvider) -> usize
  • .start_column_fast(&self, root_node: &dyn SourceTextInfoProvider) -> usize
  • .end_column_fast(&self, root_node: &dyn SourceTextInfoProvider) -> usize
  • .width_fast(&self, root_node: &dyn SourceTextInfoProvider) -> usize
  • .tokens_fast(&self, root_node: &dyn RootNode) -> &'a [TokenAndSpan]
  • .leading_comments_fast(&self, root_node: &dyn RootNode) -> CommentsIterator<'a>
  • .trailing_comments_fast(&self, root_node: &dyn RootNode) -> CommentsIterator<'a>
  • .previous_token_fast(&self, root_node: &dyn RootNode) -> Option<&TokenAndSpan>
  • .next_token_fast(&self, root_node: &dyn RootNode) -> Option<&TokenAndSpan>
  • .previous_tokens_fast(&self, root_node: &dyn RootNode) -> &'a [TokenAndSpan]
  • .next_tokens_fast(&self, root_node: &dyn RootNode) -> &'a [TokenAndSpan]

Node/Enum Node/Nodes (view cargo feature only):

  • .module(&self) -> &'a Module - Gets the root node if the view was created from a Module. Otherwise panics.
  • .script(&self) -> &'a Script - Gets the root node if the view was created from a Script. Otherwise panics.
  • .program(&self) -> Program<'a> - Gets the root node whether it be a Module or a Script.
  • .parent(&self) -> Option<Node<'a>>
  • .children(&self) -> Vec<Node<'a>>
  • .child_index(&self) -> usize
  • .ancestors(&self) -> AncestorsIterator<'a>
  • .previous_sibling(&self) -> Option<Node<'a>>
  • .next_sibling(&self) -> Option<Node<'a>>
  • .previous_siblings(&self) -> Vec<Node<'a>>
  • .next_siblings(&self) -> Vec<Node<'a>>
  • .text(&self) -> &str - Slightly slower than .text_fast(module) because it requires going up the tree to get the root node
  • .start_line(&self) -> usize
  • .end_line(&self) -> usize
  • .start_column(&self) -> usize
  • .end_column(&self) -> usize
  • .width(&self) -> usize
  • .tokens(&self) -> &[TokenAndSpan] - All the descendant tokens within the span of the node.
  • .children_with_tokens(&self) -> Vec<NodeOrToken<'a>> - Gets the children with the tokens found between the children
  • .children_with_tokens_fast(&self, root_node: &dyn RootNode) -> Vec<NodeOrToken<'a>>
  • .leading_comments(&self) -> CommentsIterator<'a>
  • .trailing_comments(&self) -> CommentsIterator<'a>
  • .kind(&self) -> NodeKind - Gets the "node kind" enum variant associated with the node (ex. NodeKind::ClassDecl).
  • .previous_token(&self) -> Option<&TokenAndSpan>
  • .next_token(&self) -> Option<&TokenAndSpan>
  • .previous_tokens(&self) -> &'a [TokenAndSpan]
  • .next_tokens(&self) -> &'a [TokenAndSpan]

Node/Enum Node (view cargo feature only):

  • .to::<NodeType>(&self) -> Option<&NodeType>
  • .expect::<NodeType>(&self) -> &NodeType
  • .is::<NodeType>(&self) -> bool

TokenAndSpan:

  • .token_index(&self, root_node: &dyn RootNode) -> usize - Gets the token index of the specified module.

Root Node (Program/Module/Script):

  • token_at_index(&self, index: &usize) - Option<&TokenAndSpan>

View Construction Functions

  • with_ast_view - Creates a view from an swc Program (either Module or Script)
  • with_ast_view_for_module - Creates a view from an swc Module
  • with_ast_view_for_script - Creates a view from an swc Script

TODO

  • Right now this only works if analyzing one file at a time. It would be good to improve the API to accept a large collection of source files (should be easy).
  • Unit tests

view - Example

Given the following parsed input code:

class MyClass { prop: string; myMethod() {}}

Code can be written like so:

// setup swc (parse an AST and optionally get the comments and tokens)
let text_info = SourceTextInfo::new(...);
let program: swc_ecmascript::ast::Program = ...;
let comments: swc_common::comments::SingleThreadedComments = ...;
let tokens: Vec<TokenAndSpan> = ...;

// setup for creating a view
let program_info = ProgramInfo {
  program: &program,
  text_info: Some(&text_info),
  // optionally provide the comments for comment related methods
  comments: Some(&comments)
  // optionally provide the tokens for token related methods
  tokens: Some(&tokens),
};

// now create and use the view
dprint_swc_ecma_ast_view::with_ast_view(program_info, |program| {
  let class = program.children()[0].expect::<ClassDecl>().class;
  println!("{:?}", class.text());

  for child in class.children() {
    println!("---------");
    println!("Child: {:?}", child.text());
    println!("Parent: {:?}", child.parent().unwrap().text());
    if let Some(prev_sibling) = child.prev_sibling() {
      println!("Previous sibling: {:?}", prev_sibling.text());
    }
    if let Some(next_sibling) = child.next_sibling() {
      println!("Next sibling: {:?}", next_sibling.text());
    }
  }
});

Outputs:

"class MyClass { prop: string; myMethod() {}}"
---------
Child: "prop: string;"
Parent: "class MyClass { prop: string; myMethod() {}}"
Next sibling: "myMethod() {}"
---------
Child: "myMethod() {}"
Parent: "class MyClass { prop: string; myMethod() {}}"
Previous sibling: "prop: string;"

dprint-swc-ext's People

Contributors

bartlomieju avatar cd-work avatar dprintbot avatar dsherret avatar lucacasonato avatar magurotuna avatar paolobarbolini avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

dprint-swc-ext's Issues

Example does not work

I tried following the example program below, but got stuck with passing the comments into a ProgramInfo struct.

// setup swc (parse an AST and optionally get the comments and tokens)
let text_info = SourceTextInfo::new(...);
let program: swc_ecmascript::ast::Program = ...;
let comments: swc_common::comments::SingleThreadedComments = ...;
let tokens: Vec<TokenAndSpan> = ...;

// setup for creating a view
let program_info = ProgramInfo {
  program: &program,
  text_info: Some(&text_info),
  // optionally provide the comments for comment related methods
  comments: Some(&comments)
  // optionally provide the tokens for token related methods
  tokens: Some(&tokens),
};

// now create and use the view
dprint_swc_ecma_ast_view::with_ast_view(program_info, |program| {
  let class = program.children()[0].expect::<ClassDecl>().class;
  println!("{:?}", class.text());

  for child in class.children() {
    println!("---------");
    println!("Child: {:?}", child.text());
    println!("Parent: {:?}", child.parent().unwrap().text());
    if let Some(prev_sibling) = child.prev_sibling() {
      println!("Previous sibling: {:?}", prev_sibling.text());
    }
    if let Some(next_sibling) = child.next_sibling() {
      println!("Next sibling: {:?}", next_sibling.text());
    }
  }
});

Here is my current code:

let text = read_to_string(file).expect("Failed to read file");
let text_info = SourceTextInfo::new(text.into());
let parsed_source = parse_module(ParseParams {
    specifier: "file:///my_file.js".to_string(),
    media_type: JavaScript,
    text_info,
    capture_tokens: true,
    maybe_syntax: None,
    scope_analysis: false,
}).expect("Should parse");

let comments = parsed_source.comments();
// let program: swc_ecmascript::ast::Program = Program::Module(parsed_source.module().to_owned());
let tokens = parsed_source.tokens();

let program_ref: dprint_swc_ext::view::ProgramRef = dprint_swc_ext::view::ProgramRef::Module(parsed_source.module());

let new_comments = comments.as_single_threaded();
let program_info = ProgramInfo {
    program: program_ref,
    text_info: Some(&text_info),
    comments: Some(&new_comments), // <-- here
    tokens: Some(&tokens)
};

It throws an error "Mismatched Types"

error[E0308]: mismatched types
  --> src/parsers/javascript/parser.rs:39:28
   |
39 |             comments: Some(&new_comments),
   |                            ^^^^^^^^^^^^^ expected struct `dprint_swc_ext::view::Comments`, found `&SingleThreadedComments`

I tried messing around with .into(), .to_owned(), but I can't seem to figure out how to convert a single threaded comment into a "Comments" object. Any help is welcome, I appreciate the work you do on this repo.

Feature Timeline

Do you have a timeline for when this feature will be implemented?

Right now this only works if analyzing one file at a time. It would be good to improve the API to accept a large collection of source files (should be easy).

Refactor the way of constructing graph to avoid undefined behavior

At the moment, the generated code has lots of codes that might lead to undefined behavior: https://github.com/dprint/dprint-swc-ecma-ast-view/blob/dfc9347015603495579eec592b27ca301abb093e/rs-lib/src/generated.rs#L7097-L7111

Here MaybeUninit::uninit().assume_init() is called, but Rust doesn't allow us to invoke assume_init() on values that are not properly initialized in memory. It looks like the code works fine for now, but there's no guarantee that it will keep working.

So I'd like to propose replacing these codes to avoid undefined behavior. I think this refactoring is not so easy or simple but it's worth it.

Here are my thoughts on how to refactor:

  • make all fields of Node structs (such as BindingIdent) private and create getter methods for them
  • change type of fields that currently use MaybeUninit::uninit().assume_init() (such as id of BindingIdent) to Cell<Option<..>>. They become None only when being constructed. Once the AST view has been completely created, they should be Some. Therefore, in the getter methods, we can safely call .unwrap()
  • rewrite the construction of the view

As a whole, BindingIdent could be refactored into the following:

// Make all fields private
pub struct BindingIdent<'a> {
  parent: Node<'a>,
  inner: &'a swc_ast::BindingIdent,
  id: Cell<Option<&'a Ident<'a>>>, // wrap in Cell
  type_ann: Cell<Option<&'a TsTypeAnn<'a>>>, // wrap in Cell
}

// Instead, expose getters for all fields
impl<'a> BindingIdent<'a> {
  pub fn parent(&self) -> Node<'a> {
    self.parent
  }

  pub fn inner(&self) -> &'a swc_ast::BindingIdent {
    self.inner
  }

  pub fn id(&self) -> &'a Ident<'a> {
    self.id.get().unwrap() // when the library users call it, this `unwrap` is totally safe
  }

  pub fn type_ann(&self) -> Option<&'a TsTypeAnn<'a>> {
    self.type_ann.get()
  }
}

fn get_view_for_binding_ident<'a>(inner: &'a swc_ast::BindingIdent, parent: Node<'a>, bump: &'a Bump) -> &'a BindingIdent<'a> {
  let node = bump.alloc(BindingIdent {
    inner,
    parent,
    id: Cell::new(None), // wrap in Cell
    type_ann: Cell::new(None), // wrap in Cell
  });
  let parent: Node<'a> = (&*node).into();
  node.id.set(Some(get_view_for_ident(&inner.id, parent.clone(), bump)));
  node.type_ann.set(match &inner.type_ann {
    Some(value) => Some(get_view_for_ts_type_ann(value, parent, bump)),
    None => None,
  });
  node
}

As a side note: In the above example, type_ann is not required to wrap in Cell as it is not related to MaybeUninit. But it's preferable because the assignment node.type_ann = ... requires mutable reference to node while parent holds immutable reference to node. The Rust compiler is supposed to warn it, but somehow it compiles (probably due to std::mem::transmute in From trait impl.) For immutable reference and mutable reference not to exist at the same time, Cell is helpful.

Let me know what you think about this refactoring :)

Is is possible to let SourceFileInfo have swc's `Program` rather than `Module`?

I've been re-implementing a few of deno_lint's rules using this crate.
deno_lint uses swc_ecmascript::ast::Program as an entry point of linting, therefore it would be nice if this crate also took Program instead of Module. That is, I'd be happy if SourceFileInfo was altered to:

pub struct SourceFileInfo<'a> {
  pub module: &'a swc_ecmascript::ast::Program, // altered!
  pub source_file: Option<&'a swc_common::SourceFile>,
  pub tokens: Option<&'a Vec<TokenAndSpan>>,
  pub comments: Option<&'a SingleThreadedComments>,
}

and the signature of with_ast_view to:

pub fn with_ast_view<'a, T>(
    source_file_info: SourceFileInfo<'_>, 
    with_view: impl FnOnce(&'a Program<'a>) -> T // altered!
) -> T

From my taking a look at the code of this crate and how the underlying code generation work, it seems like the above alternation is achievable (although it looks a bit complicated to me :-|)

TypeScript generated code

Building on #13, we should generate TypeScript types and code for deserializing a source file to these types.

  1. Generate TS classes of each node that extends from a base Node class (should be possible to do node instanceof ClassDecl)
  2. Generate code for getting a node's children.
  3. Generate code that will take a deserialized JS object, set the prototype of each object to the appropriate type, set the node's parent (in the case of source file, undefined), then traverse all the child nodes doing the same.

cc @magurotuna, I wouldn't mind working on this, but if you would like to do this work let me know! For me it might take a few weeks.

Add unit tests

It would be useful to add unit tests for the methods this crate adds.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.