Get type information of a node.js module about tern

[marijnh]

The bin/condense tool I added today might be what you're looking for. Try...

bin/condense --plugin node node_modules/acorn/acorn.js

And you'll get a json structure similar to what's found under defs/. This is still buggy and hardly tested, though.

In general, the inference engine in Tern (lib/infer.js) is a stand-alone thing that can be used by other libraries, so yes, you can plug this into something you build yourself, though it does contain a bunch of fuzzy heuristics optimized for editing, which might not be appropriate in other situations.

[myrne]

To what extend is infer.js stand-alone? Could it be a separate node module, published to npm? Now Tern is open source I think it make sense to "deaggregate" the code somewhat, to enhance reusability.

I can imagine you have other priorities though. This raises the question: What are your priorities currently? Do you have a list of things you want to accomplish? The editor plugins will take some time certainly.

Re fuzzy matching: Could there be a fuzzy: false option? Some fuzziness won't matter though I think. In principle it's not a problem if the fault-tolerance of the parser remains, because modules published to npm (in live use in projects) can be assumed to be valid Javascript. But what other fuzzy stuff is there?

[myrne]

I see you're already using npm for dependencies, so if infer.js could be extracted into a separate module, then it's easily reintegrated.

[myrne]

So I have ran condense script on a really simple node module: fs-exists. It wraps fs.exists so it's callback signature is in line with the other fs functions. Its effective signature is something like:

fsExists(stringPath, function(nullErr, boolExists) {})

bin/condense --plugin node node_modules/fs-exists/lib/fs-exists.js gives this:

{
  "!name": "node_modules/fs-exists/lib/fs-exists.js",
  "exports": "fn(path: ?, callback: ?)"
}

Why? Maybe there no info on the native fs module?
I think the "null" value of the first argument to the callback could be inferred, without.
Entire code of module, for reference.

  var fs, fsExists;

  fs = require("fs");

  module.exports = fsExists = function(path, callback) {
    return fs.exists(path, function(result) {
      return callback(null, result);
    });
  };

[marijnh]

To what extend if infer.js stand-alone? Could it be a separate node module, published to npm?

It could. But it won't be. You can include it separately (require("tern/lib/infer")) already. I'm not going to split the library into several fragments for no good reason, since it increases the maintenance burden.

I think the "null" value of the first argument to the callback could be inferred

null (and undefined) types don't exist in Tern, since they are rarely interesting.

When you inspect the fsExists function in the demo at ternjs.net, you'll see that it did give the callback argument a function type. Here it guessed that since it is called, it's likely to be a function. But the condense tool only outputs types that it actually saw, and no actual function type was fed into the function in this code.

That's a general limitation of forward-inferring systems. They'll often need code to be actually used in order to properly type it. If you include some code (by passing a +file.js to condense) that uses this module in the analysis, you'll get better output.

[myrne]

the condense tool only outputs types that it actually saw,

I don't really understand what the condense tool is doing then, i.e. what part of the analysis it leaves out. Or perhaps it's not part of the static analysis at all. (Note I'm not sure if I use the word "static analysis" properly here, I have no experience with compilers and such. What I mean by static analysis is everything that can be seen without running code.)

What I see in the fsExists code is this:
fsExists is called with two arguments, path and callback. path is given unchanged to fs.exists. fs.exists is a property of fs, which is a known node module. It can be known that fs.exists takes a string argument as its first argument, and a function argument as its second. Since path is passed unchanged (this would also be true if there was for example some concatenation of strings, or calls to String.substring beforehand), argument type of path should be an instance of String.

Just as well it's visible from the code that callback will be called. Thus callback should be a function. It's the only way to prevent an exception from being thrown.

This can be seen without running code, no?

Perhaps I'm just way-overestimating what "static analysis" can do. Maybe the above implicitly assumes a kind of omniscient strong AI to be there (I'm not being sarcastic). Maybe what I say just can't be "computed". If so, I'd love to have pointed out my error, because then I'll learn. :)

null (and undefined) types don't exist in Tern, since they are rarely interesting.

I agree that they are not that interesting, but they are interesting in knowing if Tern has been able to determine a type. Still, I could do without.

[marijnh]

It's the only way to prevent an exception from being thrown.

I guess that's the crux of the issue. In statically typed languages, you can, from the way a value is used, infer its type. In JavaScript there's A) a lot of implicit conversions going on (you could pass anything, and it'd be converted to a string in most situations that expect a string), and B) type errors are runtime errors, so the assumption that there will be no type errors in the code can't be made. Often users intentionally write code that wouldn't type in a statically typed language, relying on some more subtle logic (duck-typing conditionals, implicit conversions) to not actually trigger a type error.

Thus, propagating type information 'back' from uses does not produce very good results in JS.

[myrne]

the assumption that there will be no type errors in the code can't be made
the assumption that the caller of the function doesn't want a type error to be thrown can be made. Or could be made. Maybe, in the most general case, you want to be in some kind of state where you basically want to prove that you can in fact pass any value for the callback argument. Perhaps a test suite wants to ensure that an error in fact is being thrown if it's being passed anything else than a function. But I think that in practice a caller is interested in how to use the code properly, to get a same type of "function signature" as would be given in a module's documentation.

a lot of implicit conversions going on (you could pass anything, and it'd be converted to a string in most situations that expect a string)

You've got a point there. I can probably call fs.exists(123, cb) (native) and it will probably call my callback to tell me wether file with path "123" (a string) exists. But when I type fs.exists in an editor I want to see [String] (or so) somewhere close to the cursor. I think Tern did something like this, last time I played with it.

In any case, it tells me that String.substring takes "Number" arguments. But "abcdef".substring("2","4") === "abcdef".substring(2,4). Implicit type conversion by the friendly Javascript. So is the inferred (or hardcoded) function signature of String.substring wrong then? For me, it's not.

If someone want to build further inferences on top of a function signature that claims it takes a string argument, it can be understood that in fact anything that has a toString method could safely be passed in place of an actual string. At the same time, it could try to infer (not sure how hard that would be) if toString method is at least somewhere defined in object or the prototype chain before Object.toString, because (effectively) passing "[object Object]" is probably not what it's intended. In a stricter mode, it could also warn about passing anything that's not exactly a string.