dart-lang / source_span

A library for identifying source spans and locations.

About this package

source_span is a library for tracking locations in source code. It's designed to provide a standard representation for source code locations and spans so that disparate packages can easily pass them among one another, and to make it easy to generate human-friendly messages associated with a given piece of code.

The most commonly-used class is the package's namesake, SourceSpan. It represents a span of characters in some source file, and is often attached to an object that has been parsed to indicate where it was parsed from. It provides access to the text of the span via SourceSpan.text and can be used to produce human-friendly messages using SourceSpan.message().

When parsing code from a file, SourceFile is useful. Not only does it provide an efficient means of computing line and column numbers, SourceFile.span() returns special FileSpans that are able to provide more context for their error messages.

source_span's Issues

DDC issue "Missing concrete implementation of 'SourceSpanMixin.highlight'. "

I run bazelify init and bazelify serve on an Angular2 project. I didn't expect to succeed, just curious what would happen. If this issue is not useful or actionable, please just close it.

____[96 / 178] Compiling @yaml//:yaml with ddc
ERROR: /home/myuser/.cache/bazel/_bazel_myuser/d7cc63dbd0d8e0618a44683ac56d1bee/external/source_maps/BUILD:15:1: Compiling @source_maps//:source_maps with ddc failed: Worker process sent response with exit code: 1..
[error] Missing concrete implementation of 'SourceSpanMixin.highlight'. (package:source_maps/src/source_map_span.dart, line 39, col 7)

Dart VM version: 1.21.0-edge.66cb033ac11dfc576967025ba4a20e3aefbf41f2 (Fri Nov 18 16:50:33 2016) on "linux_x64"

bazel 0.2.2

fix analyzer warnings

In particular:
[warning] Missing concrete implementation of getter 'SourceSpanWithContext.context' (source_map_span.dart, line 39, col 7)

Windows unsupported?

It seems that the build against windows is no longer possible due to termion incompatibility to it, see e.g. kamiyaa/joshuto#172

Fix Link in GitHub Repository Description

The link in the description of this repo is incorrect. It currently points to https://pub.dartlang.org/packages/string_scanner

This should be https://pub.dartlang.org/packages/source_span

Please provide examples or use cases

This looks like it may be useful, but I don't understand what it's for. From what I read, it can help provide more information for using is logging and errors, like name of the class or line an error is in?

Missing source tag for release 1.8.1 in Pub.dev

Please tag the version of the source code that was released as version 1.8.1 in Pub.dev.

RangeError in Highlighter

This bug was probably introduced in #30

The following highlighter test currently throws an exception:

test("at a newline", () {
      var span = new SourceFile.fromString("Hello\n\n\n").span(5,7);
      expect(span.highlight(), equals('what is the correct output?'));
});

throws:

  RangeError: Invalid value: Not in range 0..1, inclusive: -1
  dart:core                                                               _StringBase.lastIndexOf
  package:source_span/src/highlighter.dart 160:30                         Highlighter._lastLineLength
  package:source_span/src/highlighter.dart 129:19                         Highlighter._normalizeTrailingNewline
  package:source_span/src/highlighter.dart 63:15                          new Highlighter
  package:source_span/src/span_mixin.dart 64:16                           SourceSpanMixin.highlight

I'm not sure what the correct highlighter output should be in this case.

Consider not using runtimeType

This causes code-size warnings when compiling to js:

Hint: Using '.runtimeType.toString()' causes the compiler to generate more code because it needs to preserve type arguments on generic classes, even if they are not necessary elsewhere.
If used only for debugging, consider using option --lax-runtime-type-to-string to reduce the code size impact.

And we only use it in:

source_span/lib/src/span_mixin.dart

Lines 82 to 83 in 24151fd

	@override
	String toString() => '<$runtimeType: from $start to $end "$text">';

Maybe we can just change it to:

String toString() => '<SourceSpan: from $start to $end "$text">';

And have subclasses overwrite toString to ensure that they keep their name.

There is no 1.2.3 tag, but 1.2.3 is published to pub

A pub get can fetch version 1.2.3, but there is no tag corresponding to this fetch. Adding the tag, but if this is wrong, please remove it again.

SourceFile sets up _lineStarts eagerly

The body of the SourceFile.decoded could be executed lazily. It would increase the performance for cases when serialisation of spans is not necessary. For example, doing successful AST transformations without needing to report errors.

source_span/lib/src/file.dart

Lines 73 to 85 in f934308

	SourceFile.decoded(Iterable<int> decodedChars, {url})
	: url = url is String ? Uri.parse(url) : url,
	_decodedChars = new Uint32List.fromList(decodedChars.toList()) {
	for (var i = 0; i < _decodedChars.length; i++) {
	var c = _decodedChars[i];
	if (c == _CR) {
	// Return not followed by newline is treated as a newline
	var j = i + 1;
	if (j >= _decodedChars.length || _decodedChars[j] != _LF) c = _LF;
	}
	if (c == _LF) _lineStarts.add(i + 1);
	}
	}

Any solution for disjoint spans?

I use this package a lot via the SpanScanner in package:string_scanner. However, one difficulty I frequently run into when writing parsers is disjoint span errors when I call union().

This happens when parsing languages that ignore whitespace. For example, I just completed a GraphQL parser, but oftentimes the VM will crash on a disjoint span error before being able to explain the cause of actual syntax errors within GraphQL sources.

Is there any sort of common practice to avoid this?

I know this isn't an issue, per se, but I couldn't think of where else to ask this. Thanks in advance!!!

Change String or Uri arguments to Uri

Adding a static type makes the API easier to use correctly. The call site can easily inline the Uri.parse if necessary.

Computed column numbers are not consistent with editors for surrogate pairs

I tested that with the error reporting of dart-sass, which relies on that package (using SourceFile.fromString).

Giving it this invalid input, it reports an error on the ;:

a {
  b: "👭a"(;
}

Error: expected ")".
  ╷
2 │   b: "👭a"(;
  │            ^
  ╵
  test.scss 2:12  root stylesheet

The error is reported as being at column 12. However, all editors I tried (VS Code, Intellij IDEs, gedit) are reporting this ; as being at column 11. They seem to compute columns based on unicode codepoints (or maybe on glyphs), not based on UTF-16 code units.

Support textless spans

Currently SourceSpans are required to have text associated with them. However, this isn't always possible; for example, spans extracted from source maps won't necessarily have the text of the source or target available, nor should they need to. We should make text optional and make union() resilient to this.

Add Travis

Used by 21 Dart packages. Would be good to have frequent checks on this...

SourceFile.getText is doing too many allocations

I am stress-testing my parser and SourceFile.getText seems to be making unnecessary allocations.
Isn't String.fromCharCodes(_decodedChars, start, end); a valid substitute? It appears to cause much less allocations this way.

source_span/lib/src/file.dart

Lines 216 to 217 in f934308

	String getText(int start, [int end]) =>
	new String.fromCharCodes(_decodedChars.sublist(start, end));

Implement hashCode in _FileSpan to make dart2js happy

...since _FileSpan implements operator ==

Highlighting bug when the end line is a power of 10

When highlighting, there is normally a space between the line numbers and the sidebar.

line 13, column 1 of example.txt:
   ,
13 | example
   | ^^^^^^^
   '

but there isn't when the 1-indexed line number is a power of 10, e.g.

line 10, column 1 of example.txt:
  ,
10| example
  | ^^^^^^^
  '

I believe this is due to this line using the 0-indexed line number to determine how much padding there should be.

Add Travis CI and Code Coverage

cc @kevmoo

Invalid SourceLocation URI on Windows

This issue seems to be blocking: dart-lang/sdk#27797

DDC passes a sourceUrl to the SourceLocation constructor as follows:

C:\project\somefile.dart

Uri.parse chokes on it and sets invalid paths. DDC in turn will fail with the exception written down in the related issue.

Minimal reproduction code:

test('absolute sourceUrl is parsed correctly on Windows', () {
  p.Context windowsContext = new p.Context(style: p.Style.windows);
  var location = new SourceLocation(0, sourceUrl: r'C:\project\file.dart');
  expect(location.sourceUrl, windowsContext.toUri(r'C:\project\file.dart'));
  print(location.sourceUrl);
});

Too many line breaks cause ArgumentError when calling highlight()

version: 1.8.2 (latest)

SourceFile.fromString('abc\r\n\r\ndef').span(2, 6).highlight()


Invalid argument(s): The context line "abc

" must contain "c
".
#0      new SourceSpanWithContext (package:source_span/src/span_with_context.dart:28:7)
#1      _Highlight._normalizeNewlines (package:source_span/src/highlighter.dart:567:12)
#2      new _Highlight.<anonymous closure> (package:source_span/src/highlighter.dart:528:21)
#3      new _Highlight (package:source_span/src/highlighter.dart:531:11)
#4      new Highlighter (package:source_span/src/highlighter.dart:62:31)
#5      SourceSpanMixin.highlight (package:source_span/src/span_mixin.dart:72:12)

analysis issues when compiled with ddc

I see two analysis issues when this package is compiled with ddc:

severe: [AnalyzerMessage] The getter '_end' is not defined for the class 'SourceSpan' (package:source_span/src/file.dart, line 237, col 19)
severe: [AnalyzerMessage] The getter '_start' is not defined for the class 'SourceSpan' (package:source_span/src/file.dart, line 237, col 34)

examples can show up on the pub site

The pub site can show example code for a package.
https://www.dartlang.org/tools/pub/package-layout#examples

Let me know if you want me to try working on this.

related issue : dart-lang/site-www#413

Convert extension methods to instance methods in the next breaking release

#49 adds some extension methods on SourceSpan. These are being added as extensions to avoid breaking downstream classes that implement SourceSpan without including SourceSpanMixin, but they would ideally be instance methods whose implementation would be provided with the mixin but could also be overridden by downstream users. The next breaking change should fix this.