dart-lang / string_scanner Goto Github PK

Used by 8 dart packages

master contains an @alwaysThrows fix for StringScanner.error (97cdbe2). Unfortunately, the CHANGELOG was updated to say 1.1.0, but the pubspec.yaml was updated to say 1.0.4. I think 1.0.4 is correct.

StringScanner lost data with the code below:

var content = 'xxxx[fn=4][fn=37][fn=36][fn=12][fn=26]xxxx';

var _scanner = StringScanner(content);
    while (!_scanner.isDone) {
      if (_scanner.scan(RegExp('\\[fn=(\\d+)]'))) {
        print(content.substring(_scanner.lastMatch!.start + 1, _scanner.lastMatch!.end - 1));
      }
      if (!_scanner.isDone) {
        _scanner.position++;
      }
    }

the result is :

I/flutter (25050): fn=4
I/flutter (25050): fn=36
I/flutter (25050): fn=26

but If I use this method directly：

var allMatches = RegExp('\\[fn=(\\d+)]').allMatches(content);
    for (var element in allMatches) {
      print('--${element.start}');
    }

I got the right answer：

I/flutter (25050): [fn=4]
I/flutter (25050): [fn=37]
I/flutter (25050): [fn=36]
I/flutter (25050): [fn=12]
I/flutter (25050): [fn=26]

something lost when I use StringScanner.

Please consider adding these methods

  /// Scans until [delimiter] matches [string] at the current position.
  ///
  /// Returns `true` if something is consumed, `false` otherwise.
  bool scanUntil(Pattern delimiter) {
    final oldPosition = position;
    while (!isDone && !matches(delimiter)) readChar();
    return oldPosition < position;
  }

  /// Scans until any of [delimiters] matches [string] at the current position.
  ///
  /// Returns `true` if something is consumed, `false` otherwise.
  scanUntilAny(List<Pattern> delimiters) {
    final oldPosition = position;
    while (!isDone && !delimiters.any(matches)) readChar();
    return oldPosition < position;
  }

Adding a static type makes the API easier to use correctly. The call site can easily inline the Uri.parse if necessary.

This package does not actually depend on path.

The following example demonstrates the issue:

import 'package:string_scanner/string_scanner.dart';

void main() {
  final text = '\u{12345}';
  final scanner = new SpanScanner(text);
  final start = scanner.state;

  while (!scanner.isDone) {
    scanner.readChar();
  }

  print(scanner.spanFrom(start));
}

This code throws:

RangeError: End 2 must not be greater than the number of characters in the file, 1.

I believe the issue is that package:string_scanner operates on code units, whereas package:source_span operates on code points (runes), resulting in this index mismatch for surrogate pairs.

I found a workaround by decoding the string first:

import 'package:source_span/source_span.dart';
import 'package:string_scanner/string_scanner.dart';

void main() {
  final text = '\u{12345}';
  final file = new SourceFile.decoded(text.codeUnits);
  final scanner = new SpanScanner.within(file.span(0));
  final start = scanner.state;

  while (!scanner.isDone) {
    scanner.readChar();
  }

  print(scanner.spanFrom(start));
}

Is the code unit and code point distinction here intentional? If so is there a better way to ensure proper UTF-16 support? The workaround seems reasonable, although having to provide a span rather than the decoded file itself is clunky. Would adding a constructor to SpanScanner which accepts a decoded file or list of code units be a suitable solution? Or could SpanScanner optionally operate on code points instead of code units?

<img src="https://avatars.githubusercontent.com/u/17034?v=3" align="left" width="96" height="96"hspace="10"> Issue by kevmoo
Originally opened as dart-lang/sdk#18469

It was fixed in https://code.google.com/p/dart/source/detail?r=35368

Need to verify this fix is sufficient.

...and bump the min-SDK to 1.4

The pub site can show example code for a package.
https://www.dartlang.org/tools/pub/package-layout#examples

Let me know if you want me to try working on this.

related issue : dart-lang/site-www#413

here is my code

class _MyHomePageState extends State<MyHomePage> {
  StringScanner _scanner = StringScanner(testStr);
  @override
  Widget build(BuildContext context) {
    print('$_scanner');
    print('${_scanner.scan(RegExp(r'/\*(.|\n)*\*/'))}');
    return Container(
      child: Text('test'),
    );
  }
}

It works great on the emulator, but when I package the release version, it will cause a stack overflow.

Complete code here：Github code

and I build release apk

apk