linkify.match gets stuck with string containing a lot of emoji about linkify-it

[puzrin]

Fixed

Thanks for your help!

New install will load fixed uc.micro, but i released a new version with bumped dependencies for sure.

[puzrin]

I've asked guys from disapora & npm to inspect this change for sure. If they will not find any edge cases, i'll do "official" release.

[puzrin]

Hm... that's really strange. There should be no such regexes and fuzzyLink: true should not be related. We cared about such situations.

online demo

See the link above, can't reproduce hang you have. Could you prepare test repo with minimal example?

[eramdam]

Oh shoot I wanted to say that I managed to avoid the issue by setting fuzzyLink to false. I wrote my issue a little took quickly 😓

I added a PoC.js file in the gist linked earlier https://gist.github.com/eramdam/d743452c916e90ff7b42720f4384088e

Try to switch fuzzyLink to false/true to see. Running node v6.2.0

[puzrin]

Hm... still can't reproduce, checked in node 6 latest and node 4 latest. Your gist has typos, here is one i used after fixes https://gist.github.com/puzrin/b573e94015f11c71e048e52a42b6db43

Difference with fuzzy on/off is almost not visible. Could you clarify your example?

[eramdam]

The two last lines on the string in PoC.js were actually very important because they are the one that triggers the issue for me. Specifically the abcd://abcdeghi.com part.

const str = 'Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis n😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡Lorem ipsum dolor sit😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈Lorem ipsum dolor sit amet, consectetur😇😇😇😇😇😇😇😇😇😄AAUID: 234567876543456 Sent via: abcd://abcdeghi.com';
const res = BaseFunc(str);

I recorded the situation if that might help https://asciinema.org/a/57ojqri3hvpuly7brol6izt1w

[eramdam]

More details:

With fuzzyLink: true

• The string Lorem ipsum dolor sit amet, consectetur😇😇😇😇😇😇😇😇😇😄AAUID: 234567876543456 Sent via: abcd://abcdeghi.com is processed finely (~20ms)
• The string 😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈Lorem ipsum dolor sit amet, consectetur😇😇😇😇😇😇😇😇😇😄AAUID: 234567876543456 Sent via: abcd://abcdeghi.com starts to slow down the process where it takes ~430ms to process it
• The string 😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡Lorem ipsum dolor sit😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈😈Lorem ipsum dolor sit amet, consectetur😇😇😇😇😇😇😇😇😇😄AAUID: 234567876543456 Sent via: abcd://abcdeghi.com starts to be pretty long to process (~3.7sec) and it gets worse the more emoji get added to the string.

And when I set fuzzyLink to false everything runs fine in 15ms.

[puzrin]

Reproduced :(. Pattern is '😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡 .com'

.com needed to pass simple check that fuzzy link may exist. @rlidwka take a look please

[puzrin]

This pattern also crashes online demo.

[puzrin]

Possible workarounds:

1. Reoptimize uc.micro regex to use u flags when available. Problem - may be can fix node (not sure), but still sucks with old browsers.
   • for old browsers we can drop astral matches before TLDs in fuzzy links.
2. Split fuzzy search to have strong start (or end) points:
   • search .TLD/valid_path. In this case pattern .TLD is simple & unique enougth to limit possible scans.
   • validate matching prefix before .TLD. In this case $ can help. If not - we can write back-direction scanner.

[eramdam]

Awesome! Glad to help, and thanks for the insanely fast response 👍

[puzrin]

Seems you've been interested in #26. Do you have stats about links with < & > ? I'm not sure that < > comes in pairs like (...) and [...].

I could suggest such kludge: if http(s):// is prepended with <, then drop in found link everything after >. But not sure how safe is it.

[eramdam]

@puzrin it's "common" in text version of automatic emails so yes > and < come in pair. The tricky thing is not to match the trailing > as part of the URL, sounds pretty safe to me since these are characters that are supposed to be in URLs AFAIK.

[puzrin]

So, you suggest to drop > only if it is the last character in link and < come before http:// ?

Under "pairs" i meaned location inside links ~ like http://example.com/foo_bar_(wiki)

[puzrin]

May be you have some stat when < or > can be used in link? Need examples to meditate over :)

[eramdam]

I don't have specific statistics, but it's used A LOT for emails also. And no I don't think >and <are supposed to be contained inside a valid URL:

given this <http://domain.com/foo_bar_(wiki)> then the parsed URL should be http://domain.com/foo_bar_(wiki).

Here are some examples I can get from the top of my head:

<http://domain.com>
<[email protected]>

[puzrin]

Try version from master. I don't know is it correct c502c8b.

< & > were disabled in links, including path & query.

[eramdam]

Will try that asap

[eramdam]

@puzrin I tried and it seems good to me, works as expected! 👍 👌