Comments (28)
Feel free to create a dummy facebook event (or even a FB post on your wall) and see what happens?
It seems like it fails to parse that link properly, instead linking to https://zh.wikipedia.org/wiki/ .
from linkify-it.
I'm sorry I'm not a Chinese speaker. Perhaps someone else can help out here. There are also other Chinese punctuation characters, like ,
from linkify-it.
More examples of how difficult these Chinese posts are to parse: link
What's your idea on this?
from linkify-it.
I will try to resolve this problem.
from linkify-it.
Could you post a permalink to demo http://markdown-it.github.io/linkify-it/ ? You can type there all examples at once, with results.
from linkify-it.
Problem is in (
, used in your example. There are NO space before it. I'm not familiar with asian punctuation. Are such (
always word terminators? Is is possible to have it in links, like in wiki?
from linkify-it.
I added another issue where a user mistakenly added http:// twice.
These are both (technically speaking) user behavior issues, but the first one especially is quite common.
Formally speaking ( is not a word terminator, however because it takes so much white space, many chinese people don't put a space in front of it (it's an act of being lazy, but it's quite common).
Perhaps I should do some string manipulation before I pass it to linkify-it
?
from linkify-it.
Example with double http:// is technically correct. That's like http://localhost
- any local domain allowed. Imho such typo do not need fixes.
Example with (
is more serious. It uses chineese unicode scopes, and can be distinguished from english scopes. But i need recommendations about grammar to be sure.
from linkify-it.
Let's leave it open until someone can formalize info for asian languages group (probably japan language has the same issues). I'm ready to fix as soon as possible, but would like to avoid kludges with inccomplete workarounds.
from linkify-it.
Seems you given wrong link, it shows default text.
from linkify-it.
Sorry, I've fixed the link now.
from linkify-it.
Thanks for examples. Posted a question at commonmark forum http://talk.commonmark.org/t/linkifier-lets-discuss-and-test/1045/9?u=vitaly
As far as i see in last example, spaces are not used at all. It's possible to to track locale change (non-english -> english), but that's not safe.
from linkify-it.
That's correct, spaces are not used. The Chinese use different punctuation marks or even don't use them because URLs are in English script and therefore the disctinction is easily visible to the eye, but more difficult for machines.
from linkify-it.
What about links with chineese chars? I can find link start somehow by [ any non english ]http://
, but can't use this rule to search link end.
PS. Anyway, it worth to start collecting test samples for china language separately. Something like this
from linkify-it.
You're absolutely right. I think the first rule makes sense, but URLs can probably have chinese characters. Perhaps the first rule by itself would help though, and somekind of smart processing for certain delimiters that are unlikely to appear within an URL, e.g.:
str.replace(/[\uff08\uff09\uff0c\uff01\u3002\uff1f\u3010\u3011\uff3b\uff3d\u3001\u300a\u300b\u2605]/g, ' ') //( ) , ! 。 ? 【 】 [ ] 、 《 》 ★
from linkify-it.
Found this URL doesn't match:
BTW, I found a website comparing different regexps for URL:
https://mathiasbynens.be/demo/url-regex
from linkify-it.
By the way, Facebook does this properly, but I'm not sure if they use a proprietary solution.
from linkify-it.
@fengmk2 implementation can be not easy, but for the first step it would be enougth to have collection of fixtures with good coverage of Chinese edge cases.
See https://github.com/markdown-it/linkify-it/tree/master/test/fixtures
from linkify-it.
Here is my example page linkifying incorrectly:
http://www.dancedeets.com/events/896736620379772/2016-r16-taiwan-x-wbc
In this case, it's a vertical bar, though to be precise it's |
and not |
(I hope github shows the difference properly). The page author uses it as a delimiter, but linkify sees it as part of the domain name (presumably it sees it as just another word character).
from linkify-it.
@mikelambert Nothing to fix. Fuzzy mode is not safe, it's an author's mistale to use linkify-it in wrong way.
from linkify-it.
I'm the website author using linkify-it to add links to raw text from a variety of sources which I didn't write myself (facebook events and other websites). And the individual source authors are not using linkify-it, and of course not writing their text with linkify-it in mind.
I recognize that fuzzy is not safe, and not perfect. However, it seems unfortunate that a vertical bar (what some authors are using as visual punctuation) is treated as part of a domain name. It seemed like the fuzziness could be smarter, even if it's still fuzzy.
So to clarify, is this a "it's not a bug, just user error" bug, or a "it's not a bug I care about fixing, but patches are welcome" bug?
from linkify-it.
@mikelambert I mean, problems with |
in that examples is because linkifier applied after layout compose, instead of before compose. But linkifier goal is to find links in natural texts, not everywhere (that's impossible).
My personal opinion is, that linkifier is used not as expected. So, this example is not good enougth as a reason for changes. May be i don't understand something, but this is my opinion for now. If i had to do such site, i would parse links first, then compose header.
Also, i understand that people can have another opinions and may wish to just quick fix something via hacks. For this case, linkifier allows to override regexps without need to fork project.
from linkify-it.
What do you mean by "layout compose" ? I assume you are referring to writing up the text and adding the |
to lay things out?
https://www.facebook.com/events/896736620379772/ is the source text I am working with. (Notice that Facebook gets the linkification correct.) I receive the raw text from the FB API, and then am trying to linkify it for use on my own website.
I understand that linkifier-it might be the wrong tool (since this is not natural text), and I am using it incorrectly (since it is applied after the author's layout composition). But unfortunately, I am not the author of the text, and so I am not able to linkify before adding the |
characters.
Thanks for your time!
from linkify-it.
Then, if you have source in known format (|
-separated), i would split it first, then apply linkifier, then join back. May be, with some additional conditions (line should be short and have chineese letters inside). Or would try to get html if possible (not familiar with FB api)
The reason to add |
support could be, if you say "humans write such way" or "that's de-facto standard", with a lot of examples.
from linkify-it.
In other words, if you have auto-generated text - consider parse/detect it's structure prior to apply linkifier. Or if you know some new uncovered de-facto patterns of human writing - create new issue with proofs (live examples), and i'll try to fix it if possible.
from linkify-it.
Thank you, that's a creative solution to this problem. I'll go with that for now.
I'll create a separate issue with the few examples I have, and you can decide if it's justified "de-facto pattern of human writin" or not there. :)
from linkify-it.
@mikelambert I'm wondering what does facebook linkify do with this following link?
link: https://zh.wikipedia.org/wiki/(
you can see what this link take you to here: https://zh.wikipedia.org/wiki/(
from linkify-it.
I am looking forward to the fix! It could be more enjoyable every time I paste links in the GitHub README with Chinese characters following it can SMARTLY display them correctly.
from linkify-it.
Related Issues (20)
- url not matched when it contains `...:` HOT 3
- options to escape a particular link? HOT 1
- Schema keys are case-insensitive, but only in one direction? HOT 3
- Add .app TLD to tlds_default HOT 6
- Parsed incorrectly when the link ends with the closing parenthesis in Chinese
- Colon after URL is included into URL, when using markdown-it with `linkify: true` HOT 1
- How to return 0 in the validate function ? HOT 1
- Links preceded by emojis are not rendered
- Router admin login IP disclosed in source code HOT 1
- Error: Namespace 'LinkifyIt' has no exported member 'LinkifyIt' HOT 3
- Change default protocol in fuzzyLink option HOT 2
- Doesn't seem to support domain-specific rules HOT 1
- Inconsistent link recognition when wrapped within html element HOT 2
- Matching URL's that contain spaces %20 HOT 2
- url with !! is not detected correctly HOT 1
- Email links followed by a hyphen are not detected HOT 5
- URLs with period and parantheses stop before closing parentheses HOT 3
- Links with "_" in the domain name are not regarded as links HOT 2
- Links that are containing '?' are not detected HOT 1
- Links with slashes capturing trailing semicolons HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from linkify-it.