alanclarke / urlite

A very small, fast, dependency free url parser and formatter for nodejs and the web

A very small, fast, dependency free url parser and formatter for nodejs and the web

• fast
• few lines of code
• 100% test coverage

why is it so small and fast?

It extracts all url fragments in a single step using one massive regex

usage

npm install --save urlite

var url = require('urlite')

url.parse('http://user:[email protected]:3000/path?query=string#fragment')

{
  auth: 'user:pass',
  hash: '#fragment',
  hostname: 'blah.com',
  href: 'http://user:[email protected]:3000/path?query=string#fragment',
  path: '/path?query=string',
  pathname: '/path',
  port: '3000',
  protocol: 'http:',
  search: '?query=string'
}

var href = window.location.href
url.format(url.parse(href)) === href

Urlite extra

An extended version of urlite is available at urlite/extra. This includes helpful features such as querystring, hash and auth parsing:

// version of urlite with additional extras like querystring and auth parsing
var url = require('urlite/extra')
var parsed = url.parse('http://user:[email protected]:3000/path?a=b#c=d')
parsed.search // -> { a: "b" }
parsed.search.a = 'c'
parsed.hash // -> { c: "d" }
parsed.hash.c = 'e'
parsed.auth // -> { user: 'user', password: 'password' }
url.format(parsed) // -> 'http://user:[email protected]:3000/path?a=c#c=e'

comparison

File size:

NAME                    SIZE        SIZE (minified)
urlite                  3.02 kB     0.957 kB
urlparser               5.82 kB     1.57 kB
url-parse               12 kB       2.89 kB
url                     46.5 kB     11.8 kB
min-url                 25.6 kB     12.6 kB
fast-url-parser         55.2 kB     15 kB
url-parse-as-address    78.7 kB     22.7 kB

Performance:

require("urlite").parse             2,210,417 ops/sec ±0.90% (95 runs sampled)
require("fast-url-parser").parse    2,047,302 ops/sec ±0.89% (95 runs sampled)
require("urlparser").parse          631,561 ops/secs ±0.87% (92 runs sampled)
require("min-url").parse            343,680 ops/sec ±1.16% (93 runs sampled)
require("url-parse")                334,385 ops/sec ±1.03% (97 runs sampled)
require("url").parse                140,836 ops/sec ±1.26% (94 runs sampled)
require("url-parse-as-address")     135,691 ops/sec ±0.94% (95 runs sampled)

Want to work on this for your day job?

This project was created by the Engineering team at Qubit. As we use open source libraries, we make our projects public where possible.

We’re currently looking to grow our team, so if you’re a JavaScript engineer and keen on ES2016 React+Redux applications and Node micro services, why not get in touch? Work with like minded engineers in an environment that has fantastic perks, including an annual ski trip, yoga, a competitive foosball league, and copious amounts of yogurt.

Find more details on our Engineering site. Don’t have an up to date CV? Just link us your Github profile! Better yet, send us a pull request that improves this project.` Contact GitHub API Training Shop Blog About

urlite's Issues

Use jsize

https://www.npmjs.com/package/jsize

> jsize urlite urlparser url-parse url min-url fast-url-parser url-parse-as-address -v
urlite   2.22 kB (initial)  1.47 kB (minify)    708 B (gzip)
urlparser    4.85 kB (initial)  1.87 kB (minify)    869 B (gzip)
url-parse    12.64 kB (initial) 3.8 kB (minify) 1.73 kB (gzip)
url      43.04 kB (initial) 12.13 kB (minify)   4.75 kB (gzip)
min-url      25.38 kB (initial) 14.2 kB (minify)    5.02 kB (gzip)
fast-url-parser      52.94 kB (initial) 15.85 kB (minify)   5.74 kB (gzip)
url-parse-as-address     74.23 kB (initial) 23.25 kB (minify)   8.63 kB (gzip)

valid property

Would be a nice little helper and avoids having to use valid-url

url.parse('hey').valid // false
url.parse('https://github.com/QubitProducts/urlite/issues/new').valid // true

Parse fails when password contains '@'

Parse fails when password contains @

--- i/test/test-parse.js
+++ w/test/test-parse.js
@@ -33,7 +33,7 @@ describe('parse', function () {
   })

   it('should handle auth', function () {
-    var url = 'proto://user:[email protected]:3000/some/pathname?query=string#fragment'
+    var url = 'proto://user:pass@[email protected]:3000/some/pathname?query=string#fragment'
     expect(parse(url)).to.eql({
       auth: 'user:password',
       hash: '#fragment',

btw, parsing the url works using new URL()

Mishandles path

Cross browser testing

problems parsing "edge cases"

According to RFC 3986 the following URLs are valid, but not parsed properly by urlite:

http://:[email protected]:123/some/directory/file.html?query=string#fragment

http://[2010:836B:4179::836B:4179]
http://[::FFFF:129.144.52.38]:80/index.html
http://[3ffe:2a00:100:7031::1]

/wiki/Help:IPA is a valid path, but urlite considers this /wiki/Help: to be the protocol (/ may not occur in an URI's scheme).

And then there's the funky stuff like the backslash issue

http://i.xss.com\\www.example.org/some/directory/file.html?query=string#fragment

Ambiguous behaviour for parse, modify, format

When you parse, modify and then format a URL, the behaviour is not defined in the API.

For example

const parts = urlite.parse('https://github.com/QubitProducts/urlite/issues/new')

parts.query = 'a=4'
urlite.format(parts)
// https://github.com/QubitProducts/urlite/issues/new

parts.search = '?a=4'
urlite.format(parts)
// https://github.com/QubitProducts/urlite/issues/new?a=4

parts.host = 'bar.' + parts.host
urlite.format(parts)
// https://github.com/QubitProducts/urlite/issues/new?a=4

parts.hostname = 'foo.' + parts.hostname
urlite.format(parts)
// https://foo.github.com/QubitProducts/urlite/issues/new?a=4

It's useful to change some of the parsed url, then stick it back together afterward. Perhaps you could do something similar to the browser API for a elements

const urlParser = document.createElement('a')
urlParser.href = 'https://github.com/QubitProducts/urlite/issues/new'
urlParser.host = 'bar.' + urlParser.host

urlParser.href
// https://bar.github.com/QubitProducts/urlite/issues/new

urlParser.host
// https://bar.github.com/QubitProducts/urlite/issues/new

urlParser.hostname
// https://bar.github.com/QubitProducts/urlite/issues/new

You'd probably need to use setters for this to work.

Security issue in regex

The regex inside lib/pattern.js file seems to be vulnerable to Regex denial of service. I used this payload to cause a 17 second delay when a URL is processed by the parse() function. Maybe it can be even longer, probably depends on the length.

'//:' + '\t:\t'.repeat(90000)+ '\t'

querystring does not encode and decode params

When you parse or format query object you should decode or encode params.

See https://github.com/defunctzombie/node-url and https://github.com/Gozala/querystring packages.

Found a possible security concern

Hello 👋

I run a security community that finds and fixes vulnerabilities in OSS. A researcher (@kevin-mizu) has found a potential issue, which I would be eager to share with you.

Could you add a SECURITY.md file with an e-mail address for me to send further details to? GitHub recommends a security policy to ensure issues are responsibly disclosed, and it would help direct researchers in the future.

Looking forward to hearing from you 👍

(cc @huntr-helper)

Feature request - add "user" & "pass" if "auth" exists

Thank you regardless for this module!

Benchmarks

Add benchmark results to readme for speed, memory and filesize

Issue with the license

This package uses the UNLICENSED license

According to npm, it means:

Finally, if you do not wish to grant others the right to use a private or unpublished package under any terms:

{
  "license": "UNLICENSED"
}

Not sure it is intended. If not, can you please clarify the license? Thanks!