balazsbotond / urlcat Goto Github PK

Is your feature request related to a problem? Please describe.

When param keys are defined in the path template, there is no static validation that they are present in the params object, which can lead to unbound path template components at runtime. The most likely case in which this occurs tends to be a casing or spelling mismatch on a path template component or params property.

Describe the solution you'd like

With the advent of template literal types in TypeScript 4.1, we can check for keys defined in the path argument and make them required in params. I've stubbed out a naïve annotation, which doesn't cover all the implemented overloads, as a proof of concept.:

type PathTemplateTokens<Rte> = Rte extends `${string}/:${infer P}/${infer Rest}`
  ? P | PathTemplateTokens<`/${Rest}`>
  : Rte extends `${string}/:${infer P}`
  ? P
  : never;

function urlcat<BaseUrl extends string,
  PathTemplate extends string,
  Params extends Record<PathTemplateTokens<PathTemplate>, string | number>
>(
  baseUrl: BaseUrl,
  pathTemplate: PathTemplate,
  params: Params) { };

urlcat("http://example.com", "org/:orgName/user/:userId", {
  foo: "bar"
}); // error! required properties missing
urlcat("http://example.com", "org/:orgName/user/:userId", {
  orgname: "SprocketCo",
  userId: "951d20a0-e188-4db4-a946-df426d3d9e91"
}); // error! suggests `orgName` when spelled `orgname`
urlcat("http://example.com", "org/:orgName/user/:userId", {
  orgName: "SprocketCo",
  userId: "951d20a0-e188-4db4-a946-df426d3d9e91"
}); // ok, everything required is there
urlcat("http://example.com", "org/:orgName/user/:userId", {
  orgName: "SprocketCo",
  userId: "951d20a0-e188-4db4-a946-df426d3d9e91",
  sortBy: "sprocketsPerWeek"
}); // ok, extra stuff goes in the query string

Describe alternatives you've considered

N/A

Additional context
Add any other context or screenshots about the feature request here.
TypeScript Playground link to stub implementation

Screenshot of TypeScript suggesting params:

Screenshot of TypeScript catching a spelling mistake:

Thanks so much for your time! I love this library.

Hey,

in some use cases (like OAuth1) we require whitespaces to be encoded as %20, instead of +.

Can you thus please add an option to encode it as %20?

Reference: https://stackoverflow.com/questions/9336267/url-encode-oauth-signature

Thanks :)

Use currently accepted best practice as the basis of the comparison instead of my own clumsy solution.

See this Reddit comment: https://www.reddit.com/r/programming/comments/iyh2yn/urlcat_build_correct_urls_easily/g6eibpq?utm_source=share&utm_medium=web2x&context=3

See screenshot:

Warnings should be treated as errors.

A lint npm script should be added.

Linting should be part of the Travis build and break the build if there are any violations.

Like this:

[![](https://badgen.net/bundlephobia/minzip/urlcat)](https://bundlephobia.com/result?p=urlcat)

Describe the bug
The README notes several key features to the project. From the README:

Features:

Friendly API
No dependencies
0.8 KB minified and gzipped
TypeScript types provided

The second and third bullets are no longer correct.

To Reproduce
Steps to reproduce the behavior:

1. Go to README
2. Scroll down to 'Features'
3. See error

Expected behavior
The updated information on key features with the latest in master.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
This should be updated to the correct minified size and no longer include a mention of zero dependencies.

Describe the bug
locally runs fine, in a docker build my code blows up with "urlcat is not a function".

Node v16. I run in ESM mode.

minimal example is create a ts project with module ES2020, run with node v16 type: "module".

import urlcat from "urlcat";
urlcat("", "")

results in the "not a function" type error. I tried import {default as urlcat} from "urlcat" too.

any idea what could be happening? I really like this module!

I'm not a native English speaker. I would be very glad if someone from an English-speaking country could review the README and the JSDoc comments and correct any spelling/grammatic/stylistic mistakes.

This issue is currently blocked by #74

If we decide to continue supporting Deno, and choose the second or third option described in #74, we will need to run unit tests on both Node and Deno (and maybe even the browser?).

A proposal for this has already been discussed in #43. We could continue that discussion here.

Describe the bug
This library states it has no dependencies, yet it uses qs as dependency.

To Reproduce
Read README file.

Expected behavior
Remove claim that this is dependency free lib. Or remove the qs dependency.

Describe the bug
I build a URL like so:

urlcat("https://twitter.com", "/i/api/2/search/adaptive.json", {
	q: query,
	tweet_search_mode: "live",
	// seems to respect numbers <100, seems to only go up to 104??
	count: 100,
});

Where query is "🦇🔊", and get the following error:

TypeError: Failed to construct 'URL': Argument 1 is not a valid ByteString
    at makeException (deno:extensions/webidl/00_webidl.js:14:12)
    at converters.ByteString (deno:extensions/webidl/00_webidl.js:324:15)
    at Array.record<ByteString, ByteString> (deno:extensions/webidl/00_webidl.js:778:30)
    at Array.webidl.converters.sequence<sequence<USVString>> or record<USVString, USVString> or USVString (deno:extensions/url/00_url.js:529:67)
    at new URLSearchParams (deno:extensions/url/00_url.js:18:87)
    at query (https://deno.land/x/[email protected]/src/index.ts:95:10)
    at urlcatImpl (https://deno.land/x/[email protected]/src/index.ts:74:25)
    at urlcat (https://deno.land/x/[email protected]/src/index.ts:63:12)
    at makeSearchLatestUrl (file:///Users/alextes/code/twitter_scraper/src/twitter_web.ts:178:3)
    at file:///Users/alextes/code/twitter_scraper/src/twitter_web.ts:191:13

Encoding with encodeURIComponent seems to fix things but I thought the lib already did that for me? I'm also not sure why this started breaking one day to the next 🤔 .

Thought I'd report, maybe it was always broken somehow and you do need to encode emoji yourself?

In the documentation, we have a contribute a section on the main page as well as in separate pages. We need to remove either one. I added separate pages so we can add more info on how to contribute or compile from the source and do discuss this were to keep these details? on the main page or on separate pages?

@balazsbotond whats is your opinion on this?

Why?

• rich autocomplete in VS Code and other editors
• automatic API documentation generation in the future

The annotations should be based on the current API section of a readme, but a bit more detailed. Examples should be included (can be copied straight from the README). Good English is important - I'm not a native speaker so I would be very happy if someone from an English-speaking country could write these. But if you aren't native either, your contribution is still very welcome.

Reference:
https://www.typescriptlang.org/docs/handbook/jsdoc-supported-types.html

I'm not a native English speaker. I would be very glad if someone from an English-speaking country could review the README and the JSDoc comments and correct any spelling/grammatic/stylistic mistakes.

While we have already received a contribution for this, further ones would be still very much appreciated.

Is your feature request related to a problem? Please describe.
Case: need multiple values for query parameter. Currently (urlcat = 2.0.4) it always turns an array into a string and that ends up being a query parameter.

Describe the solution you'd like
In that case query parameter should be mentioned multiple times: each time with another value from an array.

Describe alternatives you've considered
Back to handwriting query.

Input:

urlcat('http://example.com', '/path', { q: 'a b' })

Expected:

http://example.com/path?q=a+b

Actual:

http://example.com/path?a%20b

To reproduce the exception:

in:

urlcat("http://example.com", "//");

out:

If a parameter appears twice in the path, it is substituted twice:
     TypeError [ERR_INVALID_URL]: Invalid URL: //

Users of urlcat may not know that new URL is called internally.
So users don't expect exceptions.

The problem is when pathTemplate is variable.
Users don't know what string cause an exception until they give it to urlcat. 😰

It forces users to add try.
Or users need to read the urlcat code to prevent this.

How is adding try internally ❓

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

For exemple,

const url = 'http://myurl.com?myparams=1'
urlcat(url, {foo: 2, bar: 3})

OUTPUT: http://myurl.com?myparams=1?foo=2&bar=2
WANTED: http://myurl.com?myparams=1&foo=2&bar=2

Currently we use a custom implementation. Using these built-in classes would reduce complexity, improve readability and prevent bugs.

This library is fantastic for building URLs simply, but it would be really useful if it could also support RFC-6570 URI Templates.

So, for example, I can do the following:

const url = urlcat('https://example.com', '/users/{id}{?full}', { id: '123', full: 'true', ignored: 'here' });

to produce https://example.com/users/123?full=true

I can probably do this using this library without expansions and then using something like https://www.npmjs.com/package/url-template for the rest, but it would be awesome to have it all-in-one.

Cheers

First of all, this is a very useful library - thank you for taking the time to build it.

Describe the bug
The URLSearchParams call in index.js file causes the script to fail when running node.js < v10. For whatever reason, I only see URLSearchParams when npm installing this library - but when cloning it locally and running a build I don't:

npm version

function query(params) {
    return new URLSearchParams(params).toString();
}

build version

function query(params, config) {
    var _a, _b;
    var qsConfiguration = {
        format: (_b = (_a = config === null || config === void 0 ? void 0 : config.objectFormat) === null || _a === void 0 ? void 0 : _a.format) !== null && _b !== void 0 ? _b : 'RFC1738',
        arrayFormat: config === null || config === void 0 ? void 0 : config.arrayFormat
    };
    return qs_1.default.stringify(params, qsConfiguration);
}
exports.query = query;

To Reproduce
Steps to reproduce the behavior:

1. Switch to a node version < 10.
2. Create an empty folder and npm install urlcat.
3. Write a simple node function that calls urlcat.
4. Run the node function.
5. Watch it fail because URLSearchParams does not exist in the global scope.

Expected behavior
I was expecting to see the same code when npm installing, as when I do when building this library locally.

Desktop (please complete the following information):
N/A

Smartphone (please complete the following information):
N/A

Additional context
Without getting into too much detail about the project I'm working on - it uses SSR with a node version that is less than v10. Additionally, I don't have access to simply add a "URLSearchParams" global to act as a polyfill/workaround.

Describe the bug
Possibly incorrect queryParams output

To Reproduce
urlcat('https://google.com', '', {test: 'abc'})
returns https://google.com/test=abc

Expected behavior
Should return
https://google.com?test=abc

export type ParamMap = Record<string, any>;

The type any is expected not to cause an error, no matter what type is passed.
In fact, I passed in various types, but urlcat didn't make an error, so I thought it was wonderful.

However, the following behavior was observed.
I don't know if these are intended.

array type

in:

urlcat("http://example.com", {q: [1, 2, 3]});

out:

http://example.com/?q=1%2C2%2C3

[ and ] are removed as a result of the array being stringized.
It will then be encoded.

object type

in:

urlcat("http://example.com", {q: { key: 1 }});

out:

http://example.com/?q=%5Bobject+Object%5D

The object has been stringized to [object Object].
The string will then be encoded.

Personally, this behavior still feels rough. 😅

I think

The behavior above seems to be inconsistent with passing number and string.
urlcat should encode array and object as string containing [ and {.

I think no need to parse params in detail.
The following multi-level arguments can be supplied by the specification, so there is no end to it.

urlcat ("http://example.com" , {q: [[1, 'a', true], 2, 3]});
urlcat ("http://example.com" , {q: {key: {child_key: 1}}});

I think no need to quit any.
Because urlcat is sometimes called from JavaScript as well as TypeScript.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

• chore(deps): update actions/checkout action to v4
• chore(deps): update actions/setup-node action to v4
• chore(deps): update dependency eslint to v9
• chore(deps): update dependency node to v20
• chore(deps): update dependency tsup to v8
• chore(deps): update github/codeql-action action to v3
• 🔐 Create all rate-limited PRs at once 🔐

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update dependency @types/qs to v6.9.15
• chore(deps): update dependency eslint to v8.57.0
• chore(deps): update dependency node to v18.20.2
• chore(deps): update dependency ts-jest to v29.1.2
• chore(deps): update dependency tsup to v6.7.0
• chore(deps): update jest monorepo (@types/jest, jest)
• chore(deps): update typescript-eslint monorepo to v5.62.0 (@typescript-eslint/eslint-plugin, @typescript-eslint/parser)
• fix(deps): update dependency qs to v6.12.1
• chore(deps): update dependency typescript to v5
• chore(deps): update typescript-eslint monorepo to v7 (major) (@typescript-eslint/eslint-plugin, @typescript-eslint/parser)
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

urlcat currently works with Deno - I know because I've manually tested it before the latest release. This is obviously very boring, easy to forget and error-prone, so it would be great if we could run the test suite on Deno too (our build currently runs on Node 10, 11, 12, 13 and 14) after each commit automatically.

If you have experience with Travis CI and Docker, and/or the inclination to dig deep into their documentation and you can deliver a nice, clean solution, please tell us in a comment :)

Hi there,

I'm a longtime user of urlcat. Currently using it on many personal projects as well as at work where we see tens of thousands of uniques per day.

I recently updated to the latest version of urlcat, and I was really disappointed to see that the bundle size has increased so much. Previously, one of the things I loved about urlcat was its small footprint and how lightweight it was. But now, with the increase to 10Kb, it feels like a step backwards.

I understand that sometimes it's necessary to increase the size of a library in order to add new features or improve performance. But in this case, it's not clear to me why the bundle size had to increase so much. Could you please explain the reasoning behind this decision?

I'm also concerned about the impact this will have on the performance of my app. A larger bundle size means longer load times and potentially slower runtime performance. Is there anything being done to mitigate these potential negative effects?

Overall, I'm really disappointed with this change. I've been a big fan of urlcat for a long time, but this increase in bundle size feels like a step in the wrong direction. I hope that in future updates, the focus can be on keeping the library lightweight and efficient, while still adding new features and improvements.

Thank you for your time.

We currently use tslint for linting but as it turns out, it is deprecated in favor of eslint.

We need to switch to eslint with a minimal config we can extend later.

Resources:

• Getting Started
• eslint Config
• eslint Rules

Is your feature request related to a problem? Please describe.
I'm trying to construct a query string from an array using query but it seems like I get the JavaScript default join behavior: keys=a,b

Describe the solution you'd like
Calling query with { keys: ['a', 'b'] } (and possibly a second options parameter to control this?) should result in "keys[]=a&keys[]=b"

Describe alternatives you've considered
Since query expects an object the only way to get around this is to wrap it in another function that pulls array keys out of the params object and maps them separately then concatenates the result with query's.

Additional context
I'm making an API request to a PHP backend that uses this URL syntax for receiving array values.

Describe the bug
The path is incorrectly concatenated with stringified object.

To Reproduce

1. Use Safari 10
2. Navigate to http://incandescent-farmer.surge.sh/
3. Result is /hello/world?%5Bobject=Object%5D=

Expected behavior
Result should be /hello/world only

Screenshots

Desktop (please complete the following information):

• OS: macOS Sierra
• Safari
• 10.1.2

I just tested and it appears that this module works with Deno! For instance, the following code:

import urlcat from 'https://denopkg.com/balazsbotond/urlcat/src/index.ts';

console.log(urlcat('https://api.foo.com', ':name', { id: 25, name: 'knpwrs' }));

Had the following output:

❯ deno run test.ts
Download https://denopkg.com/balazsbotond/urlcat/src/index.ts
Download https://raw.githubusercontent.com/balazsbotond/urlcat/master/src/index.ts
Check file:///Users/knpwrs/Downloads/test.ts
https://api.foo.com/knpwrs?id=25

It might be nice to do these three things:

1. Submit this module to https://deno.land/x, that way the import https://denopkg.com/balazsbotond/urlcat/src/index.ts can just be https://deno.land/x/urlcat.
2. Add some instructions to the readme showing usage with Deno and https://deno.land/x.
3. Add some CI runs for deno as well (just for sanity checking to make sure the import works, I don't think a full test suite is necessary as long as the import works and the Node tests pass).

Once this module is submitted to https://deno.land/x, the example above becomes:

import urlcat from 'https://deno.land/x/urlcat';

console.log(urlcat('https://api.foo.com', ':name', { id: 25, name: 'knpwrs' }));

We should throw an Error instead of silently continuing and leaving the path parameter placeholder in the rendered path.

I want to use feature added on #6.
It merged Oct 3 last year but not released yet.
There are any release schedule?

See this Reddit comment: https://www.reddit.com/r/programming/comments/iyh2yn/urlcat_build_correct_urls_easily/g6ebzjr?utm_source=share&utm_medium=web2x&context=3

This would give more credibility to the project.

Several patterns resulted in results that differed from predictions.
Are these intended results?

1. "https://example.test/" , "" -> "https://example.test"
2. "https://example.test/" , "/" -> "https://example.test"
3. "https://example.test//", "//" -> "https://example.test///"

Expected

1. "https://example.test/"
"https://example.test/" -> "https://example.test/", so it is unnatural for the trailing slash to disappear.
2. "https://example.test/"
   Same as 1.
3. "https://example.test//"
   　For an empty segment, the number of slashes appears to be -1.

from v2.0.4

Our current method for measuring code coverage is rather limited. First, it works on the JavaScript output instead of the original TypeScript; second, it also measures coverage for the tests themselves. The only thing it should measure is src/index.ts. The output should be in LCOV format so we can feed it into coveralls.

I have created the current solution in a hurry; I have no time to dig deeper into how TS code coverage measurement works. I would love to learn from someone who is more experienced or simply just has the time to do this properly.

NOTE: please only work on this if you either have experience in this area or are willing to learn about it without much handholding and are able to produce a nice, clean solution.

Thank you!

package.json

{
    "type": "module",
    "dependencies": {
        "typescript": "4.9",
        "urlcat": "^3.1.0"
    }
}

tsconfig.json

{
    "compilerOptions": {
        "module": "NodeNext",
        "moduleResolution": "nodenext",
        "strict": true
    },
    "include": ["./src"]
}

src/index.ts

import urlcat from 'urlcat'
//                 ~~~~~~~~
urlcat()

Reports the following error:

Could not find a declaration file for module 'urlcat'. 'node_modules/urlcat/dist/index.mjs' implicitly has an 'any' type.
  Try `npm i --save-dev @types/urlcat` if it exists or add a new declaration (.d.ts) file containing `declare module 'urlcat';`ts(7016)

Suggested changes to fix this problem

diff --git a/package.json b/package.json
--- a/package.json
+++ b/package.json
@@ -11,8 +11,14 @@
   "types": "dist/index.d.ts",
   "exports": {
     ".": {
-      "import": "./dist/index.mjs",
-      "require": "./dist/index.js"
+      "import": {
+        "types": "./dist/index.d.mts",
+        "default": "./dist/index.mjs"
+      },
+      "require": {
+        "types": "./dist/index.d.ts",
+        "default": "./dist/index.js"
+      }
     },
     "./package.json": "./package.json"
   },

and copy ./dist/index.d.ts to ./dist/index.d.mts

The API documentation should be generated instead of being hardcoded in the README.

We need to find a tool that produces nice-looking API docs easily and also allows adding additional pages.

And a badge added to the README

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

Is your feature request related to a problem? Please describe.
We need a basic website. This would give more credibility to the project.

Describe the solution you'd like
A simple way to do this would be to just publish the README as a web site. Copy the docs folder to the github pages branch and copy the contents of the README into index.md.

Describe alternatives you've considered
We could build a fancier, more complex website but I honestly don't have either the time or the patience to do so.

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

Only the following should be allowed as path parameter values:

• string
• number
• boolean - I'm not sure about this one though but I don't want to be overly strict

The following primitive types, in my opinion, should not be allowed:

• symbol
• bigint
• undefined
• null

Complex types (arrays and object) should not be allowed at all.

The error thrown should be a TypeError and its message should contain the parameter name and a list of allowed types.

urlcat('http://example.com', '/path', { p: 1, q: [2, 3] })

currently returns:

http://example.com/path?p=1&q=2%2C3

should return:

http://example.com/path?p=1&q=2&q=3

urlcat now uses qs internally. qs is licensed under the BSD 3-clause license which is less permissive than MIT, which urlcat currently uses. We need to determine if this means that urlcat also needs to change its license - I'll try to find some info about this but any help is appreciated.

Until #51 was merged, urlcat was dependency-free and therefore it could trivially support Deno along with Node and the browser. This PR introduced qs as a dependency to support arrays and objects as query parameters and different ways of formatting them.

I asked a question on the Deno Discourse server and after discussing this situation with @wperron, I can see 4 options:

1. Create a completely separate version of urlcat based on the 2.x branch specifically for Deno

• pros:
  • very easy to support Deno
  • we don't need to check intercompatibility for each change we make, like we would do with a single codebase
• cons:
  • no rich array/object formatting options in the Deno version
  • maintenance overhead of having to support 2 libraries that do something very similar

2. Fork qs

We could maintain our own fork of qs based on the TypeScript port here. qs is very mature and the release frequency isn't insane so each new version could be integrated into urlcat.

• pros:
  • we could only use the files needed for stringification - less bloat
  • single codebase for Deno, Node and the browser
  • Deno version is full-featured
• cons:
  • maintenance overhead of integrating each new version of qs into urlcat
  • we need to check intercompatibility for each change we make

3. Use Rollup or webpack and target ESM as a transpilation target

Note from the Deno chat:

I've never tried it, but be sure to check the imports generated by the bundler, Deno will require imports to have the file extension in order to work

• pros:
  • updating qs is much easier than with the previous option (just change package.json)
  • bundlers support tree shaking - less bloat
  • single codebase for Deno, Node and the browser
  • Deno version is full-featured
  • if we already use a bundler, it would be trivial to generate a zip/CDN version of urlcat for the web
• cons:
  • we need to check intercompatibility for each change we make
  • much more complicated build system

4. Drop Deno support

• pros:
  • zero maintenance overhead of supporting Deno
  • we don't need to check intercompatibility for each change we make
• cons:
  • urlcat wouldn't be available on a platform which is getting more popular and could be the next big thing in the js world

@tcarrio, @3zk1m0, @knpwrs or anyone else who is interested - I would really appreciate if you shared your opinion about this.