denoland / deno_std

The Deno Standard Library

Deno Standard Library

High-quality APIs for Deno and the web. Use fearlessly.

NOTE: The standard library is also available on JSR - check out the @std scope on JSR here.

Get Started

import { copy } from "https://deno.land/std@$STD_VERSION/fs/copy.ts";

await copy("./foo", "./bar");

See here for recommended usage patterns.

Documentation

Check out the documentation here.

Recommended Usage

1. Include the version of the library in the import specifier.

   Good:

   import { copy } from "https://deno.land/std@$STD_VERSION/fs/copy.ts";

2. Only import modules that you require.

   Bad (when using only one function):

   import * as fs from "https://deno.land/std@$STD_VERSION/fs/mod.ts";

   Good (when using only one function):

   import { copy } from "https://deno.land/std@$STD_VERSION/fs/copy.ts";

   Good (when using multiple functions):

   import * as fs from "https://deno.land/std@$STD_VERSION/fs/mod.ts";

3. Do not import symbols with a name prefixed by an underscore (they're not intended for public use).

   Bad:

   import { _format } from "https://deno.land/std@$STD_VERSION/path/_common/format.ts";

4. Do not import modules with a directory or filename prefixed by an underscore (they're not intended for public use).

   Bad:

   import { createLPS } from "https://deno.land/std@$STD_VERSION/streams/_common.ts";

   Good:

   import { TextLineStream } from "https://deno.land/std@$STD_VERSION/streams/text_line_stream.ts";

5. Do not import test modules or test data.

   Bad:

   import { test } from "https://deno.land/std@$STD_VERSION/front_matter/test.ts";

Stability

Sub-module	Status
archive	Unstable
assert	Stable
async	Stable
bytes	Stable
cli	Unstable
collections	Stable
console	Unstable
crypto	Stable
csv	Stable
data_structures	Unstable
datetime	Unstable
dotenv	Unstable
encoding	Stable
expect	Unstable
flags	Unstable
fmt	Stable
front_matter	Stable
fs	Stable
html	Unstable
http	Unstable
ini	Unstable
io	Unstable
json	Stable
jsonc	Stable
log	Unstable
media_types	Stable
msgpack	Unstable
net	Unstable
path	Stable
permissions	Deprecated
regexp	Unstable
semver	Unstable
streams	Stable
testing	Stable
text	Unstable
toml	Stable
ulid	Unstable
url	Unstable
uuid	Stable
webgpu	Unstable
yaml	Stable

For background and discussions regarding the stability of the following sub-modules, see #3489.

Design

Minimal Exports

Files are structured to minimize the number of dependencies they incur and the amount of effort required to manage them, both for the maintainer and the user. In most cases, only a single function or class, alongside its related types, are exported. In other cases, functions that incur negligible dependency overhead will be grouped together in the same file.

Deprecation Policy

We deprecate the APIs in the Standard Library when they get covered by new JavaScript language APIs or new Web Standard APIs. These APIs are usually removed after 3 minor versions.

If you still need to use such APIs after the removal for some reason (for example, the usage in Fresh island), please use the URL pinned to the version where they are still available.

For example, if you want to keep using readableStreamFromIterable, which was deprecated and removed in favor of ReadableStream.from in v0.195.0, please use the import URL pinned to v0.194.0:

import { readableStreamFromIterable } from "https://deno.land/[email protected]/streams/readable_stream_from_iterable.ts";

Contributing

Check out the contributing guidelines here.

Releases

The Standard Library is versioned independently of the Deno CLI. This will change once the Standard Library is stabilized. See here for the compatibility of different versions of the Deno Standard Library and the Deno CLI.

A new minor version of the Standard Library is published at the same time as every new version of the Deno CLI (including patch versions).

Badge

<a href="https://deno.land/std">
  <img
    width="135"
    height="20"
    src="https://raw.githubusercontent.com/denoland/deno_std/main/badge.svg"
    alt="Built with the Deno Standard Library"
  />
</a>

[![Built with the Deno Standard Library](https://raw.githubusercontent.com/denoland/deno_std/main/badge.svg)](https://deno.land/std)

deno_std's Issues

SSE support

Add support for server-sent events in net package.

CI should use v0.2.4

deno_std/azure-pipelines.yml

Line 2 in ffd9f9d

DENO_VERSION: 'v0.2.3'

(Travis too)

compile error when respond set headers

deno: 0.2.1
v8: 7.1.302.4
typescript: 3.2.1

import { serve } from "https://deno.land/x/net/http.ts";
const s = serve("0.0.0.0:8000");

async function main() {
  for await (const req of s) {
    req.respond({ body: new TextEncoder().encode("Hello World\n") });
  }
}

main();

error TS2322: Type '{ "content-type": string; }' is not assignable to type 'Headers'.
  Object literal may only specify known properties, and '"content-type"' does not exist in type 'Headers'.

69     req.respond({ headers: {"content-type": "application/json;charset=utf8"}, body: new TextEncoder().encode("Hello World\n") });
                               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  .deno/deps/https/deno.land/x/net/http.ts:42:3
    42   headers?: Headers;
         ~~~~~~~
    The expected type comes from property 'headers' which is declared here on type 'Response'

Create branch when we bump versions of Deno

The last branch we can "pin" to was v0.2.4, a pre-ESM release which had some changes the broke. It isn't safe to externally depend on master, but it would be great to get regular branches we can fix to.

Idea: .env file parser

Maybe create .env file parser for the std library? I would like to work on it

JSON response for net module

add JSON response support

Windows CI should use a fixed version of deno

deno_std/azure-pipelines.yml

Line 26 in 2ece847

- powershell: iex (iwr https://deno.land/x/install/install.ps1)

cc @srijanreddy98

Is deno supports runtime type checking?

Wanted: irc server

Needs tests. Although this repo has been cavalier about landing code without tests, I am trying to grow it now - and that necessitates the ability to check code in CI.

Less verbose test function

Currently, the test function has the following usage:

// #1

test({
  name: "example",
  fn() {
    // test code
  }
});

// #2

test(function example() {
  // test code
});

I like #2, but I think #1 is a bit verbose.

How about this instead?

test("example", () => {
  // test code
});

crash when visiting dir with index.html

Try running 'file_server' in the deno root directory and visiting http://0.0.0.0:4500/website

> file_server
HTTP server listening on http://0.0.0.0:4500/
[2018-12-12 10:07:09] "GET /third_party HTTP/1.1" 200
[2018-12-12 10:07:15] "GET /third_party/v8 HTTP/1.1" 200
[2018-12-12 10:07:19] "GET /third_party/v8/README.md HTTP/1.1" 200
[2018-12-12 10:07:52] "GET /third_party/v8/AUTHORS HTTP/1.1" 200
[2018-12-12 10:08:56] "GET /third_party/v8/.gn HTTP/1.1" 200
[2018-12-12 10:09:02] "GET /build_extra HTTP/1.1" 200
TypeError: Cannot read property 'status' of undefined
    at serverLog (file:///Users/rld/.deno/deps/https/deno.land/x/net/file_server.ts:182:71)
    at http_1.listenAndServe (file:///Users/rld/.deno/deps/https/deno.land/x/net/file_server.ts:204:5)

Discussion: asserts

In testing we originally had two methods assert and assertEqual

but at some point I (somewhat thoughtlessly) suggested that we mirror the Node API. So we have things like assert.equal as well as assertEqual.

What do people think? I kinda like assertEqual as it's less to type and I imagine the tab completion is better in VS Code.

Obviously we should do one or the other but not both.

(Related: I'd really like to have https://github.com/bokuweb/deno-pretty-assert in core... cc @bokuweb)

Unreachable code after infinte loop inside generic function "http#serve"

Code in this line never run and listener never closing.

Re-enable cat.ts test

deno_std/examples/test.ts

Lines 12 to 23 in 8ad47e3

	/** A more complicated test that runs a subprocess. */
	/* TODO re-enable this test. Seems to be broken on Windows.
	test(async function catSmoke() {
	const p = run({
	args: ["deno", "examples/cat.ts", "README.md"],
	stdout: "piped"
	});
	const s = await p.status();
	assertEqual(s.code, 0);
	});
	*/

It was disabled in 8ad47e3 because it froze on Windows.

Idea: file/file.ts -- ergonomics for interacting with files

Extending the core with more ergonomic file functions:

example:

export function fileExists(filename: string): boolean {
  return readDirSync(".").filter((f) => f.name === filename).length > 0;
}

export function writeStringSync(filename: string, data: string) {
  const encoder = new TextEncoder();
  writeFileSync(filename, encoder.encode(data));
}

export function readStringSync(filename: string, encoding: string) {
  const decoder = new TextDecoder(encoding);
  return decoder.decode(readFileSync(filename));
}

etc.

If this seems useful and in line with deno_std then I can propose a more complete API.

Gauging interest for collections module

Hi all,

I am interested in implementing generic collections if there is a desire for it. Things like maps, sets, etc. Is this a goal for this library?

Wanted: bundler

Very simple to start - but something like rollup - that uses deno-like imports.
Please scope out ideas here before submitting PR - this is ideally something everyone can share.

Add Windows CI

Speed up testing and benching

Currently testing and benching do awaits in a loop in runTests/runBenchmarks 🐌
I got a Promise.all refactor for benching. Think the same should be done for testing.
Does not require any API changes.

Logger implementation

I'm opening this issue to start discussion on vision for logging module.

Coming from Python world and being familiar with its library, my intent for Deno logging was to emulate Python's approach.

Its mechanism is based on:

• loggers - object to call to log something obviously
• LogRecord - contains timestamps and metadata about message logged by logger
• handlers - also known as transports. One logger may utilize multiple `handlers. Eg. console handler that print output to console, file handler that streams logs to file on disk.
• filters - predicates that interact with LogRecords, allowing for very granular control. Both loggers and handlers can have filters
• formatters - a function that given LogRecord instance returns output string

Loggers are accessed using getLogger function, so wherever in code you do const logger = logging.getLogger('myLogger'); you get the same instance.

This page gives a nice overview how to use it in action.

One of more popular node's logging libraries winston takes not so different approach.

I'm really interested what community expects.

Reorg

We've been adding code without much thought to the overall directory structure. I suggest the following renames:

//net/http.ts -> //http/mod.ts
//net/file_server.ts -> //http/file_server.ts

//net/bufio.ts -> //io/bufio.ts

//net/textproto.ts -> //textproto/mod.ts

//path/index.ts -> //fs/path.ts
//mkdirp/index.ts -> //fs/mkdirp.ts

//logging/index.ts -> //log/mod.ts

//flags/index.ts -> //flags/mod.ts

//colors/index.ts -> //colors/mod.ts

Along with the associated tests.

If you're worried about breaking existing code:

1. People should link to tagged releases of deno_std ... EG "https://deno.land/x/[email protected]/net/http.ts" .. we do not make stability guarantees for master.
2. It's better to make these changes now, when deno_std is young than later.

file_server should serve Content-Type header for common file formats

Testing module compatibility

At the minute we don't really seem to follow node's assertion interfaces or anyone else's.

Is there any reference as to what convention was followed or was it thought up just for deno?

It may be worth rewriting the testing module to adhere to node's assertion module and be entirely compatible.

Basically current implementation:

assertEqual(actual, expected, msg);
assert(equal(actual, expected), msg);

Node implementation:

assert.equal(actual, expected, msg);

So:

• assertEqual doesn't seem to be needed, only for setting a nice message (deepEqual and equal would do this themselves)
• equal does a deep equality check for us but in node does an Object.is alone (as deepEqual exists for the former)
• An AssertionError is thrown rather than an Error in node
• Negated functions exist in node such as notEqual

Add time util module

something like moment or date-fns

Let it be an official library.

Reduce developer focus, focus on their code rather than due to which library to choose

is there anyone think so

Too much usage of mod.ts

I'm glad we've got rid of the usage of index.ts everywhere - however I'm not so happy with the excessive usage of "mod.ts" now. For example, in http/mod.ts:

deno_std/http/mod.ts

Lines 1 to 7 in 4283c26

	export {
	serve,
	listenAndServe,
	Response,
	setContentLength,
	ServerRequest
	} from "./http.ts";

This is just boilerplate. It would be better to have people link to http/http.ts instead.

I would like to do another disruptive rename - sorry - in which we remove the mod.ts usage.

HTTP/2

Forgive me if this is already been discussed or I am wrong, but is there a reason that Deno HTTP (and Websockets) are based on HTTP/1. Everything else in Deno seems so modern, it seems weird to support the old standard and not HTTP/2. Maybe this is already on the roadmap?

Obviously biased article supporting this viewpoint:
https://medium.com/@pgjones/http-1-should-die-81b7588d617e

ProtocolError: malformed MIME header initial line

I'm sending over a POST request and I think it's confusing the body for the header.

ProtocolError: malformed MIME header initial line: "query": "{ hello }",

Does Deno need a QueryString module?

It's common in Node and JavaScript applications to store simple state in URL query strings. But manually parsing and serialising query strings is a pain. In Node-land, packages like e.g. qs are very popular.

My question:

1. Is this functionality worth bringing into a Deno standard library?
2. If so, what behaviours, capabilities and interface(s) would be most desirable and ergonomic?

Provide a test runner

I was wondering if it would be reasonable to provide a test runner that (with some config) searched for, loaded, and ran all tests of a project? I'm personally not quite a fan of having to import all the tests manually and run them like that, and I feel like it could potentially be a little limiting. Plus, having a test runner that searches and executes tests would allow us to spin up sub-processes to handle executing a large number of test suites in parallel, and handle the output properly. It would also allow us to provide a nice interface as well, and support doing things like watching for changes on disk and re-running the tests.

I'd be quite happy with writing this runner, but I'd prefer to make sure that this would be reasonable to do first, and if so, know what it should look like before implementing anything. My first thought is to make the runner read a configuration file (e.g. test.config.ts, this would also be a command line argument to support a custom config file), and in that you give the regex/glob matcher that is used when walking the directories in search of tests. Something like:

export default {
  include: /.*\.test\.ts^/,
  exclude: /scripts/
}

I'm open to suggestions on how this would work, this was just my first thoughts on it.

Safair cannot open the page

import { serve } from 'https://deno.land/x/net@4457b04/http.ts'
const s = serve('0.0.0.0:8000')

async function main() {
  for await (const req of s) {
    req.respond({ body: new TextEncoder().encode('Hello')})
  }
}

main()

Make prettier usable from other repositories.

deno --allow-run --allow-write https://deno.land/x/prettier/main.ts

The command like the above should format the source code in any repository.

wanted: http file server program

I'd love to be able to do

alias file_server="deno https://deno.land/x/net/file_server.ts --allow-net"

and it would serve the cwd() in http. This is useful for developing HTML locally.
Like this one https://www.npmjs.com/package/http-server

file_server should support file index

When serving a directory which does not have an index.html page a list of files should be displayed. Like http-server does.

Port more remaining responsewrite tests from Go

https://github.com/golang/go/blob/master/src/net/http/responsewrite_test.go#L19
https://github.com/denoland/deno_net/blob/master/http_test.ts

file_server should support "--cors" command line argument

when it does it also supplies the following headers

access-control-allow-origin: *
access-control-allow-headers: Origin, X-Requested-With, Content-Type, Accept, Range

wanted: prettier

currently in deno_std we fork out to Node in order to run prettier:
https://github.com/denoland/deno_std/blob/master/format.ts#L30

This has several problems:

• deno_std shouldn't depend on node.
• different people may have different prettier versions installed
• forking to execute JS is not optimal - we should already be able to execute JS.

Somehow we want to either have a remote link to a version of prettier that can run in the deno (i.e. browser bundle) or (maybe better) drop a prettier bundle into the deno_std source directory with corresponding .d.ts file. Let's say //prettier/prettier.js and //prettier/prettier.d.ts

Reenable logging test

deno_std/logging/test.ts

Lines 69 to 70 in 61fdae5

	// TODO(ry) Re-enable this test. Disabled because it was failing on Linux.
	// assertEqual(testOutput, expectedOutput);

5132550

merge deno_install into this repo

Reasons:

1. To manage fewer repos
2. To make sure changes in the install script do not inadvertently break deno_std's CI
3. With the new deno install script, people might want to import it and use it in other code.
4. To potentially add a install.sh script with the latest version of deno hard coded into it. deno_std currently gets bumped for every tagged deno release - I don't want to additionally bump deno_install too.
   It's slightly odd because deno_install contains a lot of code in python and powershell, but I think that can be overlooked.

Migrate testing module

IMHO it'd be a good idea to provide testing module as part of deno_std. I can take care of migration

output test information in http.ts

import { serve } from "https://deno.land/x/http/mod.ts";
serve;

output:

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

Add fs.walk

A basic implementation:

import { FileInfo, readDirSync } from "deno";

function *walk(dir: string): IterableIterator<FileInfo> {
  const ls = readDirSync(dir);
  for (const f of ls) {
    if (f.isSymlink()) {
      // TODO add followlinks option
    } else if (f.isFile()) {
      yield f;
    } else {
      yield *walk(f.path);
    }
  }
}

Should this go in fs/walk.ts or should there be some other directory structure?
The current org of fs is a little strange.

Also, what other functions ought to be in fs? glob?

http_test.ts

chrome is ok but curl and firefox never response

./deno_osx_x64 --version
deno: 0.1.12
v8: 7.1.302.4
typescript: 3.1.6

import { serve } from "https://deno.land/x/net/http.ts";

const addr = "127.0.0.1:8090";
const s = serve(addr);
console.log(`listening on http://${addr}/`);

const body = (new TextEncoder()).encode("Hello World\n");


async function main() {
  for await (const req of s) {
  await req.respond({ status: 200, body });
  }
}

main();

curl http://127.0.0.1:8090/
curl: (56) Recv failure: Connection reset by peer

Testing: Explanation of runTests()

I'm guessing there is a reason none of the test examples include the runTests() call. I inferred from reading main.ts that there is some plan to automatically call that from deno eventually.

May I suggest, at least temporarily, that some clarification be made to the main testing/README.md that includes this runTests() call? I have forked deno_std and am prepared to attempt a documentation update to describe that potentially saving other newbies to the project from searching for (don't take offense) how to get this damn test to actually run. 😁 deno test.js happily returns nothing without the runTests() call, of course. It's just not completely obvious to absolute beginners to add that.

---

By the way, "Hello!" I'm brand new to the project but have read through all the code for the deno_std libraries and have been using Deno (over Node) to teach command line coding and porting my own stuff. (I'm not gonna lie. I absolutely despise npm, package.json, and node_modules.) I cannot say how much I agree with every point @ry makes. I'm a pretty good writer and coder and have been programming since server-side JavaScript from Netscape was a thing.

Also I know this project is new. But I have been having tremendous success with GitLab, which has subgroups (organizations), built-in devops, and automated mirroring to or from any Git repo (for GitHub mirroring). I am preparing a 30 point comparison that will publish soon for evaluation. I can already see several places where this project would strongly benefit and since it is still young I thought at least I could mention it.

I apologize for this informal content in an issue. I see there is a ticket for IRC (or Discord or whatever) and noticed the first newsletter came out. Happy to help where I can (outside of the 60 hours a week I am teaching and creating curriculum).

CI should fail if format.ts detects differences

We do this in core with https://github.com/denoland/deno/blob/master/tools/test_format.py

README looks bad on mobile

Support streaming body

For serving large files using file_server.ts we want to avoid loading whole files into memory. To prevent that we should add support for streaming response in http.ts

Web sockets.

file_server should render markdown

Add crypto module

SHA1 is required for #29

testing: add runIfMain

Once we support imports.meta.url and have a way for testing if the current script is the main script, we want to have a two line snippit of code that will allow any test to be executed as a standalone test deno foo_test.ts rather than running the main entry point.
Essentially something like this:
https://github.com/tensorflow/tensorflow/blob/c4e61ac1fd0b3b48461e2876b20fe56c09faa2ff/tensorflow/python/lib/io/tf_record_test.py#L484

The most basic form would be something like this:

if (args[1] == imports.meta.url) {
  testMain();
}

But perhaps we can make this shorter by having a function like runIfMain(imports.meta.url)

This came up in: #129