tunnckocore / opensource Goto Github PK

Delivering delightful digital solutions. Monorepo of monorepos of Open Source packages with combined ~100M/month downloads, semantically versioned following @conventional-commits. Fully powered ES Modules, @Airbnb @ESLint + @Prettier, independent & fixed versioning. Quality with @Actions, CodeQL, & Dependabot.

Home Page: https://tunnckocore.com/opensource

JavaScript 99.26% TypeScript 0.74%

modern javascript monorepo opensource jest babel typescript eslint eslint-config babel-preset

opensource's People

Contributors

Stargazers

Watchers

Forkers

barbagrigia ciju fengweijp cybernetics maurojr navjinder stefaleon vorobeyko devopsutils tinesoft kensongoo suryatmodulus iamanujsh chocolateloverraj davidtanner veeral008 sorpconitdo jontze

opensource's Issues

TypeScript support?

I'm trying to use it with typescript but I'm kinda new to it. What exactly does default export of gibbon looks like? Is it function?

Not clear what to do with such included module in bundle:

As I can tell, it's just declaring var gibon inside module and not exporting it anywhere, so module just returns empty Object.

add option for file size limiting

In file event listener add one check that checks file.size against opts.textLimit or opts.fileLimit or opts.maxFileSize. Not sure for the name of the option. 1mb by default?

ref #42

Simplify my ESLint config

Seems like some plugins or configs are doing very heavy stuff and as a whole it's damn ridiculously slow.

When using just the eslint-config-airbnb it's noticeable faster.

Desirable to be able to configure the name of ctx.request.body

Right now, I can do this and access body, files and fields

var Koa = require('koa')
var body = require('koa-better-body')
var convert = require('koa-convert')

var app = new Koa()

app
  .use(convert(body()))
  .use(async function (ctx) {
    console.log(ctx.request.body)    // if buffer or text
    console.log(ctx.request.files)   // if multipart or urlencoded
    console.log(ctx.request.fields)  // if json
  })
  .listen(8080, function () {
    console.log('koa server start listening on port 8080')
  })

But if I want to use the property named body to access the content of a json, I'd do the following:

...
app
  .use(convert(body({
    fields: 'body', // Now body contains the incoming json
    files: 'uploads', // I can even rename files
    body: 'buffer' // BUT THIS I CAN'T DO 
  })))
  .use(async function (ctx) {
    console.log(ctx.request.buffer)    // if buffer or text - THIS WON'T WORK
    console.log(ctx.request.uploads)   // if multipart or urlencoded
    console.log(ctx.request.body)  // if json
  })
...

I'd like to be able to rename the buffer/text property to something else.

Branding

Some colors for future branding. I had branding orange color on Twitter, but after the new design/mobile change I lost it, I guess. I was copying it from there for years, but anyway.

Today looked around few logo generators and have some ideas and colors.

- #00AEF0 - light blue (cyan/almost cyan), from the app.logopony.com
- #248fed - a bit darker blue (from freelogodesign.org)
- #081E50 - more darker, for cmr (from the logopony) / #252c4d
- #f0e224 / #f0e10e yellow, it's great with the second blue
- #5c2785
Titan One, Sonsie One, Shrikhand, Fugaz One
Fredoka One (cmr)

utils: testCoverage() tweaks

rename the jestCov property that we suggest to write to the package's package.json.

Support other runners than Jest is/should be possible, probably.

update docs for koa-better-body

switch example to use https://github.com/tunnckoCore/koa-better-router
add table of contents

parse-commit-message v4: show the commit hash when the error is shown

thanks for the suggestion by @prxg22. I was thinking about that too.

Cannot find module '@babel/runtime/helpers/interopRequireDefault'

am trying to install orientjs in my project which in-turn requires parse-function() npm and it throws babel error

Gather stats information

Would be cool to gather how much times it is called :D I used pretty always, every single day hundreds of times, probably.

Would be something simple, like local storage at user's home or something like that.

npm install koa-better-body failing this morning

When I ran "npm install koa-better-body" on a clean machine this morning it failed with "npm ERR! enoent undefined ls-remote -h -t https://github.com/tunnckoCore/body-parsers.git"

Perhaps "body-parsers.git" should be "koa-body-parsers.git" instead?

Catch all route / wildcard / 404 route

utils doesn't support pkg.workspaces to be an object

https://github.com/tunnckoCore/opensource/blob/master/%40tunnckocore/utils/src/index.js#L127

e.g.

{
  "workspaces": {
    "packages": ["packages/*"],
    "nohoist": ["foo/**"]
  }
}

like we use it https://github.com/tunnckoCore/opensource/blob/master/package.json#L6-L18

Create tests for gibon

Don't know, but maybe I'll try jest for first time here. I'm just not frontend guy and that's one of the few frontend libs that i have.

Published package does not include TS definitions

typings is defined in package.json, but the file it points to is not present in the distributed package:

https://unpkg.com/browse/[email protected]/package.json

  "typings": "dist/typings/index.d.ts",

❯ npm pack koa-better-body
npm notice 
npm notice 📦  [email protected]
npm notice === Tarball Contents === 
npm notice 619B   dist/main/index.js      
npm notice 477B   dist/module/index.js    
npm notice 4.9kB  dist/main/utils.js      
npm notice 4.4kB  dist/module/utils.js    
npm notice 2.5kB  package.json            
npm notice 2.2kB  dist/main/index.js.map  
npm notice 2.3kB  dist/module/index.js.map
npm notice 13.1kB dist/main/utils.js.map  
npm notice 13.3kB dist/module/utils.js.map
npm notice 19.0kB CHANGELOG.md            
npm notice 24.1kB README.md               
npm notice === Tarball Details === 
npm notice name:          koa-better-body                         
npm notice version:       3.1.13                                  
npm notice filename:      koa-better-body-3.1.13.tgz              
npm notice package size:  20.9 kB                                 
npm notice unpacked size: 86.9 kB                                 
npm notice shasum:        eab742745401eff70b8069b29c284df87be2c1a6
npm notice integrity:     sha512-744Ay3hXDobRS[...]xC4sWJbErFndg==
npm notice total files:   11                                      
npm notice 
koa-better-body-3.1.13.tgz

In DefinitelyTyped/DefinitelyTyped#39451, @types/koa-better-body was deprecated (12 days ago), so this missing typings file is making this package unbuildable with TypeScript.

www: Roadmap

Use GatsbyJS
Use Emotion components + TailwindCSS (the tailwind.macro)

{
	"gatsby-plugin-advanced-sitemap": "^1.0.6",
    "gatsby-plugin-catch-links": "^2.0.15",
    "gatsby-plugin-compile-es6-packages": "^1.1.0",
    "gatsby-plugin-emotion": "^4.0.7",
    "gatsby-plugin-feed": "^2.2.1",
    "gatsby-plugin-google-analytics": "^2.0.20",
    "gatsby-plugin-manifest": "^2.1.1",
    "gatsby-plugin-mdx": "^1.0.5",
    "gatsby-plugin-offline": "^2.1.1",
    "gatsby-plugin-page-creator": "^2.0.13",
    "gatsby-plugin-prefetch-google-fonts": "^1.4.2",
    "gatsby-plugin-react-helmet": "^3.0.12",
    "gatsby-plugin-robots-txt": "^1.4.0",
    "gatsby-plugin-twitter": "^2.0.13",
    "gatsby-remark-copy-linked-files": "^2.0.12",
    "gatsby-remark-external-links": "^0.0.4",
    "gatsby-remark-smartypants": "^2.0.9",
    "gatsby-source-filesystem": "^2.0.37",
    "gatsby-transformer-json": "^2.1.11",
    "gatsby-transformer-remark": "^2.3.12",
    "gatsby-transformer-yaml": "^2.1.12",
    "tailwind.macro": "^1.0.0-alpha.9",
    "tailwindcss": "^1.0.2"
}

Can't make the multipart work with koa-router

I'm on [email protected], [email protected] and [email protected].

Trying to disable multipart for all the routes except /profilepic.

app.use(body({
  multipart: false,
  strict: false
}))

router.post('/profilepic',
  body({
    multipart: true,
    uploadDir: path.resolve(__dirname, '../../../uploads')
  }),
  this.changeProfilePicture
)

with this code changeProfilePicture is never called. It works if I remove the body() from the router and use multipart: true on app.use.

pretty-config: add support for passing array of names

Which will behave like that

let settings = await prettyConfig('rollup', { configFiles })

if (!settings) {
  settings = await prettyConfig('rolldown', { configFiles })
}
console.log(settings)

instead of above we can hide it to look like that

const settings = await prettyConfig(['rollup', 'rolldown'], { configFiles })

if (!settings) {
  console.log('no config files found for `rollup` and `rolldown` tools')
}

Fails docker builds for images that do not have git

If you use docker and a small image like alpine that has no git and rely on npm then you will not be able to build the image because this module dependency is one obtained via git.

Offending line:
https://github.com/tunnckoCore/koa-better-body/blob/master/package.json#L24

The solution, install it from npm instead of git.

eslint-config: update/add some rules

'func-name-matching': ['off', 'always', {
  includeCommonJSModuleExports: true,
  considerPropertyDescriptor: true,
}],
max-len: [
  'error', {
    code: 80,
    comments: 80,
    ignoreUrls: true,
    ignoreStrings: false,
    ignoreRegExpLiterals: true,
    ignoreTemplateLiterals: true,
    ignoreTrailingComments: true,
    ignoreComments: false
  }
]

commonly used CJS will fail:

/* eslint func-name-matching: ["error", "always", { "includeCommonJSModuleExports": true }] */

module.exports = function foo() {};
module['exports'] = function foo() {};

// no errors:

module.exports = () => {};
module['exports'] = () => {};

How can I keep a multipart form upload in memory (skip the file)?

I need to parse many requests with images and forward them to another server. Is there a way to avoid writing each one to a file? I would rather access some sort of Buffer with the file's data in it. Here is an example request:

curl -F "image=@$HOME/Pictures/blue_red_pill.jpg" -F "username=slim" http://localhost:3234/upload

Here is what I have so far:

const form = new formidable.IncomingForm()
const koaBody = require('koa-better-body')({
    multipart: true,
    IncomingForm: form,
})

form.onPart = function(part) {
    if (!part.filename) {
        // let formidable handle all non-file parts
        form.handlePart(part)
        return
    }   

    part.on('data', buffer => {
        if (buffer.length == 0) return
        console.log('received', buffer.length, 'bytes');
        // form.handlePart(???
    })

    part.on('end', () => {
        console.log('end');
        // form.handlePart(???
    })
    // part.on('error' // what about cleanup on error?
}

const router = require('koa-router')()
router.post('/upload', koaBody, function *() {
    const buffer = this.request[???]
})

I'm receiving the data, but I don't have anywhere to store it. Is there a way to get the data I collected back into the request?

Next Major: Simplifying the API, docs and the plugins, smaller codebase, TypeScript support

What about that?

import { parse } from 'parse-function'

const pluginsArray = [
  (node, result) => {},
  (astNode, result) => {},
]

const result = parse(string|Function, parserOptions, pluginsArray)

// or without passing parser options
const result = parse(string|Function, pluginsArray)

We are just removing the extra step of creating "app" instance and then calling parse.
Also removing the use so called "smart plugins", we don't need to change anything to that "app" instance.

So now plugins are more simpler and flat, instead of (app) => (node, result) => {}

Should be noted that we can't just add plugins property to the parserOptions, because will conflict with babylon for example.

parse function is broken

The last release of parse function is broken as it is missing arrify package which is in dev dependencies, while it should be in main dependencies.
We had to urgently update our package to add it.

web(ghub.now.sh): better 404 when package not exists in the registry

try https://ghub.now.sh/jh23jk4h2kj3h4k2j3h4

rewritten memoize-fs using `cacache`

// MPL-2.0 License
const crypto = require('crypto');
const meriyah = require('meriyah');
const cacache = require('cacache');
// const serializer = require('serialize-javascript');

const DEFAULT_OPTIONS = {
  cacheId: '$$rootId',
  astBody: false,
  serialize,
  deserialize,
};

function memoizeFs(options) {
  let opts = {
    ...DEFAULT_OPTIONS,
    ...options,
  };

  if (
    !opts.cachePath ||
    (opts.cachePath && typeof opts.cachePath !== 'string')
  ) {
    throw new TypeError('options.cachePath is expected to be of type string');
  }

  return {
    fn(funcToMemoize, settings) {
      opts = { ...opts, ...settings };

      return async function memoizedFn(...args) {
        const cacheData = generateMeta(funcToMemoize, args, opts);
        const id = cacheData.hashId;
        const res = opts.force ? false : await get(id, opts);

        if (opts.force) {
          await invalidateCache(id, opts);
        }

        if (!res) {
          const fnResult = await funcToMemoize(...args);
          const metadata = { ...cacheData, result: fnResult };
          const dataString = opts.serialize(metadata);

          await cacache.put(opts.cachePath, id, dataString, {
            metadata: {
              contents: dataString,
              cacheId: opts.cacheId,
              salt: opts.salt,
            },
          });

          return fnResult;
        }

        if (opts.maxAge > 0 && Date.now() > res.time + opts.maxAge) {
          await invalidateCache(id, opts);
        }

        const deserializedValue = opts.deserialize(res.metadata.contents);
        return deserializedValue;
      };
    },
    async invalidate(id, settings) {
      const opt = { ...opts, ...settings };

      if (!id) {
        await invalidateCache(id, opt);
        return;
      }

      const info = await getInfo(id, opt);
      const items = [].concat(info).filter(Boolean);

      await Promise.all(
        items.map(async (item) => {
          await invalidateCache(item.key, opt, item.integrity);
        }),
      );
    },

    async getInfo(id, settings) {
      return getInfo(id, { ...opts, ...settings });
    },
  };
}

async function getInfo(id, opts) {
  const cache = await cacache.ls(opts.cachePath);

  const cacheList = Object.keys(cache || {}).map((k) => cache[k]);

  if (!id || (id && typeof id !== 'string')) {
    return cacheList.length === 1 ? cacheList[0] : cacheList;
  }

  const ids = cacheList.filter((x) => x.metadata.cacheId === id);
  if (ids.length === 1) {
    return ids[0];
  }
  if (opts.latest) {
    const desc = ids.sort((a, b) => b.time - a.time);
    return desc[0];
  }
  return ids;
}

async function get(id, opts) {
  const res = await cacache.get.info(opts.cachePath, id);

  if (res) {
    const meta = await cacache.get(opts.cachePath, id);
    return {
      ...res,
      metadata: { ...meta.metadata, contents: meta.data.toString() },
    };
  }

  return null;
}

async function invalidateCache(id, opts, integrity) {
  if (!id) {
    await cacache.rm.all(opts.cachePath);
    await cacache.verify(opts.cachePath);
    return;
  }

  await cacache.rm.entry(opts.cachePath, id);

  const res = integrity
    ? { integrity }
    : await cacache.get.info(opts.cachePath, id);

  if (res) {
    await cacache.rm.content(opts.cachePath, res.integrity);
  }

  await cacache.verify(opts.cachePath);
}

function generateMeta(fn, args, options) {
  const opts = {
    ...DEFAULT_OPTIONS,
    ...options,
  };

  const salt = opts.salt || '';
  let fnStr = '';
  let fnAst = null;

  if (!opts.noBody) {
    fnStr = String(fn);
    if (opts.astBody) {
      fnAst = meriyah.parse(fnStr, { jsx: true, next: true });
      fnStr = JSON.stringify(fnAst);
    }
  }

  const argsStr = opts.serialize(args);
  const hashId = crypto
    .createHash('sha256')
    .update(fnStr + argsStr + opts.cacheId + salt)
    .digest('hex');

  return {
    fn: fnAst || fn,
    fnStr,
    args,
    argsStr,
    hashId,
  };
}

function serialize(val) {
  const circRefColl = [];
  return JSON.stringify(val, (name, value) => {
    if (typeof value === 'function') {
      return; // ignore arguments and attributes of type function silently
    }
    if (typeof value === 'object' && value !== null) {
      if (circRefColl.includes(value)) {
        // circular reference found, discard key
        return;
      }
      // store value in collection
      circRefColl.push(value);
    }
    // eslint-disable-next-line consistent-return
    return value;
  });
}

function deserialize(str) {
  return JSON.parse(str).result;
}

(async function main() {
  const memoizer = memoizeFs({
    cachePath: './some-cache',
    // serialize: serializer,
    // deserialize: (str) => eval(`(${str})`),
  });

  let c = 0;
  const memoizedFn = memoizer.fn(
    async (a, b) => {
      c += a + b;
      setTimeout(() => Promise.resolve(), 1501);

      return {
        a,
        b,
        c,
        help() {
          console.log('with cacheId and salt', a, b, c);
          return c;
        },
      };
    },
    { cacheId: 'some-cache-id', salt: 'b', maxAge: 15000 },
  );

  // await memoizer.invalidate();
  // await memoizer.invalidate('$$rootId');
  // await memoizer.invalidate('some-cache-id');
  // await memoizer.invalidate('some-cache-id', { latest: true });
  console.log(await memoizedFn(1, 2));
  console.log(await memoizedFn(1, 2)); // cache hit, or fresh hit when after maxAge
  console.log(await memoizedFn(1, 2)); // always cache hit
  console.log(c);
})();

Error when POSTing large files

This error seems to be similar to helapkg/hela#3. When I try to upload a large video file (622MB), I get this error:

TypeError: Cannot read property 'fields' of undefined
    at Object.parseBody 
    (/api/node_modules/koa-better-body/utils.js:222:33)
    at parseBody.next (<anonymous>)
    at Object.plugin 
    (/api/node_modules/koa-better-body/index.js:50:21)
    at plugin.next (<anonymous>)
    at onFulfilled
    (/api/node_modules/co/index.js:65:19)
    at <anonymous>

My configs are as follow:

let bodyParserMiddleWare = bodyParser({fields: 'body', formLimit: '1000mb'});

My request is multipart and my code looks like this:

let create = async (ctx, next) => {
    let file = ctx.request.files[0]; // I have access to the file I am uploading

    await ... // code that writes file to disk

    someFunction() // never executed as the error posted above occurs while the "await" is being executed
}

router.post('/', create);

This bug doesn't occur with smaller files. Is there a file limit in koa-better-body? A request timeout?

Thanks.

Tests and update the `multipart` and `formidable` recipes

There may need more tests, PRs are always welcome.

Update formidable recipe

test cases for below
when IncomingForm instance is passed to options, you should be able to pass IncomingForm options to that options. For example (pseudo-code)

var form = new IncomingForm()
form.on('file', console.log)

app.use(body({
  IncomingForm: form,
  keepExtensions: true,
  uploadDir: path.join(__dirname, 'uploads')
}))

instead of these two ways of doing it

first

var form = new IncomingForm({
  keepExtensions: true
  // etc
})

second (as in recipe and as in formidable README)

var form = new IncomingForm()
form.keepExtensions = true

koa-better-body example

Hi,

I ended up finding koa-better-body as it allows setting my own Formidable object, which koa-body does not seem to allow. My project also uses @koa/router.

The koa-better-body example with the router however does not seem to work? I am getting:

var app = require('koa')();
                        ^

TypeError: Class constructor Application cannot be invoked without 'new'

eslint-config: actually push the upadtes to no-param-reassign

always forgetting... ah? Too much tabs and workspaces locally. DANG :D

super weird bug in docs runner

For some reason it throws this error, almost always.

(node:799081) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'config' of undefined
    at /home/charlike/github/tunnckoCore/opensource/node_modules/@jest/core/build/TestScheduler.js:289:28
    at Generator.next (<anonymous>)
    at asyncGeneratorStep (/home/charlike/github/tunnckoCore/opensource/node_modules/@jest/core/build/TestSc
heduler.js:131:24)
    at _next (/home/charlike/github/tunnckoCore/opensource/node_modules/@jest/core/build/TestScheduler.js:15
1:9)
    at /home/charlike/github/tunnckoCore/opensource/node_modules/@jest/core/build/TestScheduler.js:156:7
    at new Promise (<anonymous>)
    at /home/charlike/github/tunnckoCore/opensource/node_modules/@jest/core/build/TestScheduler.js:148:12
    at onFailure (/home/charlike/github/tunnckoCore/opensource/node_modules/@jest/core/build/TestScheduler.js:302:26)
    at onError (/home/charlike/github/tunnckoCore/opensource/@tunnckocore/create-jest-runner/src/createJestRunner.js:137:9)
    at /home/charlike/github/tunnckoCore/opensource/@tunnckocore/create-jest-runner/src/createJestRunner.js:173:15
(node:799081) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 14)

No matter which is the package. Super awful for debugging and always forgetting how we can see the actual error. Debugged it few times but... :D It's nothing related to create-jest-runner, there is no problems.

labels

labels:
  - name: duplicate
    color: "#a3a3a3"
    description: This issue or pull request topic/problem already exists

  - name: good first issue
    color: "#1d76db"
    description: Good for newcomers and first-time contributors.

  - name: Priority: Critical
    color: "#ee0701"
    description: This should be dealt with ASAP. Not fixing this issue would be a serious error.

  - name: Priority: High
    color: "#d93f0b"
    description: After critical issues are fixed, these should be dealt with before any further issues.

  - name: Priority: Medium
    color: "#fbca04"
    description: This issue may be useful, and needs some attention.

  - name: Priority: Low
    color: "#0e8a16"
    description: This issue can probably be picked up by anyone looking to contribute to the project, as an entry fix.

  - name: Status: Available
    color: "#c2e0c6"
    description: No one has claimed responsibility for resolving this issue. Generally this will be applied to bugs and enhancement issues, but may be applied to others.

  - name: Status: Accepted
    color: "#78cc35"
    description: It's clear what the subject of the issue is about, and what the resolution should be.

  - name: Status: Blocked
    color: "#d73a4a"
    description: There is another issue that needs to be resolved first, or a specific person is required to comment or reply to progress. There may also be some external blocker.

  - name: Status: Abandoned
    color: "#000000"
    description: It's believed that this issue is no longer important to the requestor and no one else has shown an interest in it.

  - name: Status: In Progress
    color: "#cccccc"
    description: This issue is being worked on, and has someone assigned.

  - name: Status: On Hold
    color: "#e99695"
    description: Similar to blocked, but is assigned to someone. May also be assigned to someone because of their experience, but it's recognised they are unable to process the issue at this time.

  - name: Status: Completed
    color: "#adff60"
    description: Issue with steps to reproduce the bug that’s been verified by at least one reviewer.

  - name: Status: Review Needed
    color: "#fbca04"
    description: The issue has a PR attached to it which needs to be reviewed. Should receive review by others in the community, and at least one member / comitter. Specifics on when merging PRs is allowed is still up for debate.

  - name: Status: Revision Needed
    color: "#006b75"
    description: At least two people have seen issues in the PR that makes them uneasy. Submitter of PR needs to revise the PR related to the issue.

  - name: Type: Bug
    color: "#ee0701"
    description: Inconsistencies or issues which will cause an issue or problem for users or implementors.

  - name: Type: Maintenance
    color: "#fbca04"
    description: Updating phrasing or wording to make things clearer or removing ambiguity, without changing the functionality.

  - name: Type: Enhancement
    color: "#d4c5f9"
    description: Most issues will probably ask for additions or changes. It's expected that this type of issue will result in a Pull Request.

  - name: Type: Question
    color: "#cc317c"
    description: A query or seeking clarification on parts of the spec. Probably doesn't need the attention of everyone, just a few to help bring clarification or explain intent.

  - name: Type: Documentation
    color: #5319e7
    description: An Issue or Pull Request for improving or updating documentation.

  - name: Status: Proposal
    color: "#d4c5f9"

koa-better-body: Recipes - we always need more.

The are couple of recipes for the moment, but we accept more ideas.

parsing xml body (using the options.extendTypes.custom and options.handler)
usage with other middlewares: for example koa-router, koa-route or koa-resource
koa@2 way
jsonStrict
strict and non strict mode and why strict is the default - also need link to IETF
extendTypes
onerror and the defaults koa().onerror method
multipart: custom files and fields names
formidable.IncomingForm instance: events handling

Shared Formidable IncomingForm instance fails on concurrent uploads

Version: 3.0.2

Description: Providing a custom Formidable IncomingForm instance via the IncomingForm option on create causes concurrent uploads to put that instance in a bad state. The concurrent uploads usually fail with the following error message. Any further uploads after this failure will not complete until the connection times out.

  Error: MultipartParser.end(): stream ended unexpectedly: state = START_BOUNDARY
      at MultipartParser.end (/Users/rob/code/influent3/docker-rhel-fail/node_modules/formidable/lib/multipart_parser.js:326:12)
      at IncomingMessage.<anonymous> (/Users/rob/code/influent3/docker-rhel-fail/node_modules/formidable/lib/incoming_form.js:130:30)
      at emitNone (events.js:86:13)
      at IncomingMessage.emit (events.js:188:7)
      at endReadableNT (_stream_readable.js:975:12)
      at _combinedTickCallback (internal/process/next_tick.js:80:11)
      at process._tickCallback (internal/process/next_tick.js:104:9)

My understanding of Formidable is that the IncomingForm instance should not be reused across requests. The problematic lines in koa-better-body appear to be here.

Steps to reproduce:
Use the example code from the Formidable recipe here: https://github.com/tunnckoCore/koa-better-body/tree/master/recipes/formidable

Use the following to get the server into a failed state:

curl -i http://localhost:4290/ -F "foo=@path_to_some_reasonable_sized_file" & curl -i http://localhost:4290/ -F "foo=@path_to_some_reasonable_sized_file"

where path_to_some_reasonable_sized_file should supply something large enough to ensure the uploads are concurrent. In my case it was a file ~ 500kb.

After the above command, any further upload will lock until the connection times out. E.g. use the recipe's example curl:

curl -i http://localhost:4290/ -F "foo=@%s/README.md" -F user=test

do not load filesystem cache in rollup runner, if not dist found

cuz gives us false positives

also, do not load cache if config is changed.

ctx.request.files remains undefined

I can't access uploaded files - ctx.request.files remains undefined, ctx.request.fields shows me all form fields (including my field's string value - but no file or contents).

I posted details on Stack-overflow as well: https://stackoverflow.com/questions/45126597/koa-better-body-file-uploading-not-working

npm install removes raw-body dependencies from koa-body-parsers triggering error

This is a situation we have been fighting with for quite some time, and we keep checking out this block from our package-locks which is obviously very hacky

We use node 10.13.0 and npm 6.4.1 with docker in all our apis, and all of them use koa-better-body
We have been unable to explain why sometimes the raw-body dependency is added, and why sometimes it is removed

which triggers Cannot find module 'raw-body'

+        "koa-body-parsers": {
+          "version": "git+https://github.com/tunnckoCore/body-parsers.git#b515504ef0bfe3e04aa3dd13502d3d90c2d60956",
+          "from": "git+https://github.com/tunnckoCore/body-parsers.git#b515504ef0bfe3e04aa3dd13502d3d90c2d60956"
         }
       }
     },
-    "koa-body-parsers": {
-      "version": "git+https://github.com/tunnckoCore/body-parsers.git#b515504ef0bfe3e04aa3dd13502d3d90c2d60956",
-      "from": "git+https://github.com/tunnckoCore/body-parsers.git",
-      "requires": {
-        "raw-body": "^2.1.0"
-      }
-    },
// removes the whole raw-body block

The koa-better body version is set to ^3.0.4
Any advice is welcome!

Thank you! :)

koa-better-body: v4 - More granular control over limits and uploading

would resolve helapkg/hela#88
would resolve #46
would resolve #43
would resolve #42
(probably) Switch to use busboy or multiparty, because gives more (and more easier) control
Easier Renaming (not touching the hard disk?)
new extendTypes.file(s), so for example allow only .txt files

new options:

fileLimit - total of bytes for each file
filesLimit - total of bytes of all files combined
fieldLimit - total of bytes for each field value (if field is file field it respects fileLimit)
fieldsLimit - total of bytes of all fields
formLimit - combined of filesLimit and fieldsLimit
maxFiles - max number of files (max number of actual files, not how much file fields form has)
maxFields - max number of fields in form
maxFieldNameSize - total bytes of each field's name

hint: if you decide multiparty, use the part event only.

docks: support multiple files + more tweaks

First, support to collect docs from multiple files. Important to note is that if there is only one file, headings shouldn't be that nested. For example if there is multiple files then the h3 heading will be the filename, h4 the exposed things. If thre is only src/index.js then h3 should not be the filename but the exposed things.

Second, do not print **Params** if there is no params.

require('gibon') returns an empty object when using webpack

There might be a problem with your UMD bundler, requiring gibon using webpack results in an empty object:

var gibon = require('gibon');

console.log(gibon); // empty object

Versions involved:

  gibon 0.3.2
   node 7.7.1
    npm 4.3.0
webpack 2.2.1

I made a gist with the steps to reproduce it:
gist.github.com/gibatronic/b4cc95f723712863a8aa31a796b86a0a

Licensing Plan

Basically. A more clear explanation soon.

LICENSING.md - describe the process & the different licenses, answer questions, and etc.
LICENSE.md - quadruple licensed (when all have SPDX IDs, put them on license field too).
- (Parity-7.0.0 AND Prosperity-3.0.0) OR Patron-1.0.0 OR Private License
- all by @licensezero
For contributors, BlueOak Model v1.0.0 (consider allowing Apache-2.0), waiving their rights to me and the other CODEOWNERS and maintainers - through CLA bot. That's because to be able to sell private licenses more securely?
For significant contributions:
- inclusion in monorepo as CODE OWNER to some/all packages dirs
- inclusion in the split of the profits from Patron and/or Private Licenses
Patron License is subscription-based, you can get it at GumRoad (cards + paypal)
- https://gumroad.com/l/tunnckocorePatronLicense
There's also "tunnckoCore Recurring Donation" as support/donation mechanism, instead of other platforms like Patreon, Paypal, Paypal.me, OpenCollective, Ko-Fi and so on.
- https://gumroad.com/l/tunnckoCoreRecurringDonation

All this is until I get Stripe account and switch to LicenseZero CLI, which will be soon when register the company in UK. Or probably we can think of something in parallel then, or I don't know, will see.

Should be noted that when subscribing for Patron License or buying Private License, you get access to all the source code and packages in this monorepo. It's not per package, but for all!

All packages, code, and documentation are under the following

Parity v7.0.0, AND
Prosperity v3.0.0, or
Patron-License v1.0.0, or
Private License, contact [email protected]

All this basically means the following:

under Parity, similar to GPL (but not so strict and viral), you should open source your work and larger work, under ANY permissive license;
under Prosperity, you cannot use it for commercial/monetary as stated there;
Patron License, subsription-based through Gumroad for 1/3/6/12 months, Patreon or PayPal
- it's possible for even more, contact [email protected]

Examples

CodeceptJS (using `parse-function`)

Since they opensource the whole of their work as MIT, they use the software under Parity.

Renovate (using `get-installed-path`)

get-installed-path is not yet in this monorepo, but will be

The whole Renovate is open-sourced under AGPL, but since it have Pro plan, assuming that it can be sold (which triggers the Prosperity's "any manner primarily intended for or directed toward commercial advantage or private monetary compensation"), this means that its usage of get-installed-path can't fall under the Parity AND Prosperity condition, so it should go for Patron License or Private license, OR stick to the last version which isn't using this licensing. Which for this case is [email protected].

As of November, Renovate is part of WhiteSource and is completely opensource and free, and so it doesn't have problem using get-installed-path or any other package.

❯ npm pack koa-better-body
npm notice 
npm notice 📦  [email protected]
npm notice === Tarball Contents === 
npm notice 2.0kB  package.json
npm notice 708B   CHANGELOG.md
npm notice 16.5kB README.md   
npm notice === Tarball Details === 
npm notice name:          koa-better-body                         
npm notice version:       3.0.7                                   
npm notice filename:      koa-better-body-3.0.7.tgz               
npm notice package size:  6.4 kB                                  
npm notice unpacked size: 19.2 kB                                 
npm notice shasum:        40f01f5c11085880b5c4d3f0acae169d555c142b
npm notice integrity:     sha512-XmXjgqqUfxn6f[...]TExixQInkIM2w==
npm notice total files:   3

❯ tar xzvf koa-better-body-3.0.7.tgz 
x package/package.json
x package/CHANGELOG.md
x package/README.md

Anyways to set the file size limit for the `multipart/form-data`?

I see there is a common problem with formidable, just want to know is there a way to deal with file size limitation by the bytesReceived from form.progress ?

web(ghub.now): bug with the `.replace('.git')` part

Examples

https://ghub.now.sh/pacote
and some of my repos

babel plugin for "tunnckocoreInterop" for SANE cjs <-> esm

Basically as seen here, similar to add-module-exports which doesn't helps.

const tunnckocoreInterop = `const ___exportsWithoutDefault = Object.keys(exports)
  .filter((x) => x !== 'default')
  .reduce((acc, key) => {
    acc[key] = exports[key];
    return acc;
  }, {});

module.exports = Object.assign(exports.default, ___exportsWithoutDefault);
`;

So, we break neither of the systems.

// ESM
import foo from 'bar';
import { qux } from 'bar';

// CJS
const foo = require('bar');
const { qux } = require('bar');

Problem when going back in browser with gibon router

It doesn't handle route change when going back. Nothing fires.

utils: add `tryExtensions` and `async tryCatch`

Repeated in couple of Jest Runners.

the tryExtensions

function tryExtensions(filepath, config) {
  const { extensions } = getWorkspacesAndExtensions(config.cwd);
  const hasExtension = path.extname(filepath).length > 0;

  if (hasExtension) {
    return filepath;
  }

  const extension = extensions.find((ext) => fs.existsSync(filepath + ext));
  if (!extension) {
    throw new Error(`Cannot find input file: ${filepath}`);
  }

  return filepath + extension;
}

the tryCatch (notice that it doesn't have hasError now) and better signature.

function isObject(val) {
  return val && typeof val === 'object' && !Array.isArray(val);
}
async function tryCatch(fn, onError) {
  try {
    return await fn();
  } catch (err) {
    if (typeof onError === 'function') {
      return {
        error: onError(err),
      };
    }

    if (isObject(onError)) {
      return {
        error: fail({
          start: onError.start,
          end: new Date(),
          test: {
            path: onError.testPath,
            title: 'Rollup',
            errorMessage: `jest-runner-rollup: ${err.stack || err.message}`,
          },
        }),
      };
    }

    const now = new Date();
    return {
      error: fail({
        start: now,
        end: now,
        test: {
          path: '___rollup-runner-failing.js',
          title: 'Rollup',
          errorMessage: `jest-runner-rollup: ${err.stack || err.message}`,
        },
      }),
    };
  }
}

utils: the `res.workspaceRootPath` is probably the same as `res.cwd`

so, no need to pollute.

utils: testCoverage calculation

This one here

opensource/@tunnckocore/utils/src/index.js

Lines 215 to 236 in 80dcb16

    
           const res = Object.keys(files) 
        
             .filter((key) => key.startsWith(pkgRoot)) 
        
             .map((filename) => files[filename]); 
        
           if (res.length > 1) { 
        
             value = res.reduce( 
        
               (accumulator, item) => { 
        
                 accumulator.statements += item.statements / res.length; 
        
                 accumulator.branches += item.branches / res.length; 
        
                 accumulator.functions += item.functions / res.length; 
        
                 accumulator.lines += item.lines / res.length; 
        
                 return accumulator; 
        
               }, 
        
               { statements: 0, branches: 0, functions: 0, lines: 0 }, 
        
             ); 
        
           } 
        
           const cov = Object.keys(value).reduce( 
        
             (accum, typeName) => accum + value[typeName], 
        
             0, 
        
           );

The other way to calc is to sum all:

item.statements + item.branches + item.functions + item.lines

and then divide it by the res.length

Still not sure. But it's probably the same, or I'm sleepy.

	const res = Object.keys(files)
	.filter((key) => key.startsWith(pkgRoot))
	.map((filename) => files[filename]);

	if (res.length > 1) {
	value = res.reduce(
	(accumulator, item) => {
	accumulator.statements += item.statements / res.length;
	accumulator.branches += item.branches / res.length;
	accumulator.functions += item.functions / res.length;
	accumulator.lines += item.lines / res.length;

	return accumulator;
	},
	{ statements: 0, branches: 0, functions: 0, lines: 0 },
	);
	}

	const cov = Object.keys(value).reduce(
	(accum, typeName) => accum + value[typeName],
	0,
	);

tunnckocore / opensource Goto Github PK

opensource's People

Contributors

Stargazers

Watchers

Forkers

opensource's Issues

Update formidable recipe

Examples

CodeceptJS (using parse-function)

Renovate (using get-installed-path)

Recommend Projects

Recommend Topics

Recommend Org

CodeceptJS (using `parse-function`)

Renovate (using `get-installed-path`)