cdnjs / cdnjs-importer

Easier way to add your libraries in cdnjs.

The #1 free and open source CDN built to make life easier for developers.

---

---

Table of Contents

• Introduction
  • Other Repositories
• Contributing
• Sponsors
• License

Introduction

This is the robot-only repository for cdnjs, where all the library assets that are hosted on cdnjs are stored. For the JSON files that control the libraries we host, please see the "human" cdnjs/packages repository.

Other Repositories

For the JSON files controlling the libraries we host on cdnjs, please take a look at the "human" cdnjs/packages repository.

For our website, please refer to the cdnjs/static-website repository.

For the cdnjs API, please refer to the cdnjs/api-server repository.

For the full cdnjs branding and brand-related assets/guidelines, please see the cdnjs/brand repository.

For our monthly CDN stats and usage reports, check out the cdnjs/cf-stats repository.

You can find all our repositories at github.com/cdnjs!

Contributing

As this repository is now considered robot-only, pull requests are no longer accepted for this repository. If you are looking to contribute to cdnjs, please take a look at the cdnjs/packages repository or any of our other open-source repositories on GitHub!

Sponsors

cdnjs wouldn't be the success that it is today without our sponsors' kind support. These companies currently support cdnjs:

• Cloudflare
• Algolia
• DigitalOcean
• Statuspage
• Sentry
• UptimeRobot

If you are interested in becoming a sponsor, please feel free to contact us!

License

Each library is released under its own license. This cdnjs repository is published under MIT license.

cdnjs-importer's Issues

Testing on Travis-CI

Should better wait for #16

Command line support

It would be nice to be able to use cdnjs-importer from a command line, passing the libs as argument(s).

For example:

$ cdnjs-importer [email protected]:jgthms/minireset.css.git

Where the command would be executed from the root directory of local cdnjs repo, so specifying its directory would not be necessary.

Support import from git repo

unit test

Add unit test for each functions so that we will not brake the functions in accident.

importer replaced "filename" with README

see cdnjs/cdnjs#6418
more specifically cdnjs/cdnjs#6418 (comment)

i ran importer (5e4f91a):

1. cdnjs-importer -p /home/users/glen/scm/cdnjs/cdnjs -g https://github.com/glensc/jquery.tiptip

my origin url had .js "filename" in package.json file: https://github.com/glensc/jquery.tiptip/blob/00ad42ba1296e94fa0f74e91f4fa328acf6b4635/package.json#L17

but importer created commit where it "filename" was replaced with "README.txt"

"npm*" keys being overwritten

see details here:
#23 (comment)
#23 (comment)

[new-version 2.0.0 test] Some problems occurred during the test.

@IonicaBizau~ ✨

I had tried some libraries and got some problems when i test the new-version v2.0.0

• Using -g when my path to cdnjs not the same with default

    $ cdnjs-importer -g [email protected]:IonicaBizau/gh.js.git
  
    /path/to/cdnjs-importer/bin/cdnjs-importer:44
    Logger.log("Couldn't find the CDNJS repository in this path: " + CDNJS_ROOT
                                                                    ^
    ReferenceError: CDNJS_ROOT is not defined

• Using -p to provide the path of cdnjs loction-->did not work and got the response below

  $ cdnjs-importer -p /path/to/cdnjs 
  Usage: cdnjs-importer [options]
  
  Options:
  -g, --git-url <git-url>  Your library git url.                   
  -p, --path <path>        The path to your cdnjs local repository.
  -h, --help               Displays this help.                     
  -v, --version            Displays version information.           
  
  Examples:
  cdnjs-importer -g [email protected]:IonicaBizau/gh.js.git # adds gh.js to cdnjs
  cdnjs-importer -g ... -p path/to/cdnjs
  
  The default cdnjs repository location is in ~/cdnjs
  
  Documentation can be found at https://github.com/IonicaBizau/cdnjs-importer
  

• cdnjs-importer -g git://github.com/SocketCluster/socketcluster-client.git -p /path/to/cdnjs did not pass the test

  • the error msg from npm test

    socketcluster-client 
      ✗ socketcluster-client: filename from package.json exists 
        » ./ajax/libs/socketcluster-client/2.3.9//tmp/tmp-4032mdSxKTrI9jPj/README.md does not exist but is referenced in package.json! // valid-packages-test.js:101 
  ✗ Broken » 14781 honored ∙ 1 broken (2.360s) 
  npm ERR! Test failed.  See above for more details.
  npm ERR! not ok code 0
  

  • the package.json

  {
  "name": "socketcluster-client",
  "description": "SocketCluster JavaScript client",
  "version": "2.3.9",
  "homepage": "http://socketcluster.io",
  "contributors": [
    {
      "name": "Jonathan Gros-Dubois",
      "email": "[email protected]"
    }
  ],
  "repository": {
    "type": "git",
    "url": "git://github.com/SocketCluster/socketcluster-client.git"
  },
  "dependencies": {
    "linked-list": "0.1.0",
    "sc-channel": "1.0.x",
    "sc-emitter": "1.0.x",
    "sc-formatter": "1.0.x",
    "ws": "0.7.2"
  },
  "readmeFilename": "README.md",
  "npmName": "socketcluster-client",
  "npmFileMap": [
    {
      "basePath": "/",
      "files": [
        "**/*"
      ]
    }
  ],
  "filename": "/tmp/tmp-4032mdSxKTrI9jPj/README.md"
  }
  

  I will keep testing!
  Fighting!:dancer::dancer::dancer:

Call cdnjs/tools/fixFormat.js before commit

cdnjs/tools/fixFormat.js is a small tool to help clean up libs' pacakge.json, remove the key-values and fix the format to what we want, it'll be good to call that tool before commit package.json!

Testing without cdnjs repo

Update is-there

filename contains the tmp path

[email protected]:chrisdavies/rlite.git

TypeError: Cannot read property 'replace' of undefined

Looking at this, I suppose it's that the file map isn't being built (for whatever reason).

dlynch@wmf1453:~/wikimedia$ cdnjs-importer -g https://gerrit.wikimedia.org/r/p/oojs/core -p ~/src/cdnjs
info  Pulling from origin.
info  Cloning https://gerrit.wikimedia.org/r/p/oojs/core into /var/folders/28/9vnnrdl97hx9zt4z9n15nxgc0000gp/T/tmp-804249v5twSnysk6U
info  Set the source path: /dist
info  Creating the version directory: /Users/dlynch/src/cdnjs/ajax/libs/oojs/1.1.10
info  Getting the library files.
/usr/local/lib/node_modules/cdnjs-importer/lib/index.js:211
                package.filename = files[0].replace(pwd, "");
                                           ^

TypeError: Cannot read property 'replace' of undefined
    at /usr/local/lib/node_modules/cdnjs-importer/lib/index.js:211:44

Use CLP as arguments parser

https://github.com/IonicaBizau/node-clp

Importer duplicates folders and files

When the source contains folders, importer would copy folders, but then also copy the files inside of those folders and duplicate them on the top level of the resulting copy.

For example with this: https://github.com/jnicol/trackpad-scroll-emulator, here is the result of importing with cdnjs-importer:

      ▾ css/
          trackpad-scroll-emulator.css
      ▾ demo/
        ▾ css/
            demo-fullscreen.css
            demo.css
        ▾ js/lib/
            jquery-1.7.1.min.js*
          fullscreen-horiz.html
          fullscreen.html
          index.html
      ▾ js/lib/
          jquery-1.7.1.min.js*
      ▾ lib/
          jquery-1.7.1.min.js*
        bower.json
        demo-fullscreen.css
        demo.css
        fullscreen-horiz.html
        fullscreen.html
        Gruntfile.js
        index.html
        jquery-1.7.1.min.js*
        jquery.trackpad-scroll-emulator.js
        jquery.trackpad-scroll-emulator.min.js
        LICENSE
        package.json
        README.md
        trackpad-scroll-emulator.css

You can see that the files below the folders are the same files that are in the folders. The source does not contain duplicates.

I investigated briefly and it seems to come from Getting the library files step, where the files array ends up with both folders and files inside those folders.

So that in Copying the library files step, I put a logger inside map and this is what I got:

info  Copying the library files
info  current file: /var/folders/d5/9j9st5w90cj5867wwr1g299h0000gn/T/tmp-2995sGtEqzj5zr31/bower.json
info  copy to: /Users/petrgazarov/workspace/cdnjs/ajax/libs/trackpad-scroll-emulator/1.0.0/bower.json
info  current file: /var/folders/d5/9j9st5w90cj5867wwr1g299h0000gn/T/tmp-2995sGtEqzj5zr31/css
info  copy to: /Users/petrgazarov/workspace/cdnjs/ajax/libs/trackpad-scroll-emulator/1.0.0/css
info  current file: /var/folders/d5/9j9st5w90cj5867wwr1g299h0000gn/T/tmp-2995sGtEqzj5zr31/css/trackpad-scroll-emulator.css
info  copy to: /Users/petrgazarov/workspace/cdnjs/ajax/libs/trackpad-scroll-emulator/1.0.0/trackpad-scroll-emulator.css
info  current file: /var/folders/d5/9j9st5w90cj5867wwr1g299h0000gn/T/tmp-2995sGtEqzj5zr31/demo
info  copy to: /Users/petrgazarov/workspace/cdnjs/ajax/libs/trackpad-scroll-emulator/1.0.0/demo
info  current file: /var/folders/d5/9j9st5w90cj5867wwr1g299h0000gn/T/tmp-2995sGtEqzj5zr31/demo/css
info  copy to: /Users/petrgazarov/workspace/cdnjs/ajax/libs/trackpad-scroll-emulator/1.0.0/css
info  current file: /var/folders/d5/9j9st5w90cj5867wwr1g299h0000gn/T/tmp-2995sGtEqzj5zr31/demo/css/demo-fullscreen.css
info  copy to: /Users/petrgazarov/workspace/cdnjs/ajax/libs/trackpad-scroll-emulator/1.0.0/demo-fullscreen.css
info  current file: /var/folders/d5/9j9st5w90cj5867wwr1g299h0000gn/T/tmp-2995sGtEqzj5zr31/demo/css/demo.css
info  copy to: /Users/petrgazarov/workspace/cdnjs/ajax/libs/trackpad-scroll-emulator/1.0.0/demo.css
info  current file: /var/folders/d5/9j9st5w90cj5867wwr1g299h0000gn/T/tmp-2995sGtEqzj5zr31/demo/fullscreen-horiz.html
info  copy to: /Users/petrgazarov/workspace/cdnjs/ajax/libs/trackpad-scroll-emulator/1.0.0/fullscreen-horiz.html
info  current file: /var/folders/d5/9j9st5w90cj5867wwr1g299h0000gn/T/tmp-2995sGtEqzj5zr31/demo/fullscreen.html
info  copy to: /Users/petrgazarov/workspace/cdnjs/ajax/libs/trackpad-scroll-emulator/1.0.0/fullscreen.html
info  current file: /var/folders/d5/9j9st5w90cj5867wwr1g299h0000gn/T/tmp-2995sGtEqzj5zr31/demo/index.html
info  copy to: /Users/petrgazarov/workspace/cdnjs/ajax/libs/trackpad-scroll-emulator/1.0.0/index.html
info  current file: /var/folders/d5/9j9st5w90cj5867wwr1g299h0000gn/T/tmp-2995sGtEqzj5zr31/demo/js
info  copy to: /Users/petrgazarov/workspace/cdnjs/ajax/libs/trackpad-scroll-emulator/1.0.0/js
info  current file: /var/folders/d5/9j9st5w90cj5867wwr1g299h0000gn/T/tmp-2995sGtEqzj5zr31/demo/js/lib
info  copy to: /Users/petrgazarov/workspace/cdnjs/ajax/libs/trackpad-scroll-emulator/1.0.0/lib
info  current file: /var/folders/d5/9j9st5w90cj5867wwr1g299h0000gn/T/tmp-2995sGtEqzj5zr31/demo/js/lib/jquery-1.7.1.min.js
info  copy to: /Users/petrgazarov/workspace/cdnjs/ajax/libs/trackpad-scroll-emulator/1.0.0/jquery-1.7.1.min.js
info  current file: /var/folders/d5/9j9st5w90cj5867wwr1g299h0000gn/T/tmp-2995sGtEqzj5zr31/Gruntfile.js
info  copy to: /Users/petrgazarov/workspace/cdnjs/ajax/libs/trackpad-scroll-emulator/1.0.0/Gruntfile.js
info  current file: /var/folders/d5/9j9st5w90cj5867wwr1g299h0000gn/T/tmp-2995sGtEqzj5zr31/jquery.trackpad-scroll-emulator.js
info  copy to: /Users/petrgazarov/workspace/cdnjs/ajax/libs/trackpad-scroll-emulator/1.0.0/jquery.trackpad-scroll-emulator.js
info  current file: /var/folders/d5/9j9st5w90cj5867wwr1g299h0000gn/T/tmp-2995sGtEqzj5zr31/jquery.trackpad-scroll-emulator.min.js
info  copy to: /Users/petrgazarov/workspace/cdnjs/ajax/libs/trackpad-scroll-emulator/1.0.0/jquery.trackpad-scroll-emulator.min.js
info  current file: /var/folders/d5/9j9st5w90cj5867wwr1g299h0000gn/T/tmp-2995sGtEqzj5zr31/LICENSE
info  copy to: /Users/petrgazarov/workspace/cdnjs/ajax/libs/trackpad-scroll-emulator/1.0.0/LICENSE
info  current file: /var/folders/d5/9j9st5w90cj5867wwr1g299h0000gn/T/tmp-2995sGtEqzj5zr31/package.json
info  copy to: /Users/petrgazarov/workspace/cdnjs/ajax/libs/trackpad-scroll-emulator/1.0.0/package.json
info  current file: /var/folders/d5/9j9st5w90cj5867wwr1g299h0000gn/T/tmp-2995sGtEqzj5zr31/README.md
info  copy to: /Users/petrgazarov/workspace/cdnjs/ajax/libs/trackpad-scroll-emulator/1.0.0/README.md

I think it could be solved in tandem with #29, where the importer only imports files that cdnjs can host... which would potentially solve this problem as well.

Should remove or fill empty field

I found that if there is empty field in package.json, importer won't removed it. But it should be removed or filled information in the field.
I take titatoggle as an example. The author field in package.json is empty, but importer do nothing for this. It need to be filled in the author info.

The library does not found on npm but importer add npm auto-update config

I take titatoggle as an example. This library does not be published on NPM, but importer added npm auto-update config in package.json. It should use git auto-update instead of npm autoupdate.

  "npmName": "TiTaToggle", 
  "npmFileMap": [  
    { 
      "basePath": "/dist",
      "files": [ 
        "**/*"  
      ] 
    } 
  ]

btw, if there is a dist folder, basePath should be "dist". We don't need the slash.

importer fill the wrong filename in package.json

I take titatoggle as an example. In this case, filename should be titatoggle-dist-min.css but I found that importer fill /_titatoggle.less into filename field.

Error when running from command-line

I have installed the module globally and then try to run it as follows:

cdnjs-importer -c /Users/erikschierboom/Programming/cdnjs [email protected]:ErikSchierboom/knockout-pre-rendered.git

I then get the following error:

/usr/local/lib/node_modules/cdnjs-importer/lib/index.js:100
 in autoupdate process (default: `[{ basePath: lib.dir , files: ["**/*"] }]`).
                                                                     ^
SyntaxError: Unexpected token *
    at Module._compile (module.js:439:25)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at Object.<anonymous> (/usr/local/lib/node_modules/cdnjs-importer/bin/cdnjs-importer:4:21)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)

fatal: Could not read from remote repository.

After running:

cdnjs-importer -c ~/code/cdnjs/ [email protected]:rileyjshaw/terra.git

I'm getting the error:

info  [21:49.31 - 12.04.2015] Clonning [email protected]:rileyjshaw/terra.git into /var/folders/45/< __redacted__ >
error [21:49.31 - 12.04.2015] Failed to clone the repository: Error: Command failed: Cloning into '.'...
Warning: Permanently added the RSA host key for IP address '192.30.252.129' to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Any thoughts?

Checkout a different branch

importer should not grab files that cdnjs does not host

I take titatoggle as an example. In this case, we need to grab titatoggle-dist.css and titatoggle-dist-min.css only. But I found that importer also grab _titatoggle.less in dist folder. We don't need to grab the files that cdnjs does not host.

DepShield encountered errors while building your project

The project could not be analyzed because of maven build errors. Please review the error messages here. Another build will be scheduled within 24 hours. If the build is successful this issue will be closed, otherwise the error message will be updated.

_{This is an automated GitHub Issue created by Sonatype DepShield. GitHub Apps, including DepShield, can be managed from the Developer settings of the repository administrators.}

Should not add unrelated files in commit.

I found that if there are some untracked files in the repo, cdnjs-importer will add them all with the commit, we should only add the files been fetched by the importer, not them all.