wearelucid / api-fetcher

Generates multiple JSON files from fetching API endpoints (i18n supported) 🛬

Api fetcher

Generates multiple JSON files from fetching (WordPress) API endpoints (i18n supported) 🚀

Installation

yarn add @wearelucid/api-fetcher

Usage

Be aware that this package uses ES6 syntax!

Add a script to your package.json

"scripts": {
  "fetch": "node fetchData.js"
}

Since node currently does not support es2015 module syntax, we need to install babel-cli and add the script like so:

"scripts": {
  "fetch": "babel-node --presets env -- fetchData.js"
}

yarn fetch

Example

Full Example (Wordpress)

import fetcher from 'api-fetcher'

const config = {
  savePath: './static/data',
  compressJSON: true, // setting this to false may help debugging :-)
  perPage: 5000, // arbitrary
  itemsPerPage: 10, // set for pagination (default will automatically be 10)
  languages: [
    { lang: 'de', locale: 'de_CH' },
    { lang: 'en', locale: 'en_US' }
  ],
  apiUrl: 'https://your-backend/api'
}

fetcher.log.printText('Lucid')
fetcher.log.printConfig(config)

// fetch paginated posts
fetcher.paginate('posts', { posts: { method: fetcher.getWPPostType, postType: 'posts', transforms: [removeFieldsFromPost] } },

// fetch bundled data
fetcher.bundle('basic', {
  pages: { method: fetcher.getWPPostType, postType: 'pages', transforms: [removeFieldsFromPost, decodeTitle], filters: [showOnlyPublished] },
  menus: { method: fetcher.getWPMenus },
  options: { method: fetcher.getWPOptionsPage, slug: 'options' },

  // If you need to get categories and custom taxonomies
  categories: { method: fetcher.getWPCategories },
  formats: { method: fetcher.getWPCustomTaxonomy, taxonomy: 'formats' }
}, config)


/**
* Filter (Note: This filter is an example. It is not needed. Wordpress by default only delivers published posts and pages)
*/
function showOnlyPublished (data) {
  return (data && data.length) ? data.filter(p => p.status === 'publish') : data
}

/**
* Delete fields we don't need (anymore)
*/
function removeFieldsFromPost (data) {
  return fetcher.applyToOneOrMany(_removeFieldsFromPost, data)
}

function _removeFieldsFromPost (data) {
  delete data._links
  return data
}

Generating Multiple Bundles

If you want to generate multiple json bundles you can invoke fetcher.bundle() with different a name like so:

fetcher.bundle('basic', {
  pages: { method: fetcher.getWPPostType, postType: 'pages', transforms: [removeFieldsFromPost, decodeTitle] },
  posts: { method: fetcher.getWPPostType, postType: 'posts', transforms: [removeFieldsFromPost, decodeTitle] }
}, config)

Generating Paginated Collections

You can also generated paginated collections like so:

fetcher.paginate('posts', { posts: { method: fetcher.getWPPostType, postType: 'posts', transforms: [removeFieldsFromPost, decodeTitle] } }, config)
fetcher.paginate('pages', { pages: { method: fetcher.getWPPostType, postType: 'pages', transforms: [removeFieldsFromPost, decodeTitle] } }, config)

In some cases you might also want to load all items of the once you loaded paginated (for having all the data):

fetcher.bundle('fileName', { posts: { method: fetcher.getWPPostType, postType: 'posts', transforms: [removeFieldsFromPost, decodeTitle] } }, config)

Default items per page will be set to 10. You can provide the variable itemsPerPage inside your config.

itemsPerPage: 10

This will generate a collection of json files (with your specified naming), in this case:

posts.de.1.json
posts.de.2.json
posts.de.3.json

api-fetcher's Issues

Don't override existing keys during ACFflattening

If you call an ACF 'link' it will override the posts default link.
First check existing keys and throw error that ACF needs another name.

Create data.json without existing file

At the moment the fetching process fails if the data.json file(s) is (are) not present already.
The data.json would be something I'd like to the gitignore file, but doing so will cause build fails, because of the issue above.
The fetcher should check if the
file exists > yes > override
or
file exists > no > create a new one