hexojs / hexo-migrator-rss

RSS migrator for Hexo.

hexo-migrator-rss

Migrate your blog from Atom/RSS to Hexo.

Install

$ npm install hexo-migrator-rss --save

Usage

Execute the following command after installed. source is the file path or URL of Atom/RSS.

$ hexo migrate rss <source> [--options]

• alias: Populates the alias setting in the front-matter, for use with the hexo-generator-alias module. This is useful for generating redirects.
• limit: Maximum number of posts to import from the feed. All posts are imported by default.
  • Example:

  $ hexo migrate rss /path/atom.xml --limit 3

• skipduplicate: Skip posts with similar title as existing ones.
  • If a feed contains a post titled 'Foo Bar' and there is an existing post named 'Foo-Bar.md', then that post will not be migrated.
  • The comparison is case-insensitive; a post titled 'FOO BAR' will be skipped if 'foo-bar.md' exists.
  • Without this option (default), this plugin will continue to migrate that post and create a post named 'Foo-Bar-1.md'

hexo-migrator-rss's Issues

"Not a feed" error when importing Squarespace feed

I'm trying to import my RSS feed from Squarespace and hexo-migrator-rss finds 0 posts and aborts with "Something's wrong", but no indication what.

I put some logging messages in, and found that the callback on both lines 38 and 40 receive an error parameter of "Error: Not a feed".

(BTW that error is never reported in the console output. Very unhelpful.)

My RSS feed validates, so I'm not sure why the migrator thinks it's not a feed.

undefined migrator plugin is not installed

after I do $ npm install hexo-migrator-rss --save
I go to my blog file and open git bash ,do $ hexo migrate rss ,it prompt ：
undefined migrator plugin is not installed.
Installed migrator plugins:

For more help, you can check the online docs: http://hexo.io/

how to solve that?please!

Handle excerpt with markup

Currently the description of a post is pass to excerpt of a post's front-matter. But this approach doesn't support markup.

So, a better approach is to prepend ${description}\n<!-- more -->\n to the post's content.

Support multiple sources

High Severity Vulnerability

This is the output of npm audit command.

                       === npm audit security report ===                        
                                                                                
┌──────────────────────────────────────────────────────────────────────────────┐
│                                Manual Review                                 │
│            Some vulnerabilities require your attention to resolve            │
│                                                                              │
│         Visit https://go.npm.me/audit-guide for additional guidance          │
└──────────────────────────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ High          │ Regular Expression Denial of Service                         │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ validator                                                    │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=3.22.1                                                     │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ hexo-migrator-rss                                            │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ hexo-migrator-rss > feedparser > resanitize > validator      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://npmjs.com/advisories/42                              │
└───────────────┴──────────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ High          │ XSS Filter Bypass via Encoded URL                            │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ validator                                                    │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=2.0.0                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ hexo-migrator-rss                                            │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ hexo-migrator-rss > feedparser > resanitize > validator      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://npmjs.com/advisories/43                              │
└───────────────┴──────────────────────────────────────────────────────────────┘

And this is the contents of the /var/www/html/node_modules/hexo-migrator-rss/package.json

{
  "_from": "hexo-migrator-rss",
  "_id": "[email protected]",
  "_inBundle": false,
  "_integrity": "sha512-A1ykFSEL6NyOlzX5ayDjikNu/DoL2ikPNg+N1JZXUZ4hAcgTrOSRwSPBMigJK0z8AjGyrvv+8hXz8w2tQ5J3MQ==",
  "_location": "/hexo-migrator-rss",
  "_phantomChildren": {},
  "_requested": {
    "type": "tag",
    "registry": true,
    "raw": "hexo-migrator-rss",
    "name": "hexo-migrator-rss",
    "escapedName": "hexo-migrator-rss",
    "rawSpec": "",
    "saveSpec": null,
    "fetchSpec": "latest"
  },
  "_requiredBy": [
    "#USER",
    "/"
  ],
  "_resolved": "https://registry.npmjs.org/hexo-migrator-rss/-/hexo-migrator-rss-0.1.4.tgz",
  "_shasum": "636156cb031a49773599a845cb230e08a9c06297",
  "_spec": "hexo-migrator-rss",
  "_where": "/var/www/diary/html",
  "author": {
    "name": "Tommy Chen",
    "email": "[email protected]",
    "url": "http://zespia.tw"
  },
  "bugs": {
    "url": "https://github.com/hexojs/hexo-migrator-rss/issues"
  },
  "bundleDependencies": false,
  "dependencies": {
    "async": "^0.9.0",
    "chai": "^3.4.1",
    "feedparser": "0.16.6",
    "request": "^2.36.0",
    "to-markdown": "0.0.1",
    "url": "^0.11.0"
  },
  "deprecated": false,
  "description": "RSS migrator plugin for Hexo",
  "homepage": "https://github.com/hexojs/hexo-migrator-rss#readme",
  "keywords": [
    "hexo",
    "rss",
    "feed",
    "atom",
    "migrator"
  ],
  "license": {
    "type": "MIT",
    "url": "https://raw.github.com/hexojs/hexo-migrator-rss/master/LICENSE"
  },
  "main": "index",
  "maintainers": [
    {
      "name": "Abner Chou",
      "email": "[email protected]",
      "url": "http://abnerchou.me"
    }
  ],
  "name": "hexo-migrator-rss",
  "repository": {
    "type": "git",
    "url": "git+https://github.com/hexojs/hexo-migrator-rss.git"
  },
  "scripts": {
    "test": "mocha test"
  },
  "version": "0.1.4"
}

I even reinstalled the plugin the issue remains.

TypeError: Cannot read property '1' of null

There is a problem when migrating blog ghost. Process is throwing TypeError: Cannot read property '1' of null. You may try it with tforia.com/blog/rss/.

has no method 'parseUrl'

I use this plugin to try to migrate my blog and I get a error with

has no method 'parseUrl'
TypeError: Object function FeedParser(options) {
  if (!(this instanceof FeedParser)) return new FeedParser(options);
  TransformStream.call(this);
  this._readableState.objectMode = true;
  this._readableState.highWaterMark = 16; // max. # of output nodes buffered

  this.init();
  this.parseOpts(options);
  // See https://github.com/isaacs/sax-js for more info
  this.stream = sax.createStream(this.options.strict /* strict mode - no by default */, {lowercase: true, xmlns: true });
  this.stream.on('error', this.handleSaxError.bind(this));
  this.stream.on('processinginstruction', this.handleProcessingInstruction.bind(this));
  this.stream.on('opentag', this.handleOpenTag.bind(this));
  this.stream.on('closetag',this.handleCloseTag.bind(this));
  this.stream.on('text', this.handleText.bind(this));
  this.stream.on('cdata', this.handleText.bind(this));
  this.stream.on('end', this.handleEnd.bind(this));
} has no method 'parseUrl'
    at async.waterfall.async.forEach.postLink (/tmp/blog/node_modules/hexo-migrator-rss/index.js:23:22)
    at fn (/tmp/blog/node_modules/hexo-migrator-rss/node_modules/async/lib/async.js:582:34)
    at Object._onImmediate (/tmp/blog/node_modules/hexo-migrator-rss/node_modules/async/lib/async.js:498:34)
    at processImmediate [as _immediateCallback] (timers.js:330:15)

It seems like there is no method.
Then I check the source and files in node_modules.
In source you use ortoo-feedparser but in files with npm installed you use feedparser.

feedparser don't exports parseString, parseFile, parseUrl, and parseStream as static functions while ortoo-feedparser does.

And the plugin version is 0.0.5.
I install plugin with

npm install hexo-migrator-rss

"not a feed" when: hexo migrate rss http://liguanghe.github.io

my ENV version

Mac 10.12
hexo: 3.3.6
hexo-cli: 1.0.2
os: Darwin 16.5.0 darwin x64
http_parser: 2.7.0
node: 6.10.1
v8: 5.1.281.95
uv: 1.9.1
zlib: 1.2.8
ares: 1.10.1-DEV
icu: 58.2
modules: 48
openssl: 1.0.2k

I do this: hexo migrate rss http://liguanghe.github.io

got this:
INFO Analyzing http://liguanghe.github.io...
INFO 0 posts migrated.
FATAL Something's wrong. Maybe you can find the solution here: http://hexo.io/docs/troubleshooting.html
Error: Not a feed
at FeedParser.handleEnd (/Users/liguanghe/liguanghe.github.io/node_modules/feedparser/main.js:129:13)
at emitNone (events.js:86:13)
at SAXStream.emit (events.js:185:7)
at Object.SAXStream._parser.onend (/Users/liguanghe/liguanghe.github.io/node_modules/feedparser/node_modules/sax/lib/sax.js:184:8)
at emit (/Users/liguanghe/liguanghe.github.io/node_modules/feedparser/node_modules/sax/lib/sax.js:615:33)
at end (/Users/liguanghe/liguanghe.github.io/node_modules/feedparser/node_modules/sax/lib/sax.js:654:3)
at Object.end (/Users/liguanghe/liguanghe.github.io/node_modules/feedparser/node_modules/sax/lib/sax.js:149:24)
at SAXStream.end (/Users/liguanghe/liguanghe.github.io/node_modules/feedparser/node_modules/sax/lib/sax.js:234:16)
at FeedParser._flush (/Users/liguanghe/liguanghe.github.io/node_modules/feedparser/main.js:1010:15)
at FeedParser. (_stream_transform.js:118:12)
at FeedParser.g (events.js:291:16)
at emitNone (events.js:86:13)
at FeedParser.emit (events.js:185:7)
at prefinish (_stream_writable.js:505:12)
at finishMaybe (_stream_writable.js:513:7)
at endWritable (_stream_writable.js:525:3)
at FeedParser.Writable.end (_stream_writable.js:490:5)
at Request.onend (internal/streams/legacy.js:44:10)
at emitOne (events.js:101:20)
at Request.emit (events.js:188:7)
at IncomingMessage. (/Users/liguanghe/liguanghe.github.io/node_modules/request/request.js:1091:12)
at IncomingMessage.g (events.js:291:16)
FATAL Not a feed
Error: Not a feed
at FeedParser.handleEnd (/Users/liguanghe/liguanghe.github.io/node_modules/feedparser/main.js:129:13)
at emitNone (events.js:86:13)
at SAXStream.emit (events.js:185:7)
at Object.SAXStream._parser.onend (/Users/liguanghe/liguanghe.github.io/node_modules/feedparser/node_modules/sax/lib/sax.js:184:8)
at emit (/Users/liguanghe/liguanghe.github.io/node_modules/feedparser/node_modules/sax/lib/sax.js:615:33)
at end (/Users/liguanghe/liguanghe.github.io/node_modules/feedparser/node_modules/sax/lib/sax.js:654:3)
at Object.end (/Users/liguanghe/liguanghe.github.io/node_modules/feedparser/node_modules/sax/lib/sax.js:149:24)
at SAXStream.end (/Users/liguanghe/liguanghe.github.io/node_modules/feedparser/node_modules/sax/lib/sax.js:234:16)
at FeedParser._flush (/Users/liguanghe/liguanghe.github.io/node_modules/feedparser/main.js:1010:15)
at FeedParser. (_stream_transform.js:118:12)
at FeedParser.g (events.js:291:16)
at emitNone (events.js:86:13)
at FeedParser.emit (events.js:185:7)
at prefinish (_stream_writable.js:505:12)
at finishMaybe (_stream_writable.js:513:7)
at endWritable (_stream_writable.js:525:3)
at FeedParser.Writable.end (_stream_writable.js:490:5)
at Request.onend (internal/streams/legacy.js:44:10)
at emitOne (events.js:101:20)
at Request.emit (events.js:188:7)
at IncomingMessage. (/Users/liguanghe/liguanghe.github.io/node_modules/request/request.js:1091:12)
at IncomingMessage.g (events.js:291:16)

What shall I do? Thanks.

YAMLException: can not read a block mapping entry; a multiline key may not be an implicit key at line 2, column 5:

seeing if I can import my blogger into hexo, and how it looks, here's what I'm running and it's public so ... hexo migrate rss https://feeds.feedburner.com/xenoterracide

INFO  Post found: pkgcore the victor
FATAL Something's wrong. Maybe you can find the solution here: http://hexo.io/docs/troubleshooting.html
YAMLException: can not read a block mapping entry; a multiline key may not be an implicit key at line 2, column 5:
    date: 2010-06-02 23:36:45
        ^
    at generateError (C:\Users\xeno\IdeaProjects\blog\node_modules\js-yaml\lib\js-yaml\loader.js:162:10)
    at throwError (C:\Users\xeno\IdeaProjects\blog\node_modules\js-yaml\lib\js-yaml\loader.js:168:9)
    at readBlockMapping (C:\Users\xeno\IdeaProjects\blog\node_modules\js-yaml\lib\js-yaml\loader.js:1045:9)
    at composeNode (C:\Users\xeno\IdeaProjects\blog\node_modules\js-yaml\lib\js-yaml\loader.js:1331:12)
    at readDocument (C:\Users\xeno\IdeaProjects\blog\node_modules\js-yaml\lib\js-yaml\loader.js:1493:3)
    at loadDocuments (C:\Users\xeno\IdeaProjects\blog\node_modules\js-yaml\lib\js-yaml\loader.js:1549:5)
    at Object.load (C:\Users\xeno\IdeaProjects\blog\node_modules\js-yaml\lib\js-yaml\loader.js:1566:19)
    at C:\Users\xeno\IdeaProjects\blog\node_modules\hexo\lib\hexo\post.js:122:18
    at tryCatcher (C:\Users\xeno\IdeaProjects\blog\node_modules\bluebird\js\release\util.js:16:23)
    at Promise._settlePromiseFromHandler (C:\Users\xeno\IdeaProjects\blog\node_modules\bluebird\js\release\promise.js:510:31)
    at Promise._settlePromise (C:\Users\xeno\IdeaProjects\blog\node_modules\bluebird\js\release\promise.js:567:18)
    at Promise._settlePromise0 (C:\Users\xeno\IdeaProjects\blog\node_modules\bluebird\js\release\promise.js:612:10)
    at Promise._settlePromises (C:\Users\xeno\IdeaProjects\blog\node_modules\bluebird\js\release\promise.js:691:18)
    at Async._drainQueue (C:\Users\xeno\IdeaProjects\blog\node_modules\bluebird\js\release\async.js:133:16)
    at Async._drainQueues (C:\Users\xeno\IdeaProjects\blog\node_modules\bluebird\js\release\async.js:143:10)
    at Immediate.Async.drainQueues (C:\Users\xeno\IdeaProjects\blog\node_modules\bluebird\js\release\async.js:17:14)
    at runCallback (timers.js:651:20)
    at tryOnImmediate (timers.js:624:5)
    at processImmediate [as _immediateCallback] (timers.js:596:5)
FATAL can not read a block mapping entry; a multiline key may not be an implicit key at line 2, column 5:
    date: 2010-06-02 23:36:45
        ^
YAMLException: can not read a block mapping entry; a multiline key may not be an implicit key at line 2, column 5:
    date: 2010-06-02 23:36:45
        ^
    at generateError (C:\Users\xeno\IdeaProjects\blog\node_modules\js-yaml\lib\js-yaml\loader.js:162:10)
    at throwError (C:\Users\xeno\IdeaProjects\blog\node_modules\js-yaml\lib\js-yaml\loader.js:168:9)
    at readBlockMapping (C:\Users\xeno\IdeaProjects\blog\node_modules\js-yaml\lib\js-yaml\loader.js:1045:9)
    at composeNode (C:\Users\xeno\IdeaProjects\blog\node_modules\js-yaml\lib\js-yaml\loader.js:1331:12)
    at readDocument (C:\Users\xeno\IdeaProjects\blog\node_modules\js-yaml\lib\js-yaml\loader.js:1493:3)
    at loadDocuments (C:\Users\xeno\IdeaProjects\blog\node_modules\js-yaml\lib\js-yaml\loader.js:1549:5)
    at Object.load (C:\Users\xeno\IdeaProjects\blog\node_modules\js-yaml\lib\js-yaml\loader.js:1566:19)
    at C:\Users\xeno\IdeaProjects\blog\node_modules\hexo\lib\hexo\post.js:122:18
    at tryCatcher (C:\Users\xeno\IdeaProjects\blog\node_modules\bluebird\js\release\util.js:16:23)
    at Promise._settlePromiseFromHandler (C:\Users\xeno\IdeaProjects\blog\node_modules\bluebird\js\release\promise.js:510:31)
    at Promise._settlePromise (C:\Users\xeno\IdeaProjects\blog\node_modules\bluebird\js\release\promise.js:567:18)
    at Promise._settlePromise0 (C:\Users\xeno\IdeaProjects\blog\node_modules\bluebird\js\release\promise.js:612:10)
    at Promise._settlePromises (C:\Users\xeno\IdeaProjects\blog\node_modules\bluebird\js\release\promise.js:691:18)
    at Async._drainQueue (C:\Users\xeno\IdeaProjects\blog\node_modules\bluebird\js\release\async.js:133:16)
    at Async._drainQueues (C:\Users\xeno\IdeaProjects\blog\node_modules\bluebird\js\release\async.js:143:10)
    at Immediate.Async.drainQueues (C:\Users\xeno\IdeaProjects\blog\node_modules\bluebird\js\release\async.js:17:14)
    at runCallback (timers.js:651:20)
    at tryOnImmediate (timers.js:624:5)
    at processImmediate [as _immediateCallback] (timers.js:596:5)

Native async/await

• Native async/await
  • Remove callback
  • Utilize request-promise-native
  • Replace stream with string
    • Replace fs.createReadStream() with fs.readFile() of hexo-fs
    • Replace feedparser with feed-furious to support string input, instead of stream.
      • Port feed-furious to support async-ed camaro v4.
• IDN support #32

Include original url as alias

It would be helpful if the original URL was included in the alias field, to support the hexo-generator-alias plugin.