tk120404 / node-rssparser Goto Github PK

View Code? Open in Web Editor NEW

63.0 6.0 20.0 48 KB

🚀RSS/ATOM feed parser that returns the requested feed urls in a simple json object

Home Page: http://www.laymansite.com

JavaScript 100.00%

atom-feed javascript node-rssparser

node-rssparser's Introduction

node-rssparser

rssparser is a RSS/ATOM feed parser that returns the requested feed urls in a simple json object

Installing

Like all node.js modules, just use npm!

npm install rssparser

Usage

Using rss parser is easy, just call:

var parser = require('rssparser');
var options = {};
//rss feeds
parser.parseURL('http://laymansite.com/rss', options, function(err, out){
	console.log(out);
});

Output

The point of rssparser is to try and hide the format of the originally requested feed. Thus RSS and ATOM feeds are returned in a common format. Similar fields (pubDate vs update) will be mapped to the same field in the output.

The 'minimal' output format is:

{
	type:"rss" or "atom"
	title: Title of the feed
	description: description or subtitle
	url: url of the feed
	last_modified: pubDate or update time of the feed
	items:[
		{
			title: Title of article
			summary	: Summary or content of article
			url	: Url of the article
			categories : Categories of the article
			published_at: published date in relative
			time_ago: time in words
			author:	name of the author	
			guid		
			{
				link : link of the article
				isPermaLink	: isPermaLink true or false
			}	
		}...
	]

Tests

Tests for rssparser can be run using the command:

npm test

Make sure that you machine has an internet connection before running the tests.

Option reference

Here is the reference for other options that can be used along with the default options

Change log

0.0.11 package versions updated
0.0.10 Minor changes
0.0.9 Minor changes
0.0.8 Handled encoded request
0.0.7 More check for the attributes like title, summary in atom feed
0.0.6 Minor changes
0.0.5 More check for the attributes like title,summary,link and categories in RSS feed
0.0.4 minor changes
0.0.3 thumbnails added
0.0.2 atom parser changes
0.0.1 initial

License

http://wtfpl.org/

node-rssparser's People

Contributors

Stargazers

Watchers

Forkers

nyoraimi madnite1 markbarrington suvene kasuganosora aidvu sdepold slimbn xverges heinzeya nathanpower indisc polarity amlovey caleuanhopkins disciplezero freshy969

node-rssparser's Issues

TypeError

Getting a type error from a few feeds:

feed

http://www.economist.com/feeds/print-sections/71/united-states.xml

obj.summary = val.comments[0] != null ? val.comments[0] : '';
TypeError: Cannot read property '0' of undefined

Build issue

Hello, I was trying out this repo and i cant get it to build correctly.
bunch of error regarding node-gyp.

0 info it worked if it ends with ok 1 verbose cli [ 'C:\\Program Files\\nodejs\\node.exe', 1 verbose cli 'C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js', 1 verbose cli 'install' ] 2 info using [email protected] 3 info using [email protected] 4 verbose npm-session e52d86570aa0326f 5 silly install runPreinstallTopLevelLifecycles 6 silly preinstall [email protected] 7 info lifecycle [email protected]~preinstall: [email protected] 8 silly install loadCurrentTree 9 silly install readLocalPackageData 10 warn read-shrinkwrap This version of npm is compatible with lockfileVersion@1, but npm-shrinkwrap.json was generated for lockfileVersion@0. I'll try to do my best with it! 11 silly install loadIdealTree 12 silly install cloneCurrentTreeToIdealTree 13 silly install loadShrinkwrap 14 http fetch GET 304 https://registry.npmjs.org/iconv 165ms (from cache) 15 silly pacote version manifest for [email protected] fetched in 190ms 16 http fetch GET 304 https://registry.npmjs.org/nan 37ms (from cache) 17 silly pacote version manifest for [email protected] fetched in 39ms 18 silly install loadAllDepsIntoIdealTree 19 http fetch GET 304 https://registry.npmjs.org/vows 27ms (from cache) 20 silly pacote range manifest for vows@* fetched in 28ms 21 http fetch GET 304 https://registry.npmjs.org/glob 28ms (from cache) 22 silly pacote range manifest for glob@~4.0.6 fetched in 30ms 23 http fetch GET 304 https://registry.npmjs.org/graceful-fs 23ms (from cache) 24 silly pacote range manifest for graceful-fs@^3.0.2 fetched in 24ms 25 warn deprecated [email protected]: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js 26 http fetch GET 304 https://registry.npmjs.org/diff 65ms (from cache) 27 http fetch GET 304 https://registry.npmjs.org/eyes 66ms (from cache) 28 silly pacote range manifest for diff@~1.0.8 fetched in 66ms 29 silly pacote range manifest for eyes@~0.1.6 fetched in 66ms 30 http fetch GET 304 https://registry.npmjs.org/natives 31ms (from cache) 31 silly pacote range manifest for natives@^1.1.0 fetched in 33ms 32 http fetch GET 304 https://registry.npmjs.org/minimatch 62ms (from cache) 33 http fetch GET 304 https://registry.npmjs.org/inherits 63ms (from cache) 34 silly pacote range manifest for minimatch@^1.0.0 fetched in 63ms 35 warn deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue 36 silly pacote range manifest for inherits@2 fetched in 65ms 37 http fetch GET 304 https://registry.npmjs.org/once 65ms (from cache) 38 silly pacote range manifest for once@^1.3.0 fetched in 66ms 39 http fetch GET 304 https://registry.npmjs.org/sigmund 26ms (from cache) 40 silly pacote range manifest for sigmund@~1.0.0 fetched in 27ms 41 http fetch GET 304 https://registry.npmjs.org/lru-cache 28ms (from cache) 42 silly pacote range manifest for lru-cache@2 fetched in 30ms 43 http fetch GET 304 https://registry.npmjs.org/wrappy 29ms (from cache) 44 silly pacote range manifest for wrappy@1 fetched in 30ms 45 silly pacote range manifest for vows@* fetched in 0ms 46 silly resolveWithNewModule [email protected] checking installable status 47 silly pacote range manifest for eyes@~0.1.6 fetched in 2ms 48 silly resolveWithNewModule [email protected] checking installable status 49 silly pacote range manifest for diff@~1.0.8 fetched in 2ms 50 silly resolveWithNewModule [email protected] checking installable status 51 silly pacote range manifest for glob@~4.0.6 fetched in 2ms 52 silly resolveWithNewModule [email protected] checking installable status 53 silly pacote range manifest for graceful-fs@^3.0.2 fetched in 2ms 54 silly resolveWithNewModule [email protected] checking installable status 55 silly pacote range manifest for inherits@2 fetched in 2ms 56 silly resolveWithNewModule [email protected] checking installable status 57 silly pacote range manifest for minimatch@^1.0.0 fetched in 3ms 58 silly resolveWithNewModule [email protected] checking installable status 59 silly pacote range manifest for once@^1.3.0 fetched in 3ms 60 silly resolveWithNewModule [email protected] checking installable status 61 silly pacote range manifest for natives@^1.1.0 fetched in 0ms 62 silly resolveWithNewModule [email protected] checking installable status 63 silly pacote range manifest for lru-cache@2 fetched in 1ms 64 silly resolveWithNewModule [email protected] checking installable status 65 silly pacote range manifest for sigmund@~1.0.0 fetched in 1ms 66 silly resolveWithNewModule [email protected] checking installable status 67 silly pacote range manifest for wrappy@1 fetched in 0ms 68 silly resolveWithNewModule [email protected] checking installable status 69 silly currentTree [email protected] 69 silly currentTree +-- [email protected] 69 silly currentTree +-- [email protected] 69 silly currentTree +-- [email protected] 69 silly currentTree +-- [email protected] 69 silly currentTree +-- [email protected] 69 silly currentTree +-- [email protected] 69 silly currentTree +-- [email protected] 69 silly currentTree +-- [email protected] 69 silly currentTree +-- [email protected] 69 silly currentTree +-- [email protected] 69 silly currentTree | +-- [email protected] 69 silly currentTree | +-- [email protected] 69 silly currentTree | | -- [email protected]
69 silly currentTree | | +-- [email protected]
69 silly currentTree | | +-- [email protected]
69 silly currentTree | | +-- [email protected]
69 silly currentTree | | +-- [email protected]
69 silly currentTree | | +-- [email protected]
69 silly currentTree | | -- [email protected] 69 silly currentTree | +-- [email protected] 69 silly currentTree | +-- [email protected] 69 silly currentTree | | -- [email protected]
69 silly currentTree | +-- [email protected]
69 silly currentTree | +-- [email protected]
69 silly currentTree | +-- [email protected]
69 silly currentTree | | -- [email protected] 69 silly currentTree | +-- [email protected] 69 silly currentTree | | +-- [email protected] 69 silly currentTree | | | +-- [email protected] 69 silly currentTree | | | +-- [email protected] 69 silly currentTree | | | +-- [email protected] 69 silly currentTree | | | | -- [email protected]
69 silly currentTree | | | +-- [email protected]
69 silly currentTree | | | | -- [email protected] 69 silly currentTree | | | -- [email protected]
69 silly currentTree | | +-- [email protected]
69 silly currentTree | | | -- [email protected] 69 silly currentTree | | +-- [email protected] 69 silly currentTree | | | +-- [email protected] 69 silly currentTree | | | +-- [email protected] 69 silly currentTree | | | | -- [email protected]
69 silly currentTree | | | +-- [email protected]
69 silly currentTree | | | -- [email protected] 69 silly currentTree | | -- [email protected]
69 silly currentTree | | -- [email protected] 69 silly currentTree | +-- [email protected] 69 silly currentTree | | +-- [email protected] 69 silly currentTree | | +-- [email protected] 69 silly currentTree | | +-- [email protected] 69 silly currentTree | | -- [email protected]
69 silly currentTree | +-- [email protected]
69 silly currentTree | | +-- [email protected]
69 silly currentTree | | +-- [email protected]
69 silly currentTree | | | +-- [email protected]
69 silly currentTree | | | +-- [email protected]
69 silly currentTree | | | -- [email protected] 69 silly currentTree | | -- [email protected]
69 silly currentTree | | +-- [email protected]
69 silly currentTree | | +-- [email protected]
69 silly currentTree | | +-- [email protected]
69 silly currentTree | | +-- [email protected]
69 silly currentTree | | +-- [email protected]
69 silly currentTree | | +-- [email protected]
69 silly currentTree | | -- [email protected] 69 silly currentTree | +-- [email protected] 69 silly currentTree | +-- [email protected] 69 silly currentTree | +-- [email protected] 69 silly currentTree | +-- [email protected] 69 silly currentTree | | -- [email protected]
69 silly currentTree | +-- [email protected]
69 silly currentTree | +-- [email protected]
69 silly currentTree | +-- [email protected]
69 silly currentTree | +-- [email protected]
69 silly currentTree | +-- [email protected]
69 silly currentTree | -- [email protected] 69 silly currentTree +-- [email protected] 69 silly currentTree +-- [email protected] 69 silly currentTree +-- [email protected] 69 silly currentTree +-- [email protected] 69 silly currentTree -- [email protected]
69 silly currentTree +-- [email protected]
69 silly currentTree -- [email protected] 69 silly currentTree -- [email protected]
70 silly idealTree [email protected]
70 silly idealTree +-- [email protected]
70 silly idealTree +-- [email protected]
70 silly idealTree +-- [email protected]
70 silly idealTree +-- [email protected]
70 silly idealTree +-- [email protected]
70 silly idealTree | -- [email protected] 70 silly idealTree +-- [email protected] 70 silly idealTree +-- [email protected] 70 silly idealTree +-- [email protected] 70 silly idealTree +-- [email protected] 70 silly idealTree +-- [email protected] 70 silly idealTree +-- [email protected] 70 silly idealTree | +-- [email protected] 70 silly idealTree | +-- [email protected] 70 silly idealTree | | -- [email protected]
70 silly idealTree | | +-- [email protected]
70 silly idealTree | | +-- [email protected]
70 silly idealTree | | +-- [email protected]
70 silly idealTree | | +-- [email protected]
70 silly idealTree | | +-- [email protected]
70 silly idealTree | | -- [email protected] 70 silly idealTree | +-- [email protected] 70 silly idealTree | +-- [email protected] 70 silly idealTree | | -- [email protected]
70 silly idealTree | +-- [email protected]
70 silly idealTree | +-- [email protected]
70 silly idealTree | +-- [email protected]
70 silly idealTree | | -- [email protected] 70 silly idealTree | +-- [email protected] 70 silly idealTree | | +-- [email protected] 70 silly idealTree | | | +-- [email protected] 70 silly idealTree | | | +-- [email protected] 70 silly idealTree | | | +-- [email protected] 70 silly idealTree | | | | -- [email protected]
70 silly idealTree | | | +-- [email protected]
70 silly idealTree | | | | -- [email protected] 70 silly idealTree | | | -- [email protected]
70 silly idealTree | | +-- [email protected]
70 silly idealTree | | | -- [email protected] 70 silly idealTree | | +-- [email protected] 70 silly idealTree | | | +-- [email protected] 70 silly idealTree | | | +-- [email protected] 70 silly idealTree | | | | -- [email protected]
70 silly idealTree | | | +-- [email protected]
70 silly idealTree | | | -- [email protected] 70 silly idealTree | | -- [email protected]
70 silly idealTree | | -- [email protected] 70 silly idealTree | +-- [email protected] 70 silly idealTree | | +-- [email protected] 70 silly idealTree | | +-- [email protected] 70 silly idealTree | | +-- [email protected] 70 silly idealTree | | -- [email protected]
70 silly idealTree | +-- [email protected]
70 silly idealTree | | +-- [email protected]
70 silly idealTree | | +-- [email protected]
70 silly idealTree | | | +-- [email protected]
70 silly idealTree | | | +-- [email protected]
70 silly idealTree | | | -- [email protected] 70 silly idealTree | | -- [email protected]
70 silly idealTree | | +-- [email protected]
70 silly idealTree | | +-- [email protected]
70 silly idealTree | | +-- [email protected]
70 silly idealTree | | +-- [email protected]
70 silly idealTree | | +-- [email protected]
70 silly idealTree | | +-- [email protected]
70 silly idealTree | | -- [email protected] 70 silly idealTree | +-- [email protected] 70 silly idealTree | +-- [email protected] 70 silly idealTree | +-- [email protected] 70 silly idealTree | +-- [email protected] 70 silly idealTree | | -- [email protected]
70 silly idealTree | +-- [email protected]
70 silly idealTree | +-- [email protected]
70 silly idealTree | +-- [email protected]
70 silly idealTree | +-- [email protected]
70 silly idealTree | +-- [email protected]
70 silly idealTree | -- [email protected] 70 silly idealTree +-- [email protected] 70 silly idealTree +-- [email protected] 70 silly idealTree +-- [email protected] 70 silly idealTree +-- [email protected] 70 silly idealTree -- [email protected]
70 silly idealTree +-- [email protected]
70 silly idealTree -- [email protected] 70 silly idealTree -- [email protected]
71 silly install generateActionsToTake
72 silly diffTrees action count 2
73 silly diffTrees add [email protected]
74 silly diffTrees add [email protected]
75 silly decomposeActions action count 16
76 silly decomposeActions fetch [email protected]
77 silly decomposeActions extract [email protected]
78 silly decomposeActions preinstall [email protected]
79 silly decomposeActions build [email protected]
80 silly decomposeActions install [email protected]
81 silly decomposeActions postinstall [email protected]
82 silly decomposeActions finalize [email protected]
83 silly decomposeActions refresh-package-json [email protected]
84 silly decomposeActions fetch [email protected]
85 silly decomposeActions extract [email protected]
86 silly decomposeActions preinstall [email protected]
87 silly decomposeActions build [email protected]
88 silly decomposeActions install [email protected]
89 silly decomposeActions postinstall [email protected]
90 silly decomposeActions finalize [email protected]
91 silly decomposeActions refresh-package-json [email protected]
92 silly install executeActions
93 silly doSerial global-install 16
94 verbose correctMkdir C:\Users\boobs\AppData\Roaming\npm-cache_locks correctMkdir not in flight; initializing
95 verbose lock using C:\Users\boobs\AppData\Roaming\npm-cache_locks\staging-2dacb3dfe951e23b.lock for D:\workspace\node-rssparser\node_modules.staging
96 silly doParallel extract 16
97 silly extract [email protected]
98 silly pacote trying nan@https://registry.npmjs.org/nan/-/nan-2.0.9.tgz by hash: sha1-0Cp3D0Z3iELM65ThfKsx/8cjSgU=
99 silly extract [email protected]
100 silly pacote trying iconv@https://registry.npmjs.org/iconv/-/iconv-2.1.11.tgz by hash: sha1-X12pOmQ1BvXOqovZx41hdPi5NRo=
101 silly pacote nan@https://registry.npmjs.org/nan/-/nan-2.0.9.tgz extracted to D:\workspace\node-rssparser\node_modules.staging\nan-3ac7c7ed by content address 27ms
102 silly pacote iconv@https://registry.npmjs.org/iconv/-/iconv-2.1.11.tgz extracted to D:\workspace\node-rssparser\node_modules.staging\iconv-5e18aa31 by content address 113ms
103 silly doReverseSerial unbuild 16
104 silly doSerial remove 16
105 silly doSerial move 16
106 silly doSerial finalize 16
107 silly finalize D:\workspace\node-rssparser\node_modules\iconv\node_modules\nan
108 silly finalize D:\workspace\node-rssparser\node_modules\iconv
109 silly doParallel refresh-package-json 16
110 silly refresh-package-json D:\workspace\node-rssparser\node_modules\iconv\node_modules\nan
111 silly refresh-package-json D:\workspace\node-rssparser\node_modules\iconv
112 silly doParallel preinstall 16
113 silly preinstall [email protected]
114 info lifecycle [email protected]preinstall: [email protected]
115 silly preinstall [email protected]
116 info lifecycle [email protected]preinstall: [email protected]
117 silly doSerial build 16
118 silly build [email protected]
119 info linkStuff [email protected]
120 silly linkStuff [email protected] has D:\workspace\node-rssparser\node_modules\iconv\node_modules as its parent node_modules
121 verbose linkBins [email protected]
122 verbose linkMans [email protected]
123 silly build [email protected]
124 info linkStuff [email protected]
125 silly linkStuff [email protected] has D:\workspace\node-rssparser\node_modules as its parent node_modules
126 verbose linkBins [email protected]
127 verbose linkMans [email protected]
128 silly doSerial global-link 16
129 silly doParallel update-linked 16
130 silly doSerial install 16
131 silly install [email protected]
132 info lifecycle [email protected]install: [email protected]
133 silly install [email protected]
134 info lifecycle [email protected]install: [email protected]
135 verbose lifecycle [email protected]install: unsafe-perm in lifecycle true
136 verbose lifecycle [email protected]install: PATH: C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin;D:\workspace\node-rssparser\node_modules\iconv\node_modules.bin;D:\workspace\node-rssparser\node_modules.bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0;C:\Program Files\Git\cmd;C:\Program Files\nodejs;C:\Program Files (x86)\Windows Kits\8.1\Windows Performance Toolkit;C:\Users\boobs\AppData\Local\Microsoft\WindowsApps;C:\Program Files\Microsoft VS Code\bin;C:\Users\boobs\AppData\Roaming\npm
137 verbose lifecycle [email protected]install: CWD: D:\workspace\node-rssparser\node_modules\iconv
138 silly lifecycle [email protected]install: Args: [ '/d /s /c', 'node-gyp rebuild' ]
139 silly lifecycle [email protected]install: Returned: code: 1 signal: null
140 info lifecycle [email protected]install: Failed to exec install script
141 verbose unlock done using C:\Users\boobs\AppData\Roaming\npm-cache_locks\staging-2dacb3dfe951e23b.lock for D:\workspace\node-rssparser\node_modules.staging
142 silly saveTree [email protected]
142 silly saveTree +-- [email protected]
142 silly saveTree | -- [email protected] 142 silly saveTree +-- [email protected] 142 silly saveTree | +-- [email protected] 142 silly saveTree | +-- [email protected] 142 silly saveTree | | -- [email protected]
142 silly saveTree | | +-- [email protected]
142 silly saveTree | | +-- [email protected]
142 silly saveTree | | +-- [email protected]
142 silly saveTree | | +-- [email protected]
142 silly saveTree | | +-- [email protected]
142 silly saveTree | | -- [email protected] 142 silly saveTree | +-- [email protected] 142 silly saveTree | +-- [email protected] 142 silly saveTree | | -- [email protected]
142 silly saveTree | +-- [email protected]
142 silly saveTree | +-- [email protected]
142 silly saveTree | +-- [email protected]
142 silly saveTree | | +-- [email protected]
142 silly saveTree | | -- [email protected] 142 silly saveTree | | -- [email protected]
142 silly saveTree | +-- [email protected]
142 silly saveTree | | +-- [email protected]
142 silly saveTree | | | +-- [email protected]
142 silly saveTree | | | +-- [email protected]
142 silly saveTree | | | +-- [email protected]
142 silly saveTree | | | | -- [email protected] 142 silly saveTree | | | +-- [email protected] 142 silly saveTree | | | | -- [email protected]
142 silly saveTree | | | -- [email protected] 142 silly saveTree | | +-- [email protected] 142 silly saveTree | | | -- [email protected]
142 silly saveTree | | +-- [email protected]
142 silly saveTree | | | +-- [email protected]
142 silly saveTree | | | +-- [email protected]
142 silly saveTree | | | | -- [email protected] 142 silly saveTree | | | +-- [email protected] 142 silly saveTree | | | -- [email protected]
142 silly saveTree | | -- [email protected] 142 silly saveTree | | -- [email protected]
142 silly saveTree | +-- [email protected]
142 silly saveTree | | +-- [email protected]
142 silly saveTree | | | -- [email protected] 142 silly saveTree | | +-- [email protected] 142 silly saveTree | | +-- [email protected] 142 silly saveTree | | -- [email protected]
142 silly saveTree | +-- [email protected]
142 silly saveTree | | +-- [email protected]
142 silly saveTree | | +-- [email protected]
142 silly saveTree | | | +-- [email protected]
142 silly saveTree | | | +-- [email protected]
142 silly saveTree | | | -- [email protected] 142 silly saveTree | | -- [email protected]
142 silly saveTree | | +-- [email protected]
142 silly saveTree | | +-- [email protected]
142 silly saveTree | | +-- [email protected]
142 silly saveTree | | +-- [email protected]
142 silly saveTree | | | -- [email protected] 142 silly saveTree | | +-- [email protected] 142 silly saveTree | | +-- [email protected] 142 silly saveTree | | -- [email protected]
142 silly saveTree | +-- [email protected]
142 silly saveTree | +-- [email protected]
142 silly saveTree | +-- [email protected]
142 silly saveTree | +-- [email protected]
142 silly saveTree | +-- [email protected]
142 silly saveTree | +-- [email protected]
142 silly saveTree | +-- [email protected]
142 silly saveTree | +-- [email protected]
142 silly saveTree | +-- [email protected]
142 silly saveTree | -- [email protected] 142 silly saveTree +-- [email protected] 142 silly saveTree +-- [email protected] 142 silly saveTree | +-- [email protected] 142 silly saveTree | +-- [email protected] 142 silly saveTree | -- [email protected]
142 silly saveTree | +-- [email protected]
142 silly saveTree | | -- [email protected] 142 silly saveTree | +-- [email protected] 142 silly saveTree | +-- [email protected] 142 silly saveTree | | +-- [email protected] 142 silly saveTree | | -- [email protected]
142 silly saveTree | -- [email protected] 142 silly saveTree | -- [email protected]
142 silly saveTree -- [email protected] 142 silly saveTree +-- [email protected] 142 silly saveTree -- [email protected]
142 silly saveTree -- [email protected] 143 warn [email protected] No license field. 144 verbose stack Error: [email protected] install: node-gyp rebuild144 verbose stack Exit status 1 144 verbose stack at EventEmitter.<anonymous> (C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\index.js:285:16) 144 verbose stack at emitTwo (events.js:126:13) 144 verbose stack at EventEmitter.emit (events.js:214:7) 144 verbose stack at ChildProcess.<anonymous> (C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\lib\spawn.js:55:14) 144 verbose stack at emitTwo (events.js:126:13) 144 verbose stack at ChildProcess.emit (events.js:214:7) 144 verbose stack at maybeClose (internal/child_process.js:925:16) 144 verbose stack at Process.ChildProcess._handle.onexit (internal/child_process.js:209:5) 145 verbose pkgid [email protected] 146 verbose cwd D:\workspace\node-rssparser 147 verbose Windows_NT 10.0.16299 148 verbose argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install" 149 verbose node v8.9.4 150 verbose npm v5.6.0 151 error code ELIFECYCLE 152 error errno 1 153 error [email protected] install:node-gyp rebuild153 error Exit status 1 154 error Failed at the [email protected] install script. 154 error This is probably not a problem with npm. There is likely additional logging output above. 155 verbose exit [ 1, true ]

How we pass Max-limit

Hey could you please tell me how we pass the Max-limit in the url.

example : I want to fetch only 5 records from the URl(http://laymansite.com/feed)

Thanks in Advance.

License

Hi, I need to write something similar for other platform - before looking at your code I'd like to know the license of it. Thanks

parseString split can cost performance hit?

parseString function splits the xmlString on < to determine open-nodes.
Doesn't it lead to performance issue.

function parseString(xml, options, callback) {
    //...
    if (xml.split('<').length >= 3) {

How about taking xml.substring(0, 255) and expecting 3 open nodes to possibly fall into it?

Doesn't work with node 4.x

Hi,

it looks like the lib doesn't work with recent nodejs versions :

Error: Cannot find module '../build/Debug/iconv.node'
    at Function.Module._resolveFilename (module.js:337:15)
    at Function.Module._load (module.js:287:25)
    at Module.require (module.js:366:17)
    at require (module.js:385:17)
    at Object. (/Users/fallanic/work/testproject/node_modules/rssparser/node_modules/iconv/lib/iconv.js:27:14)
    at Module._compile (module.js:435:26)
    at Object.Module._extensions..js (module.js:442:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:311:12)
    at Module.require (module.js:366:17)
    at require (module.js:385:17)

I tried deleting the package, and running npm install again, same problem.

Thanks in advance!

Unnecessary log output

Hi, thanks for this module, I really enjoy using it. Only thing that bothers me is a log output when reading atom feeds in file feed.js - line 180:

if (typeof channel.title === "object") {
    console.log(channel.title);
}

It would be very nice if you could change that and update your npm version :)

Thanks,

Paul

Duplicate callback invocation on error.

parser.addListener: end and error both trigger callback if there is trouble with xml.

parser.addListener('end', function(jsonDOM) {
    if (jsonDOM) {
        //...
        if (callback) callback(null, output);
    } else {
        if (callback) callback("failed to parse xml", null);
    }
}

parser.addListener("error", function(err) {
    // ...
    callback = null;
});

TypeError: First argument must be a string or Buffer

My usage:

parser.parseURL('https://www.pinterest.com/evilgeniusno1/penny-for-your-thoughts.rss/', {}, function(err, out)  {

Full error:

{
  "code": 500,
  "error": "Script generated an unhandled asynchronous exception.",
  "details": "TypeError: First argument must be a string or Buffer",
  "name": "TypeError",
  "message": "First argument must be a string or Buffer",
  "stack": "TypeError: First argument must be a string or Buffer\n    at ServerResponse.end (_http_outgoing.js:739:13)\n    at /data/io/2f4c8022-44f3-4489-be0f-f29a74411702/webtask.js:26:9\n    at Parser.<anonymous> (/data/_verquire/rssparser/0.0.11/node_modules/rssparser/lib/feed.js:80:17)\n    at emitOne (events.js:116:13)\n    at Parser.emit (events.js:211:7)\n    at SAXParser.onclosetag (/data/_verquire/rssparser/0.0.11/node_modules/rssparser/node_modules/xml2js/lib/parser.js:261:26)\n    at emit (/data/_verquire/rssparser/0.0.11/node_modules/rssparser/node_modules/sax/lib/sax.js:624:35)\n    at emitNode (/data/_verquire/rssparser/0.0.11/node_modules/rssparser/node_modules/sax/lib/sax.js:629:5)\n    at closeTag (/data/_verquire/rssparser/0.0.11/node_modules/rssparser/node_modules/sax/lib/sax.js:889:7)\n    at SAXParser.write (/data/_verquire/rssparser/0.0.11/node_modules/rssparser/node_modules/sax/lib/sax.js:1436:13)\n    at Parser.exports.Parser.Parser.parseString (/data/_verquire/rssparser/0.0.11/node_modules/rssparser/node_modules/xml2js/lib/parser.js:322:31)\n    at Parser.parseString (/data/_verquire/rssparser/0.0.11/node_modules/rssparser/node_modules/xml2js/lib/parser.js:5:59)\n    at parseString (/data/_verquire/rssparser/0.0.11/node_modules/rssparser/lib/feed.js:88:16)\n    at Request._callback (/data/_verquire/rssparser/0.0.11/node_modules/rssparser/lib/feed.js:47:25)\n    at Request.self.callback (/data/_verquire/rssparser/0.0.11/node_modules/rssparser/node_modules/request/request.js:186:22)\n    at emitTwo (events.js:126:13)\n    at Request.emit (events.js:214:7)\n    at Request.<anonymous> (/data/_verquire/rssparser/0.0.11/node_modules/rssparser/node_modules/request/request.js:1163:10)\n    at emitOne (events.js:116:13)\n    at Request.emit (events.js:211:7)\n    at IncomingMessage.<anonymous> (/data/_verquire/rssparser/0.0.11/node_modules/rssparser/node_modules/request/request.js:1085:12)\n    at Object.onceWrapper (events.js:313:30)"
}

Segmentation fault with [email protected]

The rssparser module (or perhaps one of its dependencies) is broken with [email protected]. I can consistently reproduce by running npm test in the rssparser directory. The tests pass fine with [email protected], but [email protected] returns:

jacob-> npm test

> [email protected] test /Users/jacob/workspace/one-trackers/node_modules/rssparser
> vows --spec


  ♢ bindparser 

  oddities empty xml
    ✓ returns an error && docs is null
Segmentation fault: 11

Attaching segfault-handler (note: ddopson/node-segfault-handler#9) yields the following:

PID 10241 received SIGSEGV for address: 0x8
0   segfault-handler-native.node        0x00000001008fbb78 _ZL16segfault_handleriP9__siginfoPv + 192
1   libsystem_platform.dylib            0x00007fff8af405aa _sigtramp + 26
2   node                                0x00000001002a1dee _ZN2v88internal28BufferedUtf16CharacterStream9ReadBlockEv + 62
3   node                                0x00000001001d1ae7 _ZN2v88internal17BoundsCheckBbData10CoverCheckEPNS0_12HBoundsCheckEi + 129
4   node                                0x00000001001b9611 _ZN2v88internal6HGraph30EliminateRedundantBoundsChecksEPNS0_11HBasicBlockEPNS0_16BoundsCheckTableE + 299
5   node                                0x00000001001b9779 _ZN2v88internal6HGraph30EliminateRedundantBoundsChecksEPNS0_11HBasicBlockEPNS0_16BoundsCheckTableE + 659
6   node                                0x00000001001b9779 _ZN2v88internal6HGraph30EliminateRedundantBoundsChecksEPNS0_11HBasicBlockEPNS0_16BoundsCheckTableE + 659
7   node                                0x00000001001b9779 _ZN2v88internal6HGraph30EliminateRedundantBoundsChecksEPNS0_11HBasicBlockEPNS0_16BoundsCheckTableE + 659
8   node                                0x00000001001b9779 _ZN2v88internal6HGraph30EliminateRedundantBoundsChecksEPNS0_11HBasicBlockEPNS0_16BoundsCheckTableE + 659
9   node                                0x00000001001b9779 _ZN2v88internal6HGraph30EliminateRedundantBoundsChecksEPNS0_11HBasicBlockEPNS0_16BoundsCheckTableE + 659
10  node                                0x00000001001b8fd8 _ZN2v88internal6HGraph30EliminateRedundantBoundsChecksEv + 128
11  node                                0x00000001001b8f0c _ZN2v88internal6HGraph8OptimizeEPNS0_17SmartArrayPointerIcEE + 514
12  node                                0x0000000100150852 _ZN2v88internal18OptimizingCompiler13OptimizeGraphEv + 42
13  node                                0x0000000100152745 _ZN2v88internalL12GenerateCodeEPNS0_15CompilationInfoE + 126
14  node                                0x0000000100150a56 _ZN2v88internalL8MakeCodeEPNS0_15CompilationInfoE + 38
15  node                                0x000000010015177a _ZN2v88internal8Compiler11CompileLazyEPNS0_15CompilationInfoE + 294
16  node                                0x0000000100231f14 _ZN2v88internal10JSFunction16CompileOptimizedENS0_6HandleIS1_EENS0_9BailoutIdENS0_18ClearExceptionFlagE + 60
17  node                                0x00000001002855c9 _ZN2v88internal36Runtime_CompileForOnStackReplacementENS0_9ArgumentsEPNS0_7IsolateE + 551
18  ???                                 0x00001f05de30618e 0x0 + 34110063010190

Fowlers atom feed cant be processed

http://martinfowler.com/feed.atom

The summary element is empty for example. The feed itself validates

Undefined value

I had trouble with this feed: http://www.spiegel.de/schlagzeilen/eilmeldungen/index.rss

So I had to change the block beginning in line 151 in feed.js to following:

       // Put the comments instead of the description if there is no description
        if (obj.summary === null || obj.summary === '') {
          if(val.comments[0]){
            obj.summary = val.comments[0] != null ? val.comments[0] : '';
          }
        }

This worked then...

How we pass Max-limit

Hey could you please tell me how we pass the Max-limit in the url.

example : I want to fetch only 5 records from the URl(http://laymansite.com/feed)

Thanks in Advance.

TypeError: Cannot read property '0' of undefined

I've found this error across multiple feeds. Here is one as an example:

http://hosted.ap.org/dynamic/fronts/RSS_FEEDS?SITE=AP

Stacktrace:

TypeError: Cannot read property 'title' of undefined
at formatATOM (/home/collector/node_modules/rssparser/lib/feed.js:193:16)
at Parser.<anonymous> (/home/collector/node_modules/rssparser/lib/feed.js:62:30)
at Parser.EventEmitter.emit (events.js:95:17)
at Object.saxParser.onclosetag (/home/collector/node_modules/rssparser/node_modules/xml2js/lib/xml2js.js:218:24)
at emit (/home/collector/node_modules/rssparser/node_modules/xml2js/node_modules/sax/lib/sax.js:325:33)
at emitNode (/home/collector/node_modules/rssparser/node_modules/xml2js/node_modules/sax/lib/sax.js:330:3)
at closeTag (/home/collector/node_modules/rssparser/node_modules/xml2js/node_modules/sax/lib/sax.js:547:5)
at Object.write (/home/collector/node_modules/rssparser/node_modules/xml2js/node_modules/sax/lib/sax.js:958:29)
at Parser.exports.Parser.Parser.parseString (/home/collector/node_modules/rssparser/node_modules/xml2js/lib/xml2js.js:257:29)
at Parser.parseString (/home/collector/node_modules/rssparser/node_modules/xml2js/lib/xml2js.js:6:61)
at parseString (/home/collector/node_modules/rssparser/lib/feed.js:72:16)

Potential fix:

val = flattenComments(val);
var obj = {};
if (val.title) {
  obj.title = val.title[0];
}
if (val.summary) {
  obj.summary = val.description[0];
}
if (val.link) {
  obj.url = val.link[0];
}

Missing check if response is empty

Hi,

In lib/feed.js on line 26 you access response.statusCode without checkig if response is valid. I didn't investigate where it comes from as it is hard to reproduce because I check multiple feeds every minute and it only happens from time to time but if the response is undefined a TypeError will be thrown. Additional error checking would be nice here :)

Fault tolerant parsing of xml.

Noticed few blog emitting http-headers begining of feed.
Tried the following to overcome malformed xml issue due to this.

 function parseString(xml, options, callback) {
    // tolerance toward mixed xml content at start
    var xmlStart = xml.indexOf('<?xml');
    if (xmlStart > 0) { xml = xml.substring(xmlStart); }
    ...
    // we need to check that the input in not a null input

dbl callbacks

v 0.0.7

rssparser.parserURL(url, {}, function(err, json){})

http://www.pbs.org/newshour/feed/

This feed triggers a double callback. First it returns with err. Second it returns with json. In other words, the the callback passed in above is called twice once with an err and then once with json.

Same thing here:

http://www.zdnet.com/news/rss.xml
http://www.jpost.com/Rss/RssFeedsFrontPage.aspx

Number of items parsed

Hi,

For my personal needs, I wanted to limit the number of rss / atom sent by the library.
I forked the lib and made my modif.

Is it interesting to make a pull request with it ?

Thanks !

Missing documentation for the options parameters

Where can I find the documentation about what can I use in the options parameter?

Module not found: Can't resolve '../build/Debug/iconv.node'

Getting an error when I initialize the parser.

Module not found: Can't resolve '../build/Debug/iconv.node'

Encoding issue with dependency "request"

Hi again,

request has some trouble with encoding.
This feed for example gets back with strange character: http://www.spiegel.de/schlagzeilen/index.rss

As this discussion shows: http://stackoverflow.com/questions/8332500/module-request-how-to-properly-retrieve-accented-characters
I integrated iconv-lite to rssparser and convert the fetched xml with that...

now function parseUrl looks like that:

function parseURL(feedURL, options, callback) {
if (typeof options === 'function' && !callback) {
    callback = options;
    options = {};
}

  var defaults = {uri: feedURL, jar: false, proxy: false, followRedirect: true, timeout: 1000 * 30, encoding:null};
  options = _.extend(defaults, options);
  //check that the protocal is either http or https
  var u = URL.parse(feedURL);
  if (u.protocol === 'http:' || u.protocol === 'https:') {
      //make sure to have a 30 second timeout
      var req = request(options, function (err, response, xml) {
          if (err || xml === null) {
              if (err) {
                  callback(err, null);
              } else {
                  callback('Failed to retrieve source!', null);
              }
          } else {
              if ((typeof response !== "undefined" && response !== null ? response.statusCode : void 0) != null) {
                if (response.statusCode >= 400) {
                  callback("Failed to retrieve source! Invalid response code (" + response.statusCode + ")!", null);
                } else {
                  var encodedXml = iconv.decode(new Buffer(xml), 'ISO-8859-1');
                  console.log(encodedXml);
                  parseString(encodedXml, options, callback);
                }
              } else {
                callback("Failed to retrieve source! No response code!!", null);
              }
          }
      });
  } else {
      callback({error: "Only http or https protocols are accepted"}, null);
  }
}
module.exports.parseURL = parseURL;

non-latin characters

Hi there,

it seems that there is no UTF support at the moment, Cyrillic alphabet cannot be interpreted correctly.
What's the best way to add UTF support?

Many thanks,
Max

TypeError

Feed

http://www.sacbee.com/830/index.rss

This is the error:

if (xml.split('<').length >= 3) {
TypeError: Cannot read property 'split' of undefined

How to save rss data to database (Like Mongoose)?

Hi guys the node-rssparser module looks awesome, but how can I select every individual parameter like title, link, etc. and save it to a database. I'm kinda rookie in node, and was expecting somekind of a getter method, wich would be awesome though!

Thanks in advance! ;-)

Push to npm

Hi, I noticed that your npm package is not up to par with UTF-8 formatting (issue #22). Please update the npm package by running npm publish.

How to convert an object into an XML rss feed?

If I want to go the other way around and convert an object into an XML feed can I do this with rssparser?

Check for valid channel in formatATOM.

Server crashed during a call to formatATOM with an specific (or invalid) atom feed. I have this issue every time I try to get feeds from http://fullcirclemagazine.org/feed/.

The line that cause the crash is 189:

if (channel.title){
...
}

Because channel is undefined. I will try a workaround with plain validation of channel.

TypeError thrown from internals

Our RSSParser threw a TypeError from the internals. I am currently using a try...catch block to catch it.

Here is the error:

TypeError: Cannot read property 'statusCode' of undefined
   at Request._callback (.../node_modules/rssparser/lib/feed.js:26:25)
   at self.callback (.../node_modules/rssparser/node_modules/request/index.js:142:22)
   at Request.EventEmitter.emit (events.js:95:17)
   at null._onTimeout (.../node_modules/rssparser/node_modules/request/index.js:589:12)
   at Timer.listOnTimeout [as ontimeout] (timers.js:110:15)
   at timers.js:117:18
   at process._tickCallback (node.js:415:13)

Using the parser in a for loop.

The parser has an asynchronous behavior. This is problematic as I'm trying to parse a list of links. And in that case I'm wondering how I could work around this, is there a way of making it work synchronously?

The actual problem is I want to use/compare/edit the output and it relies on external data, so or I have do do it synchronously or I need to have the ability to pass these variables into the parser as arguments.

Is this possible?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.