snazzah / duck-duck-scrape

🔎 Search from DuckDuckGo and utilize its spice APIs in Node

npm install duck-duck-scrape - yarn add duck-duck-scrape

Search from DuckDuckGo and utilize its spice APIs for things such as stocks, weather, currency conversion and more!

Available Features

• Search
  • Regular search
  • Image search
  • Video search
  • News search
• Stocks (via Xignite)
• Time for Location API (via timeanddate.com)
• Currency Conversion (via XE)
• Forecast (via Dark Sky)
• Dictionary
  • Definition
  • Audio
  • Pronunciation
  • Hyphenation

Quickstart

JavaScript

const DDG = require('duck-duck-scrape');
const searchResults = await DDG.search('node.js', {
  safeSearch: DDG.SafeSearchType.STRICT
});

// DDG.stocks('aapl')
// DDG.currency('usd', 'eur', 1)
// DDG.dictionaryDefinition('happy')

console.log(searchResults);
/**

{
  noResults: false,
  vqd: '3-314...',
  results: [
    {
      title: 'Node.js® is a JavaScript runtime built on Chrome's V8 JavaScript...',
      ...
      url: 'https://nodejs.org/',
      bang: {
        prefix: 'node',
        title: 'node.js docs',
        domain: 'nodejs.org'
      }
    },
    ...
  ]
}

*/

TypeScript

import { search, SafeSearchType } from 'duck-duck-scrape';
// import * as DDG from 'duck-duck-scrape';

const searchResults = await search('node.js', {
  safeSearch: SafeSearchType.STRICT
});

duck-duck-scrape's Issues

Failed to get the VQD for query "X"

Since Friday 10/02/2023 the library has stopped working, for any query submitted. Below you can find an example of the error stack.

Error: Failed to get the VQD for query "Hello world".
at getVQD (...\node_modules\duck-duck-scrape\lib\util.js:53:15)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async search (...\node_modules\duck-duck-scrape\lib\search\search.js:40:15)

Maybe the structure of the page is changed or the user-agent is blocked? Are you aware of this issue and willing to proceed with any updates?
Thanks!

main entry is not valid

when i try to launch any code with this module it says Error: Cannot find module '/home/emmanuel_macron/Documents/programinc/gougoule.ch/server/node_modules/duck-duck-scrape/lib/index.js'. Please verify that the package.json has a valid "main" entry

step to reproduce the bug:

1. install the module
2. try using the exmple code from the readme.md in javascript nodejs
   (sorry for bad english )

search returns only 30 results

And results are 50% different from what the browser shows. How to paginate?

const ddg = new DuckDuckScrape();

const searchPromise = ddg.search("put my keywords here", 100);

searchPromise.then(function(data) {
    console.log(data.map(item => item.url));
    console.log(data.length);
})```

safeSearch not working

safeSearch is not respected by the scraper, it is returning same results for all cases

image search is not working

USAGE

Please provide a tutorial on how to use it.

Pagination?

Hi! First of all, great project!

Second, how do I paginate? The package returns just 30 results, how do I request more? Thanks!

Take over project

I would like to take over the project

API not working (anymore)

Following are NOT working (anymore)

export * from './spices/emojipedia.ts';
export * from './spices/forecast.ts';
export * from './spices/statista.ts';

Following NOT TESTED

export * from './spices/expandUrl.ts';

Following are WORKING

export * from './search/images.ts';
export * from './search/news.ts';
export * from './search/search.ts';
export * from './search/videos.ts';
export * from './spices/currency.ts';
export * from './spices/dictionary/audio.ts';
export * from './spices/dictionary/definition.ts';
export * from './spices/dictionary/hyphenation.ts';
export * from './spices/dictionary/pronunciation.ts';
export * from './spices/dns.ts';
export * from './spices/statista.ts';
export * from './spices/stocks.ts';
export * from './spices/thesaurus.ts';
export * from './spices/time.ts';

Search fails when no rawDescription is available

When using https://www.npmjs.com/package/duck-duck-scrape the search will fail if there is no rawDescription in the search result. I noticed this line is different on this repo. Can you please update the node npm?

Line 115:
rawDescription: $('.result__snippet', result).html().replace(/&amp;/g, '&').replace(/&gt;/g, '>').replace(/&lt;/g, '<').replace(/&apos;/g, "'").replace(/&quot;/g, '"'),

W20181024-07:45:43.650(-4)? (STDERR)     at Array.from.map.result (/Users/../duck-duck-scrape/index.js:115:65)
W20181024-07:45:43.650(-4)? (STDERR)     at Array.map (<anonymous>)
W20181024-07:45:43.650(-4)? (STDERR)     at DDGScraper.search (/Users/../duck-duck-scrape/index.js:112:91)
W20181024-07:45:43.650(-4)? (STDERR)     at <anonymous>
W20181024-07:45:43.650(-4)? (STDERR)     at process._tickDomainCallback (internal/process/next_tick.js:228:7)```

Uncaught Error Error: Failed to get the VQD for query "node.js".

The response.body of getVQD() method is empty, resulting in an exception thrown by exports.VQD_REGEX.exec(response.body)[1]: Uncaught TypeError TypeError: Cannot read properties of null (reading '1').
How can this issue be resolved?

Take over

I'll take this over, I'm working on a project that requires this and I'm sure many people would like to still have this.

Search doesn't respect safesearch setting

search() doesn't respect the safeSearch setting. This seems to be because of safeSearch = -2, because if I remove the = -2 then it works.

Also, is there a reason icon was removed? (it still seems to be on the npm version though)