fongsidev / scraper

@fongsidev/scraper is a library that allows you to download various types of content from various online platforms such as TikTok, Instagram, Capcut, Google Drive, and many more and also provides very useful tools/codes :).

🌐 Socials:

💻 Tech Stack:

📊 GitHub Stats:

🏆 GitHub Trophies

✍️ Random Dev Quote

🔝 Top Contributed Repo

---

💰 You can help me by Donating

scraper's Issues

It is unable to be installed on Node version > 20

Error Log:

PS C:\Users\Umair Jameel\Desktop\python\test> npm i @fongsidev/scraper
npm WARN deprecated [email protected]: request-promise-native has been deprecated because it extends the now deprecated request package, see request/request#3142
npm WARN deprecated [email protected]: The zlib module provides APIs for brotli compression/decompression starting with Node.js v10.16.0, please use it over iltorb
npm ERR! code 1
npm ERR! path C:\Users\Umair Jameel\Desktop\python\test\node_modules\iltorb
npm ERR! command failed
npm ERR! command C:\Windows\system32\cmd.exe /d /s /c node ./scripts/install.js || node-gyp rebuild
npm ERR! info install installing standalone, skipping download.
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | win32 | x64
npm ERR! gyp info find Python using Python version 3.12.2 found at "C:\Users\Umair Jameel\AppData\Local\Programs\Python\Python312\python.exe"
npm ERR! gyp http GET https://nodejs.org/download/release/v21.6.1/node-v21.6.1-headers.tar.gz
npm ERR! gyp http 200 https://nodejs.org/download/release/v21.6.1/node-v21.6.1-headers.tar.gz
npm ERR! gyp http GET https://nodejs.org/download/release/v21.6.1/SHASUMS256.txt
npm ERR! gyp http GET https://nodejs.org/download/release/v21.6.1/win-x64/node.lib
npm ERR! gyp http 200 https://nodejs.org/download/release/v21.6.1/SHASUMS256.txt
npm ERR! gyp http 200 https://nodejs.org/download/release/v21.6.1/win-x64/node.lib
npm ERR! gyp ERR! find VS
npm ERR! gyp ERR! find VS msvs_version not set from command line or npm config
npm ERR! gyp ERR! find VS VCINSTALLDIR not set, not running in VS Command Prompt
npm ERR! gyp ERR! find VS checking VS2019 (16.11.34407.143) found at:
npm ERR! gyp ERR! find VS "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools"
npm ERR! gyp ERR! find VS - found "Visual Studio C++ core features"
npm ERR! gyp ERR! find VS - missing any VC++ toolset
npm ERR! gyp ERR! find VS could not find a version of Visual Studio 2017 or newer to use
npm ERR! gyp ERR! find VS not looking for VS2015 as it is only supported up to Node.js 18
npm ERR! gyp ERR! find VS not looking for VS2013 as it is only supported up to Node.js 8
npm ERR! gyp ERR! find VS
npm ERR! gyp ERR! find VS **************************************************************
npm ERR! gyp ERR! find VS You need to install the latest version of Visual Studio
npm ERR! gyp ERR! find VS including the "Desktop development with C++" workload.
npm ERR! gyp ERR! find VS For more information consult the documentation at:
npm ERR! gyp ERR! find VS https://github.com/nodejs/node-gyp#on-windows
npm ERR! gyp ERR! find VS **************************************************************
npm ERR! gyp ERR! find VS
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: Could not find any Visual Studio installation to use
npm ERR! gyp ERR! stack at VisualStudioFinder.fail (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:113:11)
npm ERR! gyp ERR! stack at VisualStudioFinder.findVisualStudio (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:69:17)
npm ERR! gyp ERR! stack at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
npm ERR! gyp ERR! stack at async createBuildDir (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\lib\configure.js:69:26)
npm ERR! gyp ERR! stack at async run (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js:81:18)
npm ERR! gyp ERR! System Windows_NT 10.0.19045
npm ERR! gyp ERR! command "C:\Program Files\nodejs\node.exe" "C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd C:\Users\Umair Jameel\Desktop\python\test\node_modules\iltorb
npm ERR! gyp ERR! node -v v21.6.1
npm ERR! gyp ERR! node-gyp -v v10.0.1
npm ERR! gyp ERR! not ok

npm ERR! A complete log of this run can be found in: C:\Users\Umair Jameel\AppData\Local\npm-cache_logs\2024-04-26T20_04_04_516Z-debug-0.log

This is the output i get everytime i try to install it

Scrapper.TiktokUser(UserName) not working

Here's the output:
{
error: true,
message: 'The user has not uploaded any videos',
log: { code: -1, msg: 'unique_id is invalid', processed_time: 0.1326 }
}

Won't work for some tweet urls: "Request failed with status code 500"

Code to reproduce:

import { Twitter } from '@fongsidev/scraper';

const result = await Twitter(
  'https://twitter.com/EyeonPalestine/status/1779548965605617984'
);
console.log(result);

The log shows:

{ status: 404, msg: 'Request failed with status code 500' }

But the tweet exists and can be viewed normally from website and app.

Reporting littles mistakes

Hi, There's littles mistakes in the documentation and in code.

It's written in the doc so:

const FongsiDev_Scraper = require("@fongsidev/scraper");
const FongsiDev_Scraper = await import("@fongsidev/scraper");
import FongsiDev_Scraper from "@fongsidev/scraper";

//Downloader Tiktok
FongsiDev_Scraper.Tiktok("<Url Tiktok>").then(console.log); //<Object>//

FongsiDev_Scraper.Tiktok should be FongsiDev_Scraper.TiktokVideo

In Twitter-Downloader.js , line 42
if (!global.isUrlFSPR(Url) || !/x\.com|twitter\.com/i.test(Url))
You are checking if the link as the new X.com hostname, nice
But In Twitter-Downloader.js , line 150

    } else if (url.match(/twitter/i)) {
      resolve({
        type: "twitter",
        ...(await TwitterDownloader(url)),

Which comes before calling TwitterDownloader.js, you forgot to.

I found two more, but i don't remember what was them. It was same minor errors, no big deal.
Thanks for your efforts by making this package.
Have nice day!