androz2091 / discord-data-package-explorer

🌀 What's really in your Discord Data package?

Discord Data Package Explorer

What's really in your Discord Data package? And how can this data be useful? Discord Data Package Explorer does the job for you!

Dumpus

Discord Data Package Explorer v2 is now available as Dumpus. This repository (v1) will still be maintained 💪

Example

DEMO

How to Use

• Ask for your data file in Discord (Settings > Privacy & Safety)
• Open https://ddpe.androz2091.fr
• and import your data file!

Docker

This repository provides a Docker image and compose file. Simply run docker-compose up -d in the project directory and you can access the app at http://localhost:5000.

Manual Installation

Discord Data Package Explorer is built with Svelte, and is quite easy to install.

• Clone the repository.
• Install the dependencies using npm install or yarn install.
• Start the app using npm run dev or yarn dev!

Note: for testing purposes, you may append use this link to use the mocked data.

Kaki87

This project was created after a discussion with Kaki87, an awesome developer who taught me a lot of things. He had in mind to create this app but didn't have the time for it. I decided to learn Svelte and build it as a side project :) This project would never have existed without him! 🙏

discord-data-package-explorer's Issues

Site freezes

Upon dragging the folder onto the site, it retrieves user information, user messages, and user activity, but then does nothing and asks me to submit the folder again. This is what I see in the logs.

Improve SVG filling

SVG are now filled with the secondary color of the website, #7289da. The problem is that for the $ svg for example, everything is not entirely filled because some parts are not in the path.

We have to find a way to fill the whole svg 🤔

Username and top users are Unknown

Continuation of #40. I've tried redownloading the package and using the tool with multiple browsers, including MS Edge and Brave Browser, without success.

Nothing happens when I try using it.

Im using the website. On Mac OS. Basically I just drag the package folder and all I get is "you package is loading" which seems to never end...

Crashes upon attempting to stream the file

When trying to upload the package, I always receive the same error -

I'm using Safari on mobile but this also happens on Chrome

Currencies show up wrongly

The statistic of what you spend on discord is wrong because of the new regional pricing (the currency is always USD)

Discord avatar not displaying if changed after requesting Discord data.

If you request your Discord data, and after that change your Discord avatar, nothing will be displayed.

Inaccurate numbers

Message counts under "top users" are off by a lot, I have ~5k with a friend but the explorer shows ~2k and I've seen people claiming the explorer missed users they talked a lot with. Not sure if it's an issue on your end or a limitation from the package file itself, so I thought reporting it would be best

add predicted age/gender

See https://x.com/advaithj1/status/1790235948288233810
it's in the analytics file

[Run Issue] error: 'Chart' is not exported by node_modules\frappe-charts\dist\frappe-charts.min.umd.js, imported by node_modules\svelte-frappe-charts\src\components\base.svelte

I mean there are barely any instructions, not like there needs to be more, but it seems as though it's not on my end, other people I know are having this issue too.

How is timezone for "discord hours" adjusted?

I spend a lot of time across various timezones, how is the discord hours count adjusted / calculated, and what timezone does it display in?

UI: SVG icons don't have the same height/width

When using a mobile phone, the icons don't have the same width/height:

This is a small bug not really important but feel free to open a PR to fix this

Site does not load my data at all

I have tried Chrome, Edge, Firefox on laptop, desktop and mobile an get the same error. Everyone else i know works fine but not me. Now maybe its got something to do with my account being 5 years old? im not sure

Bug, duplication top user

By using this app, one of my top user is dupliacted, I tried with an other app and it worked fine.
https://www.mediafire.com/view/yisnu3sq2hjgqgv/imageDIs.png/file

Run issue: TypeError: Reduce of empty array with no initial value

After a bunch more of the
"Files of channel 405578676089126922 can't be read. Data is true and messages are false."
messages, the resulting error is as seen in the picture.
Issue occurs for 2 data packages requested a couple days ago.

Group DMs not included

It would be nice if group DMs were included somewhere, since it's considered as neither a channel nor a DM.

Wrong user discriminator

The user discriminator is saved as a number in the user.json. We need to convert it to a string and use a pad start to fill it with 0 if it is shorter than 4 characters.

[Idea] Annual Filter so you can breakdown trends.

Not sure how far back the data dump goes but it would be nice to filter/slice down a bit more granular to see how the data changes over time.

[Request] Desktop App

The most I can see is that you can make your own website with its code, but I don't see an ability to turn it into a separate desktop application that can run without a browser.

Improve browsers compatibility

As the website uses the blob.stream() method (https://github.com/Androz2091/discord-data-package-explorer/blob/master/src/views/Loader.svelte#L25), compatibility with Opera, Safari, Internet Explorer and Firefox (only Android) is broken.
We should find a way to support them

Assistance in self-hosting DDPE on shared web hosting.

I use Namecheap Shared Hosting (Stellar Plus Package) and would like to host ddpe on the website. Would I be able to get assistance in doing so?

TypeError: network error

When loading my package folder, I get this error. It's a fresh download from the Discord email, so not entirely sure what has gone wrong here. Just hangs on loading your package file.

Browser: Chrome
OS: macOS Big Sur

Data won't load into the website

There's a lot of folders that can't be read, and I don't know what the error means either.

hard-coded URLs

The source code for this app contains hard references to your personal domain androz2091.fr in a bunch of places. This introduces a big problem: Even when hosted privately, the app makes requests to your resources. This should not happen, because that makes it impossible to truly "self"-host an instance of this app. Furthermore, this is - in my opinion - a big privacy issue, especially since there doesn't seem to be a privacy policy (depending on what the analytics part of the app does, this may actually be illegal).

There should be no hard references to any private URL in the source code. If some external service is inevitably required somewhere, it should be made configurable.

Username and Top Users are Unknown

I re-uploaded the package multiple times and It's still not working.

Crash when i import my package

Messages to user / days known

Currently the "users you chat the most with" metric is often just a metric of how long you've known that user. It'd be nice if there was a metric of messages per days spoken to. This could be as simple as date of last message - date of first message and then an average

[BUG] TypeError: Cannot read property 'stream' of null

Does this everytime i try to upload

Some data is missing in the package file

Heya, sorry if this is a repeat, I did see the previous post with a similar title but wanted to see if mine was for the same reason as well. When I attempt to go to the website (https://ddpe.androz2091.fr/) and put the package file I recieved from Discord into the website. It starts to load and hits 100% and then just stays at a blank screen and doesn't load anything. I saw the previous message stating something about a messages.csv file. I went to Package->Messages->(Picked a random file with a long string of numbers as there's a lot of them) and saw there was only a channel.json file. In every single one of them.

I'm guessing this is for the same reason as the previous post, but thought I'd post here in case it wasn't. I provided a screenshot of what the page looks like after I put the package file in the site

IDEA: Top 10 most said words/phrases

Remove Deleted Users from our Top DM'ed list

It'd be pretty useful to remove any deleted users from the top 10 people that have been DM'ed
Could easily be done by checking the name "Deleted User (random characters)" and the default pfp

For now, one could manually delete Deleted Users from their package by looking up their channel id of the DM and deleting that folder, but not everyone has access to the DM of their friend's deleted account

Payments total

Looks like payments total are not a thing anymore and it leads to a NaN :

(Suggestion) Show top 10 of words used with counts rather than just one favourite word

I think that it would be interesting to learn more than just one most used word and to also be able to see how many times they were used.

How to install with Docker ?

I tried to run this with Docker but can't find how...
I started with this Dockerfile :

FROM node
WORKDIR /usr/local/app
RUN git clone https://github.com/Androz2091/discord-data-package-explorer.git .
RUN yarn install 
EXPOSE 5000
CMD ["yarn", "dev"]

No problem building it, it seems to work on port 5000 so I tried to run it with a 5000:5000 redirection :

yarn run v1.22.5
$ rollup -c -w
rollup v1.32.1
bundles src/main.js → public/build/bundle.js...
(node:29) [DEP0148] DeprecationWarning: Use of deprecated folder mapping "./" in the "exports" field module resolution of the package at /usr/local/app/node_modules/tslib/package.json.
Update this package.json to use a subpath pattern like "./*".
(Use `node --trace-deprecation ...` to show where the warning was created)
[rollup-plugin-svelte] The following packages did not export their `package.json` file so we could not check the "svelte" field. If you had difficulties importing svelte components from a package, then please contact the author and ask them to export the package.json file.
- fflate
LiveReload enabled
created public/build/bundle.js in 5.7s
> [email protected] start
> sirv public "--dev"
Your application is ready~! 🚀
- Local: http://localhost:5000
LOGS

Everything seem fine, but connections keeps trying over and over again:

--2021-05-02 15:08:26--  (try: 8)  http://localhost:5000/
Connecting to localhost (localhost)|127.0.0.1|:5000... connected.
HTTP request sent, awaiting response... No data received.
Retrying.

So i tried to create a Dockerfile with npm:

FROM node
WORKDIR /usr/local/app
RUN git clone https://github.com/Androz2091/discord-data-package-explorer.git .
EXPOSE 5000
RUN npm install 
CMD ["npm", "run", "dev"]

Rollup module make a conflit :

[4/4] RUN npm install:
#7 3.238 npm notice
#7 3.238 npm notice New minor version of npm available! 7.10.0 -> 7.11.2
#7 3.238 npm notice Changelog: <https://github.com/npm/cli/releases/tag/v7.11.2>        
#7 3.238 npm notice Run `npm install -g [email protected]` to update!
#7 3.238 npm notice
#7 3.244 npm ERR! code ERESOLVE
#7 3.253 npm ERR! ERESOLVE unable to resolve dependency tree
#7 3.253 npm ERR!
#7 3.254 npm ERR! While resolving: [email protected]
#7 3.254 npm ERR! Found: [email protected]
#7 3.255 npm ERR! node_modules/rollup
#7 3.256 npm ERR!   dev rollup@"^1.20.0" from the root project
#7 3.256 npm ERR!
#7 3.257 npm ERR! Could not resolve dependency:
#7 3.257 npm ERR! peer rollup@">=2.0.0" from [email protected]
#7 3.258 npm ERR! node_modules/rollup-plugin-svelte
#7 3.258 npm ERR!   dev rollup-plugin-svelte@"^7.1.0" from the root project
#7 3.259 npm ERR!
#7 3.260 npm ERR! Fix the upstream dependency conflict, or retry
#7 3.260 npm ERR! this command with --force, or --legacy-peer-deps
#7 3.260 npm ERR! to accept an incorrect (and potentially broken) dependency resolution.#7 3.260 npm ERR!
#7 3.261 npm ERR! See /root/.npm/eresolve-report.txt for a full report.
#7 3.275
#7 3.275 npm ERR! A complete log of this run can be found in:
#7 3.275 npm ERR!     /root/.npm/_logs/2021-05-02T13_10_45_845Z-debug.log
------
executor failed running [/bin/sh -c npm install]: exit code: 1

So I tried with --force (and with --legacy-peer-deps another time), it could be built and started

> [email protected] dev
> rollup -c -w
rollup v1.32.1
bundles src/main.js → public/build/bundle.js...
(node:19) [DEP0148] DeprecationWarning: Use of deprecated folder mapping "./" in the "exports" field module resolution of the package at /usr/local/app/node_modules/tslib/package.json.
Update this package.json to use a subpath pattern like "./*".
(Use `node --trace-deprecation ...` to show where the warning was created)
[rollup-plugin-svelte] The following packages did not export their `package.json` file so we could not check the "svelte" field. If you had difficulties importing svelte components from a package, then please contact the author and ask them to export the package.json file.
- fflate
LiveReload enabled
created public/build/bundle.js in 6.1s
> [email protected] start
> sirv public "--dev"
Your application is ready~! 🚀
- Local: http://localhost:5000
LOGS

But again as with yarn, no answer ... Any clue ?

Chart is not exported by frappe-charts

Hi,

Tried to run this with

npm install
npm run dev

The installation succeeded, but npm run dev gives the following:

> rollup -c -w

rollup v1.32.1
bundles src/main.js → public\build\bundle.js...
[!] Error: 'Chart' is not exported by node_modules\frappe-charts\dist\frappe-charts.min.umd.js, imported by node_modules\svelte-frappe-charts\src\components\base.svelte
https://rollupjs.org/guide/en/#error-name-is-not-exported-by-module
node_modules\svelte-frappe-charts\src\components\base.svelte (3:11)
1: <script>
2:   import { onMount, afterUpdate, onDestroy } from 'svelte';
3:   import { Chart } from 'frappe-charts';

[!] Error: 'Chart' is not exported by node_modules/frappe-charts/dist/frappe-charts.min.iife.js, imported by node_modules/svelte-frappe-charts/src/components/base.svelte

When I try to run npm run dev, I get the following error:

bundles src/main.js → public/build/bundle.js...
[!] Error: 'Chart' is not exported by node_modules/frappe-charts/dist/frappe-charts.min.iife.js, imported by node_modules/svelte-frappe-charts/src/components/base.svelte
https://rollupjs.org/guide/en/#error-name-is-not-exported-by-module
node_modules/svelte-frappe-charts/src/components/base.svelte (3:11)
1: <script>
2:   import { onMount, afterUpdate, onDestroy } from 'svelte';
3:   import { Chart } from 'frappe-charts';
              ^
4: 
5:   /**
Error: 'Chart' is not exported by node_modules/frappe-charts/dist/frappe-charts.min.iife.js, imported by node_modules/svelte-frappe-charts/src/components/base.svelte
    at error (/home/johnny/discord-data-package-explorer/node_modules/rollup/dist/shared/node-entry.js:5400:30)
    at Module.error (/home/johnny/discord-data-package-explorer/node_modules/rollup/dist/shared/node-entry.js:9824:16)
    at handleMissingExport (/home/johnny/discord-data-package-explorer/node_modules/rollup/dist/shared/node-entry.js:9725:28)
    at Module.traceVariable (/home/johnny/discord-data-package-explorer/node_modules/rollup/dist/shared/node-entry.js:10163:24)
    at ModuleScope.findVariable (/home/johnny/discord-data-package-explorer/node_modules/rollup/dist/shared/node-entry.js:8770:39)
    at FunctionScope.findVariable (/home/johnny/discord-data-package-explorer/node_modules/rollup/dist/shared/node-entry.js:3065:38)
    at ChildScope.findVariable (/home/johnny/discord-data-package-explorer/node_modules/rollup/dist/shared/node-entry.js:3065:38)
    at ReturnValueScope.findVariable (/home/johnny/discord-data-package-explorer/node_modules/rollup/dist/shared/node-entry.js:3065:38)
    at ChildScope.findVariable (/home/johnny/discord-data-package-explorer/node_modules/rollup/dist/shared/node-entry.js:3065:38)
    at Identifier$1.bind (/home/johnny/discord-data-package-explorer/node_modules/rollup/dist/shared/node-entry.js:4403:40)

I'm on the latest commit of the master branch and I ran npm install beforehand.
Node version: v14.16.0
NPM version: 6.14.11

For reproduceability, this docker image also produces the error.

FROM node:14.16.0
WORKDIR /usr/local/app
COPY . .
RUN npm install
CMD ["npm", "run", "dev"]

Top Users was wrong

Hello I use this website and when i use it something make me trouble
They said i speak more with someone than another but it's not true

I wrote 2000 message with a user and this user wasn't display but a men with 1,765 message was displayed

List of servers where we have been the most active

I think it would be interesting to see the servers where we are most active in the same form as the shows and the people we talked to the most.

How to load my file?

Bonjour,
J’aimerais bien savoir ce que je dois mettre à Click or drop your package file here le ZIP ou le fichier décompresse ?
Merci de l’aide.

Captcha failed and being redirected when running locally

I've been having issues attempting to use this tool as I couldn't load the page. Whenever I attempted to load the tool, I was met with a "Captcha failed" message. I tried it on different browsers, different Chrome profiles, and incognito, but nothing fixed the issue.

I attempted to load it locally thinking recaptcha wouldn't be loaded, but I was met with another issue. I was once again met with an issue, but this time the tools page would load and immediate redirect me to your hosted tool instead of mine hosted locally. Once again where I would fail the captcha for whatever reason.

I tried looking in the code to see where the Google's Recaptcha was being loaded and to stop it from loading, but I couldn't find it anywhere.

Solution

I realized that I was going about it wrong as there was no captcha being loaded at all as it was missing from the code after all. On top of that, I was being redirected to your hosted destination of the tool. Commenting out the following line allowed me to use the tool with no issue locally.

discord-data-package-explorer/src/App.svelte

Line 34 in dbaef2b

window.location.replace(`https://diswho.androz2091.fr?returnUrl=${window.location.href}`);

Other remarks

I still have no clue why I was failing the captcha when my friends weren't have an issue. I tested my recaptcha score and I was scoring a 0.9 which led me to more confusion on what was going on. Having issues locally didn't help either.

Possibility to see more than the 10 top users and channels

It would be nice to be able to add, for example, a button under the users and channels rankings to be able to see further than the top 10 on demand

Removing a message file increased the number of messages sent

I removed the folder containing the messages to one of my DMs from the package then reanalyzed it. This caused the number of messages sent to increase from 131,545 to 132,289, when it should have accounted for a loss of 45,657