tanza3d / reddark

reddark, but it's in realtime

This is the old repository! Please check out the new one here.

Reddark

A website to watch subreddits go dark in realtime.

Subreddits

Reddark pulls the list of participating subreddits from the threads on r/ModCoord. If you are the moderator of a sub that is going dark and that is not displayed on Reddark, reply to the aforementioned thread to be counted as participating.

SubManagerBot

If you have already commented your sub below or your sub is already on the list and now going private, please do NOT send a modmail - if you comment here, your sub will be on the list.

Technologies

This is using Express to host the frontend and Socket.io to serve data. Quite simple code, and not too hard to get your head around. This is based on the Original work of D4llo with permission.

reddark's Issues

Counter?

Hi. I've wanted to suggest a simple improvement to the site - a counter showing how many subs are currently private. Big font, up top of the center of the page would be cool, although of course I'm not the one to decide where to place it or how to structure it. I just thought it'd be a really good site addition.

way to illustrate the drop off in content/traffic/subscribers?

these ~7,000 participating subreddits are a small number compared to the 100,000 active-ish ones.

Is there a way to illustrate how much oomph the loss is, like weighted per # of clicks / upvotes/downvotes / amount of content in each one? (say a 1M subreddit has 200 posts a day with 8k upvotes/downvotes each, 16,000 lost interactions)

and then comparing that amount to other large/active subreddits?

e.g., "10% of reddit's traffic is gone", or "20% of their ad revenue vanished".

reddark displaying privated subreddit as public

r/askaustin, which going by this comment has been private for at least 30 minutes, is still being displayed as public.

reddark does not properly identify restricted subs

For the reasons outlined by the moderator u/RamsesThePigeon in this post, some subreddits are not going completely dark, but instead protesting by going restricted.

For example, r/edc announced they were going restricted in response to the unsatisfying AMA from u/spez. However, reddark still identifies them as "public", despite them joining the protest.

I think it can be debated whether it's worth making a distinction between going private and going restricted, but either way, I don't think the sub should be marked as public.

(Thanks so much for the site btw ✨ Such a great idea 😄)

Detect subreddits with restricted submissions

Subreddits such as /r/Pics don't show up as closed but, in practice, are closed. Since restricted submissions is a Reddit feature, perhaps this could be detected too.

It appears when a third thread was created on /r/ModCoord (5k and under) all those subreddits were lost on reddark

Suggestion: Add relative count or percentage for (1M+?) subs

Something like the following to give more sense of scale:

Might be nicer shown as percentage, maybe also with color (how close it is to 100%).

I found the information about number of subs with certain sub count at https://subredditstats.com/list/most-subscribers.
Make sure to change the filter to include sfw+nsfw.

Since the numbers for 5M+ are quite small (<50) it might also be worth just listing all of them explicitly, not just the ones participating (for 30M, 20M, 10M, and 5M brackets only).

———

If this sounds good I can whip up a PR later

Consider client-side polling

Until the API is crippled/killed, web clients are allowed to hit the API with CORS. Instead of running a server, the logic could be run locally on the client's browser.
See https://github.com/ubershmekel/redditp or my own https://github.com/HotGarbo/dumppit as examples.

How to install it?

I would like to help styles

Any way I can work on the proj in my local machine? When I clone it, the networking code does not work since the socket io is bound to https://reddark.untone.uk/, so it is difficult to develop this. It is just stuck here

what do i do?

Suggestion: Add the countdown timer

I think it would be informative to add the countdown timer for the blackout to start and then change to the countdown for the blackout to end, that way users can see what all subreddits that became private and then which all subreddits are still private even after the blackout period has ended.

Maybe something like this

Readme is outdated and the mods of r/modcoord have different instructions

The README currently says:

If you are the moderator of a sub that is going dark and that is not displayed on Reddark, you can message the r/ModCoord moderators to request that the subreddit is added to the relevant thread.

But the mods there are following a different procedure and have requested to please NOT send a modmail.

Kindly update the README. Thanks!

r/rhinolinux not showing as private

reddit.com/r/rhinolinux is private, however is not showing as that. It went private hours ago

Showing some subreddits as private when they’re still open

Sorry if you’re already aware of this! Not sure if it’s a Reddit issue or not, but just wanted to notify you that some of the subreddits currently listed as private are actually still open (for now).

Examples:

• r/MorbidReality
• r/CrimeScene
• r/MedicalGore

Again sorry if you’re already aware of this and for spamming you with issues! Just wanted to raise it now in case it would impact the site on June 12th :)

Recommending Lemmy

According to various sources, the Lemmy team is pro Putin and CCP, and has very dubious privacy, e.g. this mastodon thread.

You are encouraging people to sign up to Lemmy in your footer. Perhaps recommending kbin would be better, it has more users anyway.

edited: wrong URL

Possible audio optimisations

I noticed that (unless I’m reading the code wrong), at the moment, a new Audio object is being created every time an audio-notifiable event takes place.

reddark/public/index.js

Lines 1 to 4 in 7ab9b51

	var audioSystem = {
	playAudio: false,
	play: function (file) {
	var audio = new Audio('/audio/' + file + ".mp3");

From my speed-reading of the MDN docs, I believe that the audio file would be fetched from the server every time a new Audio object was created - meaning that the client would potentially be redownloading the same audio file every time a subreddit goes private/public.

Don’t take my word on this because it’s been ages since I last coded in JS, but this looks like it could be solved by having two Audio objects as variables - one for the privated sound, one for the going-public sound - and having the objects created automatically when the page is loaded rather than waiting until an event is triggered. Then, when an event occurs, calling .play() on the relevant variable rather than creating a whole new Audio object.

(I went looking through the code to do with audio processing because I noticed that there was a noticeable delay on my device between tapping “Enable sound alerts” and the actual sound alert playing, including after disabling them and then enabling them again. In the absence of finding any other code that might be causing the delay my speculation is that it might be as a result of having to fetch the audio file from the server before every use - although I may well be wrong on this as the cause as I haven’t looked into it much at all.)

Struggling to cope with traffic?

I'm seeing sporadic 403s and 502s on https://reddark.untone.uk/

I am not confident in my ability to not screw everything up, but a sub has gone dark and isn't on the list.

r/blahajwithjobs is private already and isn't on the list. I don't trust myself not to screw everything up if I try to contribute so I figured I would put it here.

is there a way for the list of participating subs to be periodically refreshed?

i’m just asking as ModCoord seem to be updating their megathreads a few times a day with more subs (with the updated list now including the largest sub so far, r/funny). from what i can see, at the moment the only way would be to kill the node process hosting the site and re-run it again.

if you’d like i can try and bodge together a pr that uses setInterval to refresh the list a few times a day? (maybe 4?)

alternatively, given how (relatively) little the megathreads will need reloading, i’d understand if you just wanted to keep things how they are & just rerun the node file every now and then.

Would anyone be interested in a Discord server?

Bot would send messages every time a subreddit goes private, may or may not have a chat for discussion, depends if I feel up for moderating today or not

Also consider "restricted" status

For example r/videos has gone restricted instead of private, which has essentially the same result except old content can still be browsed/seen (incl. the announcement that explains why this was done, so maybe this is actually the better way).

Where's the source of the <50k subs list

It looks like the main modcoord thread is no longer showing subs <50k

r/PsilocybinMushrooms not showing up at all

https://subredditstats.com/r/PsilocybinMushrooms you can see here it's a 100K+ subreddit, but it doesn't show up at all. It just went private and isn't listed. Like, it doesn't show it on the list at all, not just not showing it's private...

Search doesn't accept uppercase letters and doesn't work when "hide public" is checked

Searching for diy returns the expected subreddit. DIY, however, returns nothing. In fact, entering any uppercase letters returns nothing.

When Hide public subreddits is checked, the private subreddits don't get hidden during the search. It looks like it only filters the headers (20+ million, etc...) and entirely ignores the subreddits.

Reddark.mp4

Idea: A count at the top of the page?

Just an idea I had - maybe just a sentence at the top of the page, like [X] subreddits have gone dark, [X]/[Y] subreddits have gone dark, or any different wording (there’s probably a better way of wording it that I can’t think of right now). I guess as a way of being able to tell at an immediate glance how many subreddits have gone private at that time.

Also thanks for building the site! Will be fun to watch on June 12th

Counter per scales

Each category (millions, thousands, etc) could also have a counter and percentage (#74).

memory issues after a while`

Jun 09 21:45:35 untone.uk systemd[1]: reddark.service: A process of this unit has been killed by the OOM killer.
Jun 09 21:45:36 untone.uk systemd[1]: reddark.service: Main process exited, code=killed, status=9/KILL
Jun 09 21:45:36 untone.uk systemd[1]: reddark.service: Failed with result 'oom-kill'.
Jun 09 21:45:36 untone.uk systemd[1]: reddark.service: Consumed 15min 52.352s CPU time.
Jun 09 21:45:37 untone.uk systemd[1]: reddark.service: Scheduled restart job, restart counter is at 1.
Jun 09 21:45:37 untone.uk systemd[1]: Stopped reddark.
Jun 09 21:45:37 untone.uk systemd[1]: reddark.service: Consumed 15min 52.352s CPU time.
Jun 09 21:45:37 untone.uk systemd[1]: Started reddark.

this is only after 10 minutes

Website needs some padding (at least on mobile)

Not sure how the page looks on desktop at the moment, but just wanted to let you know that on mobile (Safari on iOS), the page needs a bit of padding around the edges. Screenshot attached.

Absolutely loving the website so far! Especially given the recent update from Apollo devs, I will very much like watching live as all the subs go dark.

Typing Into Search Field Filters Staff List Also

When you type something in the search field, the expected action is to filter the list of subreddits to include your search term. This works, but the staff list at reddark.untone.uk is unexpected also filtered by that search term. I'm pretty sure that wasn't the desired effect.

Website is not working - 500 and 502 errors

Stucked with "We're probably getting blocked..."

<html>
  <head>
    <title>Too Many Requests</title>
    <style>
      body {
          font: small verdana, arial, helvetica, sans-serif;
          width: 600px;
          margin: 0 auto;
      }

      h1 {
          height: 40px;
          background: transparent url(//www.redditstatic.com/reddit.com.header.png) no-repeat scroll top right;
      }

      textarea.mushroom {
          display: none
      }
    </style>
  </head>
  <body>
    <h1>whoa there, pardner!</h1>

<p>reddit's awesome and all, but you may have a bit of a
problem. we've seen far too many requests come from your ip address
recently.</p>

<p>please wait a few minutes and try again.</p>

<p>if you're still getting this error after a few minutes and
think that we've incorrectly blocked you or you would like to discuss
easier ways to get the data you want, please contact us at <a href=mailto:[email protected]?Subject=Rate%20limiting(C)%20of%20XXX.XXX.XXX.XXX (Request ID: 369XXXXXXX68-FRA)>this email address</a>.</p>

<p>You can read Reddit's Terms of Service <a href=https://www.reddit.com/wiki/api/>here</a>.</p>

<p>when contacting us, please include your ip address which is: <strong>XXX.XXX.XXX.XXX</strong> and reddit account</p>

</body>
</html>

How can I solve this issue?

Certain debug console logs left in index.js spam the console way too much

A console log has been left in the loop that iterates over all the subs and checks their status, as well as other places. Those logs are getting quiet spammy.

delay between http calls

[not an issue, just using this as a place to ask a question/discuss]

if i’m understanding the most recent code (that introduces a delay between http requests) correctly, then it works by incrementing the delay between requests by 1ms after every request. meaning that for 5000 subreddits, one run of updateStatus would take approximately 5000 + 4999 + 4998 + … + 1 = 12,502,500ms (around 208 minutes) to complete. however, the website seems to be detecting new privated subreddits quicker than that.

am i missing something here? (i feel like i am)

as a side note, i’ve chucked a branch of my fork onto a digitalocean instance, and i’m currently experimenting with trying to find the lowest delay between requests where the reddit api doesn’t start rate-limiting. in case the info is helpful to you, i’m currently down to 80ms 51ms between requests with no rate-limiting. (i can keep you updated with how low the delay can go if it would be helpful)

Hide Private Subreddits?

We have 'hide public', can we 'hide private' as an option to see who the hold-outs are?
Might encourage users to post in the online subreddits and ask if they'd be willing to participate.

suggestion: counter should ignore data from public subreddits when "Hide public subreddits" is toggled on

atm it's only possible to see data from the 6000+ subreddits planning on going private, i think it would be quite good if when toggling "Hide public subreddits", it ignored the data from those servers to give you the current count of moderators and subscribers

Uppercase letters removed from search box

To reproduce:

Type in any uppercase letter into the search box

Expected behaviour:

The letter is entered, either uppercase or lowercase

Actual behaviour:

Uppercase letter is removed.

Firefox 115.0b3

styling slightly broken on mobile after update

since 02ca61b, the styling has slightly broken when viewing the site on mobile (iPhone 11, Safari on iOS):

1. the ‘X/Y subreddits are currently dark’ line now breaks as i scroll down the page (see screenshot)
2. there is now no longer two columns on the page, only 1 - it might be a minor thing but having 2 cols rather than 1 made it easier to take in the site

i apologise for giving you another styling headache

Why are there missing subreddits?

I noticed not all subreddits are listed. Is there a reason for this?

Don't forget to add the AGPL license

In order to be compliant with the old repository ;)

Scrollbars visible in large counter mode look bad

Sorry to make an issue without a PR but I couldn't get it to run, the scrollbars visible in large counter mode aren't consistent with the theme of the rest of the counter and probably isn't intended behaviour (?), stealing this code from w3schools but I tested in Chrome Devtools and this should work for hiding scrollbars if that's intended:

/* Hide scrollbar for Chrome, Safari and Opera */
.example::-webkit-scrollbar {
  display: none;
}

/* Hide scrollbar for IE, Edge and Firefox */
.example {
  -ms-overflow-style: none;  /* IE and Edge */
  scrollbar-width: none;  /* Firefox */
}

oops i think it broke

Refreshed the page to get this:

Using wiki url to track subreddits

Hi,
they are also using the wiki page https://www.reddit.com/r/ModCoord/wiki/index.json which should be more updated than the 2 incomplete posts.

Percentage counter

The raw data is cool, but needs more thinking to put it in context. How about showing the values also as a percentage?

Web Push API

It would be nice if WebSocket updates triggered a desktop notification as well. Shouldn't require any backend work.

https://web.dev/notifications/

Suggestion: show restricted

There are some subs that opted to go restricted instead. Could you update the site to reflect those as well? Thank you.

Edit: Sorry. I now realize it's been suggested already.

searchbar makes counter look more ugly on mobile

it doesn’t look like there’s any padding so there’s no space between the searchbar and the count, and because the count is forced onto two lines it breaks when scrolling down (and part of the search bar is cut off as well).

screenshots attached below

Style and accessibility Improvements

Issue

1. The webapp tends to fill up the whole space on a webpage, making the text stretch all the way from the left to the right.
2. The sound toggle button cannot be toggled with a keyboard. You need to hover and click on it with a mouse.

Why?

1. The app centered in the browser will aid readability as you wouldn't have to move your head to read text in a single column.

It would be great if it was nicely centered on the screen; like this.
Before:

When centered properly:

2. Improving the accessibility of the button would make sure that keyboard users can navigate the web app as well.

Missing package.json

Hi, I just wanted to let you know that you only have your package-lock.json comitted.

Site returns 404

Is the site down? It started returning 403 for like 10 minutes, now it's 404 for quite a while now.

(Resolved)

Not a huge issue, but seems a little biased

worth fixing