zomux / dlmonitor

DLMonitor: Monitoring all things happening in deep learning

Purpose

This project aims to save time and energy for deep learning folks. It monitors new things on multiple sources and find out those important to you. Currently, the data sources include:

• Arxiv papers
• Tweets
• Reddit posts

Take a look at the public server: https://deeplearn.org

Install

1. Install postgres server
2. pip install -r requirements.txt
3. sudo apt-get install poppler-utils

Setup database

1. Create a .env file in the project root.

DATABASE_USER=dlmonitor
DATABASE_PASSWD=something

TWITTER_CONSUMER_KEY=something
TWITTER_CONSUMER_SECRET=something
TWITTER_ACCESS_TOKEN=something
TWITTER_ACCESS_SECRET=something

SUPERVISORD_PASSWD=something

2. Create database

Run bash bin/create_db.sh

Install Quick Read dependencies

1. install cpan
2. install text::Unidecode in cpan
3. git clone https://github.com/brucemiller/LaTeXML
4. perl Makefile.PL; make; make install

Fetch resources

Fetch Arxiv papers and tweets.

python bin/fetch_new_sources.py all

Run test server

PYTHONPATH="." python dlmonitor/webapp/app.py

Setup production server

1. Install nginx

2. Copy configuration files for supervisord and nignx

bash bin/config_server.sh

3. Start Gunicorn processes through supervisord

bash bin/start_supervisord.sh

4. Start arxiv source loading worker

PYTHONPATH="." python bin/auto_load_arxiv.py --forever

dlmonitor's Issues

The website not work

Hi, the project is very cool. However, the website seemed to not work recently. What wrong is it? Could you restart it? Thanks!

Hot paper mechanism

Hi, great project! I was wondering about the "hot" and "super hot" labels that appear. I guess it's somehow related to the chatbox number, but how those influencing factors are evaluated? Thanks!

Hot and Super Hot labels are currently not available

Hi @zomux,

Thanks for a great project, I've been using it daily for a few years now!
It seems that the "Hot Tweets" section broke and the "Hot" and "Super Hot" labels stopped working.
It seems that the problem appeared on 2023-05-18.

Do you have an idea what could be causing the problem?

meaning of chat box number?

I came across your website and it's very cool. I'm wondering: when you click on the link for a particular paper, what is the meaning of the number next to the chat icon (just right of the author names)? Are there paper comments somewhere?

Trying to get in touch regarding a security issue

Hey there!

I'd like to report a security issue but cannot find contact instructions on your repository.

If not a hassle, might you kindly add a SECURITY.md file with an email, or another contact method? GitHub recommends this best practice to ensure security issues are responsibly disclosed, and it would serve as a simple instruction for security researchers in the future.

Thank you for your consideration, and I look forward to hearing from you!

(cc @huntr-helper)

deeplearn.org is down for a few days

Hi Raphael, the deeplearn.org has been a great place for me to catch up with the latest progress of academia, thanks for the great work! It looks like the deeplearn.org is down for the last few days, I'm wondering if the fix is on the way?

https://deeplearn.org is not worked

first of all, dlmoniter is Greatest project. so, I check https://deeplearn.org EVERYDAY.
but now, it is not worked. would you restart it?
thank you.

arXiv.org link for the papers

I like the site and try to visit it regularly.

It will be great if you include a direct link to the arXiv abstract page for each paper.
Currently, we need to click the title to go to the abstract and then click the arxiv link to go to the arxiv abstract.
By the way, some words get merged into one in your abstract page.

cache the query words

Can query words be cached somewhere？

deeplearn.org is down

Hi @zomux, looks like the deeplearn.org is down again for a few days. Are you currently working on something big? ^_^