coders4help / lagesonum Goto Github PK

Number website for Lageso

License: MIT License

Python 64.26% Smarty 26.45% HTML 9.07% Shell 0.23%

lagesonum's Introduction

A website created with Python Bottle and SQLite for entering and showing numbers with a timestamp. Used at LAGeSo in Berlin for helping refugees in the German registration process www.lagesonum.de

Problem

Um in Deutschland registriert zu werden, müssen Flüchtlinge zunächst eine Nummer bekommen. Wann diese aufgerufen wird, ist schwer abzuschätzen und bisher nur vor Ort erkennbar. Die Lösung ist eine Internetseite, auf der Nummern eingetragen und abgerufen werden können. Hintergrundartikel

Nutzerkreis

Die Internetseite soll von Flüchtlingen, die sich im LaGeSO registrieren wollen, genutzt werden, um herauszufinden, ob ihre Nummer schon aufgerufen wurde. Bisher stehen dort täglich mehrere hundert Menschen in der Warteschlange, um dies vor Ort zu erfahren. Freiwillige tragen dabei die Nummern ein.

Funktionen

Bestehende Funktionen

Status der eigenen Nummer abfragen
Nummern von Anzeigetafel eintragen
Virtuelle Anzeigetafel mit eingegebenen Nummern

Die Nummern werden mit einem Fingerabdruck versehen, um die Glaubwürdigkeit eines Eintrags anhand der Zahl verschiedener Eingeber zu messen.

Mögliche Erweiterungen

Wir freuen uns über Beiträge (Pull Requests), die unsere Issues behandeln oder lösen. Diese sind am dringendsten.

Darüber hinaus gibt es größere Erweiterungen, die sinnvoll sind:

Email-Benachrichtigung wenn Nummer aufgerufen wird
Nutzerkonten mit Passwortverwaltung (oder Login über facebook)
Validierte Eingabe vom LAGeSo (Verantwortliche können uns gerne kontaktieren, s.u.)
SSL-Verschlüsselung der Internetseite

Smartphone-Apps

Die Umsetzung als Android-App mit Push-Mitteilung geplant für Refugeehackathon.

Umsetzung

Anforderungen

Die Anforderungen sind in einem Google-Dokument detailliert.

Technik

Die technische Umsetzung erfolgt mit dem Python-Webframework Bottle und SQLite.

Deployment

Es gibt eine Anleitung, wie man das Projekt lokal zum Laufen bringt.

i18n - Übersetzung und Internationalisierung

Es gibt eine gesondertes Dokument für i18n-Deployment im locales-Ordner.

Kontakt

[email protected]

lagesonum's People

Contributors

Stargazers

Watchers

Forkers

steffenmllr christophmeissner fredicious optikfluffel mrigal chrisn777 kadrian

lagesonum's Issues

Help text for entry field about entering more than 1 number

You can enter more than 1 number seperated by spaces but this is not clear to users. There needs to be a help text explaining this.

Include basic usage statistics

Some statistics about the usage would be nice. A simple solution would be a text log file, using a standard python loggin library. Visualization can later be done with other tools. Also, existing solutions can be included.

Add password protected entry with special status

LAGeSo-staff should be enabled to enter numbers. These numbers should be treated as verified without the need for further checks.

Necessary changes

New page: LAGeSo-Eintrag

Layout

Textarea entry field for numbers
Small input for password (with dot protection)
Submit button for sending to db
Cancel button for clearing all fields

No localization of this page is needed, German can be hard coded. "Sie" on this page.

Behaviour

DB entry works as in current enter page, but in addition:

Entry works only if password is correct, then:
USER-field in table numbers is filled with "LAGeSo"

Technical realisation

Passwort comparison with hashes

Behaviour and layout of display page will be described in separate issue

Duplicate DB initialization

Wenn ich in den Kopf von bottle_app.py eine print setze, wird es zwei Mal ausgeführt. Da ist irgend etwas mit der Paketstruktur seltsam. Noch keine Idee was genau.

lazy_gettext does not work

Returned variables are not displayed on the website and HTML-expressions are escaped.

Example for numbers:

Text in excel: "These numbers have been entered since %(since)s by at least %(min_count)s different persons."
Text in .po: "These numbers have been entered since %(since)s by at least %(min_count)s different persons."
Text in html: "These numbers have been entered since %(since)s by at least %(min_count)s different persons."

In HTML: the values given by since and min_count should have been substituted by the returned variables from the def().

Example for HTML-expressions:

excel: Mehr darüber kannst du in diesem <a href="http://www.taz.de/Wartezeiten-am-Berliner-Lageso/!5228958/">taz Artikel</a> lesen.
.po Mehr darüber kannst du in diesem <a href=\"http://www.taz.de/Wartezeiten-am-Berliner-Lageso/!5228958/\">taz Artikel</a> lesen.
HTML: Mehr darüber kannst du in diesem &lt;a href=&quot;http://www.taz.de/Wartezeiten-am-Berliner-Lageso/!5228958/&quot;&gt;taz Artikel&lt;/a&gt; lesen.

lazy_gettext

It seems to me, that somehow, lazy_gettext is not working. A similar problem appeared for somebody with Django on SO.

it has worked before

As far as I can remember on branch livedisplay before the merge, everything worked fine. Also on master. After changing the msgid in .po from phrases to labels, things got messed up, somehow.

Localize Timestamps

Actual:
Timestamps all have standard format:
2015-09-26 00:29:35.463763

Expected:
Timestamps should use Locale’s appropriate date and time representation (%c)

Migrate db for next release

For details see https://github.com/fzesch/lagesonum/pull/46/files

Put navbar in header.tpl

Putting the Navbar in header.tpl would be cool, so we only have to change it in one location and not every page template.
Then setting the class="active" for appropriate styling could be done client-side using jQuery (since it's already in here from Bootstrap) with something like this:

<script type="text/javascript">
  jQuery(document).ready(function() {
    jQuery('#navbar .nav a[href="' + this.location.pathname + '"]').parent().addClass('active');
  });
</script>

Move i18n to transifex

For painfree translations we should move to Transifex. I added our project there but I need help on the following steps:

import existing translations
set up deployment on server/github

Change timestamp display to be more comprehensible

Actual:
Timestamp reads like this: 2015-09-26 00:29:35.463763
Slightly too much information ;)

Expected:
Year, Month, Day, Hour, Minute

Improve design

The design is very basic and could have some improvements.

A logo is in lagesonum/pics

There is no style guide - feel free to make a design that you consider beautiful with colors you like :)

enable SSL-encryption for potential transfer of personal data

a how to for enabling SSL-encryption on nginx is in c't 23/15

Include icon in translation selection

Include icon for indicating the availability of translations left to the language name. My proposal is to use one of these icons as recommended

Archive old numbers

Maybe we should archive old/expired numbers, so performance of primary numbers table is kept high?

Additionally I don't know about reusage of numbers, so having it already stored with old timestamp might prevent it from being entered (in bad circumstances; unlikely, but possible).

One suggestion could be a regular cron job (maybe put into a shell skript):

sqlite3 --batch --cmd "BEGIN TRANSACTION;" lagesonr.db <<EOM
CREATE TABLE IF NOT EXISTS numbers_old(time TIMESTAMP NOT NULL, numbers VARCHAR(30) NOT NULL, user VARCHAR(10) NOT NULL, place VARCHAR(20) NOT  NULL, fingerprint VARCHAR(32) NOT NULL);
CREATE INDEX IF NOT EXISTS i_old_num ON numbers(number, fingerprint);
INSERT INTO numbers_old SELECT * FROM numbers WHERE time < date('now', '-30 days');
DELETE FROM numbers WHERE time < date('now', '-30 days');
COMMIT TRANSACTION;
EOM

Create scalable automatic deployment for i18n-files (csv2po, .mo)

The page needs to be internationalized in many languages. For facilitating future releases with many languages, I would like to have an automatic deployment process fulfilling following requirements:

Accept as input csv (utf8-encoding) and create .po-files. (e.g. with csv2po) in folders according to folder structure implemented.
Create .mo-files from .po-files in (e.g. with "msgfmt -o messages.mo messages.po"
Automatic creation of language selection dropdown menu on website with languages available

For validation: Same number by same person should only be entered once

Actual:
If I submit the same number more than once both entries are accepted by the form and also entered in the database.

Expected:
Every user should only be allowed to enter any number once. Probably prevent with a cookie?
When the user enters the number more than once there should be an error according to "erroruniquenumber" in the loca doc.

Steps to Reproduce:
Query fantasy number (e.g. R567) to make sure it's not in the database yet
Enter fantasy number (e.g. R567) 3 times.
Query this number from the search page.
It is in the database 3 times.

"display" is not highlighted in navigation bar

Visiting Display nav tab ain't hightlighted, while for example Join in! does.

Wrong locale name for Esperanto

The current locale for Esperanto is eo_EO, but it is just eo. In current dev branch when choosing Esperanto as a language and going to "montrotabulo" (display page), the following error occurs:

UnknownLocaleError("unknown locale 'eo_EO'",)

  File "H:\Python Scripts\lagesonum\lagesonum\bottle_app.py", line 203, in display
    since = format_datetime(oldest_to_be_shown, 'short', locale=request.locale)
  File "C:\Python34\lib\site-packages\babel\dates.py", line 609, in format_datetime
    locale = Locale.parse(locale)
  File "C:\Python34\lib\site-packages\babel\core.py", line 315, in parse
    raise UnknownLocaleError(input_id)
babel.core.UnknownLocaleError: unknown locale 'eo_EO'

Return timestamp from DB as datetime (rather than string)

In theory all you need to do is add this argument to the .connect method:

lagesonrdb = sqlite3.connect(DB_PATH, detect_types=sqlite3.PARSE_DECLTYPES)

however it was complaining here:

  File "/Users/danielberndt/workspace/tmp/lagesonum/lagesonum/bottle_app.py", line 99, in do_query
    result = cursor.execute(select_query, values).fetchall()
  File "/usr/local/Cellar/python/2.7.10/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sqlite3/dbapi2.py", line 68, in convert_timestamp
    datepart, timepart = val.split(" ")

Entering numbers results in Error 405

Actual:
Entering numbers results in Error: 405 Method Not Allowed

Expected:
No errors

Steps to reproduce:
Go to http://lagesonum.de/en_US/enter (same with De language)
Enter any valid number
Observe error

Make site "Place-aware"

With upcoming second location for "LaGeSo" the need might arise to make this site "Place-aware".

We'd need:

Definition of default place when entering site
Definition of presentation of currently selected place
Definition of "how to choose/switch place"
Enable number storage code to use place related information for number verification
Enable query and display code to use currently selected place as additional limitation

Add HELP DESK aka "FAQ" to New Design open for Growing

LaGeSoNum HELP DESK

I have a Problem with
Pulldown or direct klicks, to what to do next or may be it s possible to work with typeform here?
Security
Authoritiez
Media

I need Food, where should ich go?
There is Food at 1pm at house J.

Questions we know the answer are answered right away.

My Number is not shown in any way since 6 weeks.

[email protected]

Error: 405 Method Not Allowed with new config

Lokal und auf Server kann man keine Daten mehr eingeben (Error: 405 Method Not Allowed). Kannst du dir mal anschauen, warum?

Have translation to Turkish, Dari and Farsi updated and check variables

check all strings whether they are still correct (original meaning may have changed)
ensure that variable numbers are not translated, but remain, e.g. %(since)s

Add information about the asylum process to website

Examples:

https://www.berlin.de/labo/willkommen-in-berlin/einreise/syrische-fluechtlinge/artikel.378899.ar.php

Contact, About - New Layout

New desktop and mobile layouts for Contact and About pages in the new format.

In Sketch: https://www.dropbox.com/s/c9c9iegu3q622jp/lageso_4.sketch?dl=0
In PDF: https://www.dropbox.com/s/0m79dn95lozx2iq/lageso_4.pdf?dl=0

As a #refugeehackathon volunteer, I didn't have write access to this repository. Please comment when someone with the appropriate permissions has accessed these files, preferably by attaching them to this issue. Danke!

Caching leak

Some actually static content is loaded from external sites. Can we provide them on our site to increase speed and caching?

In the header.tpl this is:

    <link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css">
    <script src="//code.jquery.com/jquery-1.11.3.min.js"></script>
    <script src="//maxcdn.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.min.js" crossorigin="anonymous"></script>

Add "Help translate" link to the website

The link should lead to the transifex project:
https://www.transifex.com/coders4help/lagesonum-1/

Update live-date in localization before going live

Labels:
errnotfound
pleasenote_title

=> Better solution: Add placeholder for "First entry" datetime in database that can be added to the loca files

Select language based on browser preferences

Read browser preferences for languages and deliver the page in the language preferred by the user, if possible, in English otherwhise.

Hint: usr_lang = str(request.environ.get('HTTP_ACCEPT_LANGUAGE', ''))

Datenschutzerklärung finden/anpassen

Wir brauchen eine Datenschutzerklärung, die den gesetztlichen Anforderungen genügt. § 13 des Telemediengesetzes sieht vor: „der Diensteanbieter hat den Nutzer zu Beginn des Nutzungsvorgangs über Art, Umfang und Zwecke der Erhebung und Verwendung personenbezogener Daten sowie über die Verarbeitung seiner Daten zu informieren“.

Umsetzung: Textbausteine suchen und anpassen.

Push notification by email when number is shown

User story

A user $usr enters a number $num and his email adress $mail.
He receives an email with a confirmation link (double opt-in)
He confirms by link. (if not, abort)
When the number $num is entered sufficiently often by different users (e.g. 3 times), the program sends instantly a localized email to $mail with informations about the event.

Implementation proposals

rely on existing libraries for sending email and double opt-in
keep requirements for local mail server as low as possible
store $num as hash
verify $mail address with reg exp
verify $num with reg exp

Optional features

Above, only the minimum viable product is described. But enhancements are possible:

Daily notification that number has not been drawn today (to confirm that the system is working)
Daily digest of shown numbers
Option to unsubscribe (via double opt-out)

Change size of input field on enter page to small box like on query page

The input field is too big and not well usable on small displays. An entry field like the one on the query page is enough. Can you adopt that, please?

Remove W3C validation errors

There are still some HTML-issues. It would be nice to fix them. They exist on all pages.

To see the issues, use https://validator.w3.org/

www.lagesonum.de shows obsolete version

While on http://lagesonum.de the current release is available, an old version is on www.lagesonum.de . This old version needs to be removed and requests redirected to http://lagesonum.de

Since Felix just released prod, this is a rather urgent issue. Currently, the www-site is malfunctioning, many db-entries are missing (the http-version works fine, though)

Implement new design

For now it will be enough to have the following possibility in the backend: As a Sachbearbeiter*in I want to be able to enter a number and choose a scenario (out of 3 alternatives (!).. for now) to got with that number. For simplicity reasons you can call these scenarios 1.1, 1.2 and 1.3 for now.

No nice pic shown, when link is shared

When our site is shared, e.g. on facebook or twitter, we don't have meaningful picture as a symbol. On the other hand, pictures would increase traffic, what might be undesired for those on a small 3G-plan.

Responsive Templates

Sorry I don't have the time today to do a pull request, here are some response templates:

All pages on one the same page:
https://gist.github.com/steffenmllr/436ed78287039c18f7b1

Preview: http://codepen.io/anon/pen/epJYBR

Tests fail with python 3

The tests fail with python 3. It seems as if the import of two modules (input_numper.py and dbhelper.py) does not occur properly and only works at runtime of the project. It works with Python 2, though.

My errors:

With running in PyCharm 4.5:
Testing started at 21:58 ...
Traceback (most recent call last):
File "C:\Program Files (x86)\JetBrains\PyCharm 4.5.4\helpers\pycharm\utrunner.py", line 139, in
module = loadSource(a[0])
File "C:\Program Files (x86)\JetBrains\PyCharm 4.5.4\helpers\pycharm\utrunner.py", line 41, in loadSource
module = imp.load_source(moduleName, fileName)
File "C:\Python34\lib\imp.py", line 171, in load_source
module = methods.load()
File "", line 1220, in load
File "", line 1200, in _load_unlocked
File "", line 1129, in _exec
File "", line 1471, in exec_module
File "", line 321, in _call_with_frames_removed
File "H:\Python Scripts\lagesonum\tests\lagesonum_tests.py", line 4, in
from lagesonum.bottle_app import application, DB_PATH
File "H:\Python Scripts\lagesonum\lagesonum\bottle_app.py", line 14, in
import input_number as ip
ImportError: No module named 'input_number'

Process finished with exit code 1

On command line:
Traceback (most recent call last):
File "H:\Python Scripts\lagesonum\tests\lagesonum_tests.py", line 4, in
from lagesonum.bottle_app import application, DB_PATH
ImportError: No module named 'lagesonum'

Email / SMS push notifications

Purpose:
To allow newcomers to receive an email or SMS alert when their Waiting Number is called.

Constraints:
Should not require a user account or password
Must work with email and SMS
Must require subscription confirmation
Must allow subscription cancellation
Must work with all supported languages
SMS messages are limited to 160 characters

Choices:
Only one Waiting Number allowed per subscription
No limit on subscriptions per email address or phone number
A single subscription can have both email and SMS notifications
If one subscription has both email and SMS notifications, unsubscribing from one will not affect the other

Nomenclature:
Waiting Number: the number a newcomer is tracking; issued by LASeGo
Subscription: A subscription connects one Waiting Number to a phone and/or email address.
Notification: The SMS or email message sent when a Waiting Number is called

Function A: Create a new notification
1 Require user to enter one Waiting Number
2 Require user to enter at least one of: email address, or phone number
3 Send a registration message to every subscribed notification channel.
⁃ SMS: “You have registered to receive notifications for LASeGO waiting number XXXX. Respond with YES to confirm.”
⁃ email: “You have registered to receive notifications for LASeGO waiting number XXXX. Click CONFIRM to confirm.”
4 Send a confirmation message to every subscribed notification channel.
⁃ SMS: “You are now confirmed to receive notifications for LASeGO waiting number XXXX. Reply with STOP at any time to cancel this notification.”
⁃ email: “You are now confirmed to receive notifications for LASeGO waiting number XXXX. Click UNSUBSCRIBE to cancel this notification.”
5 If the user does not confirm registration on the device, they will be not be subscribed.

Function B: Sending a notification
1 Push an alert as soon as the waiting number is first reported to the system.
⁃ SMS: “Someone reported seeing number XXXX at LASeGo at HH:MM on DD/MM. Respond with STOP to stop receiving notifications on this number.”
⁃ email: “Someone reported seeing number XXXX at LASeGo at HH:MM on DD/MM. You can continue receiving notifications about this number, or click UNSUBSCRIBE to stop.”
2 If the user responses with STOP/UNSUBSCRIBE, they will be removed from further notifications

Update arab translation

too many new, fuzzy or incomplete strings for next release

Write tests

Tests are missing for almost all functions. Given the requirements from the GoogleDoc and your brain, feel free to write them :)

Ideally, tests should include:

Testing for success
Testing for failure
Testing for sanity

While this does not add to functionality of the site, it is crucial for further development and a very important background task, suitable also for beginners in Python.

Arabic rendering

When displaying languages written from the right to the left, you have to pay attention to the css to allow the font to be displayed from the right to the left, too.

The CSS attribute is

direction: rtl;

remove date from display

Actual: "These numbers have been entered since 10/1/15, 12:26 AM by at least 3 different persons."
This date is confusing as it doesn't have any relation to the actual display the people are looking at.

Just write e.g.: "The following numbers have been entered into the system within the last XX minutes"

Change text inputs to drop downs / virtual keyboard

Actual:
Non Latin keyboards can't type in the numbers as they don't share the same alphabet.

Expected:
Add 4 dropdown menus instead of the text input fields so the input is independent from the keyboard.
[A-Z] [0-9] [0-9] [0-9]
There could also be a virtual keyboard instead, whatever is easier to do.

Steps to reproduce:
Switch keyboard to Arabic and try to enter valid Numbers.

Make use of an ORM instead of writing and maintaining raw SQL queries

An easy one would be sqlalchemy (http://docs.sqlalchemy.org/en/rel_1_0/index.html).

Add Live display to show currently active numbers in real time

Numbers to be shown there:
All numbers that have been entered 3 or more times and where the last time of entry is not older than X minutes.
For the "last time of entry age" it would be great to collect stats how long numbers are displayed in average.
Until the stats are actually being collected, we should use 15 minutes as an "inactive" time setting. There should also be a link "history" where those numbers are then listed with a note "last seen".

LAGeSo kontaktieren für Nummerneingabe

Das LAGeSo sitzt an der Quelle der Informationen. Folgende Handlungsmöglichkeiten bestehen:

Eigenes Konto und Eingabe verifizierter Nummern, die aufgerufen werden
Planung am morgen im Voraus, welche Nummern an disem Tag voraussichtlich aufgerufen werden
Nutzung der Seite, um den Prozess der Vergabe von Nummern transparent zu machen.

Änderungen im Prozess durch Umzug des LAGeSo?

Das LAGeSo wird demnächst nicht mehr in der Turmstraße, sondern in der Bundesallee sein. Ergeben sich daraus Änderungen für die Nummernformatierung oder den Prozess?

Add basic user management

We should add a way so verified or trusted users can enter numbers.
As adding real user management might take too long so I propose adding the possibility to enter a secret code into the number entry fields, e.g. Àáåß until proper user management is done.