dataforgoodfr / batch5_phenix_happymeal

Meal balancing algorithm for unsold food redistribution

Phenix project - Happy meal algorithm

The main goal of this repository is to create balanced meals from a list of products

Understanding our algorithm

Overview of the balanced meal generation process

In the following workflow, EAN means European Article Number and OFF Open Food Facts.

(The source code of this workflow can be found here)

Matching

We used the Naive Bayes method in order to automatically assign categories to articles according to their names.

Classifier

We used the Random Forest method (40 trees) to automatically assign categories to articles according to their nutrients (6 features).

Defining weights in grams

This algorithm works as follow:

• If the quantity is mentioned in OFF, then converts the quantity in grams
  • if not, then compute the average of article's category quantities
• If the article is rice, semolina, pasta or instant mashed potatoes, then multiply by a specific coefficient

Optimizer

Description can be found here.

Reallocating remaining articles

add a description here

Requirements

The numpy and openfoodfacts packages are required to properly use the repo. Tested on the following version:

import sys
import numpy, openfoodfacts
print('Python %s' % '.'.join(map(str, sys.version_info[:3])))
print('Numpy %s, Openfoodfacts %s' % (numpy.__version__, openfoodfacts.__version__))

Python 3.6.5
Numpy 1.15.0, Openfoodfacts 0.1.0

How to use it

Depending on the feature you wish to test, you may need an Internet connection (product information)

Product information

Pass a barcode (EAN) as argument of the demo script:

python product_info_demo.py 3392460480827

Meal balancing

Run a simulation with the demo script similarly as below:

python tetris_demo.py 1000 --item_max_qty 100. --portion_size 500. --overflow_thresh 0.2 --underflow_thresh 0.1

which should return the result of the algorithm:

------------
RESULT
------------
40 batches for 1 persons (portion of 500.0): 502 items
1 batches for 3 persons (portion of 500.0): 27 items
3 batches for 2 persons (portion of 500.0): 65 items
Average batch loss: 41.91587070338677
Number of remaining items: 201 portioned, 4 unportioned
Number of large items: 201
Number of unindentified items: 0

Many parameters can be adjusted with the previous arguments. To check the full list of arguments and their meaning, use the help of the parser:

python tetris_demo.py -h

Useful links

• Understanding Git
  • Git Data Transport commands
  • Git Cheat Sheet
• Project Slack
• Project Trello

TODO

• Implement an EAN - product information function
• Implement a naive meal balancing algorithm
• Explore brute force and smart optimisation techniques

batch5_phenix_happymeal's Issues

Synchronise output of optimize_baskets/input of balance_adjuster

optimize_baskets should output a dataframe containing the info necessary as input for balance_adjuster

Note: Currently optimize_baskets outputs a results json for plotting in the UI. Instead, the results dictionary should be sent as input to balance_adjuster, and balance_adjuster should update this dictionary with additional data before outputting it as a json (See issue #12)

i.e. output of optimize_baskets should be dataframe + results dictionary, and input of balance_adjuster should be dataframe + results dictionary

[get_foodGroupFromToDF] Add possibility to select group aggregation level

As we aggregate the balanced split at groupAlim_1, we need to pass an argument selecting groupAlim_1 or groupAlim_2 to get_foodGroupFromToDF.

First fix attempt didn't work as many entries go through model matching
https://github.com/dataforgoodfr/batch5_phenix_happymeal/blob/master/preprocessing/categorization.py#L111-L114

Instruction

    input_listing = ct.get_foodGroupFromToDF(input_listing,
                                             EAN_col='EAN',
                                             product_name_col='Produit_Nom',
                                             mapping_file=mapping_file,
                                             model_classifier_file=model_classifier_file,
                                             model_matching_file=model_matching_file,
                                             group_name='labelAlim_1')

Current behaviour

input_listing.columns
Index(['Produit_Nom', 'EAN', 'VALEUR EN STOCK UNITAIRE (HT) ', 'quantity',
       'TOTAL (HT)', 'phenix_grams', 'off_grams', 'weight_grams',
       'labelAlim_2', 'statutAlim_2'],
      dtype='object')

Thus the meal grouping algo fails on all cases where the group isn't in the level 1 list.

Expected behaviour

input_listing.columns
Index(['Produit_Nom', 'EAN', 'VALEUR EN STOCK UNITAIRE (HT) ', 'quantity',
       'TOTAL (HT)', 'phenix_grams', 'off_grams', 'weight_grams',
       'labelAlim_1', 'statutAlim_1'],
      dtype='object')

[OFF database] Make a docker container to access OFF

Run a docker container (perhaps postgresql) with OFF database to get a local copy that would update itself regularly.

Consider using docker-compose to have a source code container linked to the OFF container (+ Phenix container if necessary).

Working example with input and output for Yassi

Make a main.py so that Yassi can test his interface

For the moment, we hardcode the output in main.py instead of connecting to the real code.

python main.py --meal-weight 3000 --delta-auth 0.1 --csv-file-path here.csv --request-id 3243254749865

#output is in the file output/<request-id>_results.json

[Grouping] Fix meal grouping algo

As per #9, it is now possible to aggregate the listing on different levels.
This needs to be reflected in https://github.com/dataforgoodfr/batch5_phenix_happymeal/blob/master/import_demo.py

[JSON output] Make export module for meal completion interface

The result of meal completion are to be displayed for the decision tool. A proper JSON output needs to be exported with all the necessary information after getting the result of https://github.com/dataforgoodfr/batch5_phenix_happymeal/blob/master/meal_balancer/balance_adjuster.py

Find long-term working solution for meal balancing algo

The free solver GLPK works much less well than the paid solver GUROBI.

e.g. according to tests by @sarima in one case GUROBI finds 17 meals and GLPK only 2.

With both solvers we are using the same algo and constraints and more or less the same python code, so we don't have much room for improvement to make GLPK as good as GUROBI. The problem seems to be in the external library.

So we could:

• use GLPK in the webapp (but it will find poor solutions)
• try to get a free or academic licence for GUROBI so we can use it in the webapp
• go back to the "basic" algo meal_balancer.grouping
• try to make an improved version of meal_balancer.grouping without using MILP solver at all
• ??