Separate units and ingredients, add scaling about mealie

[hay-kot]

As @dpieski describes. Adding this feature would require a pretty extensive rewrite of the recipe data structure that not adhere to the standard schema, and has a lot of gotchas. That, and I think that it makes the user experience for adding ingredients/recipes more difficult puts me off on including this feature. I have no plans to address this at the moment, but I'm open to a PR or working with another developer who has a good idea on how this can work and keep the user experience "wife approved".

[hay-kot]

Hey all, I've made some significant progress over in the mealie-next branch. The next big thing I need to tackle is parsing incoming ingredients.

I was able to find this repository that does exactly that using CRF++ and some Natural Language Processing. It looks to be extremely powerful and good at correctly parsing ingredients. I was able to get a model trained and create some proof of concept code. What I would like to do is package CRF++ in the docker container and pull down a trained model from somewhere (google drive maybe? ) and include the binary and model in the built container. That way we can easily make subprocess calls to CRF and get the resulting parsed ingredient back. I have this working on my M1 mac using the brew install.

I'm hoping someone would be willing to take a look at the CI/CD to build in mealie-next branch and try to get a working CRF build in the arm and x86 build. If anyone in this thread is up for this, please get in touch and we can work on implementing this.

IMO this would be a HUGE feature and quality of life improvement for using Mealie, ingredients, and scaling.

EDIT:

Here's a link to a the trained model https://drive.google.com/file/d/1HUs0dhrLJEbNA3eBsx6PZ9DWfY-oIRtS/view?usp=sharing

[dpieski]

Some comments here:

• Plurality can be more complicated than that.
  If it is greater than 1 it should be plural so {{unit(if >1, plural}}
  If the unit is in decimals, then if it is less than 1, use plural (e.g., 0.5 apples)
  If the unit is in fractions, then if it is less than 1, use singular (e.g., 1/2 apple)
• How do we decide to use factions or decimals?
• Using something like flour instead of butter, {{extra}} could be insufficient.
  For example, 1 cup sifted flour and 1 cup flour, sifted actually mean two different things in a recipe. The first means that you sift the flour into a 1 cup measuring cup to use, whereas the second means your take one cup of flour and sift it when the recipe calls for its use. The latter would provide for a greater mass of flour than the former. Maybe have "sifted flour" and "flour" be different ingredients even though they are the same product?
• Some ingredients are countable while others are measurable.
  For example, eggs are countable. They come in discrete units. You can't easily measure out 1/2 egg yolk but maybe we don't care and let the user decide there.
• Thoughts on Units (esp. if we want to convert units in the future)
  • Should probably be a reference to another table
  • Should include a "unit system" identifier of some sort. (English units have different sizes depending on what country you're in. e.g., in AU/NZ/USA(nutrition labeling) then a Tsp ~ 5ml, in CA and UK then a Tsp ~ 5.92ml, in conventional US then a Tsp ~ 4.93ml)

Edit:

• Also, probably a field for something like "organic" for an ingredient

[phantomtypist]

My two cents: https://schema.org/recipeIngredient

[dpieski]

@phantomtypist yea I've seen that. It's the same schema as Google uses too so it would help with interactions with AI Assistants.

But that is an API schema, not necessarily how the data has to be stored. You can't do many interesting things with plain text ingredient data.

[hay-kot]

Another Update, no real news but... I've come around to the idea that this is something I'd like to support with the caveat that I don't want to change the user experience on the frontend... i.e. it still needs to be very simple to add ingredients and any additional work that gets done needs to be an advanced mode or done via some programming magic. I have 2 schools of thought on how to accomplish this.

Option 1

Do not change the data structure from string and use a "parsing" method to effectively split the data. For example "1 Cup Flour" would be assuming that the first "word" is the quantity, the second 'word' is the unit and the rest is a description. The benefits of this approach would allow the overall codebase to change very little and keep the user experience effectively unchanged. Only users interested in the advanced features of need to write the structured ingredients, others could write it however they want.

Option 2

Completely change how ingredients are stored on the backend and move away from the defined schema. Splitting them into 3 parts, amount, unit, description. This would allow for user completion the frontend and an easily defined structure to work with data and do lots of tasks with the data. The major downside in my opinion is that it turns entering text into data entry which I'd like to avoid if at all possible. anyone can come up with a clever way to simplify the entry of the data I'd be super interested.

Bottom line is this will happen eventually, but I'm still mulling over the best way to get this done without adding too much complexity

[phantomtypist]

I feel like with option 1 and the parser there are six ways to Sunday to shoot yourself in the foot.

[phantomtypist]

If you go option 2 (I'm not sure if this is there right now), but you'd need to ask the user on first run if they want US or metric standards..... or come up with a clever way to support both simultaneously.

[phantomtypist]

I just thought of something else related to option 2. What if there is no concept of stored "units". The database would just hold "amount" and then amount is tied to a base line like teaspoon. Then in the database we only just store teaspoon quantities as a decimal. Some baseline to do conversions from and then you don't have to store a unit type. Then, on the recipe page you just run that through a parser to break "up" (not down, lol) the amount of that base unit.

E.g. so let's say for one ingredient in a receipt it calls for 1 cup. We allow the person, when creating the recipe, to specify amount "1" and "cup" from a unit drop down list. Then when the recipe is saved, the unit and amount they specified gets converted/"serialized" to our base unit used in the amounts property. In this case there would be the number 48 stored for this ingredient because 48 teaspoons equal 1 cup. Then, when a person pulls up a recipe we can use a parser to convert the 48 teaspoons to something more meaningful to the user like "1 cup". It'll make it really easy to scale recipes up and down as well.

Another benefit in addition to the scaling of recipes (portions) is that there is no more need to store a "unit" for an ingredient. The amount value is always stored as some baseline type of unit, e.g. teaspoon in the aforementioned example.

This would also make it really trivial to convert the system between metric and US standard as well.

[dpieski]

I'm still mulling over the best way to get this done without adding too much complexity

@hay-kot, IMHO, I think that the best thing to do would be to keep the recipes working as they are right now for the most part. But we make an addition that, when editing a recipe, you can "convert" the Recipe to an "Advanced Recipe" or something.

So, current functionality would be maintained. The current tables would not have to be modified.
With an "Advanced Recipe" you would get these ingredient features that have been talked about a lot.
Your ingredients would be formed of four sections:

• quantity
• unit
• name
• comment
  Also shown here:
  

Option 1

I do not think would be too feasible. I have talked to a friend that is an expert in AI, he said he may play around with training an AI model to parse once I told him how challenging it is. I have accumulated at least a dozen articles and postings regarding how to parse recipes and no one has really been able to get higher than like low 80% accuracy range. I did find a really interesting Thesis from a student at Princeton (she is now at Facebook) where she was using Regex to parse and was able to get over 75% accurate. I will try to find and link her GitHub. I have it bookmarked on a different computer.

Option 2

I think the issue here is it would make it harder to manage from parsing - getting everything in the right buckets when you parse in a recipe from a site. (see Sites below).

My proposal is maintaining current functionality for standard Recipes. When you select Edit on a recipe, you have an option to "automatically convert" where we attempt to parse the recipes but have the original next to it so the user can ensure it is parsed correctly. Similarly, when you Add a recipe, you have the option of like changing to an "Advanced Recipe", which, like before, would automatically try to parse any Ingredients already entered, but also allows the user to enter the Ingredients in component form.

Sites

I do think that, if a plugin model for parsing recipe sites is implemented, a customization can be made in some situations that would allow the recipe to be parsed into an Advanced Recipe, especially if the website provides the recipe with html tags identifying the recipe components, even if it is just partially provided, like quantity, unit, ingredient.

Other thoughts

• We could prime the Unit list with the unit list from the FDA

[dpieski]

@phantomtypist

If you go option 2 (I'm not sure if this is there right now), but you'd need to ask the user on first run if they want US or metric standards..... or come up with a clever way to support both simultaneously.

You could default to locale and allow the user to adjust in settings. If locale == en-US, English Units, else SI Units. Maybe conditions for UK, AU, and NZ too?

You wouldn't have to "support" more than one either, you would just have to have conversions setup. I'm sure there is already a library made for this, such as pint. Then you could import recipes that are either SI or EN. Parsing wouldn't care about the unit or convert them, it would just put in the db. You would only convert when you view a recipe.

Consideration, when viewing a recipe, option to switch between English and SI.

Difficult consideration - convert between English and SI (volume) units to SI (weight) units.

[phantomtypist]

@dpieski column for "Amount" and another for "UnitType" UnitTypes as in "volume" and "weight".

[dpieski]

I just thought of something else related to option 2. What if there is no concept of stored "units". The database would just hold "amount" and then amount is tied to a base line like teaspoon. Then in the database we only just store teaspoon quantities as a decimal.

I do not like this idea. Not everything is easily converted to teaspoons. e.g., a "dash". I really don't see a reason to do this for the most part since it would require a calculation for each time a recipe is viewed instead of simply concatenating the ingredient back together.

Also, not every ingredient would have one of each quantity, unit, name, and comment.

Take, for example, the ingredient "One Apple"
this would parse into

quantity	unit	ingredient	comment
1		Apple

but what about "one clove of garlic"? Which should that be?
The first somewhat seem natural

quantity	unit	ingredient	comment
1	clove	garlic
1		garlic clove

But the what about "1 tsp minced garlic"?
Each of the below actually mean two different things.

quantity	unit	ingredient	comment
1	tsp	garlic	minced
1	tsp	minced garlic

Similarly each of these mean two different things:
"1 cup sifted flour" and
"1 cup flour, sifted"

[phantomtypist]

Anybody just know how All Recipes site does it or any other big guy? Pretty sure this problem has already been solved in places instead of reinventing the wheel.

[dpieski]

column for "Amount" and another for "UnitType" UnitTypes as in "volume" and "weight".

@phantomtypist I do not think this is really necessary though. You know if it is a "volume" or a "weight" based on the unit column. I have a few dozen recipe books at home, and I am not sure if I have ever seen a recipe with a weight that is not in metric - has anyone see that before? Concern here would be Oz being a weight and a volume, so one would have to distinguish Oz and fl-oz.

The concern with converting between Volume and Weight is that it is difficult to do. Densities are not always the same and some of them would just have to be defined based on the comment or have a complex-ingredient where "1 cup flour, sifted' and "1 cup sifted flour" are two different names as "flour, sifted" and "sifted flour" would have two different weights based on the differing densities. There is probably a database out there of the top xx number of ingredient-volume-weight conversions. I may look around for it if I get some free time this week.

Similarly, "1 cup sugar" and "1 cup sugar, packed" would have two different weights even though they have the same volume - the comment, or complex-ingredient depending on implementation,

[dpieski]

Anybody just know how All Recipes site does it or any other big guy?

I looked up AllRecipes but I can't figure out how to get API access. There is an apps.allrecipes.com but you have to login to go to any of the links.

Here is an interesting Recipe API article I found.

bigoven

I did find bigoven API, their ingredients example is, in part:

"Ingredients": [ 
{ 
"IngredientID": 5247317, 
"DisplayIndex": 0, 
"IsHeading": false, 
"Name": "Soy Sauce", 
"HTMLName": "Soy Sauce", 
"Quantity": 0.333333333333333, 
"DisplayQuantity": "1/3", 
"Unit": "cup", 
"MetricQuantity": 79, 
"MetricDisplayQuantity": "79", 
"MetricUnit": "ml", 
"PreparationNotes": null, 
"IngredientInfo": { 
"Name": "Soy sauce", 
"Department": "Asian", 
"MasterIngredientID": 165, 
"UsuallyOnHand": true 
}, 
"IsLinked": true 
}, 
{ 
"IngredientID": 5247318, 
"DisplayIndex": 1, 
"IsHeading": false, 
"Name": "Dry sherry", 
"HTMLName": "Dry sherry", 
"Quantity": 0.25, 
"DisplayQuantity": "1/4", 
"Unit": "cup", 
"MetricQuantity": 59, 
"MetricDisplayQuantity": "59", 
"MetricUnit": "ml", 
"PreparationNotes": null, 
"IngredientInfo": { 
"Name": "Dry sherry", 
"Department": "Wines", 
"MasterIngredientID": 156, 
"UsuallyOnHand": false 
}, 
"IsLinked": true 
}]

Edamam

Edamam.com uses this for ingredients:

Field	Type	Description
foodId	string	Food identifier
quantity	float	Quantity of specified measure
measure	Measure	Measure
weight	float	Total weight, g
food	Food	Food
foodCategory	string	Shopping aisle category

##Recipal
Recipal API showing the Recipe Ingredient structure.
EXAMPLE OBJECT

{
"recipe_ingredient": {
"id": 1204,
"quantity": 1.0,
"unit": "1 cup",
"waste": 0.0,
"ingredient_id": 77,
"recipe_id": 522,
"created_at": "2014-01-12T02:37:16Z",
"updated_at": "2014-01-12T02:37:16Z",
"name": "Butter, salted",
"display_as": "Butter, salted",
"total_grams": 227.0,
"units_and_grams": {
"1 cup": 227.0,
"1 tbsp": 14.2,
"1 pat (1\" sq, 1/3\" high)": 5.0,
"1 stick": 113.0,
"1 lb": 453.592,
"1 gallon": 3632.0,
"1 gram": 1.0,
"1 oz": 28.35
}
}
}

##Spoonacular
Spoonacular API showing their recipe information:

 "extendedIngredients": [
        {
            "aisle": "Milk, Eggs, Other Dairy",
            "amount": 1.0,
            "consitency": "solid",
            "id": 1001,
            "image": "butter-sliced.jpg",
            "measures": {
                "metric": {
                    "amount": 1.0,
                    "unitLong": "Tbsp",
                    "unitShort": "Tbsp"
                },
                "us": {
                    "amount": 1.0,
                    "unitLong": "Tbsp",
                    "unitShort": "Tbsp"
                }
            },
            "meta": [],
            "name": "butter",
            "original": "1 tbsp butter",
            "originalName": "butter",
            "unit": "tbsp"
        },
        {
            "aisle": "Produce",
            "amount": 2.0,
            "consitency": "solid",
            "id": 10011135,
            "image": "cauliflower.jpg",
            "measures": {
                "metric": {
                    "amount": 473.176,
                    "unitLong": "milliliters",
                    "unitShort": "ml"
                },
                "us": {
                    "amount": 2.0,
                    "unitLong": "cups",
                    "unitShort": "cups"
                }
            },
            "meta": [
                "frozen",
                "thawed",
                "cut into bite-sized pieces"
            ],
            "name": "cauliflower florets",
            "original": "about 2 cups frozen cauliflower florets, thawed, cut into bite-sized pieces",
            "originalName": "about frozen cauliflower florets, thawed, cut into bite-sized pieces",
            "unit": "cups"
        }]

Services

Maybe have it setup so users can input an API key for a someone who provides this service?

Service	Cost	recipes	lines	health labeling
Edamam	Free (Devs)	200/mo	1000/mo	Basic
nutritionix	Free (2 users)	-	-	-

Really Awesome bulk recipe dataset that I am not sure what I want to do with yet...
http://pic2recipe.csail.mit.edu/
Here is a link to the actual data: https://data.lip6.fr/cadene/im2recipe/

[hay-kot]

So, current functionality would be maintained. The current tables would not have to be modified.
With an "Advanced Recipe" you would get these ingredient features that have been talked about a lot.
Your ingredients would be formed of four sections:

• quantity
• unit
• name
• comment

This is the approach I'm leaning towards. However I think having the split comments would be too confusing. I can't quite think of how you would end up parsing that string to accurately know what is a comment and what is an ingredient. I think have a structure of [Qty, Unit, Ingredient, Comment] Would be the easiest. In fact, I think all I would really need to add is create a "table" editor that breaks down the string into separate parts. Then you can use a separate units/ingredient database table to validate/track what units are being used across that database for autocompletion., similar to how categories and tags work now. I think this allows the most flexibility while also keeping up with the standard schema.

[dpieski]

This is the approach I'm leaning towards. However I think having the split comments would be too confusing. I can't quite think of how you would end up parsing that string to accurately know what is a comment and what is an ingredient. I think have a structure of [Qty, Unit, Ingredient, Comment] Would be the easiest

By "split comments" do you mean like the "softened butter" ingredient in that image? I would agree with you, I think.
If we have a "convert" option, the we don't really get to choose the format of the ingredient string coming in. The "table editor" idea, even if the parser is only like 70% accurate when converting to an "Advanced" recipe, would be pretty nice.

I think this recipe parser from SousChef-backend would probably get us most of the way there with a little refactoring.

If we have a table storing units, it could/should probably be pre-populated and that list above would get most of the way there. There are obviously variations that could be made to many of them still but that can grow for the particular user as you suggested. Maybe account for letter casing if that doesn't. Some exceptions to not caring about cases could be "T." =/= "t." as the first is sometimes used for Tablespoon while the second is sometimes used for Teaspoon.

For that "table editor" - we could also autocomplete the ingredients too, no? Once a few recipes are loaded up, it could make it easier to enter a new recipe.

[hay-kot]

Minor update, I was experimenting with an old npm package I found that may get us most of the way there for supporting this. I've done some work to modernize it and try to make some changes to better suite how mealie works, but overall I was unsuccessful. You can find my attempt here and the original package here

I'm hoping to find someone with some more javascript knowledge to help get this figured out, let me know if anyone interested.

[h0rnman]

So, I know nothing of python and its intricacies, but a similar project called Gourmet Recipe Manager does something similar and allows for more 'natural' input of ingredients. I've been using it for years, but it's not received a new Windows build in a very long time and has no client/server or mobile support, so I'm looking at alternatives. You might be able to reference some of their code to help with this.

Link is here: https://github.com/thinkle/gourmet/

[abhchand]

👍🏾 on this whole idea. It's definitely a more complicated schema but I think scaling ingredients is sort of an "expected" feature of recipe apps.

In my experience, fully defining an Ingredient requires capturing the following components -

Key	Value	Description
Quantity	1/4	The value of whatever unit
Unit	cup	Can be standardized (e.g. tbsp) but could also be free-form (e.g. "cubes" when talking about sugar)
Item	Carrots	The actual ingredient name
Preparation	Finely chopped	How it should be prepared, or in what form it will be used
Variant	Organic	Optional, but sometimes you want to specify a specific brand, type, or other qualifying label

Storing these fields lets you -

1. Scale as needed
2. Concatenate all values as a single display string - 1/4 Cup Carrot (Organic), Finely Chopped

For the Unit, the form could be a dropdown or autocomplete of some common unit types (e.g. cups, tbsp, mL, etc..). The end result gets stored as a simple String so that they can also populate any custom types (e.g. cloves when talking of Garlic). I wouldn't worry a lot about validation on this field, since that greatly expands the scope of this. And we mainly care about isolating Quantity since that's what's required for scaling.

[hay-kot]

@abhchand, I think your proposal makes sense. The only thing I'd add is an alias so you can merge multiple ingredients on alias in the shopping list. I'd imagine this is a 2-4 day project to get everything all lined up and make sure the import/export work across the board. Given that I'd probably never use any of these features myself, it's not super high on my list. It's probably the last thing I'll get to before we hit 1.0.

I'm happy to help anyone who wants to take this on, otherwise it'll be a while.

[hay-kot]

I've Created a task to track progress on this over at #507

I've already started by creating the database layer required and a quick mockup of the routes we'll need to get started. Hopefully this will make it easier for others to jump in an help flesh out all the UI and backend changes we'll need. For coordinating work on this please use the new issue 👍