Integration quality scale about architecture

[kellerza]

The metrics proposed are measurable. Good UX is subjective, even more so since we might not use or even have access to certain devices.

A "Level of integration" badge in the documentation could be a nice addition, with Bronze/Silver/Gold badges. And maybe a native HASS blue badge for things like automation

Mature can also be open for interpretation, since complex code might not mean bug free (tests do help, but integrations can be temperamental!)

[balloob]

I went ahead and opened a PR for an initial version. It's based on the proposal in this issue, the comments and recently introduced functionality.

I switched to using silver, gold and platinum. The reason I don't do bronze - gold is because the 2nd tier is already a very good experience.

Preview Integration Quality Scale

[OverloadUT]

Those badges would only be useful to developers to understand what is missing, which I don't think provides a ton of value. The simple Silver, Gold, Platinum provides immediate value to the end-user, and provides a good signal to developers about which components they use have room for improvement. Great way for new developers to find something to work on to learn the ropes too.

For me, I plan on creating GHIs like "ISY994 Road to Silver" which checkboxes for each requirement. Then I'll have a "Road to Gold" GHI when that one is done.

[balloob]

I like maturity, although what I like about the word "quality" is that it is clear what it means to anyone seeing a clear . And "full" definitely sounds better than "expert"

I think that this categorization will be used in various ways:

• As a guideline to developers for possible improvements to integrations
• As a guideline to the user on how good an integration is
• As a priority which integrations to suggest in the onboarding flow that I'm planning once config entries is up and running.

[fabaff]

To get started we should keep it simple. Meaning go with three grades and refine it then or make a scale according to the fulfilled requirements.

It took me hours to get the IoT classes for the available platforms/components back then. And now we have at least doubled the amount of integrations. It would already require pretty much a review of every single integration we have just to get it into the classification without doing a license review.. Sure, some part could be automated but not all.

For the users it would only add benefit if a lot of integrations are covered otherwise it's some they will learn to ignore.

[fabaff]

I guess that we have the first task for Hacktoberfest: Add Quality Scale 😉

IoT class is already in place and "Maintained by manufacturer" is a very short list.

[Landrash]

Suggest we call it Component Integration Scale. Removes the word quality but still keeps the self explained name.

Some adaptation should be made for "native" components to indicate to indicate that they are exempt (If they are) from this.

[balloob]

I would expect this scale to only apply to integrations of third party services/devices. It would not apply to something like the automation component.

[amelchio]

Looks good to me. How do you plan to use this categorization?

I am thinking of a word like "maturity". Also, "full" rather than "expert". I expect the requirements to change over time so using an absolute term is not as bad as with USB full-speed.

[NovapaX]

I think this mixes two things:

• The quality and complexity of the code (e.g. having tests, being async, ...)
• The experience for the end user (config.yaml, discovery, config_entries, entity_registry)

Both are important, but not necessarily correlated
Bad code can still give a proper user experience (until it break because of missing tests...).
Expert level code, without auto-discovery or config_entries and a huge complex config-schema will still give mediocre user-experience imo.

[balloob]

• Tests are very important for the user experience because it guarantees that the code works and keeps working.
• Async means that the code is faster, more responsive and make it more difficult to take Home Assistant down. And a stable system too is part of the user experience.

So I think that having these be part of the quality requirements is very important.

[NovapaX]

Yes, that is true. Both should be part of the score, because both are important.
But a component not being very advanced in code should not get a lower value if it does provide a very good UX.

So I think you should start with the maximum score/level. And you substract points for bad code, not having tests when it should have, not having a nice setup experience (complex config) or other not-nice things.
That way you don't punish simple components for being simple.
You see what I mean?

But anyway, this is a great idea to give users some feedback on what to expect, and devs on what to improve.

[OverloadUT]

I'm late here, but I love this idea. I think it would be similar to the "IoT class" we have listed on a lot of platforms. I actually use that IoT Class rating as one of my main decision makers when buying a new product. Pairing the unicorn "local push" badge with the "expert quality integration" badge would be a great way for users to know that a product they are thinking about buying has best-in-class integration with Hass.

(Edit: I bring up the IoT Class badge because I think it might make sense to display both things in the same place)

[stephenmg12]

It would also be helpful to know how stable an integration is. Is it an official documented API or did we reverse engineer the app? An example would be Lutron breaking things.

[OverloadUT]

That gets a tad complicated because officially we're using third party libraries to do all of the I/O.

I do like the idea though. It definitely supports my "decide which products to buy" use-case.

[awarecan]

Platinum badge looks cheaper than gold one 😄

How about
🏆💎👑

[awarecan]

Besides a quality scale, I also suggest that we have several other badges/labels we can assign to a component

• official for which got official integration certification or sponsored, such as tuya
• official api for which used official open API/SDK for the integration, such as Nest and Hue
• black hat for which used unofficial method to integrate, such as Google Hangouts?
• license checked for which it and all its libs are compatible with Apache 2 license
• experimental for which still work in progress, such as GeoLocation in last release

We can also have some badges for single featue to give devs a little accomplishment, such as

• async
• entities_config
  etc.

[dgomes]

I think @awarecan idea is better then the scale.

We could just have several badges and the scale becomes dynamic as a result of the number of badges collected by each component.

The badge approach has the benefit that developers can tackle badges one at a time.

[awarecan]

So it will like a matrix

Scale	Badge
No rank	🏛️official, 📜api, 🎩 black hat, ✔️Apache 2 license compatible, 🚀experimental
No score	basic, configuraion.yaml
🥈	scan_interval, paltform_not_ready, auth_handle, internet_outage, device_outage, unavailable, unique_id
🥇	config_ui, config_entry, device_info, discoverable, l10n, tests, code_owner
💎	parallel_updates, config_unload, async_all, aiohttp_session

[fabaff]

To be honest I'm a bit skeptical about the badges or the message they could send. If something is official then it doesn't mean that it's better than something unofficial or that the community thing is worse than the SDK of the vendor.

[balloob]

@OverloadUT is right, this scale is aimed at the end-user. The more levels we add, the less they will understand. Silver/Gold/Platinum is very straight forward.

I think that a "Maintained by manufacturer" badge would be a good idea. It will allow companies to distinguish themselves and help improve our system in the process.

That way we will have three things for each integrations:

• Quality Scale
• IoT class
• Maintained by manufacturer

[balloob]

Yeah it will be fairly easy to spot gold/platinum. It's limited to https://github.com/home-assistant/home-assistant/blob/dev/homeassistant/config_entries.py#L138