openvoiceos / ovos-core Goto Github PK

View Code? Open in Web Editor NEW

This project forked from mycroftai/mycroft-core

106.0 8.0 15.0 135.29 MB

OpenVoiceOS Core, the FOSS Artificial Intelligence platform.

Home Page: https://openvoiceos.org

License: Apache License 2.0

Python 81.49% QML 17.89% Shell 0.45% JavaScript 0.16%

hacktoberfest

ovos-core's Introduction

OVOS-core

OpenVoiceOS is an open source platform for smart speakers and other voice-centric devices.

Mycroft was a hackable, open source voice assistant by the now defunct MycroftAI. OpenVoiceOS continues that work and ovos-core (this repo) is the central component.

All Mycroft Skills and Plugins should work normally with OVOS-core, with the exception of Common Play and other media-related skills. Other Mycroft-based assistants are also believed, but not guaranteed, to be compatible.

The biggest difference between OVOS-core and Mycroft-core is that OVOS-core is fully modular. Furthermore, common components have been repackaged as plugins. That means it isn't just a great assistant on its own, but also a pretty small library!

Installing OVOS
Skills
Getting Involved
Links

Installing OVOS

If you have an existing system that you would like to install OVOS on, we strongly suggest the ovos-installer to install OVOS and its dependencies. The full assistant requires several repositories and the installer makes it easy to install them all at once.

If you would like to install OVOS on a Raspberry Pi, we suggest using the RaspOVOS image. This image is based on Raspberry Pi OS and includes OVOS and its dependencies running in a "headless" mode (no GUI). It is designed and optimized for a Raspberry Pi 3B, so on a 4 or higher its performance is even better.

If you would like to install OVOS on embedded hardware, we suggest using ovos-buildroot. This is a buildroot configuration that can be used to build a custom Linux distribution for embedded hardware. It includes OVOS and its dependencies, and is designed to be as small and efficient as possible.

You can find detailed documentation over at the community-docs or ovos-technical-manual

This repo can be installed standalone via pip install ovos-core, which will install the bare minimum components common to all services. This is useful for developers who want to build their own custom voice assistant. For more details, please see the community docs.

Skills

OVOS is nothing without skills. There are a handful of default skills, but most need to be installed explicitly. OVOS skills are all pip-installable, and can be found on PyPI or by browsing the OVOS organization on GitHub. Most classic Mycroft skills will also work on OVOS.

Please share your own interesting work!

Getting Involved

This is an open source project. We would love your help. We have prepared a contributing guide to help you get started.

The easiest way for anyone to contribute is to help with translations! You can help without any programming knowledge via the translation portal

If this is your first PR, or you're not sure where to get started, say hi in OpenVoiceOS Chat and a team member would be happy to mentor you. Join the Discussions for questions and answers.

Credits

The OpenVoiceOS team thanks the following entities (in addition to MycroftAI) for making certain code and/or manpower resources available to us:

NeonGecko
KDE / Blue Systems

ovos-core's People

Contributors

Stargazers

Watchers

Forkers

neonjarbas smartgic builderjer andlo goldyfruit 1maxnet1 emphasize mikejgray lrq3000 tarikki voronaivan93 lbt g-henderson suvanbanerjee krawc

ovos-core's Issues

Failed to create padatious handlers on Mac OS

2023-09-14 12:05:04.428 - skills - ovos_core.intent_services:__init__:62 - ERROR - Failed to create padatious handlers ('intent_cache')
Traceback (most recent call last):
  File "/Users/goldyfruit/Virtualenvs/ovos/lib/python3.11/site-packages/ovos_core/intent_services/__init__.py", line 60, in __init__
    self.padatious_service = PadatiousService(bus, config['padatious'])
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/goldyfruit/Virtualenvs/ovos/lib/python3.11/site-packages/ovos_core/intent_services/padatious_service.py", line 161, in __init__
    intent_cache = expanduser(self.padatious_config['intent_cache'])
                              ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
KeyError: 'intent_cache'

TODO - metapackage automation

ovos-core is a minimal install, just the base library
there are metapackages that require ovos-core[XXX] and introduce additional dependencies
every time a PR is merged in core the min version should be bumped in those repos
this also triggers automated docker builds

a github action should target these repos:

note: PHAL replaces mycroft enclosure and has it's own release cycle

port - resource file handling utils

potential commit of interest in mark2/qa branch - MycroftAI@b9f814a

roadmap - 0.0.7

what is needed for release 0.0.7

nice to have:

voice service has multiple errors, DeviceAPI and Hotword engine

2022-10-18 06:51:11.891 - voice - ovos_config.models:reload:190 - ERROR - Exception fetching remote configuration: 'DeviceApi' object has no attribute '_identity_file'

2022-10-18 10:23:05.567 - voice - mycroft.listener:create_hotword_engines:395 - ERROR - Failed to load hotword: lang

port - mk2 gui refactor

potential commits of interest in mark2/qa branch

move GUI specific code out from the base enclosure class and refactor - MycroftAI@235bd0c

Initial implementation of an IdleDisplaySkill - MycroftAI@68f870d

new SkillGUI helper method to replace the page - MycroftAI@3c594fb

When I ran the ovos-cli-client I got a trace in core

Running ovos-cli-client --simple and press Enter generated a trace into the core logs.

2023-04-29 18:12:46.597 - skills - ovos_core.intent_services:_normalize_all_utterances:77 - DEBUG - Utterances: [('',)]
2023-04-29 18:12:46.601 - skills - ovos_core.intent_services.padatious_service:_match_level:79 - INFO - Jurebes Matching confidence > 0.95
2023-04-29 18:12:46.871 - skills - ovos_core.intent_services.padatious_service:calc_intent:302 - INFO -  IntentMatch(intent_name='ovos.common_play:resume.intent', confidence=0.6349888471087107, entities={})
2023-04-29 18:12:46.873 - skills - ovos_core.intent_services.padatious_service:_match_level:91 - INFO - low confidence jurebes match, discarding result!
2023-04-29 18:12:46.878 - skills - ovos_core.intent_services.fallback_service:_fallback_range:172 - DEBUG - checking for FallbackSkillsV1
2023-04-29 18:12:46.880 - skills - ovos_core.intent_services:handle_utterance:276 - ERROR - string index out of range
Traceback (most recent call last):
  File "/home/ovos/.venv/lib/python3.11/site-packages/ovos_core/intent_services/__init__.py", line 250, in handle_utterance
    match = match_func(combined, lang, message)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ovos/.venv/lib/python3.11/site-packages/ovos_core/intent_services/fallback_service.py", line 187, in high_prio
    return self._fallback_range(utterances, lang, message,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ovos/.venv/lib/python3.11/site-packages/ovos_core/intent_services/fallback_service.py", line 175, in _fallback_range
    data={'utterance': utterances[0][0],
                       ~~~~~~~~~~~~~^^^
IndexError: string index out of range

unittests/ skill installer class

    I've confirmed at least the happy path for each new bus message works. 
    Only thing missing are some tests. Also, is it deliberate that core doesn't test Python > 3.9?

Originally posted by @mikejgray in #347 (comment)

Log level set within system conf not being honoured

mycroft.conf within etc states;

"log_level": "INFO"

However DEBUG messages are shown in all the logs

is_alive() and all_loaded() service status enquiries seems broken

Tried the following on latest ovos-core dev branch git install:

For is_alive():

Messagebus Service:
Client Sending -> "mycroft.messagebus.is_alive" / " "mycroft.bus.is_alive" : Expected Response -> "mycroft.messagebus.is_alive.response" / "mycroft.bus.is_alive.response" : Got Response: None

Skills Service:
Client Sending -> "mycroft.skills.is_alive" : Expected Response -> "mycroft.skills.is_alive.response" : Expected Single Response, Got Multiple Responses one after the other for a single request, received in following order:

mycroft.skills.is_alive.response {"status":true}
mycroft.skills.is_alive.response {"status":false}

For all_loaded():

Client Sends: "mycroft.skills.all_loaded" : Expected Response -> "mycroft.skills.all_loaded.response" : Expected Single Response, Got Multiple Responses one after the other for a single request, received in the following order:

mycroft.skills.all_loaded.response {"status":true}
mycroft.skills.all_loaded.response {"status":false}

_ensure_device_is_paired() is called before load_priority skills it should be called after

skill_manager.start() which now calls load_priority() skills is called after device_primer.prepare_device() inside which _ensure_device_is_paired() is run. _ensure_device_is_paired() emits the "mycroft.not.paired" message which is to be caught by the ovos-setup skill, but since ovos-setup skill is a priority skill its never loaded in time to catch the message, therefor the pairing process never kicks in.

ignoring stt_lang, en is not in enabled languages: ['fr-fr', 'fr-fr', 'en-us']

When running the ovos-audio-transformer-plugin-fasterwhisper I can see this warning:

ovos-core-86bc9bc566-wkpxm ovos-core 2023-12-07 17:35:36.965 - skills - ovos_core.intent_services:disambiguate_lang:196 - WARNING - ignoring stt_lang, en is not in enabled languages: ['fr-fr', 'fr-fr', 'en-us']

'Session' object has no attribute 'valid_languages'

When running the ovos-audio-transformer-plugin-fasterwhisper plugin I got this error:

ovos-core-86bc9bc566-vjn5j ovos-core 2023-12-07 16:54:08.030 - skills - ovos_core.intent_services:handle_utterance:343 - ERROR - 'Session' object has no attribute 'valid_languages'
ovos-core-86bc9bc566-vjn5j ovos-core Traceback (most recent call last):
ovos-core-86bc9bc566-vjn5j ovos-core   File "/home/ovos/.venv/lib/python3.11/site-packages/ovos_core/intent_services/__init__.py", line 289, in handle_utterance
ovos-core-86bc9bc566-vjn5j ovos-core     lang = self.disambiguate_lang(message)
ovos-core-86bc9bc566-vjn5j ovos-core            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ovos-core-86bc9bc566-vjn5j ovos-core   File "/home/ovos/.venv/lib/python3.11/site-packages/ovos_core/intent_services/__init__.py", line 191, in disambiguate_lang
ovos-core-86bc9bc566-vjn5j ovos-core     if v in sess.valid_languages:
ovos-core-86bc9bc566-vjn5j ovos-core             ^^^^^^^^^^^^^^^^^^^^
ovos-core-86bc9bc566-vjn5j ovos-core AttributeError: 'Session' object has no attribute 'valid_languages'

[BUG] Classic Mycroft skill inits are not calling self.initialize()

Per findings here:
OpenVoiceOS/skill-ovos-wolfie#14

In at least one case, and possibly in the wikipedia skill too, a skill with a create_skill() function and a Skill self.initialize() method is not calling the self.initialize() method.

ovos@mark1-dev:~ $ pip list | grep ovos-core
ovos-core                            0.0.8a27

Not sure how far back this goes, but this is the version on the Mark 1 where I found the issue. Running RaspOvos image for Mark 1.

Unsupported LF language prevents intent handling

It appears that a message in a language not supported by LF raises an exception in the normalization step before intent processing. It seems like this should skip normalization and proceed to intent parsing, rather than raising an exception

2022-05-20 22:27:51.837 - neon_skill - mycroft.skills.intent_service:handle_utterance:338 - ERROR - Invalid input: ru
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/mycroft/skills/intent_service.py", line 291, in handle_utterance
    setup_locale(lang)  # set default lang
  File "/usr/local/lib/python3.8/site-packages/mycroft/configuration/locale.py", line 93, in setup_locale
    load_languages([lang_code, "en-us"])
  File "/usr/local/lib/python3.8/site-packages/mycroft/configuration/locale.py", line 82, in load_languages
    LF.load_languages(langs)
  File "/usr/local/lib/python3.8/site-packages/lingua_franca/internal.py", line 198, in load_languages
    load_language(lang)
  File "/usr/local/lib/python3.8/site-packages/lingua_franca/internal.py", line 187, in load_language
    _set_active_langs(__loaded_langs)
  File "/usr/local/lib/python3.8/site-packages/lingua_franca/internal.py", line 129, in _set_active_langs
    set_default_lang(get_full_lang_code(__loaded_langs[0]))
  File "/usr/local/lib/python3.8/site-packages/lingua_franca/internal.py", line 283, in set_default_lang
    _refresh_function_dict()
  File "/usr/local/lib/python3.8/site-packages/lingua_franca/internal.py", line 137, in _refresh_function_dict
    populate_localized_function_dict(mod, langs=__loaded_langs)
  File "/usr/local/lib/python3.8/site-packages/lingua_franca/internal.py", line 633, in populate_localized_function_dict
    primary_lang_code = get_primary_lang_code(lang_code)
  File "/usr/local/lib/python3.8/site-packages/lingua_franca/internal.py", line 300, in get_primary_lang_code
    lang = __get_primary_lang_code_deprecation_warning(lang)
  File "/usr/local/lib/python3.8/site-packages/lingua_franca/internal.py", line 340, in __get_primary_lang_code_deprecation_warning
    raise(ValueError("Invalid input: " + lang))
ValueError: Invalid input: ru

roadmap - 0.0.2

Done

In Progress

TODO

What else needs to happen?

Copyleft detected - list of deps & util to detect them

All OVOS repos intend to be "universal donors", it should be possible to use every ovos repo without dragging a viral package

A new util i wrote revealed some problems with GPL dependencies that are incompatible

Additionally and just to be on safe side we also want to avoid LGPL due to reasons discussed elsewhere

viral packages in mycroft/ovos-core

'fann2': 'LGPL',
'chardet': 'LGPL',
'unidecode': 'GPL',
`python-vlc`: `LGPL`.
'pyxdg': 'GPL-2.0'

additional viral packages in ovos_workshop / OCP

'mutagen': 'GPL-2.0-or-later'

tagging @krisgesling because this is also a concern for mycroft

TODO list

proposal: OOVSkills

these are some notes i had in a gist, just an idea not something in roadmap, but i figured it would be cool to discuss

OOVSkills framework proposal

out-of-vocabulary intents skill class - OOVSkills
triggered before common_query framework
match an intent using something like https://github.com/GT4SD/zberta
- this gives us an intent classification without a registred handler
- "handle this question" becomes "handle this intent somehow"
- OOVSkills can use this info for a richer common_query experience
send the intent_tag + utterance to OOVSkills

sync lang resouces with mycroft-core

we need to port the following PRs:

Select a new STT engine for unit tests

Tests are failing because the Selene dependency appears to be removed from PyPI. Can we pick a different engine for unit tests and open a separate PR to implement that change?

Originally posted by @NeonDaniel in #288 (comment)

planned 0.0.7 breaks platforms and distribution packaging.

Issue: Requirements.txt now forces installation of skill-ovos-setup as a plugin in requirements.txt. Platforms already shipping their own setup skill that are git installed causes the skill process to crash due to this.

2023-02-10 14:03:04.934 - skills - mycroft.skills.skill_manager:_load_new_skills:503 - INFO - skill-ovos-setup.openvoiceos plugin will be replaced by a local version: /home/aix/.local/share/mycroft/skills/skill-ovos-setup.openvoiceos
2023-02-10 14:03:04.934 - skills - mycroft.skills.skill_manager:_unload_plugin_skill:591 - INFO - Unloading plugin skill: skill-ovos-setup.openvoiceos
2023-02-10 14:03:04.934 - skills - mycroft.skills.skill_manager:_unload_plugin_skill:591 - INFO - Unloading plugin skill: skill-ovos-setup.openvoiceos
2023-02-10 14:03:04.934 - skills - mycroft.skills.skill_manager:_unload_plugin_skill:591 - INFO - Unloading plugin skill: skill-ovos-setup.openvoiceos
2023-02-10 14:03:04.934 - skills - mycroft.skills.skill_manager:_unload_plugin_skill:591 - INFO - Unloading plugin skill: skill-ovos-setup.openvoiceos
2023-02-10 14:03:04.936 - skills - mycroft.skills.skill_loader:load:399 - INFO - ATTEMPTING TO LOAD SKILL: skill-ovos-setup.openvoiceos
2023-02-10 14:03:04.937 - skills - mycroft.skills.skill_manager:run:453 - ERROR - Something really unexpected has occured and the skill manager loop safety harness was hit.
Traceback (most recent call last):
  File "/home/aix/ovos/ovos-core/mycroft/skills/skill_manager.py", line 449, in run
    self._load_new_skills()
  File "/home/aix/ovos/ovos-core/mycroft/skills/skill_manager.py", line 504, in _load_new_skills
    self._unload_plugin_skill(skill_id)
  File "/home/aix/ovos/ovos-core/mycroft/skills/skill_manager.py", line 598, in _unload_plugin_skill
    self.plugin_skills.pop(skill_id)
KeyError: 'skill-ovos-setup.openvoiceos'
2023-02-10 14:03:04.938 - skills - mycroft.skills.skill_loader:_load_skill_source:506 - ERROR - Failed to load skill-ovos-setup.openvoiceos due to a missing file.

Platforms shipping their own setup skill and platforms that don't require a setup skill are now forced to deal with this new plugin install, or need to hack packaging to fix requirements.txt
Skills should not be forced with basic requirements.txt, let distributions package their own skills and requirements for the specific platform
If platforms/distributions choose they can install the essential skills. Don't make setup skill a hard requirement for every platform out there.

Install confusion

Ive been trying to follow the docs and install procedure but its a bit of a mess. Could someone please explain how I install and run ovos-core? The documentation makes it sound like running the pip install will create a folder but I see no such thing?

Unit Tests

ovos-core needs some unit tests; here's a quick draft of what I think we need:

Implement GH Action for Coverage
Implement GH Action to run unit tests
Curate Mycroft tests and remove/rework any that don't run under ovos-core
Add unit tests per Coverage

roadmap - 0.0.8

documentation:

spec of bus messages https://github.com/OpenVoiceOS/message_spec
session
audio/utterance/metadata transformers
pipeline config
solver plugins
standalone skill launcher
ovos-docker https://openvoiceos.github.io/ovos-docker/
OpenVoiceOS/community-docs#36
OpenVoiceOS/ovos-technical-manual#16

unittests:

audio:

move all audio output handling to ovos-audio

core:

GUI:

docker:

https://github.com/OpenVoiceOS/ovos-docker

opm:

misc:

Bug - start-mycroft.sh debug is not handled.

The start-mycroft.sh debug argument isn't handled. The readme references it and ./start-mycroft.sh help as well. Should it be removed from the docs or has it been deleted by accident?

Add padatious to extra-skills.txt

When running only the skills service, minimal.txt and extra-skills.txt are required but it still miss the padatious library which returns an error.

2023-03-27 22:14:16.376 - skills - mycroft.skills.intent_services.padatious_service:__init__:150 - ERROR - Padatious not installed. Falling back to pure regex alternative

roadmap - 0.0.9

depends on

#274

what is needed for release 0.0.9

GUI:

Pipelines:

introduce the concept of intent pipeline #336
PipelinePlugin base class - OpenVoiceOS/ovos-plugin-manager#177
standardize bus api for intent service OpenVoiceOS/ovos-utils#193
migrate skills to new intent service api OpenVoiceOS/OVOS-workshop#132
persona pipeline plugin
allow persona to select pipeline matchers (via Session) #352
new pipeline plugin #273
- new pipeline plugin, use LLM to select final intent to trigger

Intent Matching:

OpenVoiceOS/OVOS-workshop#12
new pipeline plugin #310
#376

OCP:

Persona:

OpenVoiceOS/ovos-persona#4

CommonIOT:

VUI
bus api
GUI (migrated from HA PHAL plugin)
reference plugins
- migrate HA phal plugin to a IOT plugin
- https://github.com/OpenVoiceOS/ovos-iot-plugin-lan
- https://github.com/OpenVoiceOS/ovos-iot-plugin-kasa
- https://github.com/OpenVoiceOS/ovos-iot-plugin-bluetooth
expose ovos itself to other IOT frameworks https://github.com/OpenVoiceOS/ovos-PHAL-sensors

PHAL:

OpenVoiceOS/ovos-PHAL#12

Backend:

OpenVoiceOS/ovos-backend-client#11

tflite not available in py3.10

precise lite plugin will fail to install in py3.10, build_all workflow will fail in PRs until tflite_runtime becomes available

roadmap 0.1.0 - out of beta!

WIP - I'm still editing and finishing this issue, more to come under State of Things sections

OVOS Pipeline

mycroft-core -> OpenVoiceOS

one common issue with mycroft-core was that you could not run a piece of the stack without importing everything else, in a server you would be forced to run the audio stack even if you are just powering a chatbot, worse, in docker setups this dragged a lot of bloat

initially we just organized imports internally to clean this up, but eventually all the pieces ended up working fully standalone and were split into their own packages

mycroft.api + mycroft.identity + mycroft.metrics -> ovos-backend-client
mycroft.skills.mycroft_skill -> ovos-workshop
mycroft.util -> ovos-utils
mycroft.lock -> ovos_utils.process_utils
mycroft.util.combo_lock -> combo_lock
mycroft.util.parse + mycroft.util.format -> lingua-franca -> ovos-lingua-franca
mycroft.client.text -> ovos-cli-client
mycroft.client.speech -> ovos-listener
mycroft.client.enclosure -> ovos-PHAL + ovos-gui
mycroft.audio -> ovos-audio-plugin-ocp + ovos-audio
mycroft.messagebus.client -> mycroft-bus-client
mycroft.messagebus.service -> ovos-messagebus
mycroft.session -> ovos-bus-client.session
mycroft.configuration -> ovos-config
mycroft.tts -> ovos-tts-plugin-XXX + ovos-plugin-manager
mycroft.stt -> ovos-stt-plugin-XXX + ovos-plugin-manager
mycroft.audio.services -> ovos-audio-plugin-XXX + ovos-plugin-manager
mycroft.client.speech.hotword_factory -> ovos-ww-plugin-XXX + ovos-plugin-manager
mycroft-skills-manager -> ovos-skills-manager

State of Things

mycroft.deprecated

as ovos-core evolved we deprecated functionality present in mycroft-core, old classes providing either dummy behavior or warning logs has been moved to mycroft.deprecated module, this is dead code that remains in core only to avoid syntax errors in downstream projects, anything that was a public api in mycroft-core remains available in ovos-core

one of the removed things was the integration with msm which forced default skills and locked core to Mycroft marketplace

OpenVoiceOS implements a standalone skill installer utility that supports several marketplaces

mycroft-skills-manager -> ovos-skills-manager

ovos-utils

at first ovos_utils provided generic utilities to interact with mycroft-core, over time more and more pieces from mycroft.util moved there as they started being needed across ovos ecosystem, ovos_utils is a shared library across most of our repositories and by itself is enough to write simple scripts to interact with OpenVoiceOS

currently ovos-core mostly contains unused utils under mycroft.util for legacy reasons, those should be fully deprecated or moved to ovos_utils

combo_lock was useful enough to become it's own package https://github.com/forslund/combo-lock

mycroft.util -> ovos_utils
mycroft.util.combo_lock -> combo_lock
mycroft.lock -> ovos_utils.process_utils

ovos-plugin-manager

mycroft-core initially hardcoded STT/TTS/WW options, a plugin mechanism was developed but the hardcoded plugins were never migrated, OpenVoiceOS created plugins for every "native" implementation and moved all base classes and plugin loading logic to ovos-plugin-manager

mycroft.tts -> ovos-tts-plugin-XXX + ovos-plugin-manager
mycroft.stt -> ovos-stt-plugin-XXX + ovos-plugin-manager
mycroft.audio.services -> ovos-audio-plugin-XXX + ovos-plugin-manager
mycroft.client.speech.hotword_factory -> ovos-ww-plugin-XXX + ovos-plugin-manager

ovos-backend-client

as alternative backends / no backend support was added to ovos-core the mycroft.api module ended up migrated to a more generic ovos-backend-client, nowadays with Selene in risk of going down at any time this packages is the defacto client library for several OVOS backends, it can be used in any project to interact with several backends or directly with web apis

mycroft.api + mycroft.identity + mycroft.metrics -> ovos-backend-client

ovos-workshop

ovos-workshop is where skill base classes live, it started as a way to bring bug fixes and features to mycroft-core via monkey patching, it evolved to provide skill base classes and a mechanism to develop standalone applications that interact with ovos-core

mycroft.skills.mycroft_skill -> ovos-workshop

ovos-config

as configuration handling started being needed in more packages and stopped referring to a single file format or location it moved into it's own shared library, it became the key component for XDG standards adoption and downstream voice assistant compatibility and continues to expand to this day with a cli interface recently added

once we fully move mycroft.conf from ovos-core to ovos-config it will be a truly standalone package

mycroft.configuration -> ovos-config

mycroft.client

the mycroft.client was a target for removal from core since the early mycroft days, all components there lend themselves to replacement or can be considered fully optional

the debug cli utility built on curses was moved into ovos-cli-client, this component had a totally different release cycle from core and was the only one built on top of curses, some of the code in this module was among the code not touched for longer, this indicates this is a mature optional component

the mycroft.client.enclosure module was among the most controversial, as it directly tied to core to specific platforms recognized by MycroftOrg, it was very difficult to add platform support to core

today this module has been deprecated and the PHAL plugin framework is used across OpenVoiceOS instead

one of the things this module did was launch the ovos-gui service as part of the enclosure implementations, in OpenVoiceOS this was made it's own service and is the source of truth for the mycroft-gui protocol implementation, it has been extended to support platform specific gui extensions

mycroft.client.text -> ovos-cli-client
mycroft.client.enclosure -> ovos-PHAL + mycroft.gui
mycroft.gui -> ovos-gui

ovos-listener

the speech client was one of the pieces OpenVoiceOS improved the most, it supports multiple wake words, VAD plugins and listening modes. It is a standalone component since core can be used via text only, such as in server setups

mycroft.client.speech -> mycroft.listener
mycroft.listener -> ovos-listener

ovos-bus

mycroft started migrating the mycroft bus client implementation to it's own repo, but never finished migrating mycroft-core to use it

the messagebus service is a simple websocket server, as proof of concept we developed an equivalent service in C++ to test performance, the reference python implementation is also available

mycroft.messagebus.service -> ovos-messagebus / ovos-bus-server (c++)
mycroft.messagebus.client -> mycroft-bus-client

ovos-bus-client

a central piece of ovos-core design is that the message.context is passed along the pipeline and can contain additional metadata, this is a key feature for supporting multiple users and things such as a different language per query

ovos-bus-client is an extension (not a replacement) of mycroft-bus-client, it adds utils to interact with bus messages and it's metadata, the mycroft.session module has been extended and migrated to this package. this is the main design conflict between ovos-core and mycroft-dinkum, where in core the session data comes from bus messages, in dinkum this is controlled by the intent manager

mycroft.session -> ovos-bus-client.session

ovos-audio

with a screen come video capabilities, the OCP framework was created to handle all kinds of playback, it uses the old mycroft audio service internally and replaces it and the mycroft common play framework

mycroft.audio -> ovos-audio-plugin-ocp + ovos-audio

ovos-core

ovos-core is now a minimal library providing the brains of OpenVoiceOS, it is the piece that understand utterances and triggers intents, it can load skills or interface with external applications

it only depends on the messagebus which can be replaced with FakeBus for test frameworks or even with something like HiveMind. OpenVoiceOS is now several independent frameworks

TODO

Document usage

Somebody should take #4 and continue the job, adding instructions for starting OVOS-Core after a pip or a git install. I am not familiar enough with the changes to finish it myself, and we've gotten questions ("I installed it now what?")

Generally speaking, the practice of making changes and then putting up a TODO for somebody to document them needs to stop. It's hard enough to document your own code after a break. Trying to document somebody else's code is a nightmare. If somebody is less than confident in their English, write your docs and then ping me or Daniel to clean them up.

Failure when no fallback stt set

If there is no fallback_module, the mycroft-speech.service fails to load.

I believe it has something to do with this:

ovos-core/mycroft/client/speech/listener.py

Lines 357 to 359 in 883bdac

    
           if not self.fallback_stt: 
        
               clazz = self.get_fallback_stt() 
        
               self.fallback_stt = clazz()

If it was changed to this:

    359  -   self.fallback_stt = clazz()
    359  +  if clazz:
    360  +      self.fallback_stt = clazz()

I think it may work

0.0.7 Installation Errors

A but in pyyaml means that the current stable ovos-core will fail during installation. There are two workarounds:

Install pyyaml 5.4 manually first:

pip install  "cython<3.0.0"
pip install --no-build-isolation pyyaml~=5.4
pip install ovos-core[skills]  # or `pip install ovos-core[skills_lgpl]`

Skip the skills (or skills_lgpl extras) and install them manually:

pip install ovos-core
pip install adapt-parser~=0.5 padacioso~=0.1 ovos-lingua-franca>=0.4.7 ovos-phal-connectivity-events~=0.0.1
# If using Padatious (skills_lgpl extras), also:
pip install padatious~=0.4.8 fann2==1.0.7

port/fix/ intent name colision

port MycroftAI#2921

unittests hang forever test_event_container.py

OpenVoiceOS/ovos-bus-client@6eb7c56 introduced some issue where unittests hang forever test/unittests/skills/test_event_container.py .... [ 34%]

Originally posted by @JarbasAl in #336 (comment)

roadmap - 0.0.6

what is needed for release 0.0.6

Nice to have:

#232
bump version for https://github.com/Uberi/speech_recognition
bump version for https://github.com/cmusphinx/pocketsphinx in https://github.com/OpenVoiceOS/ovos-ww-plugin-pocketsphinx
#163
OpenVoiceOS/skill-ovos-setup#64
OpenVoiceOS/ovos-backend-client#11
OpenVoiceOS/ovos-lingua-franca#50
OpenVoiceOS/ovos-lingua-franca#51

current changelog: V0.0.5...dev

Implemented enhancements:

Add skills-essential optional dependencies #266 (NeonDaniel)
sync lang resouces with mycroft-core #199
Lng: Add cs-cz noise words #260 (NeonDaniel)
Refactor/skill class from workshop #246 (JarbasAl)
add "last.voc" resource #248 (emphasize)
Add display setting for menu labels #244 (AIIX)
relax lingua_franca version #242 (builderjer)
Reorg shutdown to prevent logged exceptions in skill shutdown #240 (NeonDaniel)
Add config option to retry mic init on start (default True) #238 (NeonDaniel)
page background color support for show image and show animated image #236 (AIIX)
add factory reset ui #231 (AIIX)

Fixed bugs:

Actually use the advertised audio device and sample rate with the audio test. #265 (gmsoft-tuxicoman)
Bump minimum plugin version for ocp fix #259 (NeonDaniel)
Prevent logging errors when a blacklisted skill is handled as expected #257 (NeonDaniel)
Ignore setuptools pip-audit failure #253 (NeonDaniel)
Prevent exceptions if a user doesn't reply to an ask_yesno prompt #252 (NeonDaniel)
Fix system url web view #251 (AIIX)
voice service has multiple errors, DeviceAPI and Hotword engine #216
Update factory reset message #247 (NeonDaniel)
port/fix/ intent name colision #233
Handle multiple intents with the same name (MycroftAI#2921) #235 (JarbasAl)
Fix display settings ui #237 (AIIX)
Fix show_pages index api #234 (AIIX)

msm disabled by default

we should make all default skills pip installable after #9

msm should then be disabled by default + default skills added to requirements.txt

this allows us to pin skill version appropriate for the ovos-core version and avoids conflicts between ovos/mycroft skills, it is currently confusing for users when we tell them to blacklist official skills and install our versions manually

we can also look into replacing msm with osm (let the user choose), but the integration in core is only used to download default skills, so we either include a deeper integration (appstore sync etc) or there is no point in adding osm either.

There is also some decoupling needed between msm and selene skills manifest upload before we can fully remove msm

skill migration guide

OVOS Skill Replacements

also see Adopt a project discussion and https://github.com/OVOSHatchery

Old Mycroft

migration guide

GUI skills

https://github.com/MycroftAI/skill-homescreen -> https://github.com/OpenVoiceOS/skill-ovos-homescreen or https://github.com/OpenVoiceOS/skill-homescreen-lite
https://github.com/MycroftAI/skill-camera -> https://github.com/OpenVoiceOS/skill-ovos-qml-camera
https://github.com/MycroftAI/skill-mycroft-radio -> several skill options for OCP https://github.com/OpenVoiceOS/skill-ovos-tunein
https://github.com/MycroftAI/skill-music-demo -> several skill options for OCP

Community

to be updated / forked / rewritten officially by OVOS

Not reading personal skill config

I don't think this actually belongs here, but this is where I am testing now.

When I change the settings in ~/.config/mycroft/skills/<skill_name>, it is never read. This seems to be happening on all of the skills, not just this one.

When I use the settings.json file which is created with pip install, it seems to work fine.

Is this expected behavior? Should I start creating a settings.json in all skill folders?

Mismatched converse event names

The converse_service and mycroft_skill modules have mismatched event names for handling converse:

ovos-core/mycroft/skills/intent_services/converse_service.py

Line 231 in 80db035

converse_msg = message.reply("skill.converse.request",

https://github.com/OpenVoiceOS/ovos-core/blob/dev/mycroft/skills/mycroft_skill/mycroft_skill.py#L523

Do we want to normalize to event names with or without "mycroft"? My inclination is to go without except for backwards compat

Unable to blacklist skills or change the fallback priority

I'm trying to blacklist the skill-ovos-fallback-unknown.openvoiceos but ovos-core still use it:

2023-12-21 17:52:19.936 - skills - ovos_bus_client.session:update:546 - DEBUG - replacing default session with: {'active_skills': [['skill-ovos-fallback-unknown.openvoiceos', 1703199108.4514954]], 'utterance_states': {}, 'session_id': 'default', 'lang': 'en-us', 'context': {'timeout': 120, 'frame_stack': []}, 'site_id': 'unknown', 'pipeline': ['converse', 'padatious_high', 'adapt', 'common_qa', 'fallback_high', 'padatious_medium', 'fallback_medium', 'padatious_low', 'fallback_low'], 'stt': {'plugin_id': 'ovos-stt-plugin-server', 'config': {'url': 'http://stt.smartgic.local/fasterwhisper/stt'}}, 'tts': {'plugin_id': 'ovos-tts-plugin-server', 'config': {'host': 'http://tts.smartgic.local/piper', 'voice': 'ryan-high'}}}

Here is the configuration:

{
  "log_level": "DEBUG",
  "logs": {
    "path": "stdout"
  },
  "confirm_listening": false,
  "play_wav_cmdline": "aplay %1",
  "play_mp3_cmdline": "pw-play %1",
  "lang": "en-us",
  "listener": {
    "record_wake_words": true,
    "save_utterances": true,
    "remove_silence": true,
    "VAD": {
      "module": "ovos-vad-plugin-silero"
    }
  },
  "tts": {
    "module": "ovos-tts-plugin-server",
    "ovos-tts-plugin-server": {
      "host": "http://tts.smartgic.local/piper",
      "voice": "ryan-high"
    }
  },
  "stt": {
    "module": "ovos-stt-plugin-server",
    "ovos-stt-plugin-server": {
      "url": "http://stt.smartgic.local/fasterwhisper/stt"
    }
  },
  "skills": {
    "blacklisted_skills": [
      "skill-ovos-fallback-unknown.openvoiceos"
    ],
    "fallbacks": {
      "fallback_priorities": {
        "skill-ovos-fallback-chatgpt.openvoiceos": 10,
        "skill-ovos-fallback-unknown.openvoiceos": 20
      }
    }
  }
}

[BUG] all fallback answer

Describe the bug
If you install skill-ovos-fallback-chatgpt along another fallback skill, both fallback will answer

To Reproduce
Steps to reproduce the behavior:

Install skill-ovos-fallback-unknown
Install skill-ovos-fallback-chatgpt
Ask something weird (it happen a lot when not using English)
Both fallback skills answer

Expected behavior
The first fallback that answer should close the dialogue.

Additional context
Chatgpt takes a long time to answer, this causes a time out and makes fallback service go to next skill, but the fallback that timed out is not stopped

Enable GitHub Discussion.

My rationale is available at 8go/matrix-commander#68 (comment).

More specifically, I don't see any forum for this project, and https://app.element.io/#/room/#openvoiceos:matrix.org isn't a replacement for one. Would you consider enabling Discussions so that I can ask questions without creating new issues for them? https://github.com/OpenVoiceOS/ovos-core#links just links to https://community.mycroft.ai/.

Persona Initiative - Long Live the LLM

Give OpenVoiceOS some sass with Persona!

Phrases not explicitly handled by other skills will be run by Persona, so nearly every interaction will have some response. But be warned, OpenVoiceOS might become a bit obnoxious...

Large language models are having their Stable Diffusion moment. The next few months will be filled with new useful, and potentially controversial applications, pushing incumbents and startups to innovate once again.

The last weeks were filled with exciting announcements for open-source LLMs:

Meta released LLaMA - an LLM collection of models ranging from 7B to 65B parameters.
Stanford researchers released Alpaca, an instruction fine-tuned model based on Meta’s 7B LLaMA behaving similarly to OpenAI’s powerful GPT Davinci model.
Instruction fine-tuning is powerful for language models because it allows for targeted training on a specific task or domain, resulting in improved performance on that task and enabling transfer learning to other tasks.
LAION released OpenChatKit - a 20B parameter ChatGPT-like model under the Apache-2.0 license, meaning companies can incorporate it in their commercial products. They also released the underlying 43M instructions dataset so others can further fine-tune their LLMs.

Until recently, powerful LLMs were only accessible through APIs: OpenAI and now PaLM. Now the open-source community showed how the smallest LLaMA (7B) achieves GPT Davinci-like performance.

OpenVoiceOS is already adopting these technologies, the current stretch goal of our fundraiser aims to bring the Persona project to it's first stable release

https://www.gofundme.com/f/openvoiceos

Persona

Core personality
Skills personality
Spoken Answers API + "Solver" plugins
"Chatbot" Skills

Core personality

new persona service, using neon solver plugins https://github.com/OpenVoiceOS/ovos-persona
- bus api to manage personas
- default Persona settings under mycroft.conf
- allow persona to come from message.context, similarly to lang https://github.com/OpenVoiceOS/ovos-bus-client/blob/dev/ovos_bus_client/session.py#L24
OPM integration
- resource file based personas (json configs), https://github.com/OpenVoiceOS/ovos-persona/tree/dev/personas
- solver plugins can provide default personas automatically, similar to what i have been doing for TTS voices OpenVoiceOS/ovos-plugin-manager#131
- runtime scanning of available personas
- possible "marketplace" integration for user defined personas

in mycroft.conf

"persona": {
    "gender": "male",
    "attitudes": {
        "normal": 100,
        "funny": 70,
        "sarcastic": 10,
        "irritable": 0
    },
    "solvers": [
        "ovos-solver-plugin-llamacpp", 
        "ovos-solver-plugin-personagpt",
        "ovos-solver-failure-plugin"
   ],
   "ovos-solver-plugin-llamacpp": {
        "persona": "helpful, creative, clever, and very friendly"
    },
    "ovos-solver-plugin-personagpt":{
        "facts": [
            "i am a quiet engineer.",
            "i'm single and am looking for love."
            "sadly, i don't have any relatable hobbies.",
            "luckily, however, i am tall and athletic."
            "on friday nights, i watch re-runs of the simpsons alone."
       ]
   }
}

Skills personality

full details here OpenVoiceOS/OVOS-workshop#56

New file format, .jsonl

jsonl format info: https://jsonlines.org/

{"utterance": "stick the head out of the window and check it yourself", "attitude": "mean", "weight": 0.1}
{"utterance": "current weather is X", "attitude": "helpful", "weight": 0.9}

1 - load .jsonl file if it exists, else old .dialog file
2 - select an attitude based on weights defined in mycroft.conf / current active persona
3 - filter samples per attitudes
4 - select based on weights of .jsonl file

"Solver" plugins

these plugins automatically handle language support and auto translation, they provide the base for a "spoken answers" api

each persona loads several of those plugins sorted by priority, similarly to the fallback skills mechanism, this allows to check internet sources by order of reliability/functionality

create your own chatbot persona by choosing what plugins you install

self hosted

create solvers/persona server Docker endpoint like we have for other plugin classes
integrate persona with ovos-personal-backend
- new "spoken answers" api endpoint (default plugins: wikipedia + ddg + failure)
- deprecate wolfram_alpha integration, make endpoint call "spoken answers" api for selene compat
- one less external api key needed by default
- integrate with ovos-backend-client OpenVoiceOS/ovos-backend-client#28

"Chatbot" Skills

Unify all of these under a single skill-ovos-fallback-persona
- personality settings intents, eg "increase sarcasm by 20%"
- persona intents, eg, "enable story teller persona"
deprecate old chatbot skills
- just install solvers instead of skills for equivalent offline setup
- https://github.com/OpenVoiceOS/skill-ovos-fallback-dialogpt
- https://github.com/OpenVoiceOS/skill-ovos-fallback-chatgpt
- https://github.com/OpenVoiceOS/skill-ovos-fallback-aiml
- https://github.com/OpenVoiceOS/skill-ovos-fallback-rivescript
- https://github.com/OpenVoiceOS/skill-ovos-fallback-unknown

Inspiration

MycroftAI wanted to start an initiative called ‘Persona’ - a tool to help build distinct personalities for Mycroft. Think Sassy Mycroft, Polite Mycroft and so on.

the technology just wasn't there yet and the backend implementation was never finished or made public but the beta skill is still available (non-functional) https://github.com/MycroftAI/skill-fallback-persona

Fallback for online services

Dear all,
what you are doing is amazing. I have been thinking of the cases when you hit a fallback for STT and TTS services and I think it could be an improvement to change the way fallbacks are declared. At the moment we have:

module: "",
fallback-module: ""

this case is working fine, but it is limiting because, for example, it forces the user to choose to run the fallback as local or remote service when the case to hit a fallback can be multiple.

For example, I finish my tokens in my remote service TTS , so my main service fail and I hit the fallback that could be, for example, another account with the same service or another online service. However, I could also be in the situation where I have a very limited or no connection and thus my fallback should be an offline service.

A possible solution can be to define the services differently like an ordered list:

module: [
  - online_service1,
  - online_service2,
  - offline_service
]

With this configuration, if I do not have internet, the modules keep failing until starting the offline service.

just an idea.

Can't install ovos-core and ovos-bus-client

ovos-core has the requirement:
ovos_bus_client<0.0.6a1, >=0.0.5

However ovos-bus-client is at version 0.0.6a4 (https://github.com/OpenVoiceOS/ovos-bus-client/blob/dev/CHANGELOG.md).

It appears impossible to install both at the same time!

Numbers (digits) not permitted in padatious intent files

If you use in a padatious .intent file var names with a number inside intent isn't recognized. Example:
(enter | add) room {room1} won't work but (enter | add) room {room_one}.

fix docker build

rapidfuzz is failing to install for several releases now

[ENHANCEMENT] activate_skill and deactivate_skill don't return a response

When executing the intent_services.converse_service.ConverseService methods activate_skill() and deactivate_skill(), a bus message is emitted, but not in response to a message passed to it. For neon-mana-utils and the upcoming skills management API, this means it's only a one-way communication path to enabling/disabling a skill.

The enhancement could be to pass an optional message parameter to the methods above that allows them to respond directly to a request to enable/disable a skill.

[PYPI] Flesh out readme and add long_descrition

I guess its time to up the ante.

At least the "organs" of Ovos should get the special treatment
(should referencing each other)

port - mk2 activities

new module https://github.com/MycroftAI/mycroft-core/tree/mark-ii/qa/mycroft/activity

mk2 activities https://github.com/MycroftAI/mycroft-core/tree/mark-ii/qa/mycroft/client/enclosure/mark2/activities

skill class activity util MycroftAI@a80c25e#diff-0d891356bfaa5c316f1565e9f4e3da6e193d8878c4a23bb1853c7c4f019c4581

	if not self.fallback_stt:
	clazz = self.get_fallback_stt()
	self.fallback_stt = clazz()

openvoiceos / ovos-core Goto Github PK

ovos-core's Introduction

OVOS-core

Table of Contents

Installing OVOS

Skills

Getting Involved

Credits

Links

ovos-core's People

Contributors

Stargazers

Watchers

Forkers

ovos-core's Issues

Done

In Progress

TODO

TODO list

OOVSkills framework proposal

OVOS Pipeline

mycroft-core -> OpenVoiceOS

State of Things

mycroft.deprecated

ovos-utils

ovos-plugin-manager

ovos-backend-client

ovos-workshop

ovos-config

mycroft.client

ovos-listener

ovos-bus

ovos-bus-client

ovos-audio

ovos-core

TODO

OVOS Skill Replacements

Old Mycroft

Community

Persona

Core personality

Skills personality

"Solver" plugins

self hosted

"Chatbot" Skills

Inspiration

Recommend Projects

Recommend Topics

Recommend Org