dariowho / intents Goto Github PK

AS A chatbot user
IN ORDER TO have a clear response when my messages are not understood
I WANT chatbot to handle Fallback Intent
INSTEAD OF returning None or producing errors

AS A framework user
I WANT Entities to support regex definitions
SO THAT I can match common pattens such as codes, flight numbers etc.

Details

Most services support this natively

Test Cases

tbd

AS A Chatbot user
I WANT Agents to run Intent fulfillment procedures
SO THAT I can use Agents that actually do something (e.g. book a hotel room), and support more sophisticated conversation flows

Details

Webhook fulfillment calls are at the core of many prediction services. The proposal is to equip Intent subclasses with an optional fulfill() method (signature to be defined).

• When fulfill(...) is defined on an Intent class, DialogflowEsConnector will enable fulfillment on that intent in the exported Agent.
• If there is at least one Intent with fulfillment enabled, and DialogflowEsConnector is instantiated without webhook information, a warning should be issued
• The Connector interface should include abstract methods to parse fulfillment requests and provide responses accordingly
• DialogflowEsConnector should implement fulfillment-related methods
• A new Webhook interface should specify how webhooks are called and used
• FlaskWebhook should provide a baseline implementation of the Webhook interface. It shoud take a Connector instance as a parameter, and wrap fulfillment-related methods around a simple Flask service

AS A Agent developer
I WANT TO automatically initialize projects
SO THAT I save time and reduce the risk of mistakes

AS A chatbot developer
I WANT TO have a stricter check on Intent names
SO THAT Intents can guarantee unambiguous behavior across different platforms

Details

Different services have different rules on intent naming. For instance, Alexa doesn't allow dots (.) or double underscores (__), and generally there's a limit on the maximum length of a name.

We want to apply stricter validation on Intent names, so that Intents don't get mapped to the same name, which can happen at the moment (e.g. 'smalltalk.helloandsmalltalk_hello) would both be mapped to smalltalk_hellobyAlexaConnector`.

Also, in the future we may need custom intents to handle followups and generated behaviors. A syntax is needed to produce non-ambiguous generated intent names.

TODO: define rules, taking into account both the symbol set and maximum length.

Test cases

tbd

AS A Chatbot user
I WANT Agent to remember the way I referred to an Entity in my original utterance
SO THAT I can have better feedback on the Agent's understanding

Details

In Dialogflow ES we currently take the value of an Entity from the parameters dict in the response. This only contains the "canonical" value of tagged entities; that is, if user referred to the entity value with a synonim, this will not be included in the parameters dict. The "$param.original" value is only available in responses and in context parameters (note that these will be included in the response only if the intent spawns a context)

AS A chatbot user
I WANT TO receive responses without parameter placeholders or empty-valued parameters
SO THAT I feel that responses are natural and fluent

Details

This should be native in Dialogflow and seems to be implemented in Snips. Alexa to be checked.

Also, text choiches should be filtered to only include those that match the maximum number of parameters; this is a case where it would be useful:

{
  "intent": {
    "name": "lights.TurnOffLight",
    "parameters": {
      "light_entity": "light.hue_aurelle_cucina",
      "light_area": "kitchen"
    }
  },
  "confidence": 0.7150873095446443,
  "fulfillment": {
    "plaintext": "Switching off light.hue_aurelle_cucina",
    "payloads": {
      "default": [
        {
          "choices": [
            "Couldn't understand which light to turn off..",
            "Switching off light.hue_aurelle_cucina",
            "Switching off light.hue_aurelle_cucina in kitchen",
            "Switching off all lights in kitchen"
          ]
        }
      ]
    }
  }
}

AS A chatbot developer
I WANT Agents to remember context for longer than 5 turns or forget contexts arbitrarily
SO THAT I have more control on the information and followups that are available at a given point in conversation

Details

Currently each context has a default lifespan of 5 turns that is not configurable. Sometimes we want to make that lifespan longer, to retain context parameters and possible followups longer; sometimes we want to kill intents, to clean context and reduce ambiguity when interpreting new intents. This can be achieved with the following implementation

• A Intent.lifespan class variable can control the lifespan of a given intent
• A new_lifespan parameter on the follow() relation can be used to extend the duration of a context, or removed altogether (follow(new_lifespan=0)))

AS A developer
I WANT TO public Intent class types from a single module
SO THAT I can write types imports in a single line

AS A Chatbot developer
I WANT TO make predictions on a local OSS service
SO THAT I have more control and save money on cloud services

Details

Snips NLU (https://github.com/snipsco/snips-nlu) is an open source (Apache 2.0), python/rust based, NLU software with a simple API, that is complete of all of the important features of modern NLU service.

A SnipsConnector(Connector) class should wrap around the standard module, featuring:

• Export of Intent and Entity classes as Snips datasets
• Model training in place of upload()
• Intent predictions with response rendering

Note that context management is left for future work. This will need some session persistence interface, as Snips, like Alexa, doesn't handle that aspect

AS A Language Designer
I WANT TO include 'default' responses in 'rich' response group
SO THAT I save time and reduce chance of errors when defining Language resources

Details

This can be done by adding a new "include" response type:

responses:
  default:
    ...
  rich:
    - include: default
    - text:
      ...

Test scenarios

• Default responses are included

  GIVEN Intent travels.user_ask_train_station has at least a text response in "default"
  AND it has "include: default" in "rich"
  THEN  exported agent will show Default responses in rich platform responses
  

• Invalid reference (e.g. include: asdasd) raises ValuError

  GIVEN a test intent exists with at least a text response in "default" 
  AND it has "include: asdasd" in "rich"
  THEN  ValueError is thrown when loading intent language
  

• Including "rich" in "default" raises ValueError

  GIVEN a test intent exists with both "default" and "rich" groups defined
  AND it has "include: default" in "rich"
  THEN  ValueError is thrown when loading intent language
  

• Including "default" with no reponses produces a Warning log

  GIVEN a test intent exists with no responses in the "default" group 
  AND it has "include: default" in the "rich" group
  THEN exported agent will have no extra answers in rich platform responses
  AND a warning log is produced when loading intent language resources
  

Steps to reproduce

df.predict("I want a Margherita")

Expected result

restaurant.order_pizza(pizza_type: PizzaType="Margherita", ...)

Actual result

{}

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-10-6859044985a4> in <module>
----> 1 df.predict("I want a Margherita")

~/lavoro/dialogflow-agents/intents/connectors/dialogflow_es/connector.py in predict(self, message, session, language)
    154 
    155         return self._df_response_to_prediction(df_response)
--> 156 
    157     def trigger(self, intent: Intent, session: str=None, language: str=None) -> Intent:
    158         if not session:

~/lavoro/dialogflow-agents/intents/connectors/dialogflow_es/connector.py in _df_response_to_prediction(self, df_response)
    194         return DialogflowPrediction(
    195             intent=self._df_response_to_intent(df_response),
--> 196             confidence=df_response.protobuf_response.query_result.intent_detection_confidence,
    197             fulfillment_message_dict=intent_responses(df_response.protobuf_response),
    198             fulfillment_text=df_response.protobuf_response.query_result.fulfillment_text,

~/lavoro/dialogflow-agents/intents/connectors/dialogflow_es/connector.py in _df_response_to_intent(self, df_response, build_related_cls, visited_intents)
    247         visited_intents.add(intent_cls)
    248         parameter_dict = deserialize_intent_parameters(df_parameters, intent_cls, self.entity_mappings)
--> 249         related_intents_dict = {}
    250         for related in related_intents(intent_cls).follow:
    251             if related.intent_cls in visited_intents:

~/lavoro/dialogflow-agents/intents/service_connector.py in deserialize_intent_parameters(service_parameters, intent_cls, mappings)
    208             raise ValueError(f"Found parameter {param_name} in Service Prediction, but Intent class does not define it.")
    209         param_metadata = schema[param_name]
--> 210         mapping = mappings[param_metadata.entity_cls]
    211         try:
    212             if param_metadata.is_list:

KeyError: <class 'example_agent.restaurant.PizzaType'>

AS A chatbot owner
I WANT TO be able to use intent relations in Snips
SO THAT I can run complex Agents locally

AS A chatbot user
IN ORDER TO have faster replies on simple intents
I WANT Agent to fulfill only intents that define a fulfillment procedure
INSTEAD OF calling the webhook every time

Details

This is done at export time. The webhookUsed and webhookForSlotFilling should only be set to True if the intent (or at least one of its parent classes) override their .fulfill() method.

AS A chatbot user
I WANT TO interact with rich Agents via Alexa
SO THAT I can have a hands-free interaction

AS A chatbot user
I WANT chatbot to recognize some Entities as enum values
SO THAT I get less error chances when I refer to product codes, categorical values and such

Description

Currently entities work on a "value+synonyms" basis. This means that entity value must be matched with string comparison if it needs to be used as a categorical variable (e.g. a type of pizza). This is prone to errors, first because there's no static checking with strings, and also because Connectors may strip values and synonyms depending on what Services support (e.g. Alexa won't support the * character).

Proposed solution is to implement Enum entities; they must return their Enum value in predictions (e.g. PizzaType.MARGHERITA), as well as the standard string canonical value ("Margherita"), which is to be used when rendering responses.

AS A framework user
I WANT TO publish my Intents Agent as an Alexa Skill
SO THAT I reach more users without manually porting the Agent

Details

In its simplest form, the connector should include:

• Intents
• Intent parameters (Slots)
• Example Utterances
• Example Utterances with parameter references
• Plain text Responses for later: requires webhook
• System Entities
• Custom Entities (Types)
• Multi-lingual Agents

Input/Output Contexts and Events will not be part of the basic connectors.

Also, Alexa can't be explicitly queried for predictions and triggers: these actions should raise a NotImplementedError

Test Cases

Toy Agent export

GIVEN ToyAgent is defined
AND ToyAgent only imports example_agent.smalltalk.user_name_give
AND AlexaConnector is instantiated with ToyAgent
WHEN ToyAgent is exported and imported into the Alexa Developers Console
THEN A warning is shown, stating that AlexaConnector is experimental
AND The Agent is restored correctly
AND There's a smalltalk.user_name_give intent

AS A chatbot user
I WANT Agent to handle Intents with custom business logic
SO THAT it can perform actions corresponding to my requests

Details

Fulfillment calls are used by Agents to implement more complex conversation flows, and to perform relevant actions for a given intent by calling an underlying API (add to cart, send payment request, retrieve requested information, ...). To add suppport for fulfillment calls, Intents must:

1. Add an optional fulfill() method (interface tbd) to the Intent class. This will be called by Intents when the Intent is detected.
   • fulfill() may return language.IntentResponse objects, or other Intent classes
2. Create a fulfillment framework (could be part of the connector, or a module on its own). This will receive Dialogflow fulfillment requests, build the corresponding Intent object, and run its fulfill() method.
   • Fulfillment is enabled per-intent in Dialogflow: Intents that do not define a custom fulfill() method (including those in the superclasses) should set webhookUsed=False at export time.

Test Cases

tbd

AS A chatbot developer
I WANT Intents to drop support for Intent.parameter_schema()
SO THAT I don't get pylint callable-related warnings when using Intent.parameter_schema

Details

Intent.parameter_schema() is now a property. This is just a reminder to delete in in v0.3. Since we are at it, these other deprecated features are to be removed:

• Input/Output contexts
• Custom Events for triggering Intents

Steps to reproduce

@dataclass
class a_class(Intent):
    foo: str = "ciao"
    bar: List[str] = field(default_factory=lambda: [])

Expected result

An intent with optional fields foo and bar

Actual Result

TypeError: non-default argument 'bar' follows default argument

Solution

The reason for the error is that dataclass is already applied by _IntentMetaclass.__new__(), which cannot know whether a_class will be decorated by @dataclass or not. Internally, dataclass deletes bar from dict, so that it can be handled in its generated __init__(). So, the second time it's applied it will find an annotation for "bar", but no "bar" member, causing the TypeError above.

Since @dataclass is applied programmatically, an explicit @dataclass decorator is not needed for the framework to work. However, without it IDEs will not apply type hinting and autocompletion on Intent subclasses, which is not acceptable. Possible solutions include:

• Don't apply dataclass programmatically within _IntentMetaclass.__new__(); chatbot developer must explicitly decorate with @dataclass. Pro: problem is solved without hacks nor breaking changes. Cons: dataclass can't be enforced in __new__(), ill Intent subclasses may exist at some point in the flow; also, parameters can't be checked at class definition time, because Intent.parameter_schema (like all other Intent internals) assumes class is a dataclass. Some workarounds:

  • (for parameter checking) Check parameters on Agent.register() instead of class definition time
  • (for parameter checking) Make a temporary dataclass within __new__() and call parameter check on that one.
  • (for enforcing dataclass) Check is_dataclass 1) in __new__(), for parent class when subclassing 2) in Agent.register()

• Wrap @dataclass in a custom @Intent.dataclass decorator, which is aware of Intents and doesn't apply itself twice. Pro: dataclass can be enforced in __new__() without many hacks. Cons: not all IDEs will acknowledge the decorator. While VSCode infers that @Intent.dataclass returns dataclasses.dataclass, and behaves accordingly, PyCharm is much stricter and won't hint the decorated class as a dataclass (https://youtrack.jetbrains.com/issue/PY-40584)

• Overwrite built-in @dataclass decorator with a custom one that behaves differently only when applied to Intents. Intents.__init__ has to 1) overwrite standard dataclasses.dataclass 2) detect if dataclass is already in globals(), and overwrite that one as well. Pro: problem is solved without breaking changes, dataclass can be enforced at class definition time. Cons: much hack

AS A chatbot developer
I WANT TO store structured information in the conversation session
SO THAT I don't have to setup a separate persisted storage for that

Details

Intent parameters are currently limited to entities that are tagged in User messages. However, it is often useful to store structured information in the conversation session (a location, a user profile, an item metadata, ...), to be used within the conversation scope. This information cannot be tagged in user utterances; instead, it can be provided programmatically by triggers ad fulfillment results.

We want to distinguish NLU parameters and Session parameters, the first being message entities, the latter injected in session by triggers and fulfillment procedures.

To Do

Model

• Split IntentParameterMetadata into NLU parameters and Session parameters
• Update Intent.parameter_schema to allow for Session Parameters
• Raise exception if Session parameter is tagged in example utterances
• Raise exception if example utterances are defined and there is at least one required Session parameter (intent cannot be predicted)

Dialogflow

• Export Session parameters as @sys.any intent params
• predict/fulfill: de-serialize stored parameters (string, dataclass or JSON)
• fulfillment result/trigger: serialize stored parameters (string, dataclass or JSON)

Alexa

• Skip Session parameters in export

Snips

• Skip Session parameters in export
• Add test for triggered intent w/ Session parameters

AS A chatbot user
I WANT chatbots to be tested end-to-end before release
SO THAT I have less chances to encounter regressions when software is updated

AS A chatbot user
I WANT Agent to recognize followup utterances and deal with them accordingly
SO THAT I can have richer multi-turn interactions

Details

Currently Intents supports Dialogflow-style context tags in Intent subclasses. The idea is to make it more abstract and easier to define follow-up intents, by defining Intent members within an Intent class:

class buy_fish(Intent):
    """
    User: I want to buy one fish
    """
    amount: Sys.Integer = 1

class change_amount(Intent):
    """
    User: Actually, let's make it 2
    """
    parent_order: buy_fish = follow()
    
    amount: Sys.Integer

Through parent_order it's possible to manage conversation flow, as well as access the conversation context in a more abstract and elegant way.

Test Cases

tbd

AS A framework user
IN ORDER TO access dialogue session easily and uniformly across connectors
I WANT TO use a standard interface for session contexts and parameters
INSTEAD OF accessing the native Dialogflow data within predicted intent

Details

When a service, such as Dialogflow, predicts an Intent it typically keeps a conversation session with context information. In Dialogflow, this is mainly

1. The list of active contexts and their remaining lifespans
2. A list of intent parameters and their matched values in the conversation.

We want to study a way to generalize this information across different services (Alexa, Azure, Wit.ai, ...) and provide a common interface for accessing it through the Intent class (as well as referencing context values in language resources). A specification for this is to be defined.

Test Scenarios

TBD

AS A framework user
I WANT TO have separate documentation pages per each API section
SO THAT docs access is less confusing, and it's easier to Ctrl+F in page

Details

An example: https://github.com/bigladder/kiva/tree/develop/docs

Test Cases

tbd

AS A chatbot developer
I WANT code coverage information to be consistent and up to date
SO THAT I can make a reliable judgment on the quality of the project

Description

Codecov is configured for this repo and documented in the "Development" section of the docs. However, coverage info is updated manually. We want a CI script to read codecov token from GitHub secrets and publish a new report every time something is merged in master.

AS A chatbot user
I WANT developers to have visibility and control over the slot filling process
SO THAT I can have more sophisticated interactions

Details

An Intent can be predicted without all of its required parameters. In this case Dialogflow and Alexa will enter slot filling, returning incomplete predictions and prompts for missing parameters.

At the moment, this will cause intent=None in the Prediction object, with no signal that slot filling is in progress. We want to model this case by

• Provide explicit information about slot filling being in progress (e.g. Prediction.slot_filling=True)
• Return the set of partially matched parameters (could be an Intent objects with a sentinel MISSING value on missing parameters)
• Adding a separate fulfillment procedure for slot filling

AS A developer
I WANT TO write parameter references without specifying an example value
SO THAT I write examples faster and with less syntax

Details

Currently, an example utterance that contain a parameter must define an example entity value for it. For instance:

- My name is $user_name{John}

However, most of the times the example is not informative. We want to support writing the same utterance like:

- My name is $user_name

An example value is still needed when exporting the Agent to Dialogflow. Intents will have to draw an example from the Entity definition. Note that, while this is trivial for Custom Entities, it will require adding some examples to built-in System Entities.

Test Scenarios

TBD