entities in request always empty

[EeeDotZee]

I'm submitting a...

• Bug report
• Feature request
• [ x ] Documentation issue or request
• Other... Please describe:

Expected Behavior

If i set an entity in my language model it should be filled up with a value when the request is sent.

Current Behavior

I'm trying to migrate my v3 project to v4 and everything works fine beside migrating my inputs to v4. I adjusted the language model as described in the documentation and this is how my intent looks right now:

"SetupUsernameIntent": {
        "phrases": [  "{name}",
          "my name is {name}",
          "i am {name}",
          "you can call me {name}"],
        "entities": {
          "name": {
            "type": {
              "alexa": "AMAZON.US_FIRST_NAME",
              "dialogflow": "@sys.given-name"
            }
          }
        }
      }

This is how my intent handler looks:

@Intents('SetupUsernameIntent')
setupUsername(){
    console.log(this.$entities);
}

However this.$entities is always empty if i do for example my name is John. I also see that in the Debugger in the lifecycle that now entities are being added.

What am i doing wrong? My example is pretty much the same as in documentation.

PS: the right intent is triggered, so the routing works

Your Environment

• Jovo Framework version used: 4.0.0
• Operating System: MacOS, Firefox

[m-ripper]

Hello there, can you please share the request that is supposed to fill the entities?

[EeeDotZee]

Hello there, can you please share the request that is supposed to fill the entities?

sure!

{
   "version":"4.0-beta",
   "platform":"jovo-debugger",
   "id":"2be83052-971d-4048-95c2-003df5b16644",
   "timestamp":"2021-11-23T11:57:16.705Z",
   "timeZone":"Europe/Vienna",
   "locale":"en",
   "input":{
      "type":"TEXT",
      "text":"my name is John"
   }"context":{
      "device":{
         "id":"fd64fc61-c0e2-424c-afe4-5b1f9574e41a",
         "capabilities":[
            "SCREEN",
            "AUDIO"
         ]
      }"session":{
         "id":"40462e08-99ac-4492-9531-ff16a34333a7",
         "data":{
            
         }"state":[
            {
               "component":"SetupComponent"
            }
         ]"isNew":false,
         "updatedAt":"2021-11-23T11:57:03.655Z",
         "new":false
      }"user":{
         "id":"eda4fd14-8437-4022-a51a-9c5db04d180b",
         "data":{
            
         }
      }
   }
}

[m-ripper]

Thanks for the fast response.

The problem is that the underlying NLU plugin (nlp.js) that is used by the Jovo Debugger does not have a built-in name entity-type.

This is one of the main reasons we switched the content of the template in v4.

One workaround would be to add a custom entity-type and values for potential names or use another NLU integration that is able to handle names by itself.

I hope this helps.

[EeeDotZee]

I understand. Can i define a type for the Jovo Debugger specifically?
So it would be like:

 "type": {
       "alexa": "AMAZON.US_FIRST_NAME",
       "dialogflow": "@sys.given-name"
       "jovoDebugger": "MyEntityType"
}

Because i don't want to rewrite my model everytime i build my project. I definitely need to use some of these built in types especially for number inputs aswell.

[m-ripper]

Yes that works, but you will have to specify the type for nlpjs instead of jovoDebugger, like this:

 "type": {
       "alexa": "AMAZON.US_FIRST_NAME",
       "dialogflow": "@sys.given-name"
       "nlpjs": "MyEntityType"
}

As for the dynamic types, nlpjs has some built-in ones but they're not enabled by default.
Something that could work and I haven't tested yet is the following:

app.dev.ts

import { NlpjsNlu } from '@jovotech/nlu-nlpjs';
// import could be used instead if module declarations are provided for @nlpjs/builtin-default
const { BuiltinDefault } = require('@nlpjs/builtin-default');

// initialize debugger
const jovoDebugger = new JovoDebugger();

// initialize built-in entity extraction
const builtin = new BuiltinDefault()
// add the built-in entity extractor to the container of the nlp-instance
(jovoDebugger.config.nlu as NlpjsNlu).nlpjs?.container?.register('extract-builtin-??', builtin, true);

app.use(jovoDebugger)

This is something that we're going to simplify in the future.

Let me know if this fixes the problem or does not help.

[EeeDotZee]

Thank you @m-ripper !
That definitely helped. I tried your setup code for the nlpjs/builtin-default and also read the documentation but didn't get it to work on my end. In my type i tried to do "nlpjs": "number" for testing it with a numeric input but doesn't seem to work.

However as I'm only using the debugger for quick local tests anyways, I'm fine with setting it to some pre-defined values as you described above.

PS: I highly suggest adding that "nlpjs" option for model types into the documentary. I was searching through all possible types you could set and didn't find it. In case you already have it somewhere and i just wasn't able to find it please ignore this suggestion :)

[m-ripper]

Hey @EeeDotZee,
I am glad that I could help even if the dynamic part didn't work. I will take a look into this.

Regarding the typings of nlpjs I agree, that's something we're going to add in the near future as well. Sadly, nlpjs is written in javascript and does not provide typings itself, but we can go through the code and type the most important parts, it just had a low priority so far.

It's definitely helpful, thank you for your feedback!

[tfrenken]

Sorry for reopening this but are there any updates on enabling the debugger to auto-fill entities?

The workaround with a custom entity type for the debugger works for me, the small hack proposed my @m-ripper does not - although I am trying to extract a search query, not a name.

Being able to have the debugger's NLP auto extract entities would be really nice!

[jBernavaPrah]

Hi @tfrenken and @m-ripper,

I was able to enable the entities recognition with nlp.js and Microsoft extractor.

This is the code that I used:

import { BuiltinMicrosoft } from '@nlpjs/builtin-microsoft';
import { Nlp, NlpjsNlu } from '@jovotech/nlu-nlpjs';
import { LangEn } from '@nlpjs/lang-en';
import { LangIt } from '@nlpjs/lang-it';
import { Platform } from '@jovotech/framework';

// [...]

const nlp = new NlpjsNlu({
  languageMap: {
    en: LangEn,
    it: LangIt,
  },
  enabled: true,
  setupModelCallback: async (platform: Platform, nlpjs: Nlp) => {
    nlpjs.container.register('extract-builtin-??', BuiltinMicrosoft, true);
    // eslint-disable-next-line @typescript-eslint/ban-ts-comment
    // @ts-ignore
    await nlp.addCorpusFromModelsIn('./models');
    // eslint-disable-next-line @typescript-eslint/ban-ts-comment
    // @ts-ignore
    await nlpjs.train();
  },
});


app.configure({
  plugins: [
    new FileDb({
      pathToFile: '../db/db.json',
    }),
    new JovoDebugger({
      nlu: nlp,
    }),
  ],
});

export * from './server.express';

Pay attention that currently, I think there is a bug in the Microsoft Extractor package because it missing the Italian language resource. I opened the issue here waiting for the response.

[rmtuckerphx]

@jBernavaPrah Can you provide a sample TypeScript Jovo app that shows this working?

I'm interested in the Jovo model file and how to get past the error:

src/app.dev.ts(9,34): error TS7016: Could not find a declaration file for module '@nlpjs/builtin-microsoft'. 'C:/dev/projects/personal/jovo4-test-app/node_modules/@nlpjs/builtin-microsoft/src/index.js' implicitly has an 'any' type.
  Try `npm i --save-dev @types/nlpjs__builtin-microsoft` if it exists or add a new declaration (.d.ts) file containing `declare module '@nlpjs/builtin-microsoft';`

[jBernavaPrah]

Hi @rmtuckerphx
Unfortunately I don’t have any code available now, but if my memory don’t joke me, I simply added those lines:

// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore

before the first line or any lines that give you issues with Typescript. (I know I know, but I was still testing and the project and package @npljs doesn’t have Typescript available 😅)

Anyway, at the end, I decided to move away from @buildin-microsoft. This package use internally the microsoft/Recognizers-Text repository, but the JavaScript “platform” (as they call it) was not aligned with the original .Net code. As example the Italian language was completely missing.

I decided to use the .net version (the original and fully functional code) of microsoft/Recognizers-Text as external http service and call it when needed.

[rmtuckerphx]

Thanks @jBernavaPrah