Cordova plugin on device timeout

[riyer15]

(also posted on the api.ai discussions page)

Hi! I'm trying to use the APIAi plugin with cordova for voice. Within the emulator, it all works perfectly, but once I run the app on my iPhone 6 (running ios 9.3), I run into a few issues:
1.) The first voice request is "listening"/recording voice for long after I start talking, and never stops until the api.ai formal timeout (>10 seconds)
2.) The second works perfectly, but only after requesting a few times.
3.) No subsequent requests can be fulfilled -- it stops working after the second function call.

For every subsequent request, I get a timeout error after 10 seconds. Do you have any suggestions @xVir?
I've also tried this with the ionic example @xVir made, and I've had a similar issue (though sometimes, i don't have it)

Thanks,
Rani

[sstepashka]

@riyer15 Hi,
I was trying to reproduce this error, but I can't. Demo work fine. I was add same optimizations, but i don't sure, that it will help for you.

I have pushed new version of library. Please update the library and try to reproduce this problems.

[sstepashka]

@riyer15 Hey! is it help for you?

[riyer15]

@sstepashka i'm unable to remove the cordova plugin (to reinstall it) without getting an error. When I update the library I will try again and let you know.

[riyer15]

@sstepashka i'm still having the same issue. Here's a snippet of the code i use to handle the voice request:

 $scope.voiceRequest = function () {
      console.log("button pressed");
      ApiAIPlugin.requestVoice({},
        function (response) {

                               $scope.voiceReply = JSON.stringify(response['result']['fulfillment']['speech']);
                               alert($scope.voiceReply);
                               console.log(JSON.stringify(response));

          $timeout(function(){
            $scope.speakText($scope.voiceReply);
          }, 1000);

        },
        function (error) {
          alert("Error!\n" + error);
        });
    };
$scope.speakText = function(inputText){
    TTS
      .speak({
        text: inputText,
        locale: 'en-US',
        rate: 1.6
      }, function () {
      }, function (reason) {
        alert(reason);
      });
  }; 

I'm still getting issues with this sample project on my device -- the first request runs forever, and i have to press two more times to get a functioning request. Any advice?

[riyer15]

Okay, after some tinkering I've realized that, though the first request often results in a state of constant listening, when I remove my text-to-speech all subsequent requests are processed correctly. I decided to refactor the text to speech functionality, making it a separate function that isn't automatically triggered with the voiceRequest function. Even now, if I click to speak the current voiceReply value, it prevents all future recording/calls to the API.ai plugin. Have you heard of this issue? i'm using this plugin: https://github.com/vilic/cordova-plugin-tts for my TTS.

[sstepashka]

hmm... I think, I know what the problem is. This is plugin interrupting audio session, I'll fix it tomorrow.

[sstepashka]

Fixed. Updated. Please try.

[riyer15]

@sstepashka i'm still having the issue. I believe that starting the TTS plugin interrupts the API.ai I/O session for ios, as I get this error:

2016-07-19 09:53:51.054 HelloCordova[467:77017] 09:53:51.053 ERROR: [0x1a0cce000] AVAudioSession.mm:692: -[AVAudioSession setActive:withOptions:error:]: Deactivating an audio session that has running I/O. All I/O should be stopped or paused prior to deactivating the audio session.

and then, api.ai stops accepting voice requests/stops "listening." This is the same issue that i've been having before, i've just had a better chance at isolating the problem to i/o.

[sstepashka]

hm... It's strange... I have implemented you code in our sample, you can see it here and all work fine.

Please check plugin version or try to reinstall the plugin.

[riyer15]

@sstepashka i'm realizing now that i'm on the plugin versoin 1.7.3, though the new one was 1.7.4. However, cordova isn't allowing me to remove and reinstall the plugin, unless I add it through ionic state restore -- I will attempt to reinstall again and check in.

[riyer15]

@sstepashka hi, so the recording doesn't stop anymore when I submit a request after voice, but now it is recording forever, and requires me to submit two requests (speak twice) to get any response and stop the recording. Do you know what to do?

[riyer15]

Sorry! closed by mistake

[sstepashka]

I was check our sample application, all work fine. Sometimes listening doesn't stopping because of noise. It happens.

[riyer15]

@sstepashka hi, it's not because of the sound levels, i have monitored them and seen that on the second click, it will stop once sound levels reach .05, but when i click first it never stops, it waits for the on-device timeout. This issue doesn't happen on the emulator, only on the physical device.

[sstepashka]

Do you sure that error in "api.ai" plugin?

[riyer15]

@sstepashka i believe it is -- i think this may be a separate but related issue from the i/o error. After removing the text-to-speech capabilities, I still have the issue of needing two clicks on start. It isn't a huge issue without text-to-speech, but when the AVAudioSession restarts, I get the same issue -- this makes the app work pretty slowly, I have to make 2 requests for one to go through for every request if I want TTS to work. Do you have any advice?

[sstepashka]

Yes. I was testing it on 4s iOS 9.3.2

[riyer15]

@sstepashka have you had any instances of this issue? do you think it is my device specifically? I am using an iphone 6 on iOS 9.3.2

[sstepashka]

@riyer15 Can you send me all your project or make demo for represent issue? Do you try our demo for test?

[ranimoven]

@sstepashka yes i tried testing your demo on my device, and it worked well. My app relies on Api.ai to auto-detect the end of a query (a function that works perfectly on android and ios Emulator) -- could that be the source of the issue? While I have had the request problem regardless of the volume or sound levels, I have not been able to see if being able to stop could solve the problem. this is the repository for my project https://github.com/ranimoven/apiai-demo

[ranimoven]

@sstepashka I added a "stop listening" button to my device and it works well now. Though i wish there was a way for it to auto-detect the ends of queries still -- i really liked that functionality. Have you tried implementing that recently?