1.11.0

CHANGELOG.md

@@ -1,32 +1,76 @@
 # Changelog
 
-## [1.10.1a3](https://github.com/NeonGeckoCom/neon-utils/tree/1.10.1a3) (2024-04-30)
+## [1.10.2a9](https://github.com/NeonGeckoCom/neon-utils/tree/1.10.2a9) (2024-06-25)
 
-[Full Changelog](https://github.com/NeonGeckoCom/neon-utils/compare/1.10.1a2...1.10.1a3)
+[Full Changelog](https://github.com/NeonGeckoCom/neon-utils/compare/1.10.2a8...1.10.2a9)
 
 **Merged pull requests:**
 
-- Add backwards-compat import to `neon_fallback_skill` module [\#519](https://github.com/NeonGeckoCom/neon-utils/pull/519) ([NeonDaniel](https://github.com/NeonDaniel))
+- Fix bug causing old token to be used with requests after refreshing [\#530](https://github.com/NeonGeckoCom/neon-utils/pull/530) ([NeonDaniel](https://github.com/NeonDaniel))
 
-## [1.10.1a2](https://github.com/NeonGeckoCom/neon-utils/tree/1.10.1a2) (2024-04-30)
+## [1.10.2a8](https://github.com/NeonGeckoCom/neon-utils/tree/1.10.2a8) (2024-06-17)
 
-[Full Changelog](https://github.com/NeonGeckoCom/neon-utils/compare/1.10.1a1...1.10.1a2)
+[Full Changelog](https://github.com/NeonGeckoCom/neon-utils/compare/1.10.2a7...1.10.2a8)
 
 **Merged pull requests:**
 
-- Update location\_utils to use HANA [\#518](https://github.com/NeonGeckoCom/neon-utils/pull/518) ([NeonDaniel](https://github.com/NeonDaniel))
+- Mitigate issues with expired HANA tokens [\#529](https://github.com/NeonGeckoCom/neon-utils/pull/529) ([NeonDaniel](https://github.com/NeonDaniel))
 
-## [1.10.1a1](https://github.com/NeonGeckoCom/neon-utils/tree/1.10.1a1) (2024-04-29)
+## [1.10.2a7](https://github.com/NeonGeckoCom/neon-utils/tree/1.10.2a7) (2024-05-31)
 
-[Full Changelog](https://github.com/NeonGeckoCom/neon-utils/compare/1.10.0...1.10.1a1)
+[Full Changelog](https://github.com/NeonGeckoCom/neon-utils/compare/1.10.2a6...1.10.2a7)
 
-**Implemented enhancements:**
+**Merged pull requests:**
+
+- Refactor to use pytz instead of pendulum [\#527](https://github.com/NeonGeckoCom/neon-utils/pull/527) ([NeonDaniel](https://github.com/NeonDaniel))
+
+## [1.10.2a6](https://github.com/NeonGeckoCom/neon-utils/tree/1.10.2a6) (2024-05-31)
+
+[Full Changelog](https://github.com/NeonGeckoCom/neon-utils/compare/1.10.2a5...1.10.2a6)
+
+**Merged pull requests:**
+
+- Resolve Test Failures [\#528](https://github.com/NeonGeckoCom/neon-utils/pull/528) ([NeonDaniel](https://github.com/NeonDaniel))
+
+## [1.10.2a5](https://github.com/NeonGeckoCom/neon-utils/tree/1.10.2a5) (2024-05-21)
+
+[Full Changelog](https://github.com/NeonGeckoCom/neon-utils/compare/1.10.2a4...1.10.2a5)
+
+**Merged pull requests:**
+
+- Refactor to use HANA to send email instead of MQ [\#526](https://github.com/NeonGeckoCom/neon-utils/pull/526) ([NeonDaniel](https://github.com/NeonDaniel))
+
+## [1.10.2a4](https://github.com/NeonGeckoCom/neon-utils/tree/1.10.2a4) (2024-05-11)
+
+[Full Changelog](https://github.com/NeonGeckoCom/neon-utils/compare/1.10.2a3...1.10.2a4)
+
+**Merged pull requests:**
+
+- Update deprecated import to supported path [\#524](https://github.com/NeonGeckoCom/neon-utils/pull/524) ([NeonDaniel](https://github.com/NeonDaniel))
+
+## [1.10.2a3](https://github.com/NeonGeckoCom/neon-utils/tree/1.10.2a3) (2024-05-10)
+
+[Full Changelog](https://github.com/NeonGeckoCom/neon-utils/compare/1.10.2a2...1.10.2a3)
+
+**Merged pull requests:**
+
+- Fix ovos-core 0.0.8 Common Query compat. [\#523](https://github.com/NeonGeckoCom/neon-utils/pull/523) ([NeonDaniel](https://github.com/NeonDaniel))
+
+## [1.10.2a2](https://github.com/NeonGeckoCom/neon-utils/tree/1.10.2a2) (2024-05-08)
+
+[Full Changelog](https://github.com/NeonGeckoCom/neon-utils/compare/1.10.2a1...1.10.2a2)
+
+**Merged pull requests:**
+
+- Remove old patch [\#522](https://github.com/NeonGeckoCom/neon-utils/pull/522) ([NeonDaniel](https://github.com/NeonDaniel))
+
+## [1.10.2a1](https://github.com/NeonGeckoCom/neon-utils/tree/1.10.2a1) (2024-05-07)
 
-- \[FEAT\] resolve import errors with ovos-utils 0.1.0 [\#514](https://github.com/NeonGeckoCom/neon-utils/issues/514)
+[Full Changelog](https://github.com/NeonGeckoCom/neon-utils/compare/1.10.1...1.10.2a1)
 
 **Merged pull requests:**
 
-- slight refactor for ovos-utils 0.1.0 [\#515](https://github.com/NeonGeckoCom/neon-utils/pull/515) ([mikejgray](https://github.com/mikejgray))
+- Enable `hana` backend URL to be configured [\#521](https://github.com/NeonGeckoCom/neon-utils/pull/521) ([NeonDaniel](https://github.com/NeonDaniel))
 
 
 

neon_utils/hana_utils.py

@@ -29,19 +29,39 @@
 import requests
 import json
 
+from typing import Optional
 from os import makedirs
 from os.path import join, isfile, isdir, dirname
 from time import time
 from ovos_utils.log import LOG
 from ovos_utils.xdg_utils import xdg_cache_home
 
-_DEFAULT_BACKEND_URL = "https://hana.neonaiservices.com"
+_DEFAULT_BACKEND_URL = None
 _client_config = {}
 _headers = {}
 
 
-def _get_client_config_path(url: str = _DEFAULT_BACKEND_URL):
-    url_key = hash(url)
+def set_default_backend_url(url: Optional[str] = None):
+    """
+    Set the default backend URL
+    @param url: HANA backend url to use, else read from configuration
+    """
+    global _DEFAULT_BACKEND_URL
+    if not url:
+        from ovos_config.config import Configuration
+        url = Configuration().get('hana', {}).get('url') or \
+            "https://hana.neonaiservices.com"
+    if url and url != _DEFAULT_BACKEND_URL:
+        LOG.info(f"Updating HANA backend URL to {url}")
+        _DEFAULT_BACKEND_URL = url
+        global _client_config
+        global _headers
+        _client_config = {}
+        _headers = {}
+
+
+def _get_client_config_path(url: str):
+    url_key = url.split('/')[2]
     return join(xdg_cache_home(), "neon", f"hana_token_{url_key}.json")
 
 
@@ -112,6 +132,11 @@ def _refresh_token(backend_address: str):
         raise ServerException(f"Error updating token from {backend_address}. "
                               f"{update.status_code}: {update.text}")
     _client_config = update.json()
+
+    # Update request headers with new token
+    global _headers
+    _headers['Authorization'] = f"Bearer {_client_config['access_token']}"
+
     client_config_path = _get_client_config_path(backend_address)
     with open(client_config_path, "w+") as f:
         json.dump(_client_config, f, indent=2)
@@ -126,16 +151,45 @@ def request_backend(endpoint: str, request_data: dict,
     @param server_url: Base URL of Hana server to query
     @returns: dict response
     """
+    global _client_config
+    global _headers
+    if not server_url:
+        set_default_backend_url()
+        server_url = _DEFAULT_BACKEND_URL
+    if server_url != _DEFAULT_BACKEND_URL and _client_config:
+        LOG.info(f"Using new remote: {server_url}")
+        _client_config = {}
+        _headers = {}
     _init_client(server_url)
-    if time() >= _client_config.get("expiration", 0):
+    if _client_config.get("expiration", 0) - time() < 30:
         try:
             _refresh_token(server_url)
         except ServerException as e:
             LOG.error(e)
             _get_token(server_url)
-    resp = requests.post(f"{server_url}/{endpoint.lstrip('/')}",
-                         json=request_data, headers=_headers)
+    request_kwargs = {"url": f"{server_url}/{endpoint.lstrip('/')}",
+                      "json": request_data, "headers": _headers}
+    resp = requests.post(**request_kwargs)
+    if resp.status_code == 502:
+        # This is raised occasionally on valid requests. Need to resolve in HANA
+        resp = requests.post(**request_kwargs)
     if resp.ok:
         return resp.json()
+
     else:
+        try:
+            error = resp.json()["detail"]
+            # Token is actually expired, refresh and retry
+            if error == "Invalid or expired token.":
+                LOG.warning(f"Token is expired. time={time()}|"
+                            f"expiration={_client_config.get('expiration')}")
+                _refresh_token(server_url)
+                resp = requests.post(**request_kwargs)
+                if resp.ok:
+                    return resp.json()
+        except Exception as e:
+            LOG.error(e)
+            # Clear cached config to force re-evaluation on next request
+            _client_config = {}
+            _headers = {}
         raise ServerException(f"Error response {resp.status_code}: {resp.text}")

neon_utils/location_utils.py

@@ -25,11 +25,13 @@
 # LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
 # NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 # SOFTWARE,  EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+from time import time
 
-import pendulum
+import pytz
 
 from datetime import datetime
 from typing import Optional, Union
+
 from dateutil.tz import tzlocal
 from timezonefinder import TimezoneFinder
 from re import sub
@@ -143,10 +145,11 @@ def get_timezone(lat, lng) -> (str, float):
     Note that some coordinates do not have a city, but may have a county.
     :param lat: latitude
     :param lng: longitude
-    :return: timezone name, offset from GMT
+    :return: timezone name, offset in hours from UTC
     """
     timezone = TimezoneFinder().timezone_at(lng=float(lng), lat=float(lat))
-    offset = pendulum.from_timestamp(0, timezone).offset_hours
+    offset = pytz.timezone(timezone).utcoffset(
+        datetime.now()).total_seconds() / 3600
     return timezone, offset
 
 

neon_utils/mq_utils.py

@@ -29,8 +29,10 @@
 import logging
 import uuid
 
-from ovos_utils.log import deprecated
+from ovos_utils.log import deprecated, log_deprecation
 
+log_deprecation("This module has moved to neon_mq_connector.utils.client_utils",
+                "2.0.0")
 try:
     from threading import Event
     from pika.channel import Channel
@@ -47,7 +49,7 @@
 logging.getLogger("pika").setLevel(logging.CRITICAL)
 
 _default_mq_config = {
-    "server": "api.neon.ai",
+    "server": "mq.neonaiservices.com",
     "port": 5672,
     "users": {
         "mq_handler": {
@@ -60,6 +62,8 @@
 
 class NeonMQHandler(MQConnector):
     def __init__(self, config: dict, service_name: str, vhost: str):
+        log_deprecation("Import from neon_mq_connector.utils.client_utils",
+                        "2.0.0")
         super().__init__(config, service_name)
         self.vhost = vhost
         import pika
@@ -70,13 +74,11 @@ def __init__(self, config: dict, service_name: str, vhost: str):
 @deprecated("Use `neon_mq_connector.client.send_mq_request`", "2.0.0")
 def get_mq_response(vhost: str, request_data: dict, target_queue: str,
                     response_queue: str = None, timeout: int = 30) -> dict:
-    # TODO: Remove in v1.0.0 DM
     return send_mq_request(vhost, request_data, target_queue, response_queue,
                            timeout, True)
 
 
-# TODO: Mark deprecation after stable neon_mq_connector release
-# @deprecated("Use `neon_mq_connector.client.send_mq_request`", "2.0.0")
+@deprecated("Import from neon_mq_connector.utils.client_utils", "2.0.0")
 def send_mq_request(vhost: str, request_data: dict, target_queue: str,
                     response_queue: str = None, timeout: int = 30,
                     expect_response: bool = True) -> dict:

neon_utils/skills/common_query_skill.py

@@ -65,7 +65,6 @@ def handles_visuals(platform):
     return platform in VISUAL_DEVICES
 
 
-# TODO: Consider deprecation and implementing ovos_workshop directly
 class CommonQuerySkill(NeonSkill, _CQS):
     """Question answering skills should be based on this class.
 
@@ -168,6 +167,8 @@ def __handle_query_action(self, message):
         data = message.data.get("callback_data") or {}
         if data.get("answer"):
             self.speak(data["answer"])
+        else:
+            LOG.error(f"no answer provided in: {message.data.keys()}")
         # Invoke derived class to provide playback data
         self.CQS_action(phrase, data)
         self.bus.emit(message.forward("mycroft.skill.handler.complete",
@@ -191,6 +192,7 @@ def __handle_question_query(self, message):
             level = result[1]
             answer = result[2]
             callback = result[3] if len(result) > 3 else None
+            callback["answer"] = answer
             confidence = self.__calc_confidence(match, search_phrase, level)
             self.bus.emit(message.response({"phrase": search_phrase,
                                             "skill_id": self.skill_id,

neon_utils/skills/kiosk_skill.py

@@ -198,8 +198,7 @@ def stop(self):
         if user in self._active_users:
             self.end_interaction(message)
 
-    def _on_event_error(self, error, message, handler_info,
-                        skill_data, speak_errors):
+    def _on_event_error(self, error, message, *args, **kwargs):
         """
         Override error handling to speak custom exception for active sessions
         """
@@ -208,5 +207,4 @@ def _on_event_error(self, error, message, handler_info,
             LOG.exception(error)
             self.handle_error(message)
         else:
-            super()._on_event_error(error, message, handler_info,
-                                    skill_data, speak_errors)
+            super()._on_event_error(error, message, *args, **kwargs)

neon_utils/skills/neon_fallback_skill.py

@@ -44,7 +44,6 @@
 from ovos_utils.gui import is_gui_connected
 from ovos_utils.log import LOG, log_deprecation, deprecated
 from ovos_utils.skills import get_non_properties
-from ovos_utils.skills.settings import save_settings
 from ovos_utils.xdg_utils import xdg_cache_home
 from ovos_workshop.skills import OVOSSkill
 from ovos_workshop.skills.fallback import FallbackSkillV1
@@ -53,7 +52,8 @@
 from neon_utils.file_utils import resolve_neon_resource_file
 from neon_utils.location_utils import to_system_time
 from neon_utils.message_utils import dig_for_message, resolve_message, get_message_user
-from neon_utils.skills.neon_skill import CACHE_TIME_OFFSET, DEFAULT_SPEED_MODE, SPEED_MODE_EXTENSION_TIME, NeonSkill
+from neon_utils.skills.neon_skill import CACHE_TIME_OFFSET, DEFAULT_SPEED_MODE, SPEED_MODE_EXTENSION_TIME, NeonSkill, \
+    save_settings
 from neon_utils.user_utils import get_user_prefs
 
 
@@ -875,23 +875,3 @@ def init_dialog(self, root_directory: Optional[str] = None):
         """
         log_deprecation("Use `load_dialog_files`", "2.0.0")
         self.load_dialog_files(root_directory)
-
-    def add_event(self, name: str, handler: callable,
-                  handler_info: Optional[str] = None, once: bool = False,
-                  speak_errors: bool = True):
-        # TODO: Remove with ovos-workshop==0.0.13
-        try:
-            # Patching FakeBus compat. with MessageBusClient
-            if hasattr(self.bus, "ee"):
-                emitter = self.bus.ee
-            else:
-                emitter = self.bus.emitter
-            if handler_info == "mycroft.skill.handler" and \
-                    emitter.listeners(name):
-                LOG.warning(f"Not re-registering intent handler {name}")
-                return
-        except Exception as e:
-            LOG.exception(e)
-        OVOSSkill.add_event(self, name, handler, handler_info, once,
-                            speak_errors)
-