Dream distribution with SF-based DM (#572)

* speech-function-classifier model changed

* dream_Speech_models_based dist initial added

* secret required skills removed

* more fixes

* adding sfc to configs and minor refactoring of sfc

* Dockerfile fixed

* Dockerfile fixed 1

* minor fix

* minor fix 2

* minor fix 3

* minor fix 4

* minor fix 5

* fixing 120 symbols line check in models.py

* fixing 120 symbols line check in models.py

* renamed branch and dist folder for future convenience

* branch renamed according to suggestions

* initial sf-based response selector commit

* added sf-based rs to configs

* fix: pipeline and cards - almost ready

* fix: cards

* ranking_and_sf_based_response_selector updates: component card and configs

* fixes according to the PR comments + codestyle fixes

* added sfp to configs

* context fixed

* another fix

* gunicorn -> uvicorn

* fixed some things

* another fix

* attempting to fix rs

* added dialog logging to rs

* fixed prompt-selector deps

* fixing rs

* fixed rs

* fixed critical typo in conf, FastAPI -> Flask for sfc

* reqs fixed

* reqs fixed 2

* more fixes

* fixed sfc 100%

* response selector fixed

* New SFP version

* New SFP version

* dp_formatter fixed for sfp

* sfp formatter changed & sfp FAPI -> Flask

* reqs fixed for sfp

* reqs fixed again

* uvicorn -> gunicorn for sfp

* typo in reqs fixed

* asyncs removed from sfp

* minor dp formatter and rs fix

* many pr fixes

* small sfc revamp

* another pr revampt

* pr fix

* added logging

* fixing batch sfc

* Bug fixes of SFC

* Style fix

* codestyle fixed for pr

* fixes for pr

* removed unused import

* minor pr fix

* minor config addition

* server fixed for sfc

* fix: components

* dream_sfc reverted, minor pr fixes

* fix: ranking and intent based resp selector

* zip_longest -> zip

* fix: codestyle

* sfc service and sfp formatter reverted to working state

* fix: fixes for sf

* fix: first turn

* fix: tests

* fix: tests

* fix: tests

* fix: imports

* fix: black style

* fix: logs

* fix: pipeline

* fix: sfp

* fix: codestyle black

* fix: codestyle

* fix: tests

---------

Co-authored-by: avglinsky <[email protected]>
Co-authored-by: moon-strider <[email protected]>
Co-authored-by: dilyararimovna <[email protected]>
Co-authored-by: lnpetrova <[email protected]>
Co-authored-by: Ilia Dikanskiy <[email protected]>
Co-authored-by: Lida Ostyakova <[email protected]>

annotators/kbqa/tests/test_kbqa.py

@@ -13,7 +13,7 @@
         ),
         (
             {"x_init": ["How old is Donald Trump?"], "entities": [["Donald Trump"]], "entity_tags": [[["per", 1.0]]]},
-            "Donald Trump is 77 years old.",
+            "Donald Trump is 78 years old.",
         ),
     ],
 )

annotators/speech_function_classifier/Dockerfile

@@ -4,21 +4,16 @@ WORKDIR /src
 
 RUN apt-get update && apt-get install -y --allow-unauthenticated git curl && rm -rf /var/lib/apt/lists/*
 
-RUN mkdir /models && \
-    curl https://files.deeppavlov.ai/alexaprize_data/speech_function_classifier.tar.gz --output arch.tgz && \
-    tar zxfv arch.tgz && \
-    rm -rf arch.tgz && \
-    curl https://files.deeppavlov.ai/dream/speech_function_classifier/res_cor.json --output data/res_cor.json && \
-    mv data/* /models && \
-    rm -rf data
+# RUN git clone -b feat/speech-fn https://github.com/deeppavlov/DeepPavlov.git && \
+# 	cd DeepPavlov && pip install -e . && cd .. && rm -rf DeepPavlov && python -m deeppavlov download speech_fn
 
-COPY annotators/speech_function_classifier/requirements.txt requirements.txt
-
-RUN pip install -r requirements.txt && \
-    python -c "import nltk; nltk.download('punkt')"
+RUN pip install git+https://github.com/deeppavlov/DeepPavlov.git@e2ae2d91edad414736d15520eb1820cb6c06565d && \
+        python -m deeppavlov download speech_fn
 
 COPY . ./
 
+RUN pip install -r requirements.txt
+
 ARG SERVICE_NAME
 ENV SERVICE_NAME ${SERVICE_NAME}