solutions/recommendation_system/recommendation_system.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5525a271",
   "metadata": {},
   "source": [
    "# Recommendation System"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0223ff80",
   "metadata": {},
   "source": [
    "In this example we are creating a movie recommendation system by extractung feature vectors using PaddlePaddle, importing movie vectors into Milvus, and then searching in Milvus and Redis."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b4edf8a",
   "metadata": {},
   "source": [
    "## Data\n",
    "In this project, we use [MovieLens 1M](https://grouplens.org/datasets/movielens/1m/). This dataset contains 1,000,209 anonymous ratings of approximately 3,900 movies made by 6,040 MovieLens users. \n",
    "\n",
    "We use the following files:\n",
    "- movies.dat: Contains movie information.\n",
    "- movie_vectors.txt: Contains movie vectors that can be imported to Milvus easily.\n",
    "\n",
    "File structure:\n",
    "\n",
    " - movMovieID::Title::Genres   \n",
    "\n",
    "     - Titles are identical to titles provided by the IMDB (includingyear of release)\n",
    " \n",
    "     - Genres are pipe-separated\n",
    "\n",
    "     - Some MovieIDs do not correspond to a movie due to accidental duplicate entries and/or test entries\n",
    " \n",
    "    - Movies are mostly entered by hand, so errors and inconsistencies may exist\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ceedfea",
   "metadata": {},
   "source": [
    "## Requirements"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3dfeb422",
   "metadata": {},
   "source": [
    "Due to package constraints, this notebook needs to be run using Python 3.6/3.7 . It is recommended that you use a virtual enviroment like Conda, instructions for installing Conda can be found [here](https://conda.io/projects/conda/en/latest/user-guide/install/index.html). \n",
    "\n",
    "Currently, there is a dirty workaround that you can use for python 3.8. When installing the `requirements.txt`, pip will fail to install`sentencepiece`. If you rerun the notebook after the install fails and avoid redownloading the packages, the rest of the notebook should run without any hiccups.\n",
    "\n",
    "|  Packages |  Servers |\n",
    "| --------------- | -------------- |\n",
    "| pymilvus-orm==2.0.0rc1 | milvus-2.0.0-rc1 |\n",
    "| redis           | redis          |\n",
    "| paddle_serving_app |\n",
    "| paddlepaddle==2.1.1 |\n",
    "\n",
    "\n",
    "We have included a requirements.txt file in order to easily satisfy the required packages. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af42466a-af8f-4f8b-b300-8f97e10b3617",
   "metadata": {},
   "source": [
    "## Up and Running"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b436afa7",
   "metadata": {},
   "source": [
    "### Installing Packages\n",
    "Install the required python packages with `requirements.txt`. If using Python 3.8, look at workaround in the Requirements section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8d30a0db",
   "metadata": {
    "jupyter": {
     "outputs_hidden": true
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting pymilvus-orm==2.0.0rc1\n",
      "  Using cached pymilvus_orm-2.0.0rc1-py3-none-any.whl (35 kB)\n",
      "Collecting paddle_serving_app\n",
      "  Using cached paddle_serving_app-0.6.2-py3-none-any.whl (52 kB)\n",
      "Collecting paddlepaddle==2.1.1\n",
      "  Using cached paddlepaddle-2.1.1-cp36-cp36m-macosx_10_6_intel.whl (72.1 MB)\n",
      "Collecting redis\n",
      "  Using cached redis-3.5.3-py2.py3-none-any.whl (72 kB)\n",
      "Collecting pymilvus==2.0.0rc1\n",
      "  Using cached pymilvus-2.0.0rc1-py3-none-any.whl (70 kB)\n",
      "Collecting pandas==1.1.5\n",
      "  Using cached pandas-1.1.5-cp36-cp36m-macosx_10_9_x86_64.whl (10.2 MB)\n",
      "Requirement already satisfied: six in /Users/mengjiagu/opt/anaconda3/envs/test3.6/lib/python3.6/site-packages (from paddlepaddle==2.1.1->-r requirements.txt (line 3)) (1.16.0)\n",
      "Collecting protobuf>=3.1.0\n",
      "  Using cached protobuf-3.17.3-cp36-cp36m-macosx_10_9_x86_64.whl (1.0 MB)\n",
      "Collecting Pillow\n",
      "  Using cached Pillow-8.3.1-cp36-cp36m-macosx_10_10_x86_64.whl (2.9 MB)\n",
      "Collecting gast>=0.3.3\n",
      "  Using cached gast-0.5.0-py3-none-any.whl (10 kB)\n",
      "Collecting astor\n",
      "  Using cached astor-0.8.1-py2.py3-none-any.whl (27 kB)\n",
      "Collecting decorator==4.4.2\n",
      "  Using cached decorator-4.4.2-py2.py3-none-any.whl (9.2 kB)\n",
      "Collecting numpy>=1.13\n",
      "  Using cached numpy-1.19.5-cp36-cp36m-macosx_10_9_x86_64.whl (15.6 MB)\n",
      "Collecting requests>=2.20.0\n",
      "  Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB)\n",
      "Requirement already satisfied: python-dateutil>=2.7.3 in /Users/mengjiagu/opt/anaconda3/envs/test3.6/lib/python3.6/site-packages (from pandas==1.1.5->pymilvus-orm==2.0.0rc1->-r requirements.txt (line 1)) (2.8.1)\n",
      "Collecting pytz>=2017.2\n",
      "  Using cached pytz-2021.1-py2.py3-none-any.whl (510 kB)\n",
      "Collecting grpcio<1.38.0,>=1.22.0\n",
      "  Using cached grpcio-1.37.1-cp36-cp36m-macosx_10_10_x86_64.whl (3.8 MB)\n",
      "Collecting ujson>=2.0.0\n",
      "  Using cached ujson-4.0.2-cp36-cp36m-macosx_10_14_x86_64.whl (45 kB)\n",
      "Collecting grpcio-tools<1.38.0,>=1.22.0\n",
      "  Using cached grpcio_tools-1.37.1-cp36-cp36m-macosx_10_10_x86_64.whl (2.0 MB)\n",
      "Collecting mmh3\n",
      "  Using cached mmh3-3.0.0-cp36-cp36m-macosx_10_9_x86_64.whl (12 kB)\n",
      "Requirement already satisfied: setuptools in /Users/mengjiagu/opt/anaconda3/envs/test3.6/lib/python3.6/site-packages (from grpcio-tools<1.38.0,>=1.22.0->pymilvus==2.0.0rc1->pymilvus-orm==2.0.0rc1->-r requirements.txt (line 1)) (52.0.0.post20210125)\n",
      "Collecting chardet<5,>=3.0.2\n",
      "  Using cached chardet-4.0.0-py2.py3-none-any.whl (178 kB)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /Users/mengjiagu/opt/anaconda3/envs/test3.6/lib/python3.6/site-packages (from requests>=2.20.0->paddlepaddle==2.1.1->-r requirements.txt (line 3)) (2021.5.30)\n",
      "Collecting urllib3<1.27,>=1.21.1\n",
      "  Using cached urllib3-1.26.6-py2.py3-none-any.whl (138 kB)\n",
      "Collecting idna<3,>=2.5\n",
      "  Using cached idna-2.10-py2.py3-none-any.whl (58 kB)\n",
      "Collecting sentencepiece<=0.1.92\n",
      "  Using cached sentencepiece-0.1.91-cp36-cp36m-macosx_10_6_x86_64.whl (1.1 MB)\n",
      "Collecting pyclipper\n",
      "  Using cached pyclipper-1.3.0-cp36-cp36m-macosx_10_9_x86_64.whl (131 kB)\n",
      "Collecting opencv-python<=4.2.0.32\n",
      "  Using cached opencv_python-4.2.0.32-cp36-cp36m-macosx_10_9_x86_64.whl (47.9 MB)\n",
      "Collecting shapely\n",
      "  Using cached Shapely-1.7.1-cp36-cp36m-macosx_10_9_x86_64.whl (1.0 MB)\n",
      "Installing collected packages: urllib3, protobuf, idna, grpcio, chardet, ujson, requests, pytz, numpy, mmh3, grpcio-tools, shapely, sentencepiece, pymilvus, pyclipper, Pillow, pandas, opencv-python, gast, decorator, astor, redis, pymilvus-orm, paddlepaddle, paddle-serving-app\n",
      "  Attempting uninstall: decorator\n",
      "    Found existing installation: decorator 5.0.9\n",
      "    Uninstalling decorator-5.0.9:\n",
      "      Successfully uninstalled decorator-5.0.9\n",
      "Successfully installed Pillow-8.3.1 astor-0.8.1 chardet-4.0.0 decorator-4.4.2 gast-0.5.0 grpcio-1.37.1 grpcio-tools-1.37.1 idna-2.10 mmh3-3.0.0 numpy-1.19.5 opencv-python-4.2.0.32 paddle-serving-app-0.6.2 paddlepaddle-2.1.1 pandas-1.1.5 protobuf-3.17.3 pyclipper-1.3.0 pymilvus-2.0.0rc1 pymilvus-orm-2.0.0rc1 pytz-2021.1 redis-3.5.3 requests-2.25.1 sentencepiece-0.1.91 shapely-1.7.1 ujson-4.0.2 urllib3-1.26.6\n"
     ]
    }
   ],
   "source": [
    "! pip install -r requirements.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "433f6399",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Starting Milvus Server\n",
    "\n",
    "This demo uses Milvus 2.0 Standalone with docker-compose, please refer to [Install Milvus 2.0](https://milvus.io/docs/v2.0.0/install_standalone-docker.md) for other installation options (on Kubernetes or use Milvus Cluster)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "c877e285",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2021-07-09 16:36:06--  https://raw.githubusercontent.com/milvus-io/milvus/master/deployments/docker/standalone/docker-compose.yml\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 1297 (1.3K) [text/plain]\n",
      "Saving to: ‘docker-compose.yml’\n",
      "\n",
      "docker-compose.yml  100%[===================>]   1.27K  --.-KB/s    in 0s      \n",
      "\n",
      "2021-07-09 16:36:06 (32.6 MB/s) - ‘docker-compose.yml’ saved [1297/1297]\n",
      "\n",
      "\u001b[1A\u001b[1B\u001b[0G\u001b[?25l[+] Running 0/0\n",
      "\u001b[37m ⠋ Network milvus  Creating                                                0.0s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠙ Network milvus  Creating                                                0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠹ Network milvus  Creating                                                0.2s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠸ Network milvus  Creating                                                0.3s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠼ Network milvus  Creating                                                0.4s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠴ Network milvus  Creating                                                0.5s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠦ Network milvus  Creating                                                0.6s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠧ Network milvus  Creating                                                0.7s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠇ Network milvus  Creating                                                0.8s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠏ Network milvus  Creating                                                0.9s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠋ Network milvus  Creating                                                1.0s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠙ Network milvus  Creating                                                1.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠹ Network milvus  Creating                                                1.2s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠸ Network milvus  Creating                                                1.3s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠼ Network milvus  Creating                                                1.4s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠴ Network milvus  Creating                                                1.5s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠦ Network milvus  Creating                                                1.6s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠧ Network milvus  Creating                                                1.7s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠇ Network milvus  Creating                                                1.8s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠏ Network milvus  Creating                                                1.9s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠋ Network milvus  Creating                                                2.0s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠙ Network milvus  Creating                                                2.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠹ Network milvus  Creating                                                2.2s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠸ Network milvus  Creating                                                2.3s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠼ Network milvus  Creating                                                2.4s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠴ Network milvus  Creating                                                2.5s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠦ Network milvus  Creating                                                2.6s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠧ Network milvus  Creating                                                2.7s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠇ Network milvus  Creating                                                2.8s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠏ Network milvus  Creating                                                2.9s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠋ Network milvus  Creating                                                3.0s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠙ Network milvus  Creating                                                3.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠹ Network milvus  Creating                                                3.2s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠸ Network milvus  Creating                                                3.3s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠼ Network milvus  Creating                                                3.4s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠴ Network milvus  Creating                                                3.5s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠦ Network milvus  Creating                                                3.6s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠧ Network milvus  Creating                                                3.7s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠇ Network milvus  Creating                                                3.8s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠏ Network milvus  Creating                                                3.9s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 0/1\n",
      "\u001b[37m ⠋ Network milvus  Creating                                                4.0s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l\u001b[34m[+] Running 1/1\u001b[0m\n",
      "\u001b[34m ⠿ Network milvus          Created                                         4.1s\n",
      "\u001b[0m\u001b[37m ⠋ Container milvus-minio  Creating                                        0.0s\n",
      "\u001b[0m\u001b[37m ⠋ Container milvus-etcd   Creating                                        0.0s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l\u001b[34m[+] Running 3/3\u001b[0m\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Created                                    0.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Created                                    0.1s\n",
      "\u001b[0m\u001b[37m ⠋ Container milvus-standalone  Creating                                   0.0s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   0.2s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   0.2s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   0.3s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   0.3s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   0.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   0.4s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   0.5s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   0.5s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   0.6s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   0.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   0.7s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   0.7s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   0.8s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   0.8s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   0.9s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   0.9s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   1.0s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   1.0s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   1.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   1.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   1.2s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   1.2s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   1.3s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   1.3s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   1.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   1.4s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   1.5s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   1.5s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   1.6s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   1.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   1.7s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   1.7s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   1.8s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   1.8s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   1.9s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   1.9s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   2.0s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   2.0s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   2.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   2.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   2.2s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   2.2s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   2.3s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   2.3s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   2.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   2.4s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   2.5s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   2.5s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   2.6s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   2.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   2.7s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   2.7s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   2.8s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   2.8s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   2.9s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   2.9s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   3.0s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   3.0s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   3.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   3.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   3.2s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   3.2s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   3.3s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   3.3s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 2/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-etcd        Starting                                   3.4s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-minio       Starting                                   3.5s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l\u001b[34m[+] Running 4/4\u001b[0m\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l\u001b[34m[+] Running 4/4\u001b[0m\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l\u001b[34m[+] Running 4/4\u001b[0m\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l\u001b[34m[+] Running 4/4\u001b[0m\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l\u001b[34m[+] Running 4/4\u001b[0m\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Created                                    0.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   4.0s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   4.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   4.2s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   4.3s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   4.4s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   4.5s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   4.6s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   4.7s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   4.8s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   4.9s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   5.0s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   5.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   5.2s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   5.3s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   5.4s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   5.5s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   5.6s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   5.7s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   5.8s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   5.9s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   6.0s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   6.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   6.2s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   6.3s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   6.4s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   6.5s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   6.6s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   6.7s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   6.8s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   6.9s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   7.0s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   7.1s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   7.2s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   7.3s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   7.4s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   7.5s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   7.6s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 3/4\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[37m ⠿ Container milvus-standalone  Starting                                   7.7s\n",
      "\u001b[0m\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l\u001b[34m[+] Running 4/4\u001b[0m\n",
      "\u001b[34m ⠿ Network milvus               Created                                    4.1s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-minio       Started                                    3.6s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-etcd        Started                                    3.4s\n",
      "\u001b[0m\u001b[34m ⠿ Container milvus-standalone  Started                                    7.7s\n",
      "\u001b[0m\u001b[?25h"
     ]
    }
   ],
   "source": [
    "!wget https://raw.githubusercontent.com/milvus-io/milvus/master/deployments/docker/standalone/docker-compose.yml -O docker-compose.yml\n",
    "!docker-compose up -d"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c2d62e7",
   "metadata": {},
   "source": [
    "### Starting Redis Server\n",
    "We are using Redis as a metadata storage service. Code can easily be modified to use a python dictionary, but that usually does not work in any use case outside of quick examples. We need a metadata storage service in order to be able to be able to map between embeddings and the corresponding data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "b4b9865e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "23e57c4ac4ba221189c3b51284e76b5663ba19122062df88a213b5dbfce69406\r\n"
     ]
    }
   ],
   "source": [
    "!docker run  --name redis -d -p 6379:6379 redis"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6f7c8da-ffa4-453c-bbcc-1275dfc4d444",
   "metadata": {},
   "source": [
    "### Confirm Running Servers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "6df36314-53de-437c-8ede-d8ac4431581b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CONTAINER ID   IMAGE          COMMAND                  CREATED          STATUS                             PORTS                                           NAMES\r\n",
      "23e57c4ac4ba   redis          \"docker-entrypoint.s…\"   4 seconds ago    Up 3 seconds                       0.0.0.0:6379->6379/tcp, :::6379->6379/tcp       redis\r\n",
      "582c48fa905a   77552fa61aeb   \"/tini -- milvus run…\"   30 seconds ago   Up 21 seconds                      0.0.0.0:19530->19530/tcp, :::19530->19530/tcp   milvus-standalone\r\n",
      "a4d210b6f249   61ad63875109   \"etcd -listen-peer-u…\"   30 seconds ago   Up 26 seconds (health: starting)   2379-2380/tcp                                   milvus-etcd\r\n",
      "f32fac178bc3   b849e0c26c4c   \"/usr/bin/docker-ent…\"   30 seconds ago   Up 26 seconds (health: starting)   9000/tcp                                        milvus-minio\r\n"
     ]
    }
   ],
   "source": [
    "! docker ps"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cac14f13",
   "metadata": {},
   "source": [
    "### Downloading Pretrained Models\n",
    "This PaddlePaddle model is used to transform user information into vectors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "b52f00c3",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2021-07-09 16:36:55--  https://paddlerec.bj.bcebos.com/aistudio/user_vector.tar.gz\n",
      "Resolving paddlerec.bj.bcebos.com (paddlerec.bj.bcebos.com)... 103.235.46.61\n",
      "Connecting to paddlerec.bj.bcebos.com (paddlerec.bj.bcebos.com)|103.235.46.61|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 33650924 (32M) [application/x-gzip]\n",
      "Saving to: ‘user_vector.tar.gz’\n",
      "\n",
      "user_vector.tar.gz  100%[===================>]  32.09M  2.62MB/s    in 39s     \n",
      "\n",
      "2021-07-09 16:37:36 (844 KB/s) - ‘user_vector.tar.gz’ saved [33650924/33650924]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget https://paddlerec.bj.bcebos.com/aistudio/user_vector.tar.gz --no-check-certificate\n",
    "!mkdir movie_recommender\n",
    "!mkdir movie_recommender/user_vector_model\n",
    "!tar xf user_vector.tar.gz -C movie_recommender/user_vector_model/\n",
    "!rm user_vector.tar.gz"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ab3a252",
   "metadata": {},
   "source": [
    "### Downloading Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "39a7facb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2021-07-09 16:37:42--  https://paddlerec.bj.bcebos.com/aistudio/movies.dat\n",
      "Resolving paddlerec.bj.bcebos.com (paddlerec.bj.bcebos.com)... 103.235.46.61\n",
      "Connecting to paddlerec.bj.bcebos.com (paddlerec.bj.bcebos.com)|103.235.46.61|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 171308 (167K) [application/octet-stream]\n",
      "Saving to: ‘movie_recommender/movies.dat’\n",
      "\n",
      "movies.dat          100%[===================>] 167.29K  94.2KB/s    in 1.8s    \n",
      "\n",
      "2021-07-09 16:37:45 (94.2 KB/s) - ‘movie_recommender/movies.dat’ saved [171308/171308]\n",
      "\n",
      "--2021-07-09 16:37:45--  https://paddlerec.bj.bcebos.com/aistudio/movie_vectors.txt\n",
      "Resolving paddlerec.bj.bcebos.com (paddlerec.bj.bcebos.com)... 103.235.46.61\n",
      "Connecting to paddlerec.bj.bcebos.com (paddlerec.bj.bcebos.com)|103.235.46.61|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 1095505 (1.0M) [text/plain]\n",
      "Saving to: ‘movie_recommender/movie_vectors.txt’\n",
      "\n",
      "movie_vectors.txt   100%[===================>]   1.04M   228KB/s    in 4.7s    \n",
      "\n",
      "2021-07-09 16:37:52 (228 KB/s) - ‘movie_recommender/movie_vectors.txt’ saved [1095505/1095505]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Download movie information\n",
    "!wget -P movie_recommender https://paddlerec.bj.bcebos.com/aistudio/movies.dat --no-check-certificate\n",
    "# Download movie vecotrs\n",
    "!wget -P movie_recommender https://paddlerec.bj.bcebos.com/aistudio/movie_vectors.txt --no-check-certificate"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "090710b4",
   "metadata": {},
   "source": [
    "## Code Overview\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e994eb1e-aa76-446b-98c6-02c74f050ba5",
   "metadata": {},
   "source": [
    "### Importing Movies into Milvus"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a999eeb-bcc6-4800-9039-f9c57ea399f1",
   "metadata": {},
   "source": [
    "#### 1. Connectings to Milvus and Redis\n",
    "Both servers are running as Docker containers on the localhost with their corresponding default ports."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d8de5e40",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pymilvus_orm import *\n",
    "import redis\n",
    "\n",
    "connections.connect()\n",
    "# connections.connect(\"default\", host=\"localhost\", port=\"19530\")\n",
    "r = redis.StrictRedis(host=\"localhost\", port=6379) "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3c114a7",
   "metadata": {},
   "source": [
    "#### 2. Loading Movies into Redis\n",
    "We begin by loading all the movie files into redis. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "f56cf19c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import codecs\n",
    "\n",
    "#1::Toy Story (1995)::Animation|Children's|Comedy\n",
    "def process_movie(lines, redis_cli):\n",
    "    for line in lines:\n",
    "        if len(line.strip()) == 0:\n",
    "            continue\n",
    "        tmp = line.strip().split(\"::\")\n",
    "        movie_id = tmp[0]\n",
    "        title = tmp[1]\n",
    "        genre_group = tmp[2]\n",
    "        tmp = genre_group.strip().split(\"|\")\n",
    "        genre = tmp\n",
    "        movie_info = {\"movie_id\" : movie_id,\n",
    "                \"title\" : title,\n",
    "                \"genre\" : genre\n",
    "                }\n",
    "        redis_cli.set(\"{}##movie_info\".format(movie_id), json.dumps(movie_info))\n",
    "        \n",
    "with codecs.open(\"movie_recommender/movies.dat\", \"r\",encoding='utf-8',errors='ignore') as f:\n",
    "        lines = f.readlines()\n",
    "        process_movie(lines, r)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a54a6046",
   "metadata": {},
   "source": [
    "#### 3. Creating Partition and Collection in Milvus"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "ef3ef1f7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collection & partition are successfully created.\n"
     ]
    }
   ],
   "source": [
    "COLLECTION_NAME = 'demo_films'\n",
    "PARTITION_NAME = 'Movie'\n",
    "\n",
    "pk = FieldSchema(name='pk', dtype=DataType.INT64, is_primary=True, auto_id=False)\n",
    "field = FieldSchema(name='vec', dtype=DataType.FLOAT_VECTOR, dim=32)\n",
    "schema = CollectionSchema(fields=[pk, field], description=\"movie recommendation: demo films\")\n",
    "\n",
    "if utility.get_connection().has_collection(COLLECTION_NAME): # drop the same collection created before\n",
    "    collection = Collection(COLLECTION_NAME)\n",
    "    collection.drop()\n",
    "else:\n",
    "    collection = Collection(name=COLLECTION_NAME, schema=schema)\n",
    "    partition = collection.create_partition(PARTITION_NAME)\n",
    "    print(\"Collection & partition are successfully created.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d298372e",
   "metadata": {},
   "source": [
    "#### 4. Getting Embeddings and IDs\n",
    "The vectors in `movie_vectors.txt` are obtained from the `user_vector_model` downloaded above. So we can directly get the vectors and the IDs by reading the file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "1aaee36b",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_vectors():\n",
    "    with codecs.open(\"movie_recommender/movie_vectors.txt\", \"r\", encoding='utf-8', errors='ignore') as f:\n",
    "        lines = f.readlines()\n",
    "    ids = [int(line.split(\":\")[0]) for line in lines]\n",
    "    embeddings = []\n",
    "    for line in lines:\n",
    "        line = line.strip().split(\":\")[1][1:-1]\n",
    "        str_nums = line.split(\",\")\n",
    "        emb = [float(x) for x in str_nums]\n",
    "        embeddings.append(emb)\n",
    "    return ids, embeddings\n",
    "\n",
    "ids, embeddings = get_vectors()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a6140b1",
   "metadata": {},
   "source": [
    "#### 4. Importing Vectors into Milvus\n",
    "Import vectors into the partition **Movie** under the collection **demo_films**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "4ac4cfff",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Record count in collection: 3883\n"
     ]
    }
   ],
   "source": [
    "if collection.num_entities != 0:\n",
    "    print(COLLECTION_NAME + \" is not empty!\")  \n",
    "else:\n",
    "    mr = collection.insert(data=[ids,embeddings], partition_name=PARTITION_NAME)\n",
    "\n",
    "print(\"Record count in collection: \" + str(collection.num_entities))\n",
    "# print(str(len(mr.primary_keys)) + \" ids:\\n\", mr.primary_keys[:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c84a8a2c",
   "metadata": {},
   "source": [
    "### Build Index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "a052763e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Status(code=0, message='')"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Flush collection with inserted vectors to disk\n",
    "# utility.get_connection().flush([COLLECTION_NAME])\n",
    "\n",
    "index_param = {\n",
    "    \"metric_type\": \"L2\",\n",
    "    \"index_type\":\"IVF_FLAT\",\n",
    "    \"params\":{\"nlist\":128}\n",
    "}\n",
    "\n",
    "collection.create_index(field_name=\"vec\", index_params=index_param)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e93feb30",
   "metadata": {},
   "source": [
    "### Recalling Vectors in Milvus\n",
    "#### 1. Genarating User Embeddings\n",
    "Pass in the gender, age and occupation of the user we want to recommend. **user_vector_model** model will generate the corresponding user vector.\n",
    "Occupation is chosen from the following choices:\n",
    "*  0:  \"other\" or not specified\n",
    "*  1:  \"academic/educator\"\n",
    "*  2:  \"artist\"\n",
    "*  3:  \"clerical/admin\"\n",
    "*  4:  \"college/grad student\"\n",
    "*  5:  \"customer service\"\n",
    "*  6:  \"doctor/health care\"\n",
    "*  7:  \"executive/managerial\"\n",
    "*  8:  \"farmer\"\n",
    "*  9:  \"homemaker\"\n",
    "*  10:  \"K-12 student\"\n",
    "*  11:  \"lawyer\"\n",
    "*  12:  \"programmer\"\n",
    "*  13:  \"retired\"\n",
    "*  14:  \"sales/marketing\"\n",
    "*  15:  \"scientist\"\n",
    "*  16:  \"self-employed\"\n",
    "*  17:  \"technician/engineer\"\n",
    "*  18:  \"tradesman/craftsman\"\n",
    "*  19:  \"unemployed\"\n",
    "*  20:  \"writer\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "1a35a9d4",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-07-09 16:39:29,170 - INFO - LocalPredictor load_model_config params: model_path:movie_recommender/user_vector_model/serving_server_dir, use_gpu:False, gpu_id:0, use_profile:False, thread_num:1, mem_optim:True, ir_optim:False, use_trt:False, use_lite:False, use_xpu:False, precision:fp32, use_calib:False, use_mkldnn:False, mkldnn_cache_capacity:0, mkldnn_op_list:None, mkldnn_bf16_op_list:None, use_feed_fetch_ops:False, \n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from paddle_serving_app.local_predict import LocalPredictor\n",
    "\n",
    "class RecallServerServicer(object):\n",
    "    def __init__(self):\n",
    "        self.uv_client = LocalPredictor()\n",
    "        self.uv_client.load_model_config(\"movie_recommender/user_vector_model/serving_server_dir\") \n",
    "        \n",
    "    def hash2(self, a):\n",
    "        return hash(a) % 1000000\n",
    "\n",
    "    def get_user_vector(self):\n",
    "        dic = {\"userid\": [], \"gender\": [], \"age\": [], \"occupation\": []}\n",
    "        lod = [0]\n",
    "        dic[\"userid\"].append(self.hash2('0'))\n",
    "        dic[\"gender\"].append(self.hash2('M'))\n",
    "        dic[\"age\"].append(self.hash2('23'))\n",
    "        dic[\"occupation\"].append(self.hash2('6'))\n",
    "        lod.append(1)\n",
    "\n",
    "        dic[\"userid.lod\"] = lod\n",
    "        dic[\"gender.lod\"] = lod\n",
    "        dic[\"age.lod\"] = lod\n",
    "        dic[\"occupation.lod\"] = lod\n",
    "        for key in dic:\n",
    "            dic[key] = np.array(dic[key]).astype(np.int64).reshape(len(dic[key]),1)\n",
    "        fetch_map = self.uv_client.predict(feed=dic, fetch=[\"save_infer_model/scale_0.tmp_1\"], batch=True)\n",
    "        return fetch_map[\"save_infer_model/scale_0.tmp_1\"].tolist()[0]\n",
    "\n",
    "recall = RecallServerServicer()\n",
    "user_vector = recall.get_user_vector()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e15ea6e8",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### 2. Searching\n",
    "Pass in the user vector, and then recall vectors in the previously imported data collection and partition."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "e4d91d02",
   "metadata": {},
   "outputs": [],
   "source": [
    "collection.load() # load collection memory before search\n",
    "\n",
    "topK = 20\n",
    "SEARCH_PARAM = {\n",
    "    \"metric_type\":\"L2\",\n",
    "    \"params\":{\"nprobe\": 20},\n",
    "    }\n",
    "results = collection.search([user_vector],\"vec\",param=SEARCH_PARAM, limit=topK, expr=None, output_fields=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c847608",
   "metadata": {},
   "source": [
    "#### 3. Returning Information by IDs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "90a56325",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['{\"movie_id\": \"760\", \"title\": \"Stalingrad (1993)\", \"genre\": [\"War\"]}',\n",
       " '{\"movie_id\": \"1350\", \"title\": \"Omen, The (1976)\", \"genre\": [\"Horror\"]}',\n",
       " '{\"movie_id\": \"1258\", \"title\": \"Shining, The (1980)\", \"genre\": [\"Horror\"]}',\n",
       " '{\"movie_id\": \"632\", \"title\": \"Land and Freedom (Tierra y libertad) (1995)\", \"genre\": [\"War\"]}',\n",
       " '{\"movie_id\": \"3007\", \"title\": \"American Movie (1999)\", \"genre\": [\"Documentary\"]}',\n",
       " '{\"movie_id\": \"2086\", \"title\": \"One Magic Christmas (1985)\", \"genre\": [\"Drama\", \"Fantasy\"]}',\n",
       " '{\"movie_id\": \"1303\", \"title\": \"Man Who Would Be King, The (1975)\", \"genre\": [\"Adventure\"]}',\n",
       " '{\"movie_id\": \"3920\", \"title\": \"Faraway, So Close (In Weiter Ferne, So Nah!) (1993)\", \"genre\": [\"Drama\", \"Fantasy\"]}',\n",
       " '{\"movie_id\": \"1051\", \"title\": \"Trees Lounge (1996)\", \"genre\": [\"Drama\"]}',\n",
       " '{\"movie_id\": \"1605\", \"title\": \"Excess Baggage (1997)\", \"genre\": [\"Adventure\", \"Romance\"]}',\n",
       " '{\"movie_id\": \"652\", \"title\": \"301, 302 (1995)\", \"genre\": [\"Mystery\"]}',\n",
       " '{\"movie_id\": \"1275\", \"title\": \"Highlander (1986)\", \"genre\": [\"Action\", \"Adventure\"]}',\n",
       " '{\"movie_id\": \"1126\", \"title\": \"Drop Dead Fred (1991)\", \"genre\": [\"Comedy\", \"Fantasy\"]}',\n",
       " '{\"movie_id\": \"792\", \"title\": \"Hungarian Fairy Tale, A (1987)\", \"genre\": [\"Fantasy\"]}',\n",
       " '{\"movie_id\": \"2228\", \"title\": \"Mountain Eagle, The (1926)\", \"genre\": [\"Drama\"]}',\n",
       " '{\"movie_id\": \"2659\", \"title\": \"It Came from Hollywood (1982)\", \"genre\": [\"Comedy\", \"Documentary\"]}',\n",
       " '{\"movie_id\": \"2545\", \"title\": \"Relax... It\\'s Just Sex (1998)\", \"genre\": [\"Comedy\"]}',\n",
       " '{\"movie_id\": \"1289\", \"title\": \"Koyaanisqatsi (1983)\", \"genre\": [\"Documentary\", \"War\"]}',\n",
       " '{\"movie_id\": \"3743\", \"title\": \"Boys and Girls (2000)\", \"genre\": [\"Comedy\", \"Romance\"]}',\n",
       " '{\"movie_id\": \"137\", \"title\": \"Man of the Year (1995)\", \"genre\": [\"Documentary\"]}']"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "I = []\n",
    "for x in results:\n",
    "    for y in x.ids:\n",
    "        I.append(y)\n",
    "        \n",
    "recall_results = []\n",
    "for x in I:\n",
    "    recall_results.append(r.get(\"{}##movie_info\".format(x)).decode('utf-8'))\n",
    "recall_results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4f7e3c5",
   "metadata": {},
   "source": [
    "## Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "843120ee",
   "metadata": {},
   "source": [
    "After completing the recall service, the results can be further sorted using the **movie_recommender** model, and then the movies with high similarity scores can be recommended to users. You can try this deployable recommendation system using this [quick start](QUICK_START.md)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:test3.6]",
   "language": "python",
   "name": "conda-env-test3.6-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}