Merge remote-tracking branch 'upstream/main' into add_openvoice

openvinotoolkit · Jan 31, 2024 · 5d6b33d · 5d6b33d
2 parents 1f52fe7 + cb38a76
commit 5d6b33d
Show file tree

Hide file tree

Showing 15 changed files with 1,889 additions and 1,094 deletions.
diff --git a/.ci/ignore_treon_docker.txt b/.ci/ignore_treon_docker.txt
@@ -45,6 +45,7 @@
 273-stable-zephyr-3b-chatbot
 276-stable-diffusion-torchdynamo-backend
 281-kosmos2-multimodal-large-language-model
+283-photo-maker
 301-tensorflow-training-openvino
 305-tensorflow-quantization-aware-training
 404-style-transfer-webcam
diff --git a/.ci/ignore_treon_linux.txt b/.ci/ignore_treon_linux.txt
@@ -48,4 +48,5 @@
 273-stable-zephyr-3b-chatbot
 276-stable-diffusion-torchdynamo-backend
 281-kosmos2-multimodal-large-language-model
+283-photo-maker
 404-style-transfer-webcam
diff --git a/.ci/ignore_treon_mac.txt b/.ci/ignore_treon_mac.txt
@@ -46,4 +46,5 @@
 276-stable-diffusion-torchdynamo-backend
 281-kosmos2-multimodal-large-language-model
 279-mobilevlm-language-assistant
+283-photo-maker
 404-style-transfer-webcam
diff --git a/.ci/ignore_treon_win.txt b/.ci/ignore_treon_win.txt
@@ -47,3 +47,4 @@
 273-stable-zephyr-3b-chatbot
 276-stable-diffusion-torchdynamo-backend
 281-kosmos2-multimodal-large-language-model
+283-photo-maker
diff --git a/.ci/spellcheck/.pyspelling.wordlist.txt b/.ci/spellcheck/.pyspelling.wordlist.txt
@@ -3,6 +3,7 @@ accelerometers
 accelerometer
 acknowledgement
 activations
+adaptively
 adas
 ADE
 aeroplane
@@ -477,6 +478,7 @@ Patil
 PEFT
 performant
 PersonaGPT
+PhotoMaker
 photorealism
 photorealistic
 Piotr

diff --git a/.docker/Pipfile.lock b/.docker/Pipfile.lock
diff --git a/README.md b/README.md
@@ -54,6 +54,7 @@ Check out the latest notebooks that show how to optimize and deploy popular mode
 | [MobileVLM](notebooks/279-mobilevlm-language-assistant)<br> | Mobile language assistant with MobileVLM and OpenVINO | |
 | [DepthAnything](notebooks/280-depth-anything)<br>[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?filepath=notebooks%2F280-depth-anythingh%2F280-depth-anything.ipynb)<br>[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/main/notebooks/280-depth-anything/280-depth-anything.ipynb) | Monocular Depth estimation with DepthAnything and OpenVINO | <img src=https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/a9a16658-512f-470c-a33c-0e1f9d0ae72c width=300> |
 | [Kosmos-2: Grounding Multimodal Large Language Models](notebooks/281-kosmos2-multimodal-large-language-model)<br> | Kosmos-2: Grounding Multimodal Large Language Model and OpenVINO™ | <img src=https://huggingface.co/microsoft/kosmos-2-patch14-224/resolve/main/annotated_snowman.jpg width=225> |
+| [PhotoMaker](notebooks/283-photo-maker)<br> | Text-to-image generation using PhotoMaker and OpenVINO | <img src=https://github.com/openvinotoolkit/openvino_notebooks/assets/91237924/88bccc4a-5789-42ca-8a68-f402c3e7c5a4 width=300> |
 | [OpenVoice](notebooks/284-openvoice)<br>[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?filepath=notebooks%2F284-openvoice%2F284-openvoice.ipynb)[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/main/notebooks/284-openvoice/284-openvoice.ipynb) | OpenVoice a versatile instant voice tone transferring and generating speech in various languages. |<img src=https://github.com/openvinotoolkit/openvino_notebooks/assets/5703039/ca7eab80-148d-45b0-84e8-a5a279846b51 width=300> |
 
 ## Table of Contents
@@ -236,6 +237,7 @@ Demos that demonstrate inference on a particular model.
 | [280-depth-anything](notebooks/280-depth-anything)<br>[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?filepath=notebooks%2F280-depth-anythingh%2F280-depth-anything.ipynb)<br>[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/main/notebooks/280-depth-anything/280-depth-anything.ipynb) | Monocular Depth Estimation with DepthAnything and OpenVINO |  <img src=https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/a9a16658-512f-470c-a33c-0e1f9d0ae72c width=225> |
 | [281-kosmos2-multimodal-large-language-model](notebooks/281-kosmos2-multimodal-large-language-model)<br> | Kosmos-2: Multimodal Large Language Model and OpenVINO™ | <img src=https://huggingface.co/microsoft/kosmos-2-patch14-224/resolve/main/annotated_snowman.jpg width=225> |
 | [282-siglip-zero-shot-image-classification](notebooks/282-siglip-zero-shot-image-classification)<br>[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/main/notebooks/282-siglip-zero-shot-image-classification/282-siglip-zero-shot-image-classification.ipynb) | Zero-shot Image Classification with SigLIP | <img src=https://github.com/openvinotoolkit/openvino_notebooks/assets/67365453/c4eb782c-0fef-4a89-a5c6-5cc43518490b width=500> |
+| [283-photo-maker](notebooks/283-photo-maker)<br> | Text-to-image generation using PhotoMaker and OpenVINO | <img src=https://github.com/openvinotoolkit/openvino_notebooks/assets/91237924/88bccc4a-5789-42ca-8a68-f402c3e7c5a4 width=225> | 
 
 <div id='-model-training'></div>
 

diff --git a/notebooks/108-gpu-device/108-gpu-device.ipynb b/notebooks/108-gpu-device/108-gpu-device.ipynb
@@ -1832,7 +1832,7 @@
     "    # Draw info at the top left such as current fps, the devices and the performance hint being used\n",
     "    cv2.putText(frame, f\"fps {str(round(frame_fps[current_frame], 2))}\", (5, 20), cv2.FONT_ITALIC, 0.6, (0, 0, 0), 1, cv2.LINE_AA)\n",
     "    cv2.putText(frame, f\"device {device_name}\", (5, 40), cv2.FONT_ITALIC, 0.6, (0, 0, 0), 1, cv2.LINE_AA) \n",
-    "    cv2.putText(frame, f\"hint {compiled_model.get_property('PERFORMANCE_HINT').name}\", (5, 60), cv2.FONT_ITALIC, 0.6, (0, 0, 0), 1, cv2.LINE_AA)\n",
+    "    cv2.putText(frame, f\"hint {compiled_model.get_property('PERFORMANCE_HINT')}\", (5, 60), cv2.FONT_ITALIC, 0.6, (0, 0, 0), 1, cv2.LINE_AA)\n",
     "\n",
     "    # prediction contains [image_id, label, conf, x_min, y_min, x_max, y_max] according to model\n",
     "    for prediction in np.squeeze(results[current_frame]):\n",