Merge branch 'main' into bit_nncf

.ci/ignore_treon_docker.txt

@@ -1,6 +1,5 @@
 107-speech-recognition-quantization
 108-gpu-device
-111-yolov5-quantization-migration
 117-model-server
 122-quantizing-model-with-accuracy-control
 204-segmenter-semantic-segmentation
@@ -45,6 +44,8 @@
 272-paint-by-example
 273-stable-zephyr-3b-chatbot
 276-stable-diffusion-torchdynamo-backend
+281-kosmos2-multimodal-large-language-model
+283-photo-maker
 301-tensorflow-training-openvino
 305-tensorflow-quantization-aware-training
 404-style-transfer-webcam

.ci/ignore_treon_linux.txt

@@ -1,6 +1,5 @@
 107-speech-recognition-quantization
 108-gpu-device
-111-yolov5-quantization-migration
 112-pytorch-post-training-quantization-nncf
 117-model-server
 122-quantizing-model-with-accuracy-control
@@ -48,4 +47,6 @@
 272-paint-by-example
 273-stable-zephyr-3b-chatbot
 276-stable-diffusion-torchdynamo-backend
+281-kosmos2-multimodal-large-language-model
+283-photo-maker
 404-style-transfer-webcam

.ci/ignore_treon_mac.txt

@@ -1,6 +1,5 @@
 107-speech-recognition-quantization
 108-gpu-device
-111-yolov5-quantization-migration
 117-model-server
 122-quantizing-model-with-accuracy-control
 204-segmenter-semantic-segmentation
@@ -45,5 +44,7 @@
 272-paint-by-example
 273-stable-zephyr-3b-chatbot
 276-stable-diffusion-torchdynamo-backend
+281-kosmos2-multimodal-large-language-model
 279-mobilevlm-language-assistant
+283-photo-maker
 404-style-transfer-webcam

.ci/ignore_treon_win.txt

@@ -1,6 +1,5 @@
 107-speech-recognition-quantization
 108-gpu-device
-111-yolov5-quantization-migration
 117-model-server
 122-quantizing-model-with-accuracy-control
 204-segmenter-semantic-segmentation
@@ -47,3 +46,5 @@
 272-paint-by-example
 273-stable-zephyr-3b-chatbot
 276-stable-diffusion-torchdynamo-backend
+281-kosmos2-multimodal-large-language-model
+283-photo-maker

.ci/spellcheck/.pyspelling.wordlist.txt

@@ -3,6 +3,7 @@ accelerometers
 accelerometer
 acknowledgement
 activations
+adaptively
 adas
 ADE
 aeroplane
@@ -37,6 +38,7 @@ backend
 backends
 Baevski
 BasicUNet
+bboxes
 BEiT
 Belrose
 Benchmarking
@@ -88,6 +90,7 @@ configs
 Connectionist
 ContentVec
 Contrastive
+contrastive
 ControlNet
 controlnet
 ConvE
@@ -99,6 +102,7 @@ ConvNeXt
 ConvNeXts
 Convolutional
 convolutional
+coreference
 CoSENT
 CPUs
 cpu
@@ -299,6 +303,9 @@ KiTS
 Koltun
 Kondate
 Kosaraju
+kosmos
+Kosmos
+KOSMOS
 KServe
 Kubernetes
 Kupyn
@@ -364,6 +371,8 @@ mistralai
 MLS
 mms
 MMS
+MLLM
+MLLMs
 MMVLM
 MLP
 MobileLLaMA
@@ -470,6 +479,7 @@ Patil
 PEFT
 performant
 PersonaGPT
+PhotoMaker
 photorealism
 photorealistic
 Piotr
@@ -602,6 +612,8 @@ Shazeer
 Shutterstock
 siggraph
 sigmoid
+SigLIP
+siglip
 SISR
 SlimOrca
 SlowFast

README.md

@@ -53,6 +53,8 @@ Check out the latest notebooks that show how to optimize and deploy popular mode
 |[Stable Diffusion with IP-Adapter](notebooks/278-stable-diffusion-ip-adapter)<br> | Image conditioning in Stable Diffusion pipeline using IP-Adapter | <img src=https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/182657d9-2aa3-40b3-9fc4-a90b803419fe width=300> |
 | [MobileVLM](notebooks/279-mobilevlm-language-assistant)<br> | Mobile language assistant with MobileVLM and OpenVINO | |
 | [DepthAnything](notebooks/280-depth-anything)<br>[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?filepath=notebooks%2F280-depth-anythingh%2F280-depth-anything.ipynb)<br>[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/main/notebooks/280-depth-anything/280-depth-anything.ipynb) | Monocular Depth estimation with DepthAnything and OpenVINO | <img src=https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/a9a16658-512f-470c-a33c-0e1f9d0ae72c width=300> |
+| [Kosmos-2: Grounding Multimodal Large Language Models](notebooks/281-kosmos2-multimodal-large-language-model)<br> | Kosmos-2: Grounding Multimodal Large Language Model and OpenVINO™ | <img src=https://huggingface.co/microsoft/kosmos-2-patch14-224/resolve/main/annotated_snowman.jpg width=225> |
+| [PhotoMaker](notebooks/283-photo-maker)<br> | Text-to-image generation using PhotoMaker and OpenVINO | <img src=https://github.com/openvinotoolkit/openvino_notebooks/assets/91237924/88bccc4a-5789-42ca-8a68-f402c3e7c5a4 width=300> |
 
 ## Table of Contents
 
@@ -126,7 +128,6 @@ Tutorials that explain how to optimize and quantize models with OpenVINO tools.
 | [108-gpu-device](notebooks/108-gpu-device/) | Working with GPUs in OpenVINO™                                                                                              |
 | [109-performance-tricks](notebooks/109-performance-tricks/)| Performance tricks in OpenVINO™                                                                                             |
 | [110-ct-segmentation-quantize](notebooks/110-ct-segmentation-quantize/)<br>[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?filepath=notebooks%2F110-ct-segmentation-quantize%2F110-ct-scan-live-inference.ipynb) | Quantize a kidney segmentation model and show live inference                                                                | 
-| [111-yolov5-quantization-migration](notebooks/111-yolov5-quantization-migration)<br>[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/main/notebooks/111-yolov5-quantization-migration/111-yolov5-quantization-migration.ipynb) | Migrate YOLOv5 POT API based quantization pipeline on Neural Network Compression Framework (NNCF)                           | <img src = "https://user-images.githubusercontent.com/44352144/177097174-cfe78939-e946-445e-9fce-d8897417ef8e.png"  width=225> |
 | [112-pytorch-post-training-quantization-nncf](notebooks/112-pytorch-post-training-quantization-nncf/) | Use Neural Network Compression Framework (NNCF) to quantize PyTorch model in post-training mode (without model fine-tuning) | 
 | [113-image-classification-quantization](notebooks/113-image-classification-quantization/)<br>[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?labpath=notebooks%2F113-image-classification-quantization%2F113-image-classification-quantization.ipynb)<br>[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/main/notebooks/113-image-classification-quantization/113-image-classification-quantization.ipynb) | Quantize Image Classification model                                                                                     |
 | [115-async-api](notebooks/115-async-api/)<br>[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?labpath=notebooks%2F115-async-api%2F115-async-api.ipynb)<br>[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/main/notebooks/115-async-api/115-async-api.ipynb) | Use Asynchronous Execution to Improve Data Pipelining                                                                       | |
@@ -234,6 +235,9 @@ Demos that demonstrate inference on a particular model.
 | [278-stable-diffusion-ip-adapter](notebooks/278-stable-diffusion-ip-adapter)<br> | Image conditioning in Stable Diffusion pipeline using IP-Adapter |  <img src=https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/182657d9-2aa3-40b3-9fc4-a90b803419fe width=300> |
 | [279-mobilevlm-language-assistant](notebooks/279-mobilevlm-language-assistant)<br> | Mobile language assistant with MobileVLM and OpenVINO | |
 | [280-depth-anything](notebooks/280-depth-anything)<br>[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?filepath=notebooks%2F280-depth-anythingh%2F280-depth-anything.ipynb)<br>[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/main/notebooks/280-depth-anything/280-depth-anything.ipynb) | Monocular Depth Estimation with DepthAnything and OpenVINO |  <img src=https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/a9a16658-512f-470c-a33c-0e1f9d0ae72c width=225> |
+| [281-kosmos2-multimodal-large-language-model](notebooks/281-kosmos2-multimodal-large-language-model)<br> | Kosmos-2: Multimodal Large Language Model and OpenVINO™ | <img src=https://huggingface.co/microsoft/kosmos-2-patch14-224/resolve/main/annotated_snowman.jpg width=225> |
+| [282-siglip-zero-shot-image-classification](notebooks/282-siglip-zero-shot-image-classification)<br>[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/main/notebooks/282-siglip-zero-shot-image-classification/282-siglip-zero-shot-image-classification.ipynb) | Zero-shot Image Classification with SigLIP | <img src=https://github.com/openvinotoolkit/openvino_notebooks/assets/67365453/c4eb782c-0fef-4a89-a5c6-5cc43518490b width=500> |
+| [283-photo-maker](notebooks/283-photo-maker)<br> | Text-to-image generation using PhotoMaker and OpenVINO | <img src=https://github.com/openvinotoolkit/openvino_notebooks/assets/91237924/88bccc4a-5789-42ca-8a68-f402c3e7c5a4 width=225> | 
 
 <div id='-model-training'></div>
 

README_cn.md

@@ -111,7 +111,6 @@ Jupyter notebooks 分为四个大类，选择一个跟你需求相关的开始
 | [108-gpu-device](notebooks/108-gpu-device/) | 在GPU上使用OpenVINO™                                                                                              |
 | [109-performance-tricks](notebooks/109-performance-tricks/)|  OpenVINO™ 的优化技巧                                                                                            |
 | [110-ct-segmentation-quantize](notebooks/110-ct-segmentation-quantize/)<br>[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?filepath=notebooks%2F110-ct-segmentation-quantize%2F110-ct-scan-live-inference.ipynb) | 量化肾脏分割模型并展示实时推理                                                                | 
-| [111-yolov5-quantization-migration](notebooks/111-yolov5-quantization-migration)<br>[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/main/notebooks/111-yolov5-quantization-migration/111-yolov5-quantization-migration.ipynb) | 在神经网络压缩框架(NNCF)上迁移基于YOLOv5 POT API的量化管道                           | <img src = "https://user-images.githubusercontent.com/44352144/177097174-cfe78939-e946-445e-9fce-d8897417ef8e.png"  width=225> |
 | [112-pytorch-post-training-quantization-nncf](notebooks/112-pytorch-post-training-quantization-nncf/) | 利用神经网络压缩框架(NNCF)在后训练模式下来量化PyTorch模型(无需模型微调) | 
 | [113-image-classification-quantization](notebooks/113-image-classification-quantization/)<br>[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?labpath=notebooks%2F113-image-classification-quantization%2F113-image-classification-quantization.ipynb) | 量化mobilenet图片分类模型                                                                                      |
 | [115-async-api](notebooks/115-async-api/)<br>[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?labpath=notebooks%2F115-async-api%2F115-async-api.ipynb)<br>[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/main/notebooks/115-async-api/115-async-api.ipynb) | 使用异步执行改进数据流水线                                                                        | |