[POC][OV] Support OpenVINO as Keras 3 backend #19727

rkazants · 2024-05-17T15:16:58Z

Details: Support OpenVINO as Keras 3 backend. This is inference-only backend. In order to switch on this, define environment variable as follows: os.environ["KERAS_BACKEND"] = "openvino" or use set_backend.

Here is an example how it works:
Install OpenVINO pip install openvino -U

import os
os.environ["KERAS_BACKEND"] = "openvino"
import numpy as np
import keras

# build a simple network
x1 = keras.Input(shape=(2,), dtype='float32')
x2 = keras.Input(shape=(2,), dtype='float32')
add = keras.layers.Add()([x1, x2])
sigmoid = keras.activations.sigmoid(add)
model = keras.Model(inputs={'x1': x1, 'x2': x2}, outputs={'sigmoid': sigmoid})
model.summary()

# infer using OpenVINO
out = model.predict({'x1': np.array([[1.0, 2.0]], dtype=np.float32),
                     'x2': np.array([[3.0, 4.0]], dtype=np.float32)})
print('openvino out = ', out)

Signed-off-by: Kazantsev, Roman <[email protected]>

google-cla · 2024-05-17T15:17:03Z

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

fchollet

Thanks for the PR!

keras/src/ops/function.py

fchollet · 2024-05-17T16:59:57Z

keras/src/layers/layer.py

@@ -289,6 +291,9 @@ def __init__(
        self._convert_input_args = True
        # Whether to allow non-tensors as positional arguments in `call()`.
        self._allow_non_tensor_positional_args = False
+        if backend.backend() == "openvino":


This should not be backend-specific.

Hi @fchollet, please advice how to avoid this code. Keras common requires it to be of KerasTensor type during calling op-by-op. How to relax this check and allow to pass ov type object into ops wrappers?

keras/src/backend/openvino/linalg.py

codecov-commenter · 2024-05-20T04:13:05Z

Codecov Report

Attention: Patch coverage is 0.36101% with 552 lines in your changes missing coverage. Please review.

Project coverage is 78.03%. Comparing base (d85036c) to head (facde41).
Report is 10 commits behind head on master.

Files with missing lines	Patch %	Lines
keras/src/backend/openvino/numpy.py	0.00%	178 Missing ⚠️
keras/src/backend/openvino/core.py	0.00%	114 Missing ⚠️
keras/src/backend/openvino/trainer.py	0.00%	114 Missing ⚠️
keras/src/backend/openvino/nn.py	0.00%	52 Missing ⚠️
keras/src/backend/openvino/__init__.py	0.00%	22 Missing ⚠️
keras/src/backend/openvino/math.py	0.00%	18 Missing ⚠️
keras/src/backend/openvino/random.py	0.00%	14 Missing ⚠️
keras/src/backend/openvino/linalg.py	0.00%	12 Missing ⚠️
keras/src/backend/openvino/rnn.py	0.00%	7 Missing ⚠️
keras/src/backend/exports.py	0.00%	4 Missing ⚠️
... and 6 more

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #19727      +/-   ##
==========================================
- Coverage   78.91%   78.03%   -0.88%     
==========================================
  Files         510      522      +12     
  Lines       48590    49460     +870     
  Branches     8960     9059      +99     
==========================================
+ Hits        38344    38596     +252     
- Misses       8391     9000     +609     
- Partials     1855     1864       +9

Flag	Coverage Δ
keras	`77.89% <0.36%> (-0.88%)`	⬇️
keras-jax	`61.65% <0.18%> (-0.73%)`	⬇️
keras-numpy	`56.86% <0.36%> (-0.70%)`	⬇️
keras-tensorflow	`62.92% <0.18%> (-0.81%)`	⬇️
keras-torch	`61.64% <0.18%> (-0.79%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

gbaned · 2024-06-14T06:44:59Z

Hi @rkazants Can you please resolve the conflicts? Thank you!

rkazants · 2024-06-14T09:07:08Z

Hi @rkazants Can you please resolve the conflicts? Thank you!

I am working on this PR. I am trying to resolve comments.

Best regards,
Roman

gbaned · 2024-07-12T03:05:06Z

Hi @rkazants Any update on this PR? Please. Thank you!

github-actions · 2024-07-27T01:52:10Z

This PR is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.

rkazants · 2024-07-27T12:41:41Z

I am just back from vacation. I plan to continue from the next week.

github-actions · 2024-08-22T01:56:02Z

This PR is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.

gbaned · 2024-09-09T08:37:42Z

Hi @rkazants Any update on this PR? Please. Thank you!

rkazants · 2024-09-09T08:43:15Z

Hi @rkazants Any update on this PR? Please. Thank you!

Hi @gbaned, please anticipate the update early this week. Sorry for the delay.

Thanks,
Roman

…vino_backend

Signed-off-by: Kazantsev, Roman <[email protected]>

rkazants · 2024-09-25T06:18:20Z

@fchollet, please review. all other comments are addressed.

Signed-off-by: Kazantsev, Roman <[email protected]>

rkazants · 2024-09-25T11:02:37Z

@fchollet,
Also, I would suggest separate integration tests for openvino backend. I want to have test suite for each keras operation where it creates one layer model and test inference results. As reference, we can use numpy backend's results, for example. This way we will add more op coverage support step-by-step and validate new functionality. How do you think?

Thanks,
Roman

Signed-off-by: Kazantsev, Roman <[email protected]>

rkazants · 2024-09-26T06:37:22Z

Hi @fchollet, look forward to receive your review:)
I am ready to address your comments asap these days.

Thanks in advance,
Roman

rkazants · 2024-09-27T17:11:12Z

@fchollet, kindly reminder about review.

Thank you,
Roman

[POC][OV] Support OpenVINO as Keras 3 backend

76d4c1a

Signed-off-by: Kazantsev, Roman <[email protected]>

google-ml-butler bot added the size:XL label May 17, 2024

google-ml-butler bot assigned gbaned May 17, 2024

fchollet reviewed May 17, 2024

View reviewed changes

gbaned added the stat:awaiting response from contributor label Jun 14, 2024

google-ml-butler bot removed the stat:awaiting response from contributor label Jun 14, 2024

gbaned added the stat:awaiting response from contributor label Jul 12, 2024

github-actions bot added the stale label Jul 27, 2024

google-ml-butler bot removed stale stat:awaiting response from contributor labels Jul 27, 2024

gbaned added the stat:awaiting response from contributor label Aug 7, 2024

github-actions bot added the stale label Aug 22, 2024

sachinprasadhs removed stat:awaiting response from contributor stale labels Aug 26, 2024

rkazants added 5 commits September 23, 2024 13:56

Merge remote-tracking branch 'upstream/master' into rkazants/poc_open…

af254e9

…vino_backend

Mark all unsupported ops from numpy space

3e0772f

Signed-off-by: Kazantsev, Roman <[email protected]>

Mark unsupported ops in core, image, and linalg spaces

c2c1808

Signed-off-by: Kazantsev, Roman <[email protected]>

Mark unsupported ops in math, nn, random, and rnn spaces

d26c22c

Signed-off-by: Kazantsev, Roman <[email protected]>

Fix sorting imports

9e29a73

Signed-off-by: Kazantsev, Roman <[email protected]>

rkazants added 7 commits September 24, 2024 10:45

Format imports

716a0bb

Signed-off-by: Kazantsev, Roman <[email protected]>

Fix sorting imports

c097126

Signed-off-by: Kazantsev, Roman <[email protected]>

Fix sorting imports

65fe256

Signed-off-by: Kazantsev, Roman <[email protected]>

Fix inference

fe29eb9

Signed-off-by: Kazantsev, Roman <[email protected]>

Remove openvino specific code in common part

f57847b

Signed-off-by: Kazantsev, Roman <[email protected]>

Fix typo

8fb2dc5

Clean-up code

3d8b41a

Signed-off-by: Kazantsev, Roman <[email protected]>

rkazants requested a review from fchollet September 25, 2024 06:18

google-ml-butler bot added the awaiting review label Sep 25, 2024

Recover imports

08b74dc

Signed-off-by: Kazantsev, Roman <[email protected]>

rkazants added 5 commits September 25, 2024 19:40

Sort imports properly

52408b7

Signed-off-by: Kazantsev, Roman <[email protected]>

Format source code

f41b454

Signed-off-by: Kazantsev, Roman <[email protected]>

Format the rest of source code

42364c2

Signed-off-by: Kazantsev, Roman <[email protected]>

Continue format adjustment

954ed1f

Signed-off-by: Kazantsev, Roman <[email protected]>

Add OpenVINO dependency

facde41

Signed-off-by: Kazantsev, Roman <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[POC][OV] Support OpenVINO as Keras 3 backend #19727

[POC][OV] Support OpenVINO as Keras 3 backend #19727

rkazants commented May 17, 2024 •

edited

Loading

google-cla bot commented May 17, 2024

fchollet left a comment

fchollet May 17, 2024

rkazants Sep 25, 2024

codecov-commenter commented May 20, 2024 •

edited

Loading

gbaned commented Jun 14, 2024

rkazants commented Jun 14, 2024

gbaned commented Jul 12, 2024

github-actions bot commented Jul 27, 2024

rkazants commented Jul 27, 2024

github-actions bot commented Aug 22, 2024

gbaned commented Sep 9, 2024

rkazants commented Sep 9, 2024

rkazants commented Sep 25, 2024

rkazants commented Sep 25, 2024 •

edited

Loading

rkazants commented Sep 26, 2024

rkazants commented Sep 27, 2024

[POC][OV] Support OpenVINO as Keras 3 backend #19727

Are you sure you want to change the base?

[POC][OV] Support OpenVINO as Keras 3 backend #19727

Conversation

rkazants commented May 17, 2024 • edited Loading

google-cla bot commented May 17, 2024

fchollet left a comment

Choose a reason for hiding this comment

fchollet May 17, 2024

Choose a reason for hiding this comment

rkazants Sep 25, 2024

Choose a reason for hiding this comment

codecov-commenter commented May 20, 2024 • edited Loading

Codecov Report

gbaned commented Jun 14, 2024

rkazants commented Jun 14, 2024

gbaned commented Jul 12, 2024

github-actions bot commented Jul 27, 2024

rkazants commented Jul 27, 2024

github-actions bot commented Aug 22, 2024

gbaned commented Sep 9, 2024

rkazants commented Sep 9, 2024

rkazants commented Sep 25, 2024

rkazants commented Sep 25, 2024 • edited Loading

rkazants commented Sep 26, 2024

rkazants commented Sep 27, 2024

rkazants commented May 17, 2024 •

edited

Loading

codecov-commenter commented May 20, 2024 •

edited

Loading

rkazants commented Sep 25, 2024 •

edited

Loading