Skip to content

Latest commit

 

History

History
103 lines (77 loc) · 5.73 KB

README.adoc

File metadata and controls

103 lines (77 loc) · 5.73 KB

Image Recognition

Java model inference library for the Inception, MobileNetV1 and MobileNetV2 image recognition architectures. Provides real-time recognition of the LSVRC-2012-CLS categories in the input images.

Inception 1

The ImageRecognition takes an image and outputs a list of probable categories the image contains. The response is represented by RecognitionResponse class.

The JsonMapperFunction permits converting the RecognitionResponse into JSON objects and the ImageRecognitionAugmenter can augment the input image with the detected categories (as shown in pic. 1).

Usage

Add the image-recognition dependency to the pom (use the latest version available):

<dependency>
    <groupId>org.springframework.cloud.fn</groupId>
    <artifactId>image-recognition-function</artifactId>
    <version>${revision}</version>
</dependency>

Example 1: Image Recognition

The following snippet demonstrates how to use the ImageRecognition for detecting the categories present in an input image. It also shows how to convert the result into JSON format and augment the input image with the detected category labels.

ImageRecognition recognitionService = ImageRecognition.mobilenetModeV2(
  "https://storage.googleapis.com/mobilenet_v2/checkpoints/mobilenet_v2_1.4_224.tgz#mobilenet_v2_1.4_224_frozen.pb", //(1)
  224, //(2)
  5, //(3)
  true); //(4)

byte[] inputImage = GraphicsUtils.loadAsByteArray("classpath:/images/giant_panda_in_beijing_zoo_1.jpg"); //(5)

List<RecognitionResponse> recognizedObjects = recognitionService.recognize(inputImage); //(6)

  1. Downloads and loads a pre-trained mobilenet_v2_1.4_224_frozen.pb model. Mind that on first attempt it will download few hundreds of MBs. The consecutive runs will use the cached copy (5) instead. The category labels for the MobileNetV2 are resolved from src/main/resources/labels/mobilenet_labels.txt.

  2. The wxh seize of the input normalized image.

  3. Top K result to return.

  4. Cache the model on the local file system.

  5. Load the image to recognise.

  6. Return a map of the top-k most probable category names and their probabilities.

The ImageRecognition.mobilenetModeV1 and ImageRecognition.inception factory methods help to load and configure pre-trained mobilenetModeV1 and and Inception models.

Next you can convert the result in JSON format.

String jsonRecognizedObjects = new JsonMapperFunction().apply(recognizedObjects);
Sample Image Recognition JSON representation
[{"label":"giant panda","probability":0.9946687817573547},{"label":"Arctic fox","probability":0.0036631098482757807},{"label":"ice bear","probability":3.3782739774324E-4},{"label":"American black bear","probability":2.3452856112271547E-4},{"label":"skunk","probability":1.6454080468975008E-4}]

Use the ImageRecognitionAugmenter to draw the recognise categories on top of the input image.

byte[] augmentedImage = new ImageRecognitionAugmenter().apply(inputImage, recognizedObjects); //(1)
IOUtils.write(augmentedImage, new FileOutputStream("./image-recognition/target/image-augmented.jpg"));//(2)

  1. Augment the image with the recognized categories (uses Java2D internally).

  2. Stores the augmented image as image-augmented.jpg image file.

Augmented image-augmented.jpg file

Augmented

Models

This implementation supports all pre-trained Inception, MobileNetV1 and MobileNetV2 models. Following URI notation can be used to download any of the models directly from the zoo.

http://<zoo model tar.gz url>#<frozen inference graph name.pb>

The <frozen inference graph name.pb> is the frozen model file name within the archive.

Tip
 To speedup the bootstrap performance you may consider extracting the model and caching it locally. Then you can use the file://path-to-my-local-copy URI schema to access it.
Note
 It is important to use the labels that correspond to the model being used! Table below highlights this mapping.