feat: support react-native-vision-camera v3 #32
Conversation
ismaelsousa
changed the title
|
Just tested this on ios with v3 of vision camera and it's working good! I've been struggling to implement ml-kit text recognition frame processor for v3 for past 2 days and had no luck. Appreciate your work @ismaelsousa |
ismaelsousa
changed the title
|
Hello, How to update to this version? |
|
@ismaelsousa sorry for the dumb question, but how do I use your changes in my project? Could you maybe publish a package with your updates? |
|
@crly @DeluxeOwl you can directly install the branch on your deps. like this: |
I'd like to the owner publish a new version. I contacted him, let's see :) |
Can not install |
|
@ismaelsousa I'm running on Android。 Frame Processor threw an error: Exception in HostFunction: java.lang.IllegalArgumentException: Only JPEG and YUV_420_888 are supported now 
Does it only support Only JPEG and YUV_420_888 formats? |
"vision-camera-ocr": "https://github.com/ismaelsousa/vision-camera-ocr/tree/94dc64b211b38976b654651c602554741c514a24" |
|
@suwu150 Currently yes, you need to add: |
error https://github.com/ismaelsousa/vision-camera-ocr/tree/94dc64b211b38976b654651c602554741c514a24: Extracting tar content of undefined failed, the file appears to be corrupt: "Invalid tar header. Maybe the tar is corrupted or it needs to be gunzipped?" |
What is your node version? Mine is 16.14.0. Try this version. |
|
Awesome stuff ismael! |
|
@PereiraMatheuSilva also receiving the same error + same versions |
|
Possible fix patch: diff --git a/node_modules/@ismaelmoreiraa/vision-camera-ocr/ios/VisionCameraOcr.m b/node_modules/@ismaelmoreiraa/vision-camera-ocr/ios/VisionCameraOcr.m
index d922d2a..22dc677 100644
--- a/node_modules/@ismaelmoreiraa/vision-camera-ocr/ios/VisionCameraOcr.m
+++ b/node_modules/@ismaelmoreiraa/vision-camera-ocr/ios/VisionCameraOcr.m
@@ -4,16 +4,4 @@
#import "VisionCameraOcr-Swift.h"
-@interface OCRFrameProcessorPlugin (FrameProcessorPluginLoader)
-@end
-
-@implementation OCRFrameProcessorPlugin (FrameProcessorPluginLoader)
-
-+ (void)load
-{
- [FrameProcessorPluginRegistry addFrameProcessorPlugin:@"scanOCR" withInitializer:^FrameProcessorPlugin * _Nonnull(NSDictionary * _Nullable options) {
- return [[OCRFrameProcessorPlugin alloc] initWithOptions:options];
- }];
-}
-
-@end
+VISION_EXPORT_SWIFT_FRAME_PROCESSOR(OCRFrameProcessorPlugin, scanOCR) |
Doesn't work. Also, isn't that for IOS? I (we?) get the error while building for android |
|
That's my error: |
|
ismaelsousa#6 Updated diff --git a/node_modules/@ismaelmoreiraa/vision-camera-ocr/android/src/main/java/com/visioncameraocr/OCRFrameProcessorPlugin.kt b/node_modules/@ismaelmoreiraa/vision-camera-ocr/android/src/main/java/com/visioncameraocr/OCRFrameProcessorPlugin.kt
index d87251b..a5bda18 100644
--- a/node_modules/@ismaelmoreiraa/vision-camera-ocr/android/src/main/java/com/visioncameraocr/OCRFrameProcessorPlugin.kt
+++ b/node_modules/@ismaelmoreiraa/vision-camera-ocr/android/src/main/java/com/visioncameraocr/OCRFrameProcessorPlugin.kt
@@ -17,7 +17,7 @@
import com.mrousavy.camera.frameprocessor.FrameProcessorPlugin
import com.mrousavy.camera.types.Orientation
- class OCRFrameProcessorPlugin(options: MutableMap<String, Any>?) : FrameProcessorPlugin(options) {
+ class OCRFrameProcessorPlugin(options: MutableMap<String, Any>?) : FrameProcessorPlugin() {
private fun getBlockArray(blocks: MutableList<Text.TextBlock>): List<HashMap<String, Any?>> {
val blockArray = mutableListOf<HashMap<String, Any?>>()
diff --git a/node_modules/@ismaelmoreiraa/vision-camera-ocr/android/src/main/java/com/visioncameraocr/VisionCameraOcrPackage.kt b/node_modules/@ismaelmoreiraa/vision-camera-ocr/android/src/main/java/com/visioncameraocr/VisionCameraOcrPackage.kt
index 00cc15e..c6c7a54 100644
--- a/node_modules/@ismaelmoreiraa/vision-camera-ocr/android/src/main/java/com/visioncameraocr/VisionCameraOcrPackage.kt
+++ b/node_modules/@ismaelmoreiraa/vision-camera-ocr/android/src/main/java/com/visioncameraocr/VisionCameraOcrPackage.kt
@@ -12,8 +12,8 @@
override fun createNativeModules(reactContext: ReactApplicationContext): List<NativeModule> {
if(!OCRFrameProcessorPlugin.isRegistered){
OCRFrameProcessorPlugin.isRegistered = true
- FrameProcessorPluginRegistry.addFrameProcessorPlugin("scanOCR"){
- OCRFrameProcessorPlugin(it)
+ FrameProcessorPluginRegistry.addFrameProcessorPlugin("scanOCR"){ _ , options ->
+ OCRFrameProcessorPlugin(options)
}
}
return emptyList()
diff --git a/node_modules/@ismaelmoreiraa/vision-camera-ocr/ios/VisionCameraOcr.m b/node_modules/@ismaelmoreiraa/vision-camera-ocr/ios/VisionCameraOcr.m
index d922d2a..22dc677 100644
--- a/node_modules/@ismaelmoreiraa/vision-camera-ocr/ios/VisionCameraOcr.m
+++ b/node_modules/@ismaelmoreiraa/vision-camera-ocr/ios/VisionCameraOcr.m
@@ -4,16 +4,4 @@
#import "VisionCameraOcr-Swift.h"
-@interface OCRFrameProcessorPlugin (FrameProcessorPluginLoader)
-@end
-
-@implementation OCRFrameProcessorPlugin (FrameProcessorPluginLoader)
-
-+ (void)load
-{
- [FrameProcessorPluginRegistry addFrameProcessorPlugin:@"scanOCR" withInitializer:^FrameProcessorPlugin * _Nonnull(NSDictionary * _Nullable options) {
- return [[OCRFrameProcessorPlugin alloc] initWithOptions:options];
- }];
-}
-
-@end
+VISION_EXPORT_SWIFT_FRAME_PROCESSOR(OCRFrameProcessorPlugin, scanOCR)
|
|
Hey guy, sorry for the confusion. From now on, please always check the example project dependencies to see which versions are compatible. @danieloprado thanks again Please, bump all versions to the latest like this: deps Release available for this one a minor change: 2.3.0 |
|
O Windows PowerShell Instale o PowerShell mais recente para obter novos recursos e aprimoramentos! https://aka.ms/PSWindows PS D:\Projetos_Freela\appMuffato> yarn android
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation. info 💡 Tip: Make sure that you have set up your development environment correctly, by running npx react-native doctor. To read more about doctor command visit: https://github.com/react-native-community/cli/blob/main/packages/cli-doctor/README.md#doctor FAILURE: Build failed with an exception.
ld: error: undefined symbol: RNWorklet::JsiWorkletContext::runtimeMappings
ld: error: undefined symbol: RNWorklet::JsiWorkletContext::invokeOnWorkletThread(std::__ndk1::function<void (RNWorklet::JsiWorkletContext*, facebook::jsi::Runtime&)>&&)
ld: error: undefined symbol: RNWorklet::JsiWorkletContext::invokeOnJsThread(std::__ndk1::function<void (facebook::jsi::Runtime&)>&&)
ld: error: undefined symbol: RNWorklet::JsiWorkletContext::defaultInstance
ld: error: undefined symbol: RNWorklet::JsiHostObject::JsiHostObject()
ld: error: undefined symbol: vtable for RNWorklet::JsiWorkletContext
ld: error: undefined symbol: RNWorklet::JsiWorkletContext::~JsiWorkletContext()
ld: error: undefined symbol: RNWorklet::JsiWorkletContext::JsiWorkletContext(std::__ndk1::basic_string<char, std::__ndk1::char_traits, std::__ndk1::allocator > const&, facebook::jsi::Runtime*, std::__ndk1::function<void (std::__ndk1::function<void ()>&&)>, std::__ndk1::function<void (std::__ndk1::function<void ()>&&)>)
ld: error: undefined symbol: RNWorklet::JsiHostObject::~JsiHostObject()
ld: error: undefined symbol: RNWorklet::JsiWrapper::wrap(facebook::jsi::Runtime&, facebook::jsi::Value const&, RNWorklet::JsiWrapper*)
ld: error: undefined symbol: RNWorklet::JsiHostObject::get(facebook::jsi::Runtime&, facebook::jsi::PropNameID const&)
ld: error: undefined symbol: RNWorklet::JsiHostObject::set(facebook::jsi::Runtime&, facebook::jsi::PropNameID const&, facebook::jsi::Value const&)
ld: error: undefined symbol: RNWorklet::JsiHostObject::getPropertyNames(facebook::jsi::Runtime&)
ld: error: undefined symbol: typeinfo for RNWorklet::JsiHostObject
C++ build system [build] failed while executing:
BUILD FAILED in 25s Boa tarde, não estou conseguindo rodar o plugin, está gerando o erro acima meu package.json está assim: { Será que é algum problema na minha configuração? |
|
@PereiraMatheuSilva olá, parece que o vision camera não está conseguindo compilar. |
|
Hey everyone, I managed to solve the issue I was facing here. I resolved it by installing all the dependencies first and only then performing the build. It worked for me here. |
|
Has someone found a solution for the orientation in IOS? Edit: I created a patch that works for me; I'll try to open a PR for @ismaelsousa 's version. |
I faced a similar issue. Fixed it by deleting @ismaelsousa Thanks for continuously maintaining this plugin. @aarongrider Do merge in the main library |
|
@ismaelsousa the camera is too slow, it's like low fps. I noticed that when I comment the const data = scanOCR(frame); it works fine. can anyone help ? |
|
@mayran-logmanager are you adjusting the camera fps based on the fps suggested? |
|
@ismaelsousa I'm using the same code from example in the project. But I just fixed it by setting enableGpuBuffers={true}. now its fluid again |
Beware that
Thats in the JDOCS, and its confirmed that this param cause crash in samsumg. Full JDOCS for /**
* Enables or disables GPU-sampled buffers for the video stream. This only takes effect when using a {@linkcode frameProcessor}.
*
* When recording a Video ({@linkcode video}) while a Frame Processor is running ({@linkcode frameProcessor}),
* the {@linkcode Frame | Frames} will need to be forwarded to the Media Encoder.
*
* - When `enableGpuBuffers` is `false`, the Video Pipeline will use CPU buffers causing an additional copy
* from the Frame Processor to the Media Encoder, which potentially results in increased latency.
* - When `enableGpuBuffers` is `true`, the Video Pipeline will use shared GPU buffers which greatly increases
* it's efficiency as an additional buffer copy is avoided.
* (See [`USAGE_GPU_SAMPLED_IMAGE`](https://developer.android.com/reference/android/hardware/HardwareBuffer#USAGE_GPU_SAMPLED_IMAGE))
*
* In general, it is recommended to set this to `true` if possible, as this can increase performance and efficiency of the Video Pipeline.
*
* @experimental This is an experimental feature flag, use at your own risk. Some devices (especially Samsungs) may crash when trying to use GPU buffers.
* @platform Android (API 29+)
* @default false
*/
enableGpuBuffers?: booleaninstead of using this.. |
|
@pedrol2b thx for the advice man. I tried what you said and it worked as expected. |
|
build in android is failing in react vision camera v4 unresolved reference: |
I am updating right now :) soon will be available |
im waiting for you, ios works but notice that andorid didnt hahaha, thank you so much for you very quick response. |
This issue with the Android build is due to the latest update of the react-native-vision (v4) Before: import com.mrousavy.camera.frameprocessor.Frame
import com.mrousavy.camera.frameprocessor.FrameProcessorPlugin
import com.mrousavy.camera.types.Orientation
Now: import com.mrousavy.camera.frameprocessors.Frame
import com.mrousavy.camera.frameprocessors.FrameProcessorPlugin
import com.mrousavy.camera.core.types.Orientation
Same as the orientation method |
|
testing.... |
|
it is working like a charm, in andorid i dont know why with a 2 fps runinng seems like doing some blocking task thatt freezes the camera por miliseconds, that happened before too, just comenting... |
|
Thanks for letting me know it |
Please, see the release page: releases
Download it via npm:
Vision camera v4
for the vision camera v4 use this library from 3.0.0 and above