Image recognition for android devices
Explore the docs »
Report Bug
·
Request Feature
You will find the APK in app/build/outputs/apk/support/debug
.
If you wanto to generate another APK file take a look on the following guide: https://code.tutsplus.com/tutorials/how-to-generate-apk-and-signed-apk-files-in-android-studio--cms-37927
The repository consists of two parts:
- Python
- Android
The python part is used to generate sqlite files from an image dataset. You should use and edit this part only if you want to use another neural network or image dataset. The android application is ready to use and you should change it just to add files to the monuments guides.
You will find all the instuction you need just below.
Python library required:
- numpy
- cv2
- tflite
- faiss (Anaconda required)
- sklearn
The repository contains the file Python/build_sqlite.py
which must be executed by adding the argument -i
or --images
indicating the path to the dataset folder as in the following example:
python build_sqlite.py -i datasetFolder
IMPORTANT: The indicated folder must contains one folder per monument and each of which contains the images, as in the following example:
datasetFolder
├───Battistero_SanGiovanni
│ img1.jpg
│ img2.jpg
│ img3.jpg
│
├───Campanile_Giotto
│ img1.jpg
│ img2.jpg
│ img3.jpg
│
├───Cattedrale_Duomo
│ img1.jpg
│ img2.jpg
│ img3.jpg
│
└───Palazzo_Vecchio
img1.jpg
img2.jpg
img3.jpg
The file build_sqlite.py
will create three .sqlite
and .pck
files. Each pair of files is created with a different neural network. To change the neural network see dedicated paagraph.
IMORTANT: Do not change the names of the files created.
Go to the models\src\main\assets\guides
folder. Inside it there is the folder Template Monument
which is to be used as a template, so without altering its structure. It is only possible to change the name of the folder with the name of the monument which, however, must be the same uilized in the dataset folder.
Text file and an image can be placed in this folder. Text file must be placed in the folder corresponding to the language.
IMPORTANT: New files must have the same name as the files in the template folder.
NOTE: For the time being, Italian and English languages are supported.
Audio and video files are played providing an URL.
IMPORTANT: If you want to add an audio or a video to the guide, first of all you have to upload the file on a drive platform (ex. Dropbox).
Open app/src/main/java/org/tensorflow/lite/examples/classification/GuideActivity.java
.
Add a case to the switch construct with the name of the monument (must be the same as the name of the dataset folder), and replace pathVideo
and pathAudio
with the URL of the new files, as in the example:
case "Palazzo Vecchio":
if(language.equals("English"))
pathVideo += "https://driveexample.com/xxx/videoEn.mp4";
pathAudio += "https://driveexample.com/xxx/audioEn.mp4";
else
pathVideo += "https://driveexample.com/xxx/videoEn.mp4";
pathAudio += "https://driveexample.com/xxx/audioIt.mp4";
break;
Place the neural network model .tflite
in Models/src/main/assets
.
Open the file build_sqlite.py
: add the name of the neural network and the model path to the "types" list. Example:
types = [ #neural networks
('MobileNetV3_Large_100', '../models/src/main/assets/lite-model_imagenet_mobilenet_v3_large_100_224_classification_5_default_1.tflite'),
('MobileNetV3_Large_075', '../models/src/main/assets/lite-model_imagenet_mobilenet_v3_large_075_224_classification_5_default_1.tflite'),
('MobileNetV3_Small_100', '../models/src/main/assets/lite-model_imagenet_mobilenet_v3_small_100_224_classification_5_default_1.tflite'),
('newNeuralNetworkModel.tflite', 'pathNewNeauralNetworkModel.tflite')
]
Run 'build_sqlite.py'.
Go to lib_support\src\main\java\org\tensorflow\lite\examples\classification\tflite
.
- Create a class that extends the
Classifier
class with a name that indicates the new neural network. TIP: There is a template class namedClassifierNewNeuralNetworkClass
. Rename the class and change thegetModelPath()
method with the filename of the new neural network. - Modify the file
Classifier.java
by adding a name indicating the new model as in the example:
/** The model type used for classification. */
public enum Model {
MOBILENET_V3_LARGE_100,
MOBILENET_V3_LARGE_075,
MOBILENET_V3_SMALL_100,
NEWMODEL_NAME
}
- Modify the
create(Activity, Model, Device, int)
method by adding an else if with the previously created class and model:
if (model == Model.MOBILENET_V3_LARGE_100) {
return new ClassifierMobileNetLarge100(activity, device, numThreads);
} else if (model == Model.MOBILENET_V3_LARGE_075) {
return new ClassifierMobileNetLarge075(activity, device, numThreads);
} else if (model == Model.MOBILENET_V3_SMALL_100) {
return new ClassifierMobileNetSmall100(activity, device, numThreads);
}else if (model == Model.NEWMODEL_NAME) {
return new ClassifierNewClassName(activity, device, numThreads);
} else {
throw new UnsupportedOperationException();
}
- Modify
Retrievor.java
by adding an if in theRetrievor(Context, Cassifier.model)
constructor method as in the example:
if (model == Classifier.Model.MOBILENET_V3_LARGE_100) {
dbName = "MobileNetV3_Large_100_db.sqlite"
} else if (model == Classifier.Model.MOBILENET_V3_LARGE_075) {
dbName = "MobileNetV3_Large_075_db.sqlite"
} else if (model == Classifier.Model.MOBILENET_V3_SMALL_100) {
dbName = "MobileNetV3_Small_100_db.sqlite"
}else if (model == Classifier.Model.NEWMODEL_NAME) {
dbName = "newDatabaseFile_db.sqlite"
} else {
throw new UnsupportedOperationException();
}
newDatabaseFile.sqlite
is the new sqlite file that is the created in the previous paragraph.
- Finally edit the file
app/src/main/res/values/strings.xml
by inserting the name of the new model instring-array name="tfe_ic_models"
as in the example:
<string-array name="tfe_ic_models" translatable="false">
<item>MobileNet_V3_Large_100</item>
<item>MobileNet_V3_Large_075</item>
<item>MobileNet_V3_Small_100</item>
<item>NewModel_Name</item>
</string-array>
IMORTANT: The model name must be the same as entered in step 2) of this paragraph, capitalization not required. Once you have completed these steps you can use the new model by selecting it directly in the application menu.
Lorenzo Massa - [email protected]
Project Link: https://github.com/lorenzo-massa/SmartTourism