This application is hosted on AWS, relevant credentials can only be requested from AWS Administrators. Currently this is Ian Emsens and John Deprez.
The application is hosted on a t2.small EC2 instance which is configured to have multiple security groups which allow inbound tcp traffic on the following ports:
22- SSH80- HTTP443- HTTPS8080- Application API
An Elastic IP is set up to ensure the IPv4 address remains static across restarts of the instance.
No automated deploys have been set up. Meaning you won't be able to find the deploy definiton on Envoyer or anything of the sort.
Instead, deployments are done manually by using SSH to access the machine. Below are the steps you should take to update the application. I refer to parts of the application by their service name as defined in docker-compose.yml
$ cd ambar$ git pull origin master- Run local builds on parts you've changed
- e.g.
frontendrequires a local build to be done which comes down to:$ cd FrontEnd$ nvm install$ npm install$ npm run build
- e.g.
$ docker-compose build <service-name>- Make sure you're in the folder containing the
docker-compose.ymlfile when running this
- Make sure you're in the folder containing the
$ docker-compose restart <service-name>- If you don't see changes propogated use:
$ docker-compose down && docker-compose up -d
- If you don't see changes propogated use:
If nothing broke along the way you should now have a functional and updated Ambar project, check below for help troubleshooting and some useful commands.
In general, the below commands should allow you to troubleshoot most issues in this setup
$ docker stats- shows a list of all running containers on the docker daemon and shows their memory and CPU usage
$ docker-compose build- builds or rebuilds all containers
$ docker-compose up- will start or build and start all services defined in
docker-compose.yml - Tip: use
-dto run this command in detached mode (allows you to close your terminal without killing the application)
- will start or build and start all services defined in
$ docker-compose down- does the inverse of
$docker-compose up
- does the inverse of
$ docker-compose ps- shows a list of all running containers managed by the docker-compose file in your current directory. This list contains information about service health
$ docker-compose logs- will show all logging information normally shown by
$ docker-compose up
- will show all logging information normally shown by
When the es service is consistently restarting, check its memory usage using $ docker stats and then check $ docker-compose logs and look for the below error
es_1 | ERROR: [1] bootstrap checks failed
es_1 | [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
Resolve this by following instructions in this issue. This setting resets upon instance restart.
Note: this can most likely be made into an automated process within AWS
Ambar is an open-source document search engine with automated crawling, OCR, tagging and instant full-text search.
Ambar defines a new way to implement a full-text document search into yor workflow:
- Easily deploy Ambar with a single
docker-composefile - Perform a Google-like search through your documents and images contents
- Ambar supports all popular document formats, performs OCR if needed
- Tag your documents
- Use a simple REST Api to integrate Ambar into your workflow
Tutorial: Mastering Ambar Search Queries
- Fuzzy Search (John~3)
- Phrase Search ("John Smith")
- Search By Author (author:John)
- Search By File Path (filename:*.txt)
- Search By Date (when: yesterday, today, lastweek, etc)
- Search By Size (size>1M)
- Search By Tags (tags:ocr)
- Search As You Type
- Supported language analyzers: English
ambar_en, Russianambar_ru, Germanambar_de, Italianambar_it, Polishambar_pl, Chineseambar_cn, CJKambar_cjk
Ambar 2.0 only supports local fs crawling, if you need to crawl an SMB share of an FTP location - just mount it using standard linux tools. Crawling is automatic, no schedule is needed since the crawler monitors fs events and automatically processes new files.
- Ambar supports large files (>30MB)
- ZIP archives
- Mail archives (PST)
- MS Office documents (Word, Excel, Powerpoint, Visio, Publisher)
- OCR over images
- Email messages with attachments
- Adobe PDF (with OCR)
- OCR languages: Eng, Rus, Ita, Deu, Fra, Spa, Pl, Nld
- OpenOffice documents
- RTF, Plaintext
- HTML / XHTML
- Multithread processing
Notice: Ambar requires Docker to run, it can't run w/o Docker
You can build Docker images by yourself or buy prebuilt Docker images for $50 here.
- The installation instruction for prebuilt images can be found here
- Tutorial on how to build images from scratch will be available soon
If you want to see how Ambar works w/o installing it, try our live demo. No signup required.
All of the images required to run Ambar can be built by the user. In general, each image can be built by navigating into the directory of the component in question, performing any compilation steps required, then building the image like so:
# From project root
$ cd FrontEnd
$ docker build . -t <image_name>
The resulting image can be referred to by the name specified, and run by the containerization tooling of your choice.
In order to use a local Dockerfile with docker-compose, simply change the image option to build, setting the value to the relative path of the directory containing the dockerfile. Then run docker-compose build to build the relevant images. For example:
# docker-compose.yml from project root, referencing local dockerfiles
pipeline0:
build: ./Pipeline/
image: chazu/ambar-pipeline
localcrawler:
image: ./LocalCrawler/
Note that some of the components require compilation or other build steps be performed on the host before the docker images can be built. For example, FrontEnd:
# Assuming a suitable version of node.js is installed (docker uses 8.10)
$ npm install
$ npm run compile
Yes, it's fully open-source.
Yes, it is forever free and open-source.
Yes, it performs OCR on images (jpg, tiff, bmp, etc) and PDF's. OCR is perfomed by well-known open-source library Tesseract. We tuned it to achieve best perfomance and quality on scanned documents. You can easily find all files on which OCR was perfomed with tags:ocr query
Supported languages: Eng, Rus, Ita, Deu, Fra, Spa, Pl, Nld. If you miss your language please contact us on [email protected].
Yes!
Yes, it can search through any PDF, even badly encoded or with scans inside. We did our best to make search over any kind of pdf document smooth.
It's limited by amount of RAM on your machine, typically it's 500MB. It's an awesome result, as typical document managment systems offer 30MB maximum file size to be processed.
Request a dedicated support session by mailing us on [email protected]