object-search

Object search

In this example, we use fasterrcnn_resnet50_fpn for object detection and then index these cropped object images with MobileNetV2. You can use this demo system to index an image dataset and query the most similar object in those images. The query should be a cropped object image.

Features that come out of the box:

• Interactive query
• Parallel replicas
• Index with shards
• Containerization
• REST and gRPC gateway
• Dashboard monitor

To save you from dependency hell, we'll use the containerized version in these instructions. That means you only need to have Docker installed. No Python virtualenv, no Python package (un)install.

The code can of course run natively on your local machine, please read the Jina installation guide for details.

Table of Contents

• Download and Extract Data
• Index Image Data
• Start the Server
• Query via REST API
• Troubleshooting
• Documentation
• Community
• License

Download and Extract Data

Use Flickr8k

You can try this example with Flickr8k object detection dataset. You can also use other datasets like COCO & Open Images 2019

Download the Flickr8k from Kaggle

kaggle datasets download adityajn105/flickr8k
unzip flickr8k.zip 
rm flickr8k.zip
mv Images data/f8k/images

Note: Flickr8k is not an ideal dataset but we are using due to its small size.

Index Image Data

Index 1000 images. This can take some time and you can try a smaller number as well.

python app.py -task index -n 1000 -overwrite True

If it's running successfully, you should be able to see logs scrolling in the console and in the dashboard:

Start the Server

Start server which returns `original` images. The matching of query happens with all indexed object images and returns the original parent image in which the indexed object was found.

python app.py -task query -r original

Start server which returns object images. The matching of query happens with all indexed object images and returns them.

python app.py -task query -r object

Query via REST API

When the REST gateway is enabled, Jina uses the data URI scheme to represent multimedia data. Simply organize your picture(s) into this scheme and send a POST request to http://0.0.0.0:45678/api/search, e.g.:

curl --verbose --request POST -d '{"top_k": 10, "mode": "search",  "data": ["data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAICAIAAABLbSncAAAA2ElEQVR4nADIADf/AxWcWRUeCEeBO68T3u1qLWarHqMaxDnxhAEaLh0Ssu6ZGfnKcjP4CeDLoJok3o4aOPYAJocsjktZfo4Z7Q/WR1UTgppAAdguAhR+AUm9AnqRH2jgdBZ0R+kKxAFoAME32BL7fwQbcLzhw+dXMmY9BS9K8EarXyWLH8VYK1MACkxlLTY4Eh69XfjpROqjE7P0AeBx6DGmA8/lRRlTCmPkL196pC0aWBkVs2wyjqb/LABVYL8Xgeomjl3VtEMxAeaUrGvnIawVh/oBAAD///GwU6v3yCoVAAAAAElFTkSuQmCC", "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAICAIAAABLbSncAAAA2ElEQVR4nADIADf/AvdGjTZeOlQq07xSYPgJjlWRwfWEBx2+CgAVrPrP+O5ghhOa+a0cocoWnaMJFAsBuCQCgiJOKDBcIQTiLieOrPD/cp/6iZ/Iu4HqAh5dGzggIQVJI3WqTxwVTDjs5XJOy38AlgHoaKgY+xJEXeFTyR7FOfF7JNWjs3b8evQE6B2dTDvQZx3n3Rz6rgOtVlaZRLvR9geCAxuY3G+0mepEAhrTISES3bwPWYYi48OUrQOc//IaJeij9xZGGmDIG9kc73fNI7eA8VMBAAD//0SxXMMT90UdAAAAAElFTkSuQmCC"]}' -H 'Content-Type: application/json' 'http://0.0.0.0:45678/api/search'

JSON payload syntax and spec can be found in the docs.

This example shows you how to feed data into Jina via REST gateway. By default, Jina uses a gRPC gateway, which has much higher performance and rich features. If you are interested in that, go ahead and check out our other examples and read our documentation on Jina IO.

Troubleshooting

Memory Issues

If you are using Docker Desktop, make sure to assign enough memory for your Docker container, especially when you have multiple replicas. Below are my MacOS settings with two replicas:

Documentation

The best way to learn Jina in depth is to read our documentation. Documentation is built on every push, merge, and release event of the master branch. You can find more details about the following topics in our documentation.

• Jina command line interface arguments explained
• Jina Python API interface
• Jina YAML syntax for executor, driver and flow
• Jina Protobuf schema
• Environment variables used in Jina
• ... and more

Community

• Slack channel - a communication platform for developers to discuss Jina
• Community newsletter - subscribe to the latest update, release and event news of Jina
• LinkedIn - get to know Jina AI as a company and find job opportunities
• - follow us and interact with us using hashtag #JinaSearch
• Company - know more about our company, we are fully committed to open-source!

License

Copyright (c) 2020 Jina AI Limited. All rights reserved.

Jina is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.