![Gitter](https://badges.gitter.im/Join Chat.svg)
Hello everybody!
I'm very interested in Mathias Eitz's works on SIGGRAPH 2012, so I developed OpenSSE, Just for fun.
This is an open source sketch search engine for 3D object retrieval based on sketch image as input. In fact, it can retrieve more, such as images, videos, point cloud models and volume models, you just need get their line drawing images than I know you can.
In order to clearly show how to work, I used single thread and ascii encoding for file storage.
Zhang Dongdong
This demo deals with mesh models. See more search results, click demo videos.
See more search results, click demo videos.
You just need include one line in your .pro
file.
include(../../opensse.pri)
There are a series of tools for you under opensse/tools/
. When compiled, those tools are under build-xxx-Release/bin
.
This is gui demo for you under opensse/gui
.
Database in my demo comes from SHREC 2012, download 3D target dataset(~112MB). We use models under directory Watertight_dataset/Extended
.
$ mv /Users/zdd/Download/Watertight_dataset/Extended /Users/zdd/Database/SHREC12/
Using my another project, you will get 102 views for each model.
- Step 1: Generate line draing images filelist
$ ./generate_filelist -d /Users/zdd/Database/SHREC12/ -f "*.jpg" -o /Users/zdd/Database/SHREC12/2012_filelist
- Step 2: Extract descriptors
$ ./extract_descriptors -d /Users/zdd/Database/SHREC12/ -f /Users/zdd/Database/SHREC12/2012_filelist -o /Users/zdd/Database/SHREC12/2012_
you will gain two files: 2012_features
and 2012_keypoints
.
- Step 3: Generate vocabulary (optional)
$ ./generate_vocabulary -f /Users/zdd/Database/SHREC12/2012_features -n 1000 -o /Users/zdd/Database/SHREC12/vocabulary
It will take you some times, but only need run once. or use my vocabulary under ../opensse/data/
.
- Step 4: Quantize feature
$ ./quantize -v /Users/zdd/Database/SHREC12/vocabulary -f /Users/zdd/Database/SHREC12/2012_features -o /Users/zdd/Database/SHREC12/2012_samples
When dealing with large scale database, we can directly generate samples using extract_quantize tool to reduce disk usage without feature files.
$ ./extract_and_quantize -d /Users/zdd/Database/SHREC12/ -f /Users/zdd/Database/SHREC12/2012_filelist -v /Users/zdd/Database/SHREC12/2012_filelist -v /Users/zdd/Database/SHREC12/vocabulary -o /Users/zdd/Database/SHREC12/2012_samples
- Step 5: Create inverted index file
$ ./create_index -s /Users/zdd/Database/SHREC12/2012_samples -o /Users/zdd/Database/SHREC12/2012_index_file
Now all data are ready.
You can test opensse through command line
$ ./sketch_search -i /Users/zdd/Database/SHREC12/2012_index_file -v /Users/zdd/Database/SHREC12/vocabulary -d /Users/zdd/Database/SHREC12/ -f /Users/zdd/Database/SHREC12/2012_filelist -n 10 -o output
You will get a interactive interface, like:
>> open sketch search :
>> input absolute path, like "/Users/zdd/zddhub.png"
>> input q exit
>> good luck!
>> /Users/zdd/Database/SHREC12/Extended/D00057view/1.jpg
0.985509 /Users/zdd/Database/SHREC12/Extended/D00057view/1.jpg
0.953196 /Users/zdd/Database/SHREC12/Extended/D00050view/1.jpg
0.938705 /Users/zdd/Database/SHREC12/Extended/D00041view/1.jpg
0.856004 /Users/zdd/Database/SHREC12/Extended/D00050view/2.jpg
0.852683 /Users/zdd/Database/SHREC12/Extended/D00057view/2.jpg
0.849087 /Users/zdd/Database/SHREC12/Extended/D00041view/2.jpg
0.823276 /Users/zdd/Database/SHREC12/Extended/D00041view/0.jpg
0.821133 /Users/zdd/Database/SHREC12/Extended/D00050view/10.jpg
0.811917 /Users/zdd/Database/SHREC12/Extended/D00057view/10.jpg
0.810096 /Users/zdd/Database/SHREC12/Extended/D00050view/0.jpg
>> q
You must config params.json
file using your data path, like:
{
"searcher":
{
"indexfile": "/Users/zdd/Database/SHREC12/2012_index_file",
"filelist": "/Users/zdd/Database/SHREC12/2012_filelist",
"vocabulary": "/Users/zdd/Database/SHREC12/vocabulary",
"rootdir": "/Users/zdd/Database/SHREC12",
"results_num": "12",
"views_num": "102"
}
}
You'd better to use the absolute path. And then, set params.json
in ../opensse/gui/SketchSearchDemo/mainwindow.cpp
:
boost::property_tree::read_json("/Users/zdd/Database/SHREC12/params.json", params);
Compiled and You will get my demo. Good luck!
Notice: To make sure line drawing images correspond to 3d models, please copy ../opensse/data/view/
to you execute file path.
By the way, press 'C' in sketch pad, You will clear the sketch and draw next one.
OpenSSE uses Qt 5.1+, OpenCV 2.4.7+ and boost 1.55.0, it's a cross-platform library.
You need config INCLUDEPATH
and LIBS
in ../opensse/opensse.pri
file related your path.
Now compile it, so easy!
Someone still meets compile issue, so I add docker support. Please compare your environment with docker when you meet compile issue.
Run on mac:
# Install socat and xquartz if you want to run GUI demo.
brew install socat
brew cask install xquartz
# Start XQuartz
open -a XQuartz
# Expose local xquartz socket via socat on a TCP port
socat TCP-LISTEN:6000,reuseaddr,fork UNIX-CLIENT:\"$DISPLAY\"
# in another window under our opensse repo folder.
docker build -t opensse . # will takes ~15 minutes to build
docker run -it opensse
# # or use inet ip, like:
# docker run -it -e DISPLAY=$(ipconfig getifaddr en0):0 opensse
# Run command lind demo
docker run -it opensse /bin/bash
cd ~/opensse/build/tools/bin
# You will find all opensse tools
# If you don't want to compile code, You can use zddhub/opensse-demo to run demo:
docker pull zddhub/opensse-demo
docker run -it zddhub/opensse-demo
Docker images: zddhub/opensse-demo
You can use dataset and evaluation methodology to evaluate opensse on SHREC 2012, SHREC 2013, SHREC 2014.
You can generate data use test_search
tool in tests/test_search
.
Usage:
test_search -p paramsfile -i searchfilelist -r searchfileroot -o resultdir
-
Could not resolve SDK path for 'macosx10.9'
There is a workaround: Navigate to where you installed Qt (default /Users/your username/Qt) using finder Go to the subdirectory
.../5.3/clang_64/mkspecs
directoryOpen the file called qdevice.pri with a text editor Change the line !host_build:QMAKE_MAC_SDK = macosx10.9 to:
- !host_build:QMAKE_MAC_SDK = macosx10.10 if you are on OS X 10.10 (Yosemit
- !host_build:QMAKE_MAC_SDK = macosx10.11 if you are on OS X 10.11 (EI Captian))
- A sketch collect, share and evaluate platform
- A better search framework
- A new rank algorithm
Code is under the Apache License, Version 2.0.
I accept tips through Alipay. Scan below qr code to donate:
If you have any question or idea, please email to me. Or search 'zddhub' to find me in social networking platform.