parvashah-create / image-search

Visual Search Implementation for Fashion Products

This project is developed as a part of the assignment for Algorithmic marketing analyst role at QU analytics. The client for this project is a large e-tailer, SmartFashion, who wants to enhance the user-experience of their clientele by providing rich and engaging interfaces without leaving their couches. Visual search is one of the ways to achieve this objective.

Project Specifications

The project is divided into three tasks:

Task 1: Image Search

In this task, we will use the approach discussed in https://github.com/matsui528/sis to implement a search for a dataset from the https://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html site. The task requires us to develop an application using Streamlit that searches objects based on uploaded images. We will use 100 images to keep the problem small enough.

Task 2: Image Generation

In this task, we will use Dall-e to build a site to customize fashion accessories. We will use Streamlit and Dall-e to prototype an application.

Task 3: Image Search Using Pinecone

In this task, we will use Pinecone's vector database to speed up the search for images. We will integrate Pinecone's approach with Streamlit for the data used in Task 1. We will also compare the performance of Task 1 and Task 3 to see the improvement in search speed.

Deliverables

The deliverables for this project are as follows:

• Working code on Github - https://github.com/parvashah-create/image-search
• Deployed application on Streamlit - https://parvashah-create-image-search-app-nx8kxu.streamlit.app/
• Documentation on Codelabs - https://codelabs-preview.appspot.com/?file_id=1LPZ37K-IXY1ouRWRyelt--iZLZyXlMpLjT04QVGe-MM#5

Project Team

This project is developed by Sri Krishnamurthy as a part of the assignment for Algorithmic marketing analyst role at QU analytics.

References

• https://github.com/matsui528/sis
• https://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html
• https://shop.ccs.com/products/dall-e-ai-generated-t-shirt
• https://github.com/pinecone-io/examples/blob/master/search/image/image-search/image-search.ipynb

image-search's Issues

Code Labs

Each person will document the part of the code they worked on.

Task 1: Image search

Description

You plan to use the approach discussed in https://github.com/matsui528/sis to
implement a search for a dataset from the
https://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html site. Develop an
application using Streamlit that searches objects based on uploaded images. You
can use 100 images to keep the problem small enough.

Steps

1. Install the necessary libraries: You will need to install Streamlit, PyTorch, Torchvision, NumPy, and Pillow. You can install them using pip.

2. Download the DeepFashion dataset: Download the dataset from https://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html and extract it to a folder.

3. Download the pre-trained model: Download the pre-trained model from https://drive.google.com/file/d/1QFRs_MsT6T_3UNrSJdDfJbW8XZKZXU6b/view and extract it to a folder.

4. Load the model: Load the pre-trained model into your application using PyTorch.

5. Create a Streamlit interface: Use Streamlit to create a user interface where users can upload an image and search for similar images in the DeepFashion dataset.

6. Preprocess the uploaded image: Preprocess the uploaded image by resizing it to the size required by the pre-trained model and normalizing it.

7. Encode the image: Use the pre-trained model to encode the image into a feature vector.

8. Search for similar images: Search the DeepFashion dataset for images that are similar to the uploaded image based on the encoded feature vector. You can use the approach discussed in the https://github.com/matsui528/sis repository to perform the search.

9. Display the results: Display the similar images found in the DeepFashion dataset to the user in the Streamlit interface.

10. Deploy the application: Deploy the application to a server or platform where it can be accessed by users.

Readme File

Task: Create readme file

Create a readme file that documents

• Code
• Files
• Documentation
• Deliverables

Task 2: Image generation

Description

You plan to use Dall-e to build a site to customize fashion accessories (See
https://shop.ccs.com/products/dall-e-ai-generated-t-shirt ) for example. Use streamlit and
Dall-e to prototype an application.

Steps

1. Install the necessary libraries: You will need to install Streamlit, the OpenAI API, and the Requests library. You can install them using pip.

2. Register for the OpenAI API: You will need to register for an API key from OpenAI in order to use Dall-e. You can do this at https://beta.openai.com/signup/.

3. Create a Streamlit interface: Use Streamlit to create a user interface where users can customize fashion accessories using Dall-e.

4. Collect user input: Collect user input in the form of text descriptions of the fashion accessories they would like to create.

5. Call the Dall-e API: Use the Requests library to call the Dall-e API with the user input.

6. Display the generated images: Display the images generated by Dall-e to the user in the Streamlit interface.

7. Allow for customization: Allow the user to customize the generated images by adjusting various parameters, such as color, size, and shape.

8. Save the customized image: Allow the user to save the customized image to their device or to a cloud storage service.

9. Test the application: Test the application to ensure that it is functioning as expected and that the generated images look appropriate.

10. Deploy the application: Deploy the application to a server or platform where it can be accessed by users.

Task 3: Image search using Pinecone

Description

Task 3: Image search using Pinecone
You realize search is slow in Task 1 when you increase the images from 100 to 10000.
You are approached by Pinecode to use their vector database to speed up search.
They provide this ref example:
https://github.com/pinecone-io/examples/blob/master/search/image/image-search/i
mage-search.ipynb
Use this approach and integrate with Streamlit for the data used in Task 1. Compare
performance of Task 1 and Task 3

Steps

1. Install the necessary libraries: You will need to install Streamlit, Pinecone, PyTorch, Torchvision, NumPy, and Pillow. You can install them using pip.

2. Download the DeepFashion dataset: Download the dataset from https://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html and extract it to a folder.

3. Download the pre-trained model: Download the pre-trained model from https://drive.google.com/file/d/1QFRs_MsT6T_3UNrSJdDfJbW8XZKZXU6b/view and extract it to a folder.

4. Load the model: Load the pre-trained model into your application using PyTorch.

5. Create a Pinecone index: Use Pinecone to create an index for the feature vectors of the images in the DeepFashion dataset.

6. Preprocess the images: Preprocess the images in the DeepFashion dataset by resizing them to the size required by the pre-trained model and normalizing them.

7. Encode the images: Use the pre-trained model to encode the images into feature vectors.

8. Add feature vectors to Pinecone index: Add the feature vectors to the Pinecone index.

9. Create a Streamlit interface: Use Streamlit to create a user interface where users can upload an image and search for similar images in the DeepFashion dataset.

10. Preprocess the uploaded image: Preprocess the uploaded image by resizing it to the size required by the pre-trained model and normalizing it.

11. Encode the image: Use the pre-trained model to encode the image into a feature vector.

12. Search for similar images: Use Pinecone to search the index for images that are similar to the uploaded image based on the encoded feature vector.

13. Display the results: Display the similar images found in the DeepFashion dataset to the user in the Streamlit interface.

14. Compare performance: Measure the performance of the Pinecone-based image search and compare it to the performance of the approach used in Task 1.

15. Deploy the application: Deploy the application to a server or platform where it can be accessed by users.