jyshtty / autocomplete

Implementation of Autocomplete System Design for Large Scale - using Docker Compose

Implementation: Autocomplete/Typeahead Suggestions System Design

Implementation of a large scale autocomplete/typeahead suggestions system design, like the suggestions one gets when typing a Google search. Using Docker Compose.

More details in this blog article

Requirements

Docker and Docker Compose are required to run this project.

Usage

Step 1 Run the system:

$ make run

Step 2 Go to a different terminal. Add the basilar znodes and HDFS folders:

$ make setup

Step 3 Send some phrases to the search endpoint, in order to simulate multiple user search submissions to the system. This way we will have a data starting point:

$ make populate_search

Step 4 The map reduce tasks will reduce the top searches into a single HDFS file, which will be later transformed automatically by the system into a trie, and distributed to the backend:

$ make do_mapreduce_tasks

After the tries are distributed to the backend, please wait about a minute for the trie-backend-applier service to kick in. This service will assign the new backend nodes to receive the ensuing top-phrases requests.

Step 5 Visit http://localhost/ in your browser.

Write some text in the input form, and you should start to see some suggestions popping up.

You can also submit your search queries, which will be fed into the assembler. After you've submitted some entries, run make do_mapreduce_tasks again, and your queries will be considered for the next batch of suggestions. Enjoy!