microsoft / ai-chat-app-hack

HackTogether: The Microsoft Python Chatbot Hack | Register, Hack, Win

Hack Together: The AI Chat App Hack

Introduction

🛠️ Build, innovate, and #HackTogether!🛠️ It's time to get started building your first AI Chat App, using the power of RAG (Retrieval Augmented Generation). 🤖 + 📚 = 🔥

GPT models are powerful language generators, but they don't know everything about the world. RAG (Retrieval Augmented Generation) combines the power of GPT with the knowledge of a search engine. This allows you to ask questions of your own data, and get answers that are relevant to the context of your question. We'll show you how to build a RAG Chat App using Azure OpenAI, Azure AI Search, and Python, and extend it for your own data sources.

Hack Together is your playground for coding and experimenting with RAG Chat Apps. With mentorship from Microsoft experts and access to the latest tech, you will learn how to build apps using the most exciting technology - LLMs and vector search databases! The possibilities are endless for what you can create... plus you can submit your hack for a chance to win exciting prizes! 🥳

The hackathon starts on January 29th and ends on February 12th. It is recommended for participants to follow the Hack Together Roadmap for a successful hackathon.

Hack Together Roadmap 🗺️

Follow the steps below to successfully complete the hackathon.

#00 - Pre-requisites

Register to Hack Together: Register Here

Introduce yourself, we'd like to get to know you! 🥳 GitHub Discussions | Let's get to know each other 🎉

Apply for Azure OpenAI access: Request access with this form. If your access request isn't approved, you can instead use OpenAI public API or a local LLM.

#01 - Start hacking on January 29th

Either team up (max 3 team members) or fly solo to join the hacking! Make sure to register for Hack Together before starting to build your project.

#02 - Join the English language live sessions for learning, inspiration, and lots of fun!

Here is a playlist of all the completed sessions: YouTube Playlist

• Mon 1/29/2024 10:00 AM PST - Join live: Building a RAG Chat App in Python (Slides)
• Mon 1/29/2024 3:00 PM PST - Join live: Connecting a RAG Chat App to Azure Cosmos DB (Slides)
• Tue 1/30/2024 9:30 AM PST - Join live: Customizing your RAG Chat App (Slides)
• Wed 1/31/2024 1:00 PM PST - Join live: Azure AI Search Best Practices (Slides)
• Thu 2/1/2024 2:00 PM PST - Join live: GPT-4 with Vision (Demo Repo)
• Mon 2/5/2024 7:00 AM PST - Join live: RAG Chat Web Components (Slides)
• Mon 2/5/2024 11:00 AM PST - Join live: Access Control in RAG Chat Apps (Blog Post)
• Tue 2/6/2024 11:00 AM PST - Join live: Evaluating a RAG Chat App (Slides)
• Wed 2/7/2024 4:00 PM PST - Join live: Chat Completion API Tools & Functions in RAG Chat Apps (Resources)
• Thu 2/8/2024 11:00 AM PST - Join live: Continuous Deployment of your Chat App (Slides)
• Fri 2/9/2024 9:00 AM PST - Join live: Content Safety for Azure OpenAI (Slides)
• Fri 2/9/2024 10:00 AM PST - Join live: Building a Chat on your Business Data without writing a line of code (Resources)
• Mon 2/12/2024 2:00 PM PST - Join live: Project Showcase (Projects)

#03 - Also, don’t miss out on our amazing regional live sessions!

Hack Together - The AI Chat App Hack en Español:

• Wed 1/31/2024 9:00 PST - Join live: En Español Session #1
• Fri 2/2/2024 9:00 PST - Join live: En Español Session #2

Hack Together - The AI Chat App Hack em Português:

• Mon 2/5/2024 2:00 PM PST - Join live: Em Português Session #1
• Tue 2/6/2024 2:00 PM PST - Join live: Em Português Session #2

Hack Together - 人工智能聊天应用程序黑客 (Chinese):

• Mon 1/29/2024 Session #1
• Fri 2/2/2024 Session #2

#04 - Submit your project anytime before February 12th 23:59 PM PST

You may submit your project here when it's ready: 🚀 Project Submission

Check out this video for step by step project submission guidance: Project Submission Video

Code samples to get you started ✨

Most of our live streams will showcase this app solution, which uses a Python backend and React frontend:

https://github.com/Azure-Samples/azure-search-openai-demo/

However, you are free to use any language or framework you like, as long as you make an AI RAG Chat App using at least one Azure service.

To find more samples, check out the following resources:

• Azure AI samples (Python)
• Azure AI samples (JavaScript)
• Azure AI samples (Java)
• Azure AI samples (C#)
• Azure AI samples (Go)

Recommended learning materials 📚

AI

• AI learning and community hub
• Cloud skills challenge: Using Azure OpenAI Service
• Generative AI for Beginners
• Fundamentals of Generative AI
• Retrieval Augmented Generation in Azure AI Search
• Workshop - Create your own ChatGPT with Retrieval-Augmented-Generation
• OpenAI documentation
• Azure AI Search
• Azure OpenAI Service
• Comparing Azure OpenAI and OpenAI
• Azure Communication Services Chat SDK
• AI-in-a-Box
• Join the Azure AI Discord!

Judging and Prizes 🏆

Projects will be evaluated by a panel of judges, including Microsoft engineers, product managers, and developer advocates. Judging criteria will include innovation, impact, technical usability, and alignment with corresponding hackathon category.

Each winning team in the categories below will receive a cash price of $500. 💸

• Best overall: The best app will combine both a compelling data source and significant refinements to the RAG flow to improve the user experience and LLM answer quality.

• Best data source: Bring in a data source that makes for such a compelling chat that we wonder how we ever survived without it.
• Best in your own language: Show us a RAG Chat App that uses non-English data sources. Bonus points for documenting any improvements you had to make to improve search and LLM performance for your app's language.

• Most helpful community member: We're looking for hackers that are super engaged in the streams, answering other participant's questions in the forums and issue trackers, contributing code improvements, and are generally bringing good vibes to our Hack Together. 🥰

🏅 All hackathon participants who submit an app will receive a digital badge.

ai-chat-app-hack's Issues

Project: Videocall your AI

Project name

callyour.ai

Description

Textual interfaces are the norm for chatbot interaction. This is not optimal for many users. This is why I created callyour.ai, where you can have a videocall with your AI, which is a more natural way of conversation for many users.

Language

Almost all common languages

Project Repository URL

callyour.ai

Deployed Endpoint URL

No response

Project video

https://youtu.be/SQHI_G8UUmo?si=MjJLJBKlsVx_SFCi

Team members

Jonasdawi

Showcase Consent

Yes

Project: Widget Feedback Analyser

Project name

Widget Feedback Analyser

Description

The widget feedback analyser will analyse emails that contain customer feedback on the Widgets. Azure Open AI has been leveraged using a RAG architecture and a user can ask questions on the emails. Currently the data source is 15 sample emails in a text file, but this could be further developed to source emails from a marketing mailbox using the GraphAPI.

Language

English

Project Repository URL

https://github.com/mkabuage/widget-feedback-analyser

Deployed Endpoint URL

No response

Project video

https://startedsolo.sharepoint.com/:v:/s/AceTeamSite/Eea0v52BaMBEnTN0LnoH_4UBCsKeW2yvLCw7pfy9rzubWw?e=pShTDL

Team members

mkabuage

Showcase Consent

Yes

Project: Science Tutor AI ChatApp

Project name

Science Tutor

Description

We have developed a Science Tutor AI Chat App specifically designed for higher secondary students following the NCERT syllabus in India.

Our data sources include manually developed content in Hindi based on NCERT Class 11 and 12 science and maths textbooks.

We have utilized Azure's OpenAI GPT-3.5 Turbo and Text Embedding Ada-002 to enhance the model's performance. Additionally, we have incorporated hybrid searching techniques, increased the rate limits, and implemented proper indexing and custom field mapping.

This app is beneficial for both higher secondary teachers and students in India, as well as anyone globally interested in science and maths.

Language

English (as it is deployed as a webapp from azure openai studio it supports almost all languages, our datasource is in hindi language)

Project Repository URL

https://github.com/Sreya-E-P/Science-Tutor-AI-ChatAPP

Deployed Endpoint URL

https://subjectmentor.azurewebsites.net/

Project video

https://www.youtube.com/watch?v=WSUTJ485-Pc

Team members

Sreya E P, Ricky Rodrigues

Showcase Consent

Yes

Project: FoodclopediaNG

Project name

FoodclopediaNG

Description

FoodclopediaNG is an app that uses Generative AI, as well as Azure open AI to offer conversational-based responses to users. The project serves as an encyclopedia for African-based dishes giving you information on its recipes and their nutritional benefits.

Language

English

Project Repository URL

https://github.com/PeaceSuccess/Foodclopedia

Deployed Endpoint URL

https://foodclopediang.azurewebsites.net/chat

Project video

https://drive.google.com/drive/folders/1csdZ0SBgsHCwZ37ArIqmnVMCkuQm-iBX?usp=sharing

Team members

Peace, Olorunnishola, Kayode

Showcase Consent

No

Project: Stock Trading Using RAG

Project name

Stock Trading Using RAG

Description

Demo of Stock Trading Using RAG: Leveraging Diverse Data Sources for Enhanced Trading Insights

Explore how the Retrieval-Augmented Generation (RAG) model transforms stock trading by integrating insights from various data sources into your investment strategy. This demonstration highlights the use of RAG in conjunction with Azure AI Search, showcasing the model's ability to provide targeted trading recommendations based on your personal stock portfolio.

Key Data Sources Utilized:

Reddit: Gathers community sentiment and discussions on stock movements.
Yahoo Finance: Provides up-to-date financial data and market trends.
US Securities and Exchange Commission (SEC) 10-Q Reports: Offers detailed quarterly financial reports for comprehensive analysis.
Personal Portfolio: Tailors advice by considering your specific investment holdings.
News API: Delivers real-time news to capture market-moving events.
This demo illustrates the application of RAG for customizing trading strategies by analyzing a broad range of data sources. By uploading custom documents to Azure AI Search and utilizing tailored chat prompts, users receive personalized investment insights, enhancing decision-making in stock trading.

Language

English

Project Repository URL

https://github.com/01graut/stock-analyzer-rag

Deployed Endpoint URL

No response

Project video

https://youtu.be/9zlDbEaJO38

Team members

01graut, cheetiralasatyanarayana

Showcase Consent

Yes

Project: Enhancing Historical Research with RAG Chatbots

Project name

Archive Companion | Uncover Insights of your sources

Description

Enhancing Historical Research with RAG Chatbots

Context

The project idea revolves around the development of a chatbot designed to assist researchers in navigating through vast and complex sources. Specifically, the focus is on historical newspapers, such as the "Jüdische Zeitung," a pre-war Jewish newspaper from Vienna. These documents are printed in the "Fraktur" typeface, which, while offering a rich resource for historians, presents significant readability challenges for modern German speakers due to its antiquated design.

Goals

1. Accessibility: The primary goal is to reduce the mental effort and time historians currently invest in studying newspapers printed in antiquated typefaces. By simplifying the process of reading through such materials, the chatbot aims to significantly enhance the efficiency of historical research.

2. Search by Topics: Considering the weekly publication frequency of resources like the "Jüdische Zeitung," a major challenge for historians is locating relevant articles amidst vast amounts of information. The chatbot will enable users to search for articles by specific topics, streamlining the research process.

3. Uncover Hidden Gems: During wartime, newspapers were subjected to censorship, forcing editors to employ subtle methods of reporting on sensitive issues. These "codes" are often overlooked in the original sources. A chatbot, powered by a Long-Short Term Memory-Recurrent Attention Gate (LLM-RAG) model, could prove invaluable in identifying these hidden messages.

Challenges & Learnings

1. (Optical) Character Recognition: A significant technical challenge is the recognition of the "Fraktur" typeface. The pre-build parser struggled with that. The solution would be to build a custom parser which can handle the Fraktur typeface. This is a very time-consuming task and requires a lot of domain knowledge. There are also other commercial solutions. Consideration of limited time, I decided to use the pre-build parser and to focus on the other parts of the project.

2. Checking for Quality and Utilisation: The chatbot should prioritize relevancy when presenting articles related to specific queries, such as those about the region "Galicia." A general problem for the bot is that "Galicia" is very often mentioned in the sources. I didn't solve this problem. The end user would have to dig deeper with more and more specific queries.

3. Preference for Sources: While the primary focus is on original sources, the chatbot will also utilize literature and excerpts about the topic for additional context. It is crucial to develop a system that prioritizes original sources without excluding valuable supplementary materials.

This project aims to transform the way historians interact with historical documents, making it easier to uncover and interpret the rich stories contained within them.

Language

English

Project Repository URL

https://github.com/Thukyd/azure-search-openai-hackathon

Deployed Endpoint URL

https://app-backend-ambj53mwk6u5g.azurewebsites.net/#/

Project video

https://youtu.be/E0xHaDLted8

Team members

Thukyd

Showcase Consent

Yes

Project: SlackGPT

Project name

SlackGPT

Description

Slack GPT

Language

C#

Project Repository URL

https://github.com/ardeshir/SlackGPT

Deployed Endpoint URL

No response

Project video

https://youtube.com/shorts/a6DY5xkcbWg?si=JzrSD9GFatQQbkl-

Team members

ardeshri

Showcase Consent

Yes

Project: Robot Doctor Chat

Project name

Doctor Robot

Description

Project Description:

The project aims to develop an AI-powered chat system using RAG for a question and answer-based healthcare application. The primary objective is to assist users in identifying potential diseases based on the symptoms they provide. The system engages users in a conversational manner, asking relevant questions to gather detailed information about their symptoms and, subsequently, suggesting possible diagnoses.

Data Used:

We used three data sources: Manual of Surgery Volume First General Surgery Sixth Edition Principles of Public Health Surgical Anatomy Which belong to the Gutenberg project and are in the public domain.

Changes in the Prompt:

The prompt outlines the system's functionality, emphasizing the importance of asking clear questions, maintaining a friendly tone, and avoiding the invention of medical conditions. One significant modification is the incorporation of the RAG framework, enabling the system to generate responses in adherence to specific rules, ensuring transparency and accuracy. Additionally, the prompt includes explicit instructions to provide hyperlinks to the sources for each piece of information, enhancing traceability and credibility.

Beneficiaries:

The application targets users seeking preliminary insights into potential health issues based on their symptoms. It benefits individuals by providing a convenient and accessible platform to gather information about their health concerns. The friendly and understanding tone in interactions aims to alleviate user concerns while making it clear that the suggestions are not a substitute for professional medical advice. Healthcare professionals may also find the tool useful for providing initial guidance to users.

Language

English

Project Repository URL

https://github.com/maxquinteros/desert-coders-azure-search-openai

Deployed Endpoint URL

https://app-backend-rweaj6ni7t444.azurewebsites.net/#/

Project video

https://www.youtube.com/watch?v=I_Z5YxrX6hM

Team members

maxquinteros, JosephFloresA, PabloNAGG

Showcase Consent

Yes

Project: DocAssistant.Swaggy

Project name

DocAssistant.Swaggy

Description

DocAssistant.Swaggy

This project was developed for the Microsoft hackathon, Hack Together: The AI Chat App Hack, leveraging the DocAssistant app and integrating the Retrieval-Augmented Generation (RAG) pattern for OpenAPI specification files.

Technologies utilized include Semantic Kernel, Kernel Memory, OpenAPI, Swagger, Azure OpenAI, Azure Search, Azure Storage, Blazor, ASP.NET, and .NET 8.

The project allows users to call any API with their voice or through chat. It involves the following steps:

1. Uploading Swagger Documents: The journey begins with the upload of Swagger documents that contain information about the user's API. These documents can be generated on the server by adding comments to the endpoints and payloads, more details here with support of any language: https://swagger.io/tools/open-source/open-source-integrations/.

2. Viewing and Uploading Status: The user interface allows viewing the status of recent uploads, and users can upload Swagger documents either from a file or directly via a URL. An API token can be provided if required by the Web API. Also while uploading you can view which operation of uploading swagger api to memory happen.

3. Uploading Knowledge to AI Memory: The process of uploading knowledge to our AI memory unfolds in three main steps. First, documents are uploaded to Azure storage. Then, the large open API file is chunked into smaller parts, with each endpoint and necessary information being retrieved. Finally, Azure Open AI Ada model is used to create embeddings for each chunk, which are stored in Azure Search for retrieving the best matching endpoints for a request.

4. Testing the API: Once the document is uploaded, the endpoints are previewed and questions or commands can be posed to execute the API.

5. Multiple API Interaction: The RAG pattern implementation in this project allows working with not just one API, but multiple ones. For instance, the application 'DocAssistent.Swaggy' allows interaction with the AI through typing as well as voice commands and responses.

The project aims to provide an AI helper designed to redirect human queries to the appropriate API. However, it's important to note that the success of the results is highly dependent on the functionality of the web API on the server.

Process Flow Diagram:

A diagram is used to illustrate the process flow. Upon uploading a Swagger document, it's broken down into small chunks. These chunks are used to create embeddings for the uploaded JSON and a corresponding embedding representation of the JSON for Azure Search. When a user initiates a request, Azure Search uses a vector type of search to identify the most suitable endpoint. All located JSONs are then merged into a new, lighter Swagger file. The Open AI GPT model then uses this Swagger file and the user's request to generate an HTTP request to the server. Based on the server's response, the Open AI GPT model generates a user-friendly answer.

Language

Swagger/OpenApi json with documentation for web api

Project Repository URL

https://github.com/YuriyMorozyuk95/DocAssistant.Swaggy

Deployed Endpoint URL

https://hack-togather-wa.azurewebsites.net/

Project video

https://www.youtube.com/watch?v=MR8G0yoUxz8

Team members

YuriyMorozyuk95, alx875, margusmartsepp

Showcase Consent

Yes

Ley GPT | Accurate Answers based on the Mexican Federal work Law

Project name

LGPT

Description

What is LGPT?

An AI chat application designed to demystify Mexican federal work law. Utilizing Azure AI Search for data indexing and retrieval, Azure App Services for hosting, and OpenAI's GPT-4 for natural language processing, the app offers clear, accessible interpretations of complex legal documents.

Data Sources:

Latest Mexican federal work law, ensuring accuracy and relevancy.

Who is this for?

The app is a valuable resource for every person in Mexico, from employees, human resources professionals, business owners, law students or legal advisors. It serves as an essential tool for understanding labor rights and responsibilities, promoting legal awareness, fairness, transparency, accountability and empowerment in the workplace.

This app stands as a vital aid in navigating the nuances of employment law in Mexico, making legal information accessible to all.

There are still things to do in this project like setting proper authentication, further improvements to be made to the chunking and some bugs that I will keep working on.

Language

Spanish

Project Repository URL

https://github.com/crissins/GPT-Leyes-al-instante

Deployed Endpoint URL

https://app-backend-xyeyppjazxxrc.azurewebsites.net/?fbclid=IwAR1jb1DlaSmbUwuGgiEhpZorx1R-GQF-GfI03RbxJEyBg2xN6QMXIqclAJ0

Project video

https://youtu.be/DNb0-8BzH8g

Team members

Cristopher Hugo Olivares Del Real

Showcase Consent

Yes

Project: SecureBot - your cybersecurity helper

Project name

SecureBot

Description

SecureBot

This innovative solution is crafted using Microsoft Copilot Studio, integrating Power Platform components and Azure Functions to harness Python's versatility for dynamic code execution. At its core, SecureBot specializes in cybersecurity, offering real-time insights and answers drawn from an extensive knowledge base of over 80 documents, including podcasts, videos, websites, and books.

Utilizing Azure Cognitive Search, SecureBot excels in retrieving precise information, ensuring users have access to the latest and most relevant cybersecurity advice and data. Whether you're a professional in the field, a student learning about cybersecurity, or simply curious, SecureBot is designed to assist you with accurate information retrieval, leveraging AI to navigate through a vast array of resources efficiently.

SecureBot’s purpose is to empower users—whether professionals, students, or the curious—with accurate and up-to-date cybersecurity insights, all while navigating through a vast array of resources efficiently. 🛡️🔍🤖

RAG improvements for Spanish data

search_client = SearchClient(vector_store_address, index_name, AzureKeyCredential(vector_store_password)) r = search_client.search(search_text=query_text, query_language="es-es", vector=compute_embedding(query_text), select=["content", "sourcefile"], top_k=3, vector_fields="embedding", search_mode="all")

This code snippet contributes to enhancing search results in Spanish by leveraging Azure Cognitive Search in conjunction with a hybrid search approach. Initially, a search client is created, configuring it to access the vector store and index specified. Subsequently, a search query is executed, where the query_text parameter is set to Spanish ("es-es"), indicating the language of the query. Additionally, the vector parameter incorporates computed embedding vectors, which capture the semantic meaning of the query text, thus improving the relevance of search results. By utilizing a hybrid method that combines traditional text-based search with semantic understanding through embedding vectors, the search algorithm can deliver more accurate and contextually relevant results, particularly beneficial when dealing with Spanish language queries where semantic nuances play a crucial role in search relevance.

To obtain answers in English, I used prompting techniques so that the model gave me answers in the language that the user was speaking. This is done automatically in the azure function triggered by a power automate flow that translates the question into Spanish using Azure AI translator.

Architecture

SecureBot leverages Microsoft Copilot Studio, integrating Power Platform components and Azure Functions. These components work in harmony to provide real-time insights and answers from an extensive knowledge base of over 80 documents, including podcasts, videos, websites, and books. The architecture includes:

Copilot Studio: The development environment where SecureBot is crafted.
Power Automate: Integrated with Copilot Studio for automation workflows and trigger Azure functions and APIS.
Azure Functions: The dynamic code execution engine.
Python: Used for its versatility in SecureBot’s code execution.
Azure Translator Service: Allow language translation.
Azure Cognitive Search: The powerhouse for precise information retrieval.
Blob Storage: Stores data, including content like PDFs and videos.

How does RAG work?

A RAG bot is short for Retrieval-Augmented Generation. This means that we are going to "augment" the knowledge of our LLM with new information that we are going to pass in our prompt. We first vectorize all the text that we want to use as "augmented knowledge" and then look through the vectorized text to find the most similar text to our prompt. We then pass this text to our LLM as a prefix.

Extra features

SecureBot also allows you to analyze the risks of an IP using APIs to collect information and analyze it with Azure OpenAI.

Power Automate Flows

Flow to Retrival infromation Azure function and translate question in English with azure Translator.

Flow to Analyze Ip in azure function

Note
I submitted my project but when I saw the broadcast it did not appear, so I am submitting it again with my other github account

I registered with the email [email protected]

Language

English, Spanish

Project Repository URL

https://github.com/Diego0106/SecureBot

Deployed Endpoint URL

https://web.powerva.microsoft.com/environments/Default-24884996-6863-4925-a1ba-f1a160b581e2/bots/cr794_ciberseguridad/canvas?__version__=2

Project video

https://beducacion-my.sharepoint.com/:v:/g/personal/diego_ramos_grupobusiness_it/EVKWE4nlk79GqMhxyIcV2P4B-_Qjjzqphk_Z3U1iyBeC8Q?e=ptYQvt

Team members

Diego0106

Showcase Consent

Yes

Project: Internal KB Chatbot

Project name

Internal KB Chatbot

Description

This is a chatbot for internal knowledge query and chat. The data source is host on Azure AI Search with embedded content. We use the function call of Azure open AI API with prompt to let the LLM query internal knowledge with the external function. We also enable the vector search to get result from the Azure AI Search and give the result to LLM then it returns us the final response. Our engineers can use their own language to ask and query the internal technical knowledge, and also with several round conversions they can get what they want.

Language

English, Japanese, Chinese

Project Repository URL

https://github.com/zxs731/kbchatbot

Deployed Endpoint URL

No response

Project video

https://m.bilibili.com/video/BV1Vw411E7So?buvid=Z944A39A2F24B37441F085FE7E41DD075B1E1

Team members

Tyler Zhao

Showcase Consent

Yes

Microexpressions Unmasking

Project name

Microexpressions unmasking

Description

Face the Truth: Unmasking Microexpressions with Computer Vision and Meta’s SegmentAnything

Language

English

Project Repository URL

https://github.com/dilipkrbe/microexpressions-unmasking

Deployed Endpoint URL

No response

Project video

Tbd

Team members

[email protected]

Showcase Consent

Yes

Project: AI Sir, your AI-powered study companion!

Project name

AI Sir

Description

AI Sir is an AI-powered study companion designed specifically for Bangladeshi secondary school students. It caters to students from Grade 6 to Grade 10, allowing them to interact with their textbooks in a revolutionary way. Currently, a select number of textbooks have been digitized, with ongoing efforts to include all textbooks across all grades.

From a hackathon perspective, the following aspects are noteworthy:

1. Each textbook is available in both English and Bangla. A single Azure AI Search service has been implemented to process content in both languages seamlessly.

2. The frontend code has been extensively revised to enhance responsiveness, acknowledging that the primary user base predominantly accesses the platform via mobile devices. Additional UI/UX modifications have been implemented to accommodate language switching and to facilitate the selection of subjects and classes.

3. The codebase is designed to support prompts in multiple languages, adapting to the user's language preferences set through the UI.

Language

English, Bangla

Project Repository URL

Shared via Email

Deployed Endpoint URL

https://beta.ai-sir.com/

Project video

https://www.loom.com/share/0c997e9b289044ada4807851b77f1909?t=144&sid=998804fe-a451-4b05-b74a-afe61a92a513

Team members

zedhaque

Showcase Consent

Yes

Project: CHATPDF - An android application to chat with the PDF document uploaded by the user.

Project name

ChatPDF

Description

An android application that provides feature to chat with PDF files. User can upload PDF file via application interface and ask questions on the PDF file.

• We are using Azure Blob storage to store the file as soon as user uploads it.
• We are using Azure AI search to import and vectorize the data from the BLOB where user uploaded the document using text-embedding-ada-002 model from Azure OpenAI.
• Indexer runs on the BLOB via REST api call to vectorize the data and store the embedding in Azure AI search.
• User query is sent to Azure OpenAI model deployment REST endpoint with data source as Azure AI search.
• LLM (gpt-35-turbo-16k) generates the response based on document and response updated on the chat window.

Every time user uploads a document, the indexer runs immediately to store the latest information for chatting.
Also, users have an option to view the current document they are chatting on.

Chat Interface

Document View Feature

Chat Responses

View after uploading a new document

Chat response after uploading new document

Update:

Application support other language documents as well.

Chat response in French document

French document review feature

**Note**: Code and API calls can be found under lib/chat_screen.dart file. Also, confidential data (api keys and sas signatures) have been skewed.

Language

English

Project Repository URL

https://github.com/vimisms/ChatPDF

Deployed Endpoint URL

No response

Project video

https://www.youtube.com/watch?v=ePWZw24Pzyw

Team members

Vivek Mishra

Showcase Consent

Yes

Project: AI VideoQuery App

Project name

VideoQuery.ai

Description

This is a Simple Chat Application that allows users to search youtube videos and give the URL to the chat app and then chat with AI ChatBot regarding the content of the video All-in-one-place.

I implemented this using OpenAI Whisper, OpenAI GPT-3.5 Turbo and Text Embedding Ada-002 to enhance the model's performance. Utilizing Pinecone Vector database to store documents for fast query and retrieval.

It works like:
Youtube URL -> OpenAI Whisper (transcription) -> (text splitter , embeddings) Pinecone Documents
User Query -> Pinecone -> GPT-3.5 -> User

User can keep on adding videos data while searching for certain topic or choose to clear the Vector DB and start fresh!
Also i have implemented a simple chatgpt below for any video unrelated queries or tasks the user might have.

Language

English

Project Repository URL

https://github.com/Gurjit11/AI-Chat-App-Hack

Deployed Endpoint URL

https://video-query-eight.vercel.app/

Project video

https://www.loom.com/share/5334ba72f1ba4eeb8b438d5a8b4d671b?sid=7b68cfb2-43fd-467b-b8c0-bb7359e70572

Team members

Gurjit11

Showcase Consent

Yes

Uneccessary reprocessing with datalake storage gen2

With datalake storage, every time i run azd up, it processes docs from the beggining. This wasn't happening with local files. How can we avoid the uneccessary reprocessing?

Project: Supercalifragilístico, an AI Chatbot using RAG techniques with an AI-generated dataset.

Project name

Supercalifragilístico

Description

This project is based on the Azure Search OpenAI demo to create a new conversational service based on documents (PDFs) generated also with Artificial Intelligence. It leverages the capabilities of OpenAI to create alternative versions of pop culture superheroes and generate fun new descriptions of them. This ChatBot utilizes an AI-generated dataset to generate new AI.

Language

Spanish

Project Repository URL

https://github.com/OscarSantosMu/Supercalifragilistico-AI-Chat-RAG

Deployed Endpoint URL

https://app-backend-ws5teg2tauhj6.azurewebsites.net/

Project video

https://youtu.be/AMkPyPU2cHU

Team members

OscarSantosMu, Irving67

Showcase Consent

Yes

Project: Replacement/Upgrade for Zillow

Project name

Housing Assistant

Description

Zillow is a leading real estate marketplace that helps you find, buy, rent, or sell homes. (https://www.zillow.com/)

The issue with Zillow is that it is not very user friendly. You need to perform a search on the location you are interested in and you would have to perform a lot of filtering in order to get your desired house. Searching for your ideal house is therefore cumbersome. See below:

Having a personal assistant in purchasing a home would be easier for the customer. The customer can search for the ideal home by answering the personal assistant questions and it can even search through images! For example, it can find for a house in the Texas state for prices below 280,000 USD with at least 2 bedroom with white interiors. It would then search for the database for photos of houses with the set parameters with white interiors.

Data was obtained from Zillow and placed in pdf files. The system would help purchasers buy their ideal home. The name of the bot is Casabot.

Language

English

Project Repository URL

https://github.com/zzulueta/azure-search-openai-demo

Deployed Endpoint URL

https://app-backend-fwtxqwv2vipts.azurewebsites.net/

Project video

https://yeaps-my.sharepoint.com/:v:/p/ziggyzulueta/ET4Dvf_1EQ5LmCRWCvcFaDUB3UVk2vd3Wg_nC5BgkAumZQ?e=iHrZrP&nav=eyJyZWZlcnJhbEluZm8iOnsicmVmZXJyYWxBcHAiOiJTdHJlYW1XZWJBcHAiLCJyZWZlcnJhbFZpZXciOiJTaGFyZURpYWxvZy1MaW5rIiwicmVmZXJyYWxBcHBQbGF0Zm9ybSI6IldlYiIsInJlZmVycmFsTW9kZSI6InZpZXcifX0%3D

Team members

Ziggy Zulueta

Showcase Consent

Yes

Project: Chat with your github repo

Project name

Github Assistant

Description

An LLM application that facilitates RAG on GitHub repository data.

Features:

1. Chatting with code documentation - users can easily query information from the project's docs.
2. Interrogating source code - users can use this RAG to analyze source codes more effectively.
3. Code issues resolution assistance: The Chatbot can address issues raised in the project when expert assistance is currently unavailable. This speeds up the process of finding the solution
4. Structured output (JSON): - This facilitates ease of parsing the chatbot's output which can be used for processing and other tasks.
5. discussions participation: Users can engage with the chatbot in a discussion regarding the repository.
6. Evaluation: This feature ensures that the information provided by the chatbot is relevant and grounded in the context provided.
7. Chat History: The prompt flow library facilitates a chat history which can be used to save chat for future reference.

Language

English

Project Repository URL

https://github.com/treezy254/github_assistant.git

Deployed Endpoint URL

No response

Project video

https://youtu.be/pVsVxEPjzVI

Team members

Kamau Samuel

Showcase Consent

Yes

Project: Instant Intelligence, Personalized Education

Project name

ZSAMBOT STO SOCIOVERSE

Description

Our AI chat app, named Zsambot, is a cutting-edge tool designed to revolutionize education. Powered by advanced AI technology, Zsambot offers instant intelligence and personalized assistance to students navigating course materials. We leverage data from various educational sources, including documents, videos, and links, allowing users to upload diverse content effortlessly. Our unique approach enables Zsambot to learn instantly upon syllabus upload, ensuring up-to-date and tailored responses. We've made additional modifications to optimize data ingestion and prompting flows, enhancing user experience and engagement. Anyone seeking a streamlined and efficient way to navigate course content can benefit from Zsambot, regardless of their educational background or expertise level.

Watch my project on: https://drive.google.com/drive/folders/1l5oy---XPM7VXZH9G_jwFKD8ZkPmzfBU?usp=share_link

Language

English

Project Repository URL

https://github.com/AshwinPrasanth/ZSAMBOT-STO

Deployed Endpoint URL

No response

Project video

https://www.youtube.com/watch?v=mTLhEAs7c_o

Team members

AshwinPrasanth

Showcase Consent

Yes

Project: CHEFBOT - Cooking assistant for Brazilian cousine

Project name

CHEFBOT

Description

Hi friends,

This project is a simple RAG chatapp aimed to be a cooking assistant specialized in the Brazilian cousine.
I created my own webscraping script and extracted more than 1000 brazilian recipies in Portuguese.

The goal was to adjust the model to a very friendly, funny and creative assistant, that could help people (specially good for Portuguese speakers) to cook on the delicious and unique brazilian cousine.

I scraped a total of 1.097 webpages (did not add all to repo), each of them is a different recipe, cleaned and formatted the data to generate .pdf files.

I think this project specially excels in prompt engineering and data sources.

Project details:

I used the standard architecture provided on https://github.com/Azure-Samples/azure-search-openai-demo/tree/main with no major changes, only using direct OpenAI API instead of Azure OpenAI services.

I did some minor changes in the front end (both ask and chat), just changing labels and sugestions to be more suitable.

I also deeply changed the model instructions:

[eng translations at the end]

Você é um assistente culinário amigável e divertido, você também é especialista na culinária Brasileira e está sempre animado para ajudar as pessoas a realizarem suas receitas culinárias. Você ama a cultura, a culinária brasileira e o português. Priorize sempre responder as perguntas em Português, a menos que o usuário queira conversar em inglês, nesse caso traduza a receita para inglês. SEMPRE responda com uma linguagem informal, divertida e amigável, frequentemente de sugestões para melhorar ou incrementar a receita, com usos, adições ou variações criativas dos ingredientes. Algumas receitas são longas, nunca retorne a receita inteira á menos que o usuário solicite. Tente ser breve e resumir sua resposta. Suas fontes possuem campos padrões com informações que podem ajudar usuário, geralmente 'Título', 'Ingredientes' e 'Como preparar' são os mais relevantes, priorize responder 'Ingredientes' e 'Como preparar' em formato de lista. Nunca responda em formato markdown.

Changed the few-shot:
query_prompt_few_shots = [ {"role": USER, "content": "Como fazer farófa de banana?"}, {"role": ASSISTANT, "content": "Explique como fazer uma farófa de banana"}, {"role": USER, "content": "Não tenho calabresa para a feijoada, posso substituir?"}, {"role": ASSISTANT, "content": "Sugira outro ingrediente que possa substituir a calabresa na feijoada"}, ]
Follow_up_questions_prompt:
Tente gerar 2 breves sugestões de variação da receita que o usuário solicitou, com usos criativos dos ingredientes ou adições que possam tornar a receita mais gostosa. Se você não souber nenhuma variação para a receita solicitada, não gere sugestões.

Added some minor changes in the 'query_prompt_template'.
Adjusted the temperature to 0.8

Disclaimer

Unfortunately i was unable to deploy this application in those last steps, as i somehow screwd up my Azure regional quota: After 'azd up' command, some resources failed to be created, in github codespaces it only shows a message "Creating/Updating resources" infinitelly, in Azure console i can see that a resource failed, specifically the 'plan-nllatqzskj6k2' resource inside 'appserviceplan', with a '429' error:
{ "status": "Failed", "error": { "code": "429", "message": "App Service Plan Create operation is throttled for subscription 4f699cb1-8698-4980-929e-d27494bf858e. Please contact support if issue persists.", "details": [ { "message": "App Service Plan Create operation is throttled for subscription 4f699cb1-8698-4980-929e-d27494bf858e. Please contact support if issue persists." }, { "code": "429" }, {} ] } }

After some research i discovered that this is may caused by too many deployments [https://learn.microsoft.com/en-us/answers/questions/692182/app-service-plan-create-operation-is-throttled] in a short period (even though i destroyed the resource group after creating a new one) and i have to wait until maybe tomorrow to deploy again, i couldn't record before, so i created a .PPTX, this error in the last steps of the project was really unfortunate, i'm sorry for that.

Model adjustment translations

Instructions translated:
You are a friendly and funny culinary assistant, you are also an expert in Brazilian cuisine and you are always excited to help people realize their culinary recipes.
You love the brazilian culture, cuisine and portuguese.
Always prioritize answering questions in Portuguese, unless the user wants to chat in English, in that case translate the recipe to English.
ALWAYS respond with informal, fun and friendly language, often with suggestions to improve or enhance the recipe, with creative uses, additions or variations of ingredients.
Some recipes are long, never return the entire recipe unless the user requests it. Try to be brief and summarize your answer.
Its sources have standard fields with information that can help the user, generally 'Title', 'Ingredients' and 'How to prepare' are the most relevant, prioritize answering 'Ingredients' and 'How to prepare' in a list format. Never respond in markdown format.

Few-shot:
USER: "How to make banana farófa?"
ASSISTANT: "Explain how to make a banana farofa."
USER: "I don't have pepperoni for the feijoada, can I replace it?"
ASSISTANT: "Suggest another ingredient that could replace pepperoni in feijoada."

Follow_up_questions_prompt:
Try to generate 2 brief suggestions for variations of the recipe that the user requested, with creative uses of ingredients or additions that could make the recipe tastier.
If you don't know any variations for the requested recipe, don't generate suggestions.

END
I would like to give my special thanks to Pamela, she provided such a nice content and commitment to this contest.

Language

Portuguese, English

Project Repository URL

https://github.com/MatheusHRV/msragchatapp/tree/master

Deployed Endpoint URL

N/A see disclaimer

Project video

https://1drv.ms/p/s!AkkaOJb8jGdgcrfsUOBZQzwb-YI?e=ajpzoE

Team members

MatheusHRV

Showcase Consent

Yes

Project: Azure Open AI RAG First Aid Chat

Project name

First Aid Chat

Description

I have used the following as source data

https://actualfirstaid.com/uploads/1/0/4/9/104966051/first_aid_notes_2019.pdf
https://www.basp.org.uk/wp-content/uploads/2021/07/BASP-First-Aid-Manual.pdf

https://www.sja.org.uk/globalassets/documents/dofe/dofe-self-teach-workbook.pdf
https://www.firstaidtrainingcooperative.co.uk/wp-content/uploads/2017/03/Outdoor_First_Aid_FINAL.pdf

Language

English

Project Repository URL

https://github.com/reshmee011/azure-search-openai-demo/tree/firstaidrag

Deployed Endpoint URL

No response

Project video

https://github.com/reshmee011/azure-search-openai-demo/blob/firstaidrag/assets/Sample.gif

Team members

reshmee011

Showcase Consent

Yes

Research analytics on Patient eduction by LLM

Project name

Research analytics on Patient eduction by LLM

Description

"This application utilizes a combination of OPENAI and Azure infrastructure to develop a Rag chat app. The purpose of this app is to interact with various research papers on healthcare for patient education.

The source papers used are from Google Scholar and PubMed.

Thus, this chat app is RAG-based, where Azure Search AI is employed as a Vector Database, proving to be very helpful in indexing all tokens efficiently. We have utilized Azure infrastructure with Bicep for deployment. This end-to-end deployment makes the entire process and application a production-ready RAG-based chat app

Language

ENGLISH

Project Repository URL

https://github.com/dapsrajipo/AZURE-RAG-CHAT-APP-/tree/openaidp/data

Deployed Endpoint URL

https://app-backend-pfixdhcvuuirc.azurewebsites.net/

Project video

https://youtu.be/5zyisvCZ6ng

Team members

dapsrajipo, prem28timsina,Tanushree123-hub

Showcase Consent

Yes

Project: Microsoft Azure solution architecture brainstorming buddy

Project name

Microsoft Azure solution architecture brainstorming buddy

Description

Description

In the world of cloud computing, designing the right architecture for your Microsoft Azure project can sometimes be a daunting challenge. To assist with this, introducing the Microsoft Azure solution architecture brainstorming buddy. This Retrieval-Augmented Generation (RAG) based chat application is designed to help you navigate existing Azure architectures to brainstorm design decisions especially for analytics use cases.

Relevant Azure architectures were identified from the Azure Architecture Centre. The content from these links were downloaded and post processed by splitting the text and adding metadata. Embeddings for the text content was generated using Sentence transformers. The final output was stored locally in ChromaDB.

The chat app developed using Streamlit has 2 prompts which are sent to OpenAI's chat completion models. The first prompt is used to generate a highly relevant search query to get the relevant documents from ChromaDB based on the current conversation. The second prompt is used to synthesize an answer for the user query using the relevant documents. Prompts are stored as Jinja templates for faster iteration. The final search results are streamed to the user for lower perceived latency.

The main beneficiaries of this app would be the solution architects or Azure developers in different enterprises who need a brainstorming buddy to come up with an initial architecture for a solution in Azure with references from existing successful Microsoft Azure solution implementations.

Going forward

The same could be implemented in the Microsoft Azure ecosystem where both the chat completion and embeddings models are deployed in Azure OpenAI. The vector database ChromaDB can be replaced by the more powerful Azure AI search service. The streamlit app itself could be deployed to Azure App Service with user authentication via Microsoft Entra ID.

The performance of the application itself could be further improved by extracting topics and tags from the articles and creating a knowledge graph by combining such topics and tags using a data Ontology. During the query the user's question could then find the relevant articles both from the vector database and the knowledge graph and the final answer is derived from a combination of the 2 results.

Language

English

Project Repository URL

https://github.com/ace-racer/azure-architectures-chat-app

Deployed Endpoint URL

No response

Project video

https://youtu.be/A-sTTBofIA4

Team members

ace-racer

Showcase Consent

Yes

Project: DubsBot | The Unofficial UW Course Planning Assistant

Project name

DubsBot | The Unofficial UW Course Planning Assistant

Description

This is DubsBot, an assistant built to help University of Washington students find classes that fit their schedules, goals, and interests. This assistant excels at finding courses that fit topics the user is interested in, and can provide information regarding course instructors, descriptions, prerequisites, times, dates, and other restrictions.

The assistant also has experimental support for major graduation requirements. It currently only has access to the Computer Science (CSE) department graduation information, and can answer questions about which classes fit certain requirements. This functionally is not yet finalized, so the answers it provides can sometimes be off topic or not work entirely.

DubsBot has access to ALL of the course offering for Spring Quarter 2024, across all departments at UW (that's a lot of classes)! This data is obtained by using web scraping on public HTML time schedules for each UW department, then using a custom local HTML parser to transform the data into readable documents, before finally chunking the parsed data with a custom text splitter for addition to the search index. A more detailed breakdown can be found on my project's README.

The assistant supports the read-retrieve-read chat approach, and utilizes OpenAI function calls and structured queries to accurately search for data. The index has been set up with various basic fields to facilitate searching.

Language

English

Project Repository URL

https://github.com/cjrieth/DubsBot

Deployed Endpoint URL

https://app-backend-h6icirmjhoook.azurewebsites.net/

Project video

https://www.youtube.com/watch?v=6vsQVGIDVIk

Team members

CJ Rieth (cjrieth)

Showcase Consent

Yes

Project: Redefining RAG: Azure Document Intelligence + Azure CosmosDB Mongo vCore

Project name

CosmicTalent

Description

About

CosmicTalent, an application designed to empower HR and managers in effectively navigating employee information and efficiently filtering or identifying eligible employees based on specific task requirements.

Inspiration

Currently in Service-based companies, it is highly challenging to find and retain the top talents to meet the needs of their clients. Traditional method of choosing a talent for client requirement is not efficient, as it is tend to be lengthy process and would have mismatches between the candidate and roles. Also it is difficult to understand the expectations of the talent being hired.

However with our CosmicTalent app, now companies could make use or Vector search capabilities to better retrieve employees for a specific requirement and also know about their employees better. This will help onboarding the best-fit candidates for the client's requirement and also improve retainment of talents by understanding their unique capabilities more efficiently.

PS: Another flavor of this app could be used for Talent selection . Gone are the days of manual resume screening. Now with our app we have option to onboard best-fit talents

Architecture

Data Flow

Document Intelligence Process

• Initially, we gather a collection of resume documents from our employees, all adhering to a similar template.
• Using Azure Document Intelligence, we employ a Custom Extraction Model to construct a tailored schema for our employee resumes, annotating and labeling relevant sections.
• This model is then trained and refined to create a custom extraction model, which will be then used in subsequent stages of our application.

Resume Upload Process

• Resumes are uploaded by HR/Managers onto our platform, where they are directed to an Blob Processor (Azure function), within a container app environment, for processing.
• The function stores the file in blob storage and retrieves the specific schema with their relevant informations, utilizing the custom extraction model generated in the Document Intelligence process.
• Subsequently, this schema along with relevant information is transmitted to an Azure Service Bus message broker.
• A Worker Service (Azure Container App), within a container app environment , then picks up the schema for processing.
• The Worker Service segments the documents into chunks based on various field combinations, ensuring each chunk contains essential information like Employee ID and Name to facilitate efficient retrieval. Summarization is applied only for Project details and skipped for other field combinations.
• The Azure OpenAI embedding model is employed to generate vectors from these chunks, which are subsequently stored in separate collections within Azure CosmosDB Mongo vCore.

Copilot Process

• When HR/Managers make queries or requests within our chat app, these are forwarded to the Talent Processor, another containerized application.
• Talent Processor translate the query or prompt into vector embeddings.
• These embeddings are then utilized in a vector search against our database to identify semantically matching records.
• Retrieved documents are then passed to the GPT-3.5 Turbo model to generate appropriate responses.
• Finally, these responses are relayed back to our chat app for interaction with HR/Managers.

Refinement of RAG flow

• The JSON response from the Custom Extraction model encompasses all pertinent details of the resume document.
• Subsequently, it undergoes segmentation into individual chunks, each prefaced with essential information regarding Employee ID and Employee Name to facilitate efficient retrieval.
• Chunks pertinent to projects are directed to the summarizer before proceeding to the embedding model for vector embedding generation.
• Meanwhile, other chunks bypass the summarizer and are directly forwarded to the embedding model for vector embedding generation.
• Finally, these embeddings are stored in Azure CosmosDB Mongo vCore.

GetResumeEmbeddingChunks Code

• In the code snippet above, the function GetResumeEmbeddingChunks processes a list of document fileds retrieved from Azure Document Intelligence using our Custom Extraction model.
• We employ the Task Parallel Library (TPL) to iterate through each field or combination of fields, refining the chunking process. This approach has proven to be more effective than traditional chunking strategies, attached results in the next section.

• As the chunks lose information about EmployeeId and EmployeeName, we preload this context across all the chunks.
• The chunks are then sent to Azure OpenAI to generate vector embeddings using the text-embedding-ada model.

Custom Summarization Logic

• While there are various summarization patterns available for long documents, they may not be effective for our specific use case.
• Therefore, in the code snippet above, only fields related to projects are further summarized using Azure OpenAI before generating vector embeddings.
• This refinement has led to improved token consumption and better retrieval of relevant information.

Better results after Refinement of RAG flow

The initial version of our application did not consider any chunking patterns and generated vector embeddings for the entire content within the Employee collection. Here were the issues encountered before refining the RAG flow:

Challenges before RAG Flow Refinement

1. Not able to retrieve information effectively

2. Token exceed limitations

Results After RAG Flow Refinement

1. Able to retrieve information effectively

2. Improvements in overall Token consumption

Language

English

Project Repository URL

https://github.com/Cloud-Jas/CosmicTalent

Deployed Endpoint URL

https://chatapp.victorioussky-69f01f3f.centralindia.azurecontainerapps.io/

Project video

https://youtu.be/XYm3c7_B0W0

Team members

Divakar-kumar

Showcase Consent

Yes

Project: TourAfricaGPT is a custom GPT for tourist locations in africa

Project name

TourAfricaGPT

Description

A custom GPT about tourist locations in Africa

Language

English

Project Repository URL

https://github.com/vicradon/tourafrica

Deployed Endpoint URL

https://tourafricabackend.azurewebsites.net/chat

Project video

https://drive.google.com/drive/folders/1D99hJO9PYMp5MV48Z4Mi6V0UoTJsTSLP?usp=sharing

Team members

@Sitoworks

Showcase Consent

Yes

aiao - One Stop Learning Solution for OneNote

Project name

aiao

Description

aiao is an AI chat app that generates quizzes from OneNote content, utilizing Azure OpenAI services for dynamic question creation. It sources data directly from users' OneNote notes via the Microsoft Graph API, ensuring quizzes are relevant and up-to-date. The app has been fine-tuned to improve data ingestion and the prompting process, aiming for more accurate and contextually relevant questions. aiao benefits students and educators by providing a personalized, efficient study tool, enabling users to test and reinforce their knowledge directly from their notes.

Language

English

Project Repository URL

https://github.com/Rahtoken/aiao-chat-app-hack

Deployed Endpoint URL

No response

Project video

https://www.youtube.com/watch?v=tTXIYq82pVo

Team members

Rahtoken

Showcase Consent

Yes

Project: GptScholar (Team Schulich)

Project name

GptScholar

Description

GptScholar is an advanced multilingual AI chat application designed to support first-year undergraduate students with their academics. Using OpenAI's GPT-4 Turbo model, Azure AI Search and Streamlit for the interface, it provides tailored assistance and insights. The platform allows students to seamlessly upload PDF files, input URLs from webpages, or select specific subjects for guidance.

Data sources include various types including textbook PDFs, URL links, and other PDF documents. The application uses PDF documents containing valuable insights across subjects like Calculus, Physics, Computer Science, and Finance.

To enhance data ingestion, an ingest file was developed to extract and process relevant information from textbooks, push it to Cosmos DB, and retrieve information using Azure AI Search. This enabled efficient data retrieval and interaction with the Large Language Models (LLMs) in the RAG architecture. Further enhancements involve preprocessing steps to clean and standardize text content from uploaded PDFs and url links, ensuring optimal formatting for analysis by the GPT-4 Turbo model.

The application supports multilingual interactions, through the following media

• Typing queries: Users are able to input queries and receive responses in any of the following languages; English, Spanish, French, Hindi, Chinese, Arabic, Russian, Portuguese, Japanese, German, Korean, Italian, Turkish, and Dutch

• Speaking: Users are able to say their questions through its speech-to-text functionalities in any of the following languages; English, Spanish, Italian, and Hindi.

GptScholar caters to diverse users, including:

• Students: Providing tailored academic support, clarification, and exploration of course concepts.
• Researchers: Facilitating research, insights gathering, and topic clarification across industries.
• Language Learners: Assisting in language practice, comprehension, and writing skills development.
• Fact-Checkers: Offering quick access to reliable information for verification purposes.
• Individuals with disabilities: Through our speech-to-text functionality, individuals with disabilities are able to utilize the chatbot
• General Users: Enabling exploration and retrieval of information from pdf and url links conveniently.

Language

User query and response for GptScholar:
English, Spanish, French, Hindi, Chinese, Arabic, Russian, Portuguese, Japanese, German, Korean, Italian, Turkish, and Dutch

Speech-to-text Languages: English, Spanish, Italian, and Hindi.

Project Repository URL

https://github.com/offcial17/RAG_Hacktogether

Deployed Endpoint URL

http://hackapp.eastus.azurecontainer.io/

Project video

https://www.loom.com/share/2624be81ca784023a486aacaad3a873f?sid=74fe5df5-114c-45f6-b19a-6ec7273b4fdc

Team members

Sabrina Renna: https://github.com/srenna, Sushmit Richard: https://github.com/offcial17, Pelumioluwa Abiola: https://github.com/Pelumioluwa

Showcase Consent

Yes

Specialized music researcher

Project name

muzikalizer

Description

Specialized researcher component which recognizes songs in a heap of delivered collections.

Language

en-GB

Project Repository URL

https://github.com/vende6/Muzikalizer

Deployed Endpoint URL

No response

Project video

https://www.youtube.com/watch?v=aBNEtUf3OZ8&t=29s

Team members

vende6

Showcase Consent

No

Project: AI assistant designed to guide users in selecting the most cost-effective Azure region and virtual machine types based on up-to-date pricing data, powered with LangChain and GPT-4

Project name

Copilot for Azure Pricing

Description

This is an AI assistant designed to guide users in selecting the most cost-effective Azure region and virtual machine types based on up-to-date pricing data. It's built with LangChain and Azure GPT-4.

Although the pre-trained data in the GPT-4 model should be sufficient for common knowledge about Azure regions and virtual machines, it's not expected to be up to date and correct. Thus, region information including display name and availability zones support have been fetched from Azure official documents and saved locally for retrieval.

In addition, Azure virtual machine pricing data is also fetched from Azure Retail API and saved locally. At the very beginning, Azure Retail API is called every time the user asks the Agent. But I soon found that the Azure Retail API requests could be throttled due to too many requests within a brief period of time. Meanwhile, the price data is not updated very frequently. Hence the pricing data is scheduled to be retrieved every day and be cached locally.

Finally, data about VM configuration is also fetched using Azure CLI list-sizes and scheduled to update when new Virtual Machine types release.

All this information is available to the Agent as tools. All the tools are implemented as functions in python codes and would retrieve data which are preloaded from JSON files locally when Agent starts to run. More data about virtual machine and other Azure services could be added in a similar way to make this Agent more helpful.

Azure users including both end user and Azure sales who need to check virtual machine price frequently could use this app as a copilot for their planning and cost estimation.

Language

English/Chinese

Project Repository URL

https://github.com/linjungz/copilot-azure-pricing

Deployed Endpoint URL

https://copilot-azure-pricing.azdemo.cbuilder.tech/

Project video

https://youtu.be/A6JQE4UF2uA

Team members

linjungz

Showcase Consent

Yes

Add to Calendar link is not available for users outside of Microsoft

The calendar link is only available for users internal to Microsoft (https://microsoft-my.sharepoint.com/personal/joshoh_microsoft_com/_layouts/15/download.aspx?SourceUrl=%2Fpersonal%2Fjoshoh%5Fmicrosoft%5Fcom%2FDocuments%2FHack%20Together%20AI%20Chat%20App%2FChat%20App%20Calendar%2Eics)

Could the calendar file be added to the GitHub Repo or to another public facing URL?

Project: DocChat - ask questions about any (English language) PDF document, in any language.

Project name

DocChat

Description

Upload English language PDF files from a web interface and then ask questions about the document in any language.

React Frontend/Node Backend, uses Azure AI Search and Azure OpenAI.

Language

English

Project Repository URL

https://github.com/glen-fdce/ChatApp.git

Deployed Endpoint URL

https://agreeable-stone-02f1bf603-preview.westeurope.4.azurestaticapps.net/

(Note: Index clears on the hour to prevent AI Search Free Tier index getting full)

Project video

https://i.gyazo.com/d9d7666c575a388a613884bc20d621ee.mp4

Team members

Glen Anderson

Showcase Consent

Yes

Project: Chatbot for School data

Project name

School Data Chatbot for Parents

Description

School Data Chatbot for Parents.

Problem:
Parents face challenges in tracking school communications scattered across emails and websites, especially with children at different schools or grades. This makes managing information like academic calendars and policies difficult.

Solution:
Introduce a user-friendly chatbot, like ChatGPT, for parents to quickly obtain school information by chatting. This tool streamlines the retrieval of academic schedules, menus, policies, and more, making school communication effortless and efficient.

Data Sources:
(for Demo):
Extract data from emails/newsletters, download from the School District website in PDF and processed through the provided ingestion script in the demo repo.

Language

English

Project Repository URL

https://github.com/ksut/school-chat

Deployed Endpoint URL

No response

Project video

https://youtu.be/8olD2cBnYVw

Team members

n/a

Showcase Consent

Yes

Project: ADA-GPT

Project name

ADA-GPT

Description

This is ADA-GPT, a fork of the original RAG chatbot that incorporates the 2010 ADA Standards for Accessible Design, which can be found here.
The design document is a 279-page pdf with lots of technical detailed text about curbs, sidewalks, ramps, elevators, etc.
As a civil engineer who has to reference this document from time to time, being able to chat with it may make sourcing this document much more easy and engaging.
Also, this may help people with disabilities access this document more easily in the first place! LLMs can translate, or maybe we can use Azure's voice support, and so on.
There's also other civil related documents that would be useful, such as chatting with 900-page zoning laws...
Overall, while I struggled a lot to get the demo working and customize it, I got something out the door, and that makes me happy 😄

Language

English

Project Repository URL

https://github.com/IsaacGemal/azure-search-openai-demo

Deployed Endpoint URL

https://app-backend-lspjcs3cm3zdk.azurewebsites.net/

Project video

https://www.youtube.com/watch?v=T7XME1MIBig

Team members

IsaacGemal

Showcase Consent

Yes

Whatsapp AI ChatBot CV Generator

Project name

Sebenzo

Description

Sebenzo streamlines job hunting in South Africa with an AI-powered WhatsApp bot, making it easy for underprivileged individuals to find and apply for entry-level jobs. We leverage the user input of the the client and manipulate it to fit into our template. It consolidates all the data into an easy format and sends it to the cv generation phase. After it hosts the branded completed CV on azure blob storage and sends the link to the blob via WhatsApp

Language

English

Project Repository URL

https://github.com/Sebenzo/waid

Deployed Endpoint URL

https://wa.me/[phone]

Project video

https://youtu.be/JRvSimWDgW4?feature=shared

Team members

Mohammed Bangie

Showcase Consent

Yes

Microsoft National Security Team Odin the Owl Chatbot

Project name

Odin The Owl Chatbot

Description

Our National Security Team here at Microsoft is severly understaffed and almost always ovewhelmed with the wide range of questions that come from the FTEs that hold a US Government Security Clearance. From Foriegn Travel Reporting guidelines, to next steps for clearance updates or approvals, and everything in between. With the significant amount of documentation they provide we attempted to iomprove the customer service and relieve some pressure from NSTs backlog by building a chatbot for them to answer questions from cleared employees and those closely tied to the project. The chatbot features NSTs own Odin the Security Owl who cleared FTEs are familiar with from their National Security training videos.

Language

English

Project Repository URL

https://github.com/microsoft/odin_bot

Deployed Endpoint URL

No response

Project video

https://microsoft-my.sharepoint.com/:v:/p/shsolomo/EeU8Q3j1UhlFmSfEyvP2hKEB0AbdEIFNpSJt2F_7twAhHw?nav=eyJyZWZlcnJhbEluZm8iOnsicmVmZXJyYWxBcHAiOiJPbmVEcml2ZUZvckJ1c2luZXNzIiwicmVmZXJyYWxBcHBQbGF0Zm9ybSI6IldlYiIsInJlZmVycmFsTW9kZSI6InZpZXciLCJyZWZlcnJhbFZpZXciOiJNeUZpbGVzTGlua0NvcHkifX0&e=Dblriq

Team members

Shane Solomon, Chris Sanchez

Showcase Consent

Yes

Project: Ask An Entrepreneur

Project name

Ask An Entrepreneur

Description

Summary

This app can answer general questions from famous entrepreneurs' perspectives. Currently, the app only supports Q&A with Jack Ma and Elon Musk. The answers are based on transcripts of interviews with the subject. The application uses a Retrieval-Augmented Generation pattern running in Azure, using Azure AI Search for retrieval and OpenAI large language models. You can ask questions in the format of "What does Elon Musk think of space flight?"

Improvements made for LLM performance for Chinese

• The original text splitting parameters do not optimize chunking for CJK languages. We implemented a function to change the section length for chunking size depending on the language. The section length needed to be considerably smaller as Chinese text contains more information in a shorter amount of text.
• For the model to look up multiple languages more accurately, we implemented a translation function that translated the user prompt so the prompt is in both English and Chinese.
• Modified punctuation for detecting word breaks better by incorporating Chinese punctuation characters.

P.S. thanks pamelafox

P.P.S It could be the free tier of Azure deployments causes some delay when loading the endpoint. Please try again if it does not load for the first time.

Language

English, Chinese

Project Repository URL

https://github.com/AshleyZhao/ai-rag-entrepreneur-qa/tree/main

Deployed Endpoint URL

https://app-backend-edxc4c4wxgjxo.azurewebsites.net/

Project video

https://www.youtube.com/watch?v=02SXIUjNn6U

Team members

AshleyZhao, katieyang

Showcase Consent

Yes