azure / intro-to-intelligent-apps

This repository introduces and helps organizations get started with building Intelligent Apps and incorporating Large Language Models (LLMs) via AI Orchestration into them.

Introduction to Building AI Apps

This repository introduces and helps organizations get started with building AI Apps and incorporating Large Language Models (LLMs) into them.

Workshop Agenda

The objective of this workshop is to practice realistic AI orchestration scenarios and to learn how to build intelligent apps. At the end of the workshop you will:

• Know how to use prompt engineering techniques for effective generative AI responses on OpenAI
• Understand the implications of the usage of tokens and embeddings when interacting with an LLM
• Have experience in leveraging AI orchestrators like Langchain/ Semantic Kernel with Azure OpenAI
• Have evaluated different vector stores like Qdrant or Azure AI Search to enhance LLM responses with your data and context
• Know how to turn a business scenario with data, context and user input into an intelligent application on Azure

🌅 Morning (9:00 – 12:15)

Focus: Introduction, First Steps & Prompt Engineering

• 📣 Intro (30min)
  • Introductions & Setting Expectations
  • Use Case Ideation & Brainstorming
• 📣 Intro to Azure OpenAI, Prompt Engineering & Demos (105min)
  • Azure OpenAI Service
  • Demo(s)
  • Break
  • 🧑🏼‍💻 Lab #1 - Hands-on with Prompt Engineering Exercises
• 📣 Intro to AI Orchestration (60min)
  • AI Orchestration
  • Demo(s)

🌆 Afternoon (1:15 – 4:30)

Focus: Building AI Apps & Incorporating LLMs

• 📣 Intro to AI Orchestration Continued (135min)
  • 💻 Lab #2 - Hands-on with Integrating AI Exercises
  • 💻 Lab #3 - Hands-on with AI Orchestration Exercises
  • 💻 Lab #4 - Hands-on with Deploying AI Exercises
  • Break
• Wrapping-up (60min)
  • Use Case Validation
  • QnA & Closing Remarks

Getting Started with Workshop Preparation

The steps in this section will take you through setting up Azure OpenAI and some configuration files so that you can complete all of the hands-on labs successfully.

• Preparation

Post Workshop Next Steps

When you're done with this workshop and ready to move on, the following may be useful.

• Next Steps

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

intro-to-intelligent-apps's Issues

Fix chainlit

Wrap load_dotenv() into print

Validate that the env var loading works and the user sees that it worked

Wrong env variable name from the template

The env variable name in "labs/03-orchestration/01-Tokens/tokens.ipynb" is called as "AZURE_OPENAI_API_ENDPOINT" but in the .env template it is "AZURE_OPENAI_ENDPOINT".

I think this issue is also present in subsequent files under section 03.

Incorrect calculation of tokens

In the example, we are stating, that the text requires 6 tokens. In reality, it is 7, as the comma , is counted as a separate token.

intro-to-intelligent-apps/labs/03-orchestration/01-Tokens/tokens.ipynb

Line 18 in 97f8b30

" world,\n",

(Found by one of the participants!)

adopt function calling to new api

Update Semantic Kernel to latest version

Currently, the labs use the outdated version 0.17.230711.7-preview of the .NET Semantic Kernel. It should be updated to the latest version.

Prep guide

All required resources described:

• OpenAI (with the right model version)
• Codespaces or Local dev container
• AI Search (with admin key)
• MongoDB on CosmosDB

.NET interactive notebooks need .NET 7.0

Python notebooks work. But .NET ones now require .NET 7, when only .NET 6 is in the dev container.

I have suggested to change:

1. In docker-compose.yml line 13 to "VARIANT: 7.0.jammy"
2. In Dockerfile line 3 tp ARG VARIANT="7.0.jammy"

Works for me.

Would do a pull request if I knew how :-)

Add microsoft chat

https://github.com/microsoft/chat-copilot

Rename "Azure Cognitive Search" to "Azure AI Search".

There are few places, but in labs/03-orchestration/README.md we need to rename it.

Codespaces setup video

Create a short video to help participants get up and running with Codespaces prior to the event. This can be distributed a few days before the event to allow people to get Codespaces ready to go.

Make crear, that you can theoretically train your own LLM models

The text says, I can't bring my own data into an LLM because the models are pre-trained, so the only way to get more information in is to retrain the model, which is an expensive and time consuming process.

intro-to-intelligent-apps/labs/03-orchestration/02-Embeddings/embeddings.ipynb

Line 82 in cf155fc

"The thing is, you can't actually do that. The models are pre-trained, so the only way to get more information in is to retrain the model, which is an expensive and time consuming process.\n",

So this statement is contradictory. I can do it, it's just very expensive and resource intensive and time consuming. I suggest to adjust that to avoid confusion.

Clean up vector store lab

Merge 03 and 04
Clean up duplicates in Qdrant, Mongo and AI
Move up 05
Delete 06 lab

Tasks

Beta Give feedback

1. Clean up Qdrant vector store lab
2. Move lab 05 to 04
3. Delete lab 06
4. Move and cleanup AI Search Lab
5. Move and cleanup Mongo Lab

Update intro-to-intelligent-apps-workshop.pdf please!

Can you upadate https://github.com/Azure/intro-to-intelligent-apps/blob/main/presentations/intro-to-intelligent-apps-workshop.pdf please as promised in the OOP2024 WS?

Pre-requisites for local execution of labs

The codespaces / devcontainer setup should allow participants using that method to have a trouble-free experience. For those who wish to fork / clone the repo and run locally, need to provide a detailed list of pre-requisite components that need to be installed.

Azure AD authentication in labs

We looked at adding Azure AD support to the labs for customers that wish to secure access to Azure OpenAI using that method. Whilst possible, it did add a degree of extra unnecessary complexity to the labs. There were also potential issues when using Codespaces with getting AAD auth to work.

To simplify things, this workshop will expect API keys to be used to provide access to the Azure OpenAI service. However, it would be good to add an optional lab which shows how to use Azure AD based authentication, so that participants interested in this method of access control can see how it works.

improvement excercises in single jupyter notebook

Improvement:
Put the excercises into 1 jupyther notebook.
Removes repetitive starting of codespaces python kernels
In case of errors in the .env file you can restart the code in one jupyter file

Add details on preparing a .env file as a pre-requisite

For most deliveries of this workshop, customers will likely use a single instance of the Azure OpenAI service for all participants. The .env file should be preconfigured with the correct API key, endpoint etc. values prior to the event. It can then be distributed to participants, which will simplify the setup process and reduce the likelihood of participants not being able to connect to Azure OpenAI to run the labs.

add lab microsoft chat and cosmosdb

openai version is not getting updated to a new one 'for some reason'.

This wont allow Lab 02 - get available model deploys will fail due to inconsistency

Devcontainer on Mac with Apple Silicon fails due to no AARCH64 support in Azure Functions Core Tools

Running the dev container locally on a Mac with Apple Silicon will fail during the build of the container. The error will be similar to the following:

6.133 E: Unable to locate package azure-functions-core-tools
6.135 ERROR: Feature "Azure Functions Core Tools" (ghcr.io/
[2023-08-15T11:13:12.890Z] jlaundry/devcontainer-features/azure-functions-core-tools) failed to install! Look at the documentation at https://github.com/devcontainers/features/tree/main/src/azure-functions-core-tools for help troubleshooting this error.

This is because there is currently no official release of Azure Functions Core Tools that supports installation inside of a container when running on Apple Silicon:

jlaundry/devcontainer-features#7

Azure/azure-functions-core-tools#3112