azure / azure-dev Goto Github PK

Update CI/CD to support multiple environments and establish baseline for application environment promotion.

Dev > Test > Staging > Production

What needs to happen in between?

• Integration tests?
• Manual approval gates?
• Canary or blue/green rollout?

A user can scaffold out an application via templates.

Right now that list is hard-coded.

We need to be able to query and API to get the index of templates.

Cache that locally.

i.e. user executes

az dev init -t foo -l csharp

That code will try to find that template.

1. search local cache if it exists
2. query an API to try to find it
3. if not found, let them know.

We also need to inform them when a template has a new version and allow them to update the template index.

az dev templates update - to update the index
az dev templates list - to show all the templates available

I don't want to standup a service to host this list, so I would like to keep this index on GitHub.

Message drive applications are one of the key use cases for Azure. We need to create a simple application that leverages best practices for message-based apps.

When deploying to Azure App Service, you can decide if you would like Azure App Service to build your application (in which case the zip package you upload contains the source code for your application and App Service runs language specific actions like npm install and npm build or pip install) or a prebuilt package that Azure App Service just unpacks and runs directly.

Today, we don't allow the user to configure this behavior and the tool does a source deploy for python packages and a binary deploy for node and dotnet packages.

We should figure out a way to make this customizable.

Jon and I were discussing this over teams recently:

[Yesterday 5:32 PM] Matt Ellis
Was there a reason that you explicitly call npm install as part of deploy for node apps but you don't run something like pip install for python?

[Yesterday 7:01 PM] Jon Gallant
b/c i'm using the scm features

[Yesterday 7:01 PM] Jon Gallant
pip install is called by app service on deploy

[Yesterday 7:34 PM] Jon Gallant
for react apps we need to call npm install because we need to build it to produce the assets we are going to zipto build those assets we need to install 'react-scripts'we do not use scm on deploy for react

[Yesterday 7:34 PM] Jon Gallant
for python we use scm on deploy and don't need to build anything so pip install not required

[Yesterday 7:35 PM] Jon Gallant
we'll likely need to think about when we call install versus not, i.e. we probably don't need to call npm install when we have a nodejs app that will use scm on deploy

One possible design is the tool could just inspect the properties of the target app service it is deploying to and see if it expects to do SCM deploys or not and package the project accordingly.

Our Python POC supports many env var overrides, so that we are opinionated but flexible/overrideable.

Please have a look at the env vars here and ensure we support them as overrides. https://github.com/Azure/azure-dev-cli-extension#environment-variables

Make it easy to deploy your apps out to multiple regions.

Consider concept up bundling up your IaC into a scale unit with multiple deployments

• ex) East US, West US
• ex) Multiple instances in high demand regions
• Automatic single endpoint with Azure front door / traffic manager or similar

Identify how to handle dependencies between projects.

Standard use case

• Solution with 2 apps, frontend + backend.
• Need to deploy backend changes before frontend, etc.

We are looking to bridge the gap between an ML developer building an algo and model and implementing that in an application.

We should investigate use of the ez CLI.

Once azd up/down/provision/deploy are completed, print out how long the operation took.

It is both an anti-pattern, and unnecessary application code complication.

Instead applications should leverage AppService and KeyVault integration

azd generate

Generates the IaC code ARM templates from bicep, k8s manifest from Tye, so the user can save it in GitOps repo or inspect it as needed.

Create a base policy set that defines the best practices for a given resource type, e.g. encryption at rest. Apply these when the resources are deployed.

Use Azure Policy to ensure deployments comply with company standards
Overview of Azure Policy - Azure Policy | Microsoft Docs
https://docs.microsoft.com/en-us/azure/governance/policy/overview

• Prevent deploying resources that do not comply with company security policy (e.g. VMs with RDP or SSH enabled ports)
• prevent deploying resources in 1 resource group to multiple regions or to business approved regions (e.g. Azure Gov)

Similar to Create React App a customer may at some point in time want to take more control over the e2e process. At this time we should make it easy to eject the application into a set of scripts that can then be fully edited and maintained by the developers.

Similar to https://create-react-app.dev/docs/available-scripts#npm-run-eject

I've noticed that I can run the CLI on WSL, so I setup my Windows side to support everything and I immediately noticed two things:

• My API build failed because of CRLF
• The build and deploy took a while compared to running the build in Oryx

This appears to be a conscious decision since SCM_DO_BUILD_DURING_DEPLOY is disabled. We've spent a lot of effort optimizing Oryx for node/python builds and that should be taken advantage of here as well.

JavaScript, Python, and static web apps, have many popular frameworks, e.g. Express, FastAPI, Hugo, that help developers get started building applications. Each of the frameworks has their own distinct way of configuring, building, and deploying their solutions. In many scenarios, these frameworks are the starting point for developers. We will need to determine a subset of language frameworks to "devify" as way to simplify deploying these solutions on Azure.

For Phase 1, we'll need to select one or two language frameworks for each language. Within these, we will look at the popular example applications and select one or two of these examples. In addition, we'll also do the same with static web apps, e.g. Gatsby, Hugo. Although these are JavaScript, they are not the same kind of thing as an Express or FastAPI app.

Phase 1

Fork & devify existing language framework solution
Azure/azure-dev-pr#505, Azure/azure-dev-pr#506, & Azure/azure-dev-pr#508

Phase 2

Track & manage changed to the community framework solutions
(link to epic)

Phase 3

Automatically "devify" a solution based on a supported template
(link to epic)

Once azd provision completes, we write any outputs of the deployment into the .env file (and in the case of dotnet projects, we also run dotnet user-secrets add to add them to the user secrets store). However, we don't ever remove anything from this list, so if you remove an output from your infrastructure template, we'll continue to have the old value from the last deployment in the .env file.

We should be cleaning these up, I think. We can simply record the values from the last deployment in a side file and consult that after the current deployment to see outputs which were removed, and remove them from the .env file.

The template readmes are very similar across projects. Let's figure out a way to reuse content and make them dynamic. For example, on azd init -t {{ template-name }}

And reuse code from a shared library of MD files.

Have dynamic pre-reqs.

Would could potentially use this: https://github.com/marketplace/actions/dynamic-readme

Sections for reuse

• known issues
• reporting issues / feedback

During the provision flow, we look at the template we are going to deploy and ensure all the required parameters are set. If some are missing, we prompt the user for a value and then save that value both to the .parameters.json file in the working directory we'll pass az deployment ... and the .env file for future use.

However, the .parameters.json file in the working directory will be replaced on the next deployment (by taking the template in the iac folder and doing environment substitutions on it, so on the next call to provision we will prompt for all these same values.

Perhaps we should modify the root parameters file to add a parameter who's value is just something like ${PARAMETER_NAME_TO_UPPER_CASE}? But this would be our first case of the tool adding something to one of these files and it feels a little weird.

We could also become more intelligent as to how often we update the .parameters.json file in the working directory, perhaps replacing it only when the source version changes.

Some services (like Azure SQL) are not exposed to the internet by default. When developing an application that uses Azure SQL a firewall rule needs to be manually added to allow remote access.

It would be nice if azd was able to simplify this scenario somehow. One way might be to have a well-known parameter name DEV_EXTERNAL_IP and use it to conditionally populate the firewall rules in local dev scenarios.

Describe the bug
Expect to be able to rerun command and should skip provision if resources already provisioned

Additional context
Feedback by Cody

See the discussion in #542 (comment). When we do this, we should go fix the glob back up and also ensure that the context menu commands show up for azure.yml in addition to azure.yaml.

The command will be something like:

azd pipeline config --auth-type oidc

Please see this implementation to see everything we need to script to get OIDC setup:
https://github.com/jongio/github-azure-oidc/blob/main/oidc.sh

We'll want to include a fics.json file to help the devs understand what they are doing and is a good way to visualize what the FICs look like in JSON format.
https://github.com/jongio/github-azure-oidc/blob/main/fics.json

Here's a video that walks through all the concepts involved with OIDC: https://youtu.be/r5QdsjjdRDs

We have a dependency on the Related: Azure/azure-cli#20582

But we can call the graph REST API directly as I do in the above script until that is implemented.

We will use the azure/login action to support this, so the workflow included with the template will need to be updated to support OIDC or AZURE_CREDENTIALS. We might need to have different workflow files based on the authtype.

During the code review of the monitor spike we started to discuss cases where we have to do multiple azure operations which could happen in parallel (for example, listing resources in an application that contains multiple resource groups means we'll have to call a single list call per resource group) but are done today serially.

We decided to defer this work for now, since in the common case we don't have a bunch of resource groups and hence the ability to exploit parallelism but this would be something that should be helpful which improves performance. This issue tracks that work.

See #391 (review) for more discussion.

Leverage the VS Code - CodeTour to orient developers to the template when they are using VS Code.

Today provision only supports subscription level deployments. Our templates are structured such that at the subscription level we create a resource group (based on the BASENAME of the project) and the template does a nested deployment into that resource group. This is nice because it's one less piece of setup the user has to do to get started.

However, this requires the end user has permissions to do subscription level deployments, which may not be common in larger organizations, where a central team may provision resource groups for individual teams use and grant permissions to users scoped to a specific resource group. As part of our customer development, we should try to understand how arduous this requirement is.

I suspect we are going to end up in a place where we will also want to support deployments which are targeted at a specific resource group. There's a lot to think about what that means, here's some quick thoughts off the top of my head as a jumping off point to think about:

1. How do we detect if the user wants a subscription or resource level deployment (it feels like we should be able to infer this from the compiled arm template we are going to deploy, since it contains information about the deployment scope).
2. Where do we store information about the target resource group, for cases where a deployment targets a specific resource group.
3. What does deprovision mean in such a world? It seems like deleting the resource group there is a bad idea. Should deprovision walk over the resource group and just remove everything in it? Should it instead just remove the resources that were listed as outputs from the most recent deployment?

Today we only support zip deployment. Let's also support git deployment.

Let's ensure that azd monitor works for all supported hosts and dev environments. Does it launch the browser, and if not, does it fail gracefully?

Comment from @ellismg.

When I was writing the automated tests for the function deployment I noticed there was a delay between the provision and being able to deploy to the function.

This isn't great (the check code itself and the need for the check) so we should take a look and see if we can improve this.

If anything it'd be nice to use an API to do the check, rather than just continually launching and failing in the CLI since we don't differentiate failures there so we could be retrying on something very silly.

Originally posted by @ellismg in #460 (comment)

Currently the init command does two things: creates the azure directory with default settings as well as handling the template logic, which includes:

• evaluating the passed in name
• directory check (if empty)
• fetching and unzipping

I propose we reduce the complexity of init and move the template logic to a new command: generate. This follows what other frameworks do (namely Rails and Express among others). With a dedicated generate command, we could build something more interactive and friendly, including showing the end user what's available for each language/template.

Describe the bug

Certain services, such as Azure Static Web Apps, are only available in a subset of regions. When using azd provision the full list of Azure regions is displayed.

To Reproduce

1. Create a template which uses a service available in a subset of regions
2. Run azd provision
3. The full list of regions is displayed

Expected behavior

When running azd provision, only the valid regions should display.

Environment

N/A

Additional context

I'd add it might be worthwhile considering having a reduced list of regions regardless of the service. The list of available regions is quite long, and can require a fair bit of scrolling to find the right one.

Easy way to define a resource tagging strategy outside of direct ARM/bicep/terraform updates.

This scenario should leverage docker-compose and container for the solution

Describe the bug
This is not a problem with Dev CLI - rather I want to bring awareness so Dev CLI can improve the current experience.

When deploying a basic Python app (Django or Flask) via Azure CLI or VS Code, it can be hard to track down application response logs that show 2xx and 4xx responses.

Expected behavior
When a new app is deployed, I expect the app template to have any required code, and any necessary logging to be enabled so I can quickly see 2xx and 4xx responses via CLI, VS Code or Portal.

Environment
Information on your environment:
* Python 3.8+
* VS Code (current version)

As a developer, it should be clear in the code structure how/where I can add my own business logic. Interfaces between the layers should be well defined and documented so that it is easy to extend or completely replace the sample code and reuse all the infrastructure that has been configured.

Some devs won't have docker desktop, so let's figure out an alternative recommendation.

We have a Java API working in Azure/azure-dev-pr#459 and need to update the azdev CLI to support packaging and deployment.

We have references here:
https://github.com/microsoft/nubesgen-actions/blob/main/gitops-build-java-maven/action.yml
https://github.com/microsoft/nubesgen-actions/blob/main/gitops-deploy-to-app-service-java/action.yml

And @jdubois Can help with any questions.

A user has landed on the samples browser and they want to know which samples are "blue badge", are being tested on a regular basis, and have a high chance of success for me to run and deploy.

Pluggable support for legacy customer to continue to use their ARM templates vs forcing them to upgrade to bicep.

Pluggable support to allow developer to choose Terraform if they want vs. bicep

Adds support for Terraform in AZD CLI and templates

AZD CLI Requirements

• #227
• #234

Template Requirements

• #231
• #232

Add a Java version of the ToDo API

Figure out how to support shared infrastructure.

ex) Multiple apps with shared networking infra (vnets, firewall, etc)

Ideas:

• Infra only project that lays down foundational components
• Layer on top app specific concerns
• Stitch together via project dependencies

• #398
• #287
• #425

Bicep outputs should be stored as dotnet user secrets for dotnet projects. That way a dotnet developer can reference secrets in their idiomatic way and not have to use a dotenv lib to get to the values needed to run their application.

See this for reference: https://github.com/Azure/azure-dev-cli-extension/blob/main/azext_dev/__init__.py#L838