Coder Social home page Coder Social logo

azure-extensions-cli's Introduction

azure-extensions-cli

⚠️⚠️ This tool is intended only for publishers of Azure VM extensions. If you are not an Azure employee or a whitelisted extension publisher, there is nothing here for you.

This command line interface is a simple wrapper on top of RDFE Extension Publishing APIs.

With this command-line interface you can:

  • Upload an extension .zip package as a new extension or a new version.
  • Promote versions to required rollout slices.
  • List all extension versions published.
  • Get replication status of an extension version.
  • Mark a version as internal and delete a version.

Usage

Make sure you have:

  • a subscription ID that has extension publishing enabled
  • and a .pem file used as the subscription management certificate

Instead of passing these arguments over and over to the CLI, you can simply set environment variables:

export SUBSCRIPTION_ID=xxxx-xxxxx-xxxxxx...
export SUBSCRIPTION_CERT=/path/to/cert.pem
export MANAGEMENT_URL=https://management.core.windows.net

Please use the following management URLs to cloud mappings:

If you are always operating on the same extension, you can also set:

export EXTENSION_NAMESPACE=Microsoft.Azure.Extensions
export EXTENSION_NAME=FooExtension

Then use help to explore the commands and arguments.

./azure-extensions-cli --help

NOTE: If you are not familiar with extension publishing process (i.e. slices, behaviors of extension pipeline) you should read the relevant documentation first.

CLI

This might not be up-to-date, but to give an idea, here are the subcommands

$./azure-extensions-cli
NAME:
   azure-extensions-cli - This tool is designed for Microsoft internal extension publishers
    to release, update and manage Virtual Machine extensions.

USAGE:
   azure-extensions-cli [global options] command [command options] [arguments...]

COMMANDS:
   new-extension-manifest   Creates an XML file used to publish or update extension.
   new-extension		    Creates a new type of extension, not for releasing new versions.
   new-extension-version    Publishes a new type of extension internally.
   promote                  Promote published internal extension to one or more PROD Locations.
   promote-all-regions      Promote published extension to all PROD Locations.
   list-versions		    Lists all published extension versions for subscription
   replication-status		Retrieves replication status for an uploaded extension package
   unpublish-version		Marks the specified version of the extension internal. Does not delete.
   delete-version		    Deletes the extension version. It should be unpublished first.
   help, h	                Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --help, -h		show help
   --version, -v	print the version 

Installing (or building from source)

You can head over to the Releases section to download a binary built for various platforms.

If you need to compile from the source code, make sure you have Go compiler 1.6+ installed. Check out the project, set the GOPATH environment variable correctly (if necessary) and run go build. This should compile a binary.

Overview

The CLI makes it easy (easier) to publish an Azure extension. An example workflow is provided below. This workflow assumes an extension type already exists, which is why the command new-extension-version is used. (If the type does not exist use substitute for the command new-extension instead.)

Not all command line parameters are shown for each command, only the salient options are shown.

Step 1 - create an extension manifest.

  1. ./azure-extensions-cli new-extension-manifest

Step 2 - publish an extension internally.

  1. ./azure-extensions-cli new-extension-version

Step 3 - rollout the extension to Azure, by slowly including more and more regions. It is recommended that you pause 24 hours between regions.

Every time a new region is added, the previous regions must be included with the promote command.

  1. ./azure-extensions-cli promote --region "West Central US"
  2. ./azure-extensions-cli promote --region "West Central US" --region "North Central US"
  3. ./azure-extensions-cli promote --region "West Central US" --region "North Central US" --region "West US"
  4. ./azure-extensions-cli promote ...

Step 4 - promote the extension to all Azure regions.

  1. ./azure-extensions-cli promote-all-regions

Regions

As of 13-Sept-2017 this is the list of regions available for the public cloud.

Service Management Resource Manager
Australia East australiaeast
Australia Southeast australiasoutheast
Brazil South brazilsouth
Canada Central canadacentral
Canada East canadaeast
Central India centralindia
Central US centralus
Central US EUAP centraluseuap
East Asia eastasia
East US eastus
East US 2 eastus2
East US 2 EUAP eastus2euap
Japan East japaneast
Japan West japanwest
Korea Central koreacentral
Korea South koreasouth
North Central US northcentralus
North Europe northeurope
South Central US southcentralus
South India southindia
Southeast Asia southeastasia
UK South uksouth
UK West ukwest
West Central US westcentra
West Europe westeurope
West India westindia
West US westus
West US 2 westus2

TODO

  • make replication-status exit with appropriate code if replication is not completed.
  • make replication-status --wait arg to poll until replication completes.
  • add replication-status --json flag to output for a programmable output.

License

See LICENSE.


This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

azure-extensions-cli's People

Contributors

ahmetb avatar boumenot avatar hglkrijger avatar microsoft-github-policy-service[bot] avatar psap avatar vrdmr avatar yantang-msft avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

azure-extensions-cli's Issues

Add Support for Key Vault

azure-extensions-cli should be able to fetch secrets from Key Vault. This would better support existing work flows.

Random crashes

Running into random crashes:

$ ./tools/azure-cli/azure-extensions-cli_darwin_amd64 replication-status --subscription-id XXXXXXXXXX --subscription-cert azure-management-cert.pem --namespace "XXXXXXX" --name "XXXXXXX" --version 2.0.1
failed MSpanList_Insert 0x8a8000 0x1c33b966f2cd 0x0 0x0
fatal error: MSpanList_Insert

runtime stack:
runtime.throw(0x5ac9d0, 0x10)
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/panic.go:530 +0x90 fp=0x7fff5fbff040 sp=0x7fff5fbff028
runtime.(*mSpanList).insert(0x7e8948, 0x8a8000)
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/mheap.go:933 +0x293 fp=0x7fff5fbff070 sp=0x7fff5fbff040
runtime.(*mheap).freeSpanLocked(0x7e8140, 0x8a8000, 0x100, 0x0)
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/mheap.go:809 +0x4be fp=0x7fff5fbff0d8 sp=0x7fff5fbff070
runtime.(*mheap).grow(0x7e8140, 0x8, 0x0)
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/mheap.go:675 +0x2a0 fp=0x7fff5fbff130 sp=0x7fff5fbff0d8
runtime.(*mheap).allocSpanLocked(0x7e8140, 0x1, 0x0)
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/mheap.go:553 +0x4e3 fp=0x7fff5fbff188 sp=0x7fff5fbff130
runtime.(*mheap).alloc_m(0x7e8140, 0x1, 0x15, 0x0)
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/mheap.go:437 +0x119 fp=0x7fff5fbff1b8 sp=0x7fff5fbff188
runtime.(*mheap).alloc.func1()
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/mheap.go:502 +0x41 fp=0x7fff5fbff1e8 sp=0x7fff5fbff1b8
runtime.systemstack(0x7fff5fbff208)
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/asm_amd64.s:307 +0xab fp=0x7fff5fbff1f0 sp=0x7fff5fbff1e8
runtime.(*mheap).alloc(0x7e8140, 0x1, 0x10000000015, 0x1eb0f)
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/mheap.go:503 +0x63 fp=0x7fff5fbff238 sp=0x7fff5fbff1f0
runtime.(*mcentral).grow(0x7e9d40, 0x0)
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/mcentral.go:209 +0x93 fp=0x7fff5fbff2a0 sp=0x7fff5fbff238
runtime.(*mcentral).cacheSpan(0x7e9d40, 0x7e2858)
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/mcentral.go:89 +0x47d fp=0x7fff5fbff2e0 sp=0x7fff5fbff2a0
runtime.(*mcache).refill(0x8a4000, 0x15, 0x7fff5fbff348)
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/mcache.go:119 +0xcc fp=0x7fff5fbff318 sp=0x7fff5fbff2e0
runtime.mallocgc.func2()
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/malloc.go:642 +0x2b fp=0x7fff5fbff338 sp=0x7fff5fbff318
runtime.systemstack(0x7fff5fbff3d8)
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/asm_amd64.s:307 +0xab fp=0x7fff5fbff340 sp=0x7fff5fbff338
runtime.mallocgc(0x180, 0x554a80, 0x0, 0x800000000)
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/malloc.go:643 +0x869 fp=0x7fff5fbff418 sp=0x7fff5fbff340
runtime.newobject(0x554a80, 0x7e2e10)
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/malloc.go:781 +0x42 fp=0x7fff5fbff440 sp=0x7fff5fbff418
runtime.malg(0x8000, 0x7e31c0)
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/proc.go:2634 +0x27 fp=0x7fff5fbff478 sp=0x7fff5fbff440
runtime.mpreinit(0x7e39c0)
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/os1_darwin.go:140 +0x1f fp=0x7fff5fbff490 sp=0x7fff5fbff478
runtime.mcommoninit(0x7e39c0)
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/proc.go:494 +0x105 fp=0x7fff5fbff4d8 sp=0x7fff5fbff490
runtime.schedinit()
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/proc.go:434 +0x79 fp=0x7fff5fbff520 sp=0x7fff5fbff4d8
runtime.rt0_go(0x7fff5fbff550, 0xc, 0x7fff5fbff550, 0x0, 0xc, 0x7fff5fbff768, 0x7fff5fbff79c, 0x7fff5fbff7af, 0x7fff5fbff7c1, 0x7fff5fbff7e6, ...)
	/home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/asm_amd64.s:138 +0x132 fp=0x7fff5fbff528 sp=0x7fff5fbff520

If I re-run it, it succeeds.

Stuck on error: "Another operation on extension with namespace XXX, type XXX is already in progress."

Hi,

When I tried to upload a new version of my extension I got a 404:

<Error xmlns="http://schemas.microsoft.com/windowsazure" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<Code>ConflictError</Code>
<Message>Another operation on extension with namespace XXX, type XXX is already in progress.</Message>
</Error>

Or if I check my current operation calling: "https://management.core.windows.net/<SUBSCRIPTION_ID>/operations?StartTime=2017-08-01&EndTime=2017-12-01", all my request are liste as Failed or Succeeded (no InProgress) and all have an OperationCompletedTime.

How can I leave this state ?

Thanks.

new-extension-manifest only supports classic storage accounts

Creating a new extension manifest fails if one doesn't use a 'classic' storage-account.

ARM storage accounts just don't seem to work. And it fails with a kinda non-descriptive error message:
FATA[0001] Could not fetch keys for storage account. Make sure it is in publisher subscription. Error: Error response from Azure. Code: ResourceNotFound, Message: The storage account '<redacted>' was not found.

Would be great if ARM accounts would be supported too. And at least the error message should get updated.

new-extension-manifest does not generates a proper XML output

While trying to execute the following command with CLI Version 1.2.4, we do get a proper expected XML output. But it doesn't works well with later versions.

Command

azure-extensions-cli_linux_amd64 new-extension-manifest \
--management-url URL \
--subscription-id ID \
--subscription-cert Cert \
--package Package \
--storage-base-url core.windows.net \
--storage-account Account \
--namespace Namespace \
--name Name \
--version Version \
--label Custom-Label \
--description Custom-Desc \
--eula-url EULA-URL \
--privacy-url Privacy-URL \
--homepage-url Homepage \
--company My-Company \
--supported-os windows

Output with CLI v1.2.7 & 1.2.6:

<ExtensionImage xmlns="">
  <ProviderNameSpace>Namespace</ProviderNameSpace>
  <Type>Name</Type>
  <Version>Version</Version>
  <Label>label</Label>
  <HostingResources></HostingResources>
  <MediaLink>ZIP</MediaLink>
  <Endpoints></Endpoints>
  <Description>description</Description>
  <LocalResources></LocalResources>
  <IsInternalExtension>true</IsInternalExtension>
  <Eula>eula-url</Eula>
  <PrivacyUri>privacy-url</PrivacyUri>
  <HomepageUri>homepage-url</HomepageUri>
  <IsJsonExtension>true</IsJsonExtension>
  <CompanyName>company</CompanyName>
  <SupportedOS>supported-os</SupportedOS>
</ExtensionImage>

Expected Output (Working till v1.2.4)

<?xml version="1.0" encoding="utf-8" ?>
<ExtensionImage xmlns="http://schemas.microsoft.com/windowsazure"  xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
  <!-- WARNING: Ordering of fields matter in this file. -->
  <ProviderNameSpace>Namespace</ProviderNameSpace>
  <Type>Name</Type>
  <Version>Version</Version>
  <Label>Custom-Label</Label>
  <HostingResources>VmRole</HostingResources>
  <MediaLink>Zip</MediaLink>
  <Description>Custom-Desc</Description>
  <IsInternalExtension>true</IsInternalExtension>
  <Eula>EULA-URL</Eula>
  <PrivacyUri>Privacy-URL</PrivacyUri>
  <HomepageUri>Homepage</HomepageUri>
  <IsJsonExtension>true</IsJsonExtension>
  <CompanyName>My-Company</CompanyName>
  <SupportedOS>windows</SupportedOS>
  <!--%REGIONS%-->
</ExtensionImage>

What we see here is, only few user inputs are considered. We don't see any changes in help topics of new-extension-manifest. Is there anything we need to take care of or this PR might need to be revisited

Normalize Region Input

The API is very unforgiving of how regions are specified. Users should not be forced to handle "US Central" vs "USCentral" vs "uscentral" vs "Central US" etc. The tool should just take care of this.

Latest code build fails

Tried to build latest code today, it fails -

azure-extensions-cli]# go build

_/home/pkhare/azure/new_azure_extension_cli/src/azure-extensions-cli

./publish.go:112: bs.CreateContainerIfNotExists undefined (type storage.BlobStorageClient has no field or method CreateContainerIfNotExists)
./publish.go:116: bs.CreateBlockBlobFromReader undefined (type storage.BlobStorageClient has no field or method CreateBlockBlobFromReader)
./publish.go:119: bs.GetBlobURL undefined (type storage.BlobStorageClient has no field or method GetBlobURL)

cannot find package "github.com/Azure/azure-sdk-for-go/management"

Hey,
it seems that the part that imports "github.com/Azure/azure-sdk-for-go/management"
can not find this directory, if you will go to the repo - https://github.com/Azure/azure-sdk-for-go , you can see that there is no /management dir.
i'm getting this -
main.go:9:2: cannot find package "github.com/Azure/azure-sdk-for-go/management" in any of:
and when i try to "go get github.com/Azure/azure-sdk-for-go/management"
i'm getting the same issue.
Thanks.

promote-all-regions not working

Thanks for the great tool, and sorry if I'm not understanding this usage, but my promote-all-regions usage is failing. Looking at the code, I wonder if line 31 in promote.go needs to be changed to use checkFlag(c, regions) or something like that, instead of just regions?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.