Cannot delete snapshots - Authentication Error about azure-storage-python

[emgerner-msft]

I'm guessing that the auth error is because you're sending an empty snapshot which is not valid. In future versions this will be handled better, but at the moment if you don't send snapshot unless it's actually populated I think you'll see results that make more sense.

Another issue with this code is that list_blobs does not follow continuation tokens so you may be missing pages of results. This is again something that will be made automatic in newer versions, but for the moment you'll need to implement that code and pass the marker parameter.

include='snapshots' is the correct way to specify including snapshots, so if you've tried the two above steps and still are not seeing expected results, please update your code and we can keep debugging. That being said, we do have a test for this functionality. Note this test does not show how to follow continuation tokens, though there is a test nearby that does if you want to see.

[ahirner]

Thanks for the well laid out suggestion. I amended the code to work with continuation tokens.

blob_service = BlobService(AZURE_STORAGE_ACCOUNT, AZURE_STORAGE_ACCESS_KEY)
blobs = blob_service.list_blobs(CONTAINER, include='snapshots')
while True:
    for b in blobs: 
        print b.name + " | " + b.snapshot
    if not blobs.next_marker: break
    blobs = blob_service.list_blobs(CONTAINER, None, blobs.next_marker, include='snapshots')

None of the results yield a snapshot, meaning all snapshot fields are zero length strings. This either means there is a bug in the API retrieving snapshots or a bug when creating snapshots from Azure portal in that it doesn't tag them as snapshots. I've created the latest on a few days ago.

What other possibilities are there to get rid of orphan snapshots in a timely manner?

[emgerner-msft]

What other possibilities are there to get rid of orphan snapshots in a timely manner?

Maybe this is the confusion? You cannot delete a blob without deleting its snapshots. If you check out the delete_blob API, if the blob has snapshots you must specify the x_ms_delete_snapshots option as include or only, meaning delete the blob and all of its snapshots or delete just the snapshots. You can also delete a specific snapshot by specifying the 'snapshot' parameter. However, note that you cannot delete just the base blob and not it's snapshots. Thus, no orphaned snapshots.

Is above the confusion? If not, can you include more info on exactly how you're creating a blob and snapshots?

[ahirner]

Thx for following up!
I try to delete snapshots only as outlined in the original issue. However, the snapshot parameter set to a zero length string because this is what the API returns. This led me to the conclusion it is either a bug while retrieving snapshot names/timestamps or during creation. I created them via the UI in the Azure portal by clicking "capture VM".
Oh and sorry, I actually tried it with x_ms_delete_snapshots="only" or "include" to no avail.
blob_service.delete_blob('vhds', blobs[1].name, snapshot=blobs[1].snapshot, x_ms_delete_snapshots="only")

[emgerner-msft]

Ah, we got misdirected with the whole snapshot/auth error thing in the first place. Per this blog the format of VM names and their captures is as follows:
"Once you have a virtual machine set up and configured as you want, you can capture the instance as a VM Image. During the capture process, all relevant properties of the virtual machine and disks are stored and copies of the backing VHD page blob(s) are made. The copy of each VHD is stored in the same storage account and container as the original VHD being copied. If you are interested in finding the copied VHDs in Azure Storage, search for page blobs with the following naming convention: for the OS VHD, we use -os-YYYY-MM-DD<-ZZ> and for the copied data VHDs, we use -datadisk--YYYY-MM-DD(-ZZ). The date is when the VM Image was captured and the -ZZ is a number, added only if there is a collision, to make the name unique."

Is this what you see?

[ahirner]

Partially. All the blobs with the snapshots in them are on the same VHD storage. So this expected behavior. However, querying with the API returns an empty string. To infer a string manually is unclear because one blob's name is for example 'backup1-os-2015-12-28.vhd' with a last modified date of '12/27/2015 6:22 PM'. Since per the blog post "the -ZZ is a number, added only if there is a collision" and "for the OS VHD, we use -os-YYYY-MM-DD<-ZZ>", I was tempted to delete the snapshot with two string identifiers:

blob_service.delete_blob('vhds', blobs[1].name, snapshot="backup1-os-2015-12-28.vhd", 
x_ms_delete_snapshots="only")
blob_service.delete_blob('vhds', blobs[1].name, snapshot="backup1-os-2015-12-27.vhd", x_ms_delete_snapshots="only")

Both of them threw this error:

AzureHttpError: Value for one of the query parameters specified in the request URI is invalid.
<?xml version="1.0" encoding="utf-8"?>
<Error><Code>InvalidQueryParameterValue</Code><Message>Value for one of the query parameters specified in the request URI is invalid.
RequestId:
Time:2016-02-10T00:28:53.4610242Z</Message><QueryParameterName>snapshot</QueryParameterName><QueryParameterValue>backup1-os-2015-12-27.vhd</QueryParameterValue></Error>

It seems to me, that snapshot identifiers are buried somewhere in the system while the API doesn't return them or they weren't stored at the time of creation.

[emgerner-msft]

VM captures are not snapshots. :) Per the blog, the blob name has the date in it. So, that you're not getting snapshots back from list_blobs and that deleting them isn't working makes perfect sense. Check the output of list and the naming format and you should see that many of the blobs have the capture format the blog lists. Then you can delete them without any of the snapshot tooling.

[emgerner-msft]

Is this working for you?

[ahirner]

Thank you for follow-up. Indeed all of my blobs I wanted to delete were non-snapshot blobs how the API sees them (despite ambiguity also in the blog post). However, none of them could be selectively deleted from vhds containers on a running and even stopped machine due to lease locks on every blob in the vhds (not only the currently used ones). I think this inconvenience is somewhat common. I concluded to deprovision and tear the machine down was fastest method going forward.

On 23.02.2016, at 14:06, Emily Gerner [email protected] wrote:

Is this working for you?

—
Reply to this email directly or view it on GitHub #112 (comment).

[emgerner-msft]

Okay, sounds like you figured it out then. I'll close the issue.

FYI: If they're storage leases and you absolutely need to break them manually you can use the break_blob_lease method (in 0.30.0) or in older versions, lease_blob with the lease action set to break. That being said, your method is probably safer and more recommended, especially on active machines.