apache / cloudstack-go

CloudStack Go SDK

cloudstack-go

A CloudStack API client enabling Go programs to interact with CloudStack in a simple and uniform way

Status

This package covers the complete CloudStack API and is well tested. There may be some untested corner cases as there are over 600 API commands, but over all, it's safe to use this package.

To be able to find the API command you want, they are grouped by 'services' which match the grouping you can see/find on the CloudStack API docs website.

Usage

The cloudstack package is always generated against the latest stable CloudStack release (currently v4.18.x). As long as the API changes were minimum across subsequent CloudStack releases it was possible to have the generator code handle API changes such that they were backward compatible. However, over time, with frequent API changes observed across releases of Apache CloudStack(ACS), we will have the SDK releases tagged based on the ACS version.

Please see the package documentation on go.dev.

Example

// Create a new API client
cs := cloudstack.NewAsyncClient("https://cloudstack.company.com", "your-api-key", "your-api-secret", false)

// Create a new parameter struct
p := cs.VirtualMachine.NewDeployVirtualMachineParams("service-offering-id", "template-id", "zone-id")

// Set the name
name := "server-1"
p.SetName(name)

// Set the display name
p.SetDisplayname("Test server 1")

// Set any other options required by your setup in the same way

// Create the new instance
r, err := cs.VirtualMachine.DeployVirtualMachine(p)
if err != nil {
	log.Fatalf("Error creating the new instance %s: %s", name, err)
}

fmt.Printf("UUID or the newly created machine: %s", r.Id)

Features

Apart from the API commands CloudStack itself offers, there are a few additional features/functions that are helpful. For starters, there are two clients, a normal one (created with NewClient(...)) and an async client (created with NewAsyncClient(...)). The async client has a builtin waiting/polling feature that waits for a configured amount of time (defaults to 300 seconds) on running async jobs. This is very helpful if you do not want to continue with your program execution until the async job is done.

There is also a function that can be called manually (GetAsyncJobResult(...)) that does the same, but then as a separate call after the async job has started.

Another nice feature is the fact that for every API command you can create the needed parameter struct using a New...Params function, like for example NewListTemplatesParams. The advantage of using this functions to create a new parameter struct, is that these functions know what the required parameters are for every API command, and they require you to supply these when creating the new struct. Every additional parameter can be set after creating the struct by using the appropriate setters, e.g., SetName().

Last but not the least, there are a lot of helper functions that will try to automatically find a UUID for you for various resources (disk, template, virtualmachine, network...). This makes it much easier and faster to work with the API commands and in most cases you can just use then if you know the name instead of the UUID.

Developer Guide

The SDK relies on the generate.go script to auto generate the code for all the supported APIs listed in the listApis.json file. The listAPIs.json file holds the output of listApis command for a specific release of CloudStack.

# Run it via the Makefile
make all

Getting Help

Please try to see if the module documentation can provide some answers first!

• If you have an issue: report it on the issue tracker

History

Sander van Harmelen ([email protected]) was the original author of this repo which was donated to the Apache CloudStack project under an IP clearance process.

License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

cloudstack-go's Issues

Issue with ListVirtualMachines

Hi, i tring to get all VMs which belongs to user with this function
ListVirtualMachines

Here is my code:
con := cloudstack.InitCloudStack()
p3 := con.VirtualMachine.NewListVirtualMachinesParams()
//p3.SetAccount("testuser")
p3.SetListall(true)
ret, err := con.VirtualMachine.ListVirtualMachines(p3)
fmt.Println(ret)
if err != nil {
fmt.Println(err.Error())
}
After execution this code I got this error:
json: cannot unmarshal string into Go struct field VirtualMachine.virtualmachine.ostypeid of type int64

How to fix or can you please tell proper way to get all VMs which belongs to user

Thx

DomainService Fails to Unmarshal DomainChildren

Hello,

In using CS-Go, I noticed that listDomainChildren fails to unmarshal DomainChildren.

I modified the JSON tag and it worked perfectly.

    DomainChildren []*DomainChildren `json:"domain"`

I don't know if this is an issue across all versions of the CloudStack API. I personally work against 4.16.

I'd be happy to PR a change, but again I'm not sure if it would be backward compatible with older API versions.

HostService response types are broken

When running against Cloudstack 4.15 with the latest code, the cpuloadaverage and hostha fields in the response types within HostService have the wrong types causing the client to fail at parsing responses from the Cloudstack API.

The cpuloadaverage field defaults to string but needs to be a float64 (missing handling of double in the mapType function of generate.go).

The hostha field defaults to string but needs to be a custom HostHAResponse type.

The cpuloadaverage is easy to fix, but I don't immediately understand how the custom field type generation works in order to implement a fix for hostha.

createUser/updateUser APIs need to be POST

cloudstack-go/generate/generate.go

Line 1701 in 0d335ef

if a.Name == "deployVirtualMachine" || a.Name == "login" || a.Name == "updateVirtualMachine" {

The generation code uses post calls for a subset of APIs that are sensitive or large. Eventually the listApis should advertise which APIs these are, but until then we need to switch these createUser and updateUser calls (and any other that may send sensitive info) over to use POST.

Decouple test configuration and enable running tests against a real backend

Currently, in the test set the test configuration is (partly?) hardcoded using endpoint mocking approach with httptest:

func TestVirtualMachineService_DeployVirtualMachine(t *testing.T) {
	server := httptest.NewServer( ...

It would be great to have some decoupling here to provision endpoint configurations in a flexible way, also enabling automated testing against (multiple) real endpoints.

bug: async client don't support timeout

I'm trying to set an async timeout of 1 sec with the following code:

func MakeClient() *cloudstack.CloudStackClient {

       acsTimeout := 1 // sec

	cs := cloudstack.NewAsyncClient(apiURL, apiKey, secretKey, false)
	cs.AsyncTimeout(acsTimeout)

	return cs
}

However, when I run a test on the real infra, I'm getting this:

=== RUN   TestVPC_create
INFO[2022-05-25T18:36:01+03:00] Success                                      
--- PASS: TestVPC_create (3.38s)

So this test took 3.38 sec without being timed out by the ACS client. I also look 👀 into the impl and found out it to be rather poor.

Question: could anyone confirm the valid functionality of async client? Otherwise, I'm taking a lead to update the async part to match the expected functionality 🙄

CreateZoneResponse empty

CreateZone always returns an empty response. The struct should be nested with the "zone" key.

// Create zone
r, err := cs.Zone.CreateZone(p)

Example response from API

{
  "zone": {
    "id": "6a36f514-e6ea-4a16-85ff-747b063a156e",
    "name": "test",
    "dns1": "xx.xx.xx.xx",
    "dns2": "xx.xx.xx.xx",
    "ip6dns1": "",
    "ip6dns2": "",
    "internaldns1": "1xx.xx.xx.xx",
    "internaldns2": "xx.xx.xx.xx",
    "domain": "foo.example.com",
    "networktype": "Advanced",
    "securitygroupsenabled": false,
    "allocationstate": "Disabled",
    "zonetoken": "d9609197-6586-3fe3-8c4f-efa15a375e8f",
    "dhcpprovider": "VirtualRouter",
    "localstorageenabled": false,
    "tags": [],
    "hasannotations": false
  }
}

Generate and tag against 4.15.2

Hi @pdion891 I've updated the listApis.json using 4.15.2.0 with d6d4529. On running the generator it creates changes in cloudstack/HostService.go that overrides some of your manual changes:

cd generate
go build
./generate -api listApis.json

Can you do the above and see if you can address/fix your manual changes so either the generator is fixed or you simply use strings? (cloudstack/HostService.go)

GetKubernetesClusterConfigResponse is incorrect resulting in empty parsing

type GetKubernetesClusterConfigResponse struct {
	Configdata string `json:"configdata"`
	Id         string `json:"id"`
	JobID      string `json:"jobid"`
	Jobstatus  int    `json:"jobstatus"`
	Name       string `json:"name"`
}

Should be:

type GetKubernetesClusterConfigResponse struct {
	ClusterConfig struct {
		Configdata string `json:"configdata"`
		Id         string `json:"id"`
		JobID      string `json:"jobid"`
		Jobstatus  int    `json:"jobstatus"`
		Name       string `json:"name"`
	} `json:"clusterconfig"`
}

Since the response is this (showing only the start since that is the only relevant part):

CloudStack version: 4.18

CI/CD support

Discussion

It's a good time to add a CI/CD to this project. Which one to pick? Our options are

• gitlab
• github actions
• circle-ci
• travis
• teamcity
• etc

Should we vote before picking up one?

Missing API support

Here's a list of the APIs in 4.16 not yet added to the SDK

acquirePodIpAddress
addAnnotation
addCiscoAsa1000vResource
addCiscoVnmcResource
addExternalFirewall
addExternalLoadBalancer
addF5LoadBalancer
addKubernetesSupportedVersion
addSrxFirewall
addUserToProject
addVmwareDc
archiveSnapshot
cancelHostAsDegraded
cloudianIsEnabled
configureF5LoadBalancer
configureSrxFirewall
createKubernetesCluster
createManagementNetworkIpRange
createProjectRole
createSnapshotFromVMSnapshot
declareHostAsDegraded
deleteCiscoAsa1000vResource
deleteCiscoNexusVSM
deleteCiscoVnmcResource
deleteExternalFirewall
deleteExternalLoadBalancer
deleteF5LoadBalancer
deleteKubernetesCluster
deleteKubernetesSupportedVersion
deleteManagementNetworkIpRange
deleteNetscalerControlCenter
deleteProjectRole
deleteResourceIcon
deleteServicePackageOffering
deleteSrxFirewall
deleteUserFromProject
deployNetscalerVpx
destroyVolume
disableCiscoNexusVSM
disableHAForHost
enableCiscoNexusVSM
getDiagnosticsData
getKubernetesClusterConfig
getRouterHealthCheckResults
getSolidFireVolumeAccessGroupIds
getUploadParamsForIso
getUserKeys
importRole
importUnmanagedInstance
importVsphereStoragePolicies
issueCertificate
linkAccountToLdap
listAnnotations
listCaCertificate
listCAProviders
listCiscoAsa1000vResources
listCiscoNexusVSMs
listCiscoVnmcResources
listDetailOptions
listElastistorInterface
listElastistorPool
listElastistorVolume
listExternalFirewalls
listExternalLoadBalancers
listF5LoadBalancerNetworks
listF5LoadBalancers
listHostHAProviders
listHostHAResources
listInfrastructure
listKubernetesClusters
listKubernetesSupportedVersions
listManagementServers
listNetscalerControlCenter
listProjectRoles
listRegisteredServicePackages
listResourceIcon
listSrxFirewallNetworks
listSrxFirewalls
listStoragePoolsMetrics
listUnmanagedInstances
listVmwareDcs
listVolumesMetrics
listVsphereStoragePolicies
listVsphereStoragePolicyCompatiblePools
listZonesMetrics
migrateNetwork
migrateSecondaryStorageData
migrateVPC
moveNetworkAclItem
moveUser
provisionCertificate
recoverVolume
registerNetscalerControlCenter
registerNetscalerServicePackage
releasePodIpAddress
removeAnnotation
removeVmwareDc
revokeCertificate
revokeTemplateDirectDownloadCertificate
runDiagnostics
scaleKubernetesCluster
startKubernetesCluster
startRollingMaintenance
stopKubernetesCluster
stopNetScalerVpx
syncStoragePool
unmanageVirtualMachine
updateAnnotationVisibility
updateImageStore
updateKubernetesSupportedVersion
updatePodManagementNetworkIpRange
updateProjectRole
updateSecurityGroup
updateSiocInfo
updateStorageCapabilities
updateVlanIpRange
updateVmwareDc
upgradeKubernetesCluster
uploadResourceIcon
uploadTemplateDirectDownloadCertificate

Fail to list VPCs due to invalid json schema

Environment:

ACS: 4.16.0
OS: Ubuntu 21.10
SDK: v.2.12.0

Issue:
SDK fails to unmarshall listVPCs result when there are network tiers attached to a VPC. Here's my output:
VPCService_test.go:69: Failed to list VPCs due to: json: cannot unmarshal object into Go struct field VPC.vpc.network of type string

GetTemplateByName with zoneid constraint fails on template in multiple zones

Using GetTemplateByName on a template that is available in multiple zones results in

There is more then one result for Template UUID: 06145677-058a-456a-89a0-af4afd6fffcf!

The problem is that the function calls GetTemplateID with the zoneid constraint, and then calls GetTemplateByID without a zoneid contraint.

Error retrieving ID of template: json: cannot unmarshal object into Go struct field alias.childtemplates of type []interface {}

This issue is a follow-up of xanzy/terraform-provider-cloudstack#72

This issue is still exist.
So cloudstack-terraform-provider cannot be used for older versions of CloudStack and CloudPlatform.

Adding multiple details to registerTemplate, updateTemplate doesn't work

When making a registerTemplate or updateTemplate API call, the accepted API format for the map is to always use details[0]. Otherwise, the other fields don't work. Currently we increment an iterator in the client and it causes the API to ignore all but the first detail.

e.g.

(localhost) 🐱 > register template name=test displaytext=test hypervisor=Simulator url=http://foo.com/image.vhd format=vhd zoneid=-1 ostypeid=da13519b-5491-11ed-820c-0242ac120002 details[0].uefi=legacy details[1].other=foo
{
  "count": 1,
  "template": [
    {
      "account": "admin",
      "bits": 0,
      "created": "2022-11-08T20:13:59+0000",
      "crossZones": true,
      "deployasis": false,
      "details": {
        "uefi": "legacy"
      },

The above is missing the "other" detail. Compare with:

(localhost) 🐱 > register template name=test displaytext=test hypervisor=Simulator url=http://foo.com/image.vhd format=vhd zoneid=-1 ostypeid=da13519b-5491-11ed-820c-0242ac120002 details[0].uefi=legacy details[0].other=foo
{
 "count": 1,
 "template": [
   {
     "account": "admin",
     "bits": 0,
     "created": "2022-11-08T20:14:27+0000",
     "crossZones": true,
     "deployasis": false,
     "details": {
       "other": "foo",
       "uefi": "legacy"
     },

Adding "hooks" for logging/tracing/...

This issue is a follow-up of xanzy/go-cloudstack#120.

That issue was opened by @synergiator with the title "How to change the log level for go-cloudstack module?" and asked for a way to have more detailed logging.

I proposed a broader solution:

I also think that adding logging would be useful, but as there's never consensus on what to log or which logging library to use (log, zap, logrus, etc.), what would you think of adding some "hooks", optional functions that would be called before and after the actual http request? The user of CloudStack client could then provide its own logging or tracing logic.

IMHO, the best would be to also add an optional context.Context to all API functions, in order to make it possible to do contextual tracing for example.

A possible signature for these hooks could be:

// PreRequestHook would be called in newRequest, just before the HTTP request call.
type PreRequestHook func(ctx context.Context, api string, params url.Values)

// PostRequestHook would be called in each generated client function,
// just after the Unmarshal step.
type PostRequestHook func(ctx context.Context, rawResp json.RawMessage, err error, resp interface{})

I can contribute and submit a PR if you are ok with that.

Any thoughts?

quota api calls are missing

The following quota API's are not enabled in the sdk

quotaBalance
quotaCredits
quotaStatement
quotaSummary
quotaTariffCreate
quotaTariffDelete
quotaTariffList
quotaTariffUpdate
quotaUpdate
quotaEmailTemplateList
quotaEmailTemplateUpdate

Only the "quotaIsEnabled" api is working

Secret Key

Hello, for cluster-api-provider-cloudstack, I'm hoping to be able to login as a sub-domain user from a domain admin account.

To do so I need to be able to fetch a sub-domain user's api-key and secret-key.

Using ListUsers doesn't populate secret key. Using getUser I can't seem to find the user, and I don't see the CloudStack API getUserKeys equivalent in the cloudstack-go api.

Please provide guidance on how to get the sub-domain user's secret key.

Thanks!

v2.15.0 breaking change regarding displayText handling for CreateNetwork

PR #53 introducing CS 4.18 support removed displayText as a parameter to NewCreateNetworkParams(). This makes sense when talking to CS 4.18 (the parameter is no longer mandatory in CS 4.18), but when talking to CS < 4.18, this introduced a breaking change in the old behavior, resulting in Unable to execute API command createnetwork due to missing parameter displaytext.

cloudstack-go/cloudstack/NetworkService.go

Line 1103 in 3995b5e

func (s *NetworkService) NewCreateNetworkParams(name string, networkofferingid string, zoneid string) *CreateNetworkParams {

I am not entirely sure whether this is something that needs to be fixed here, or in implementations of cloudstack-go, but its a undocumented breaking change nonetheless.