controlm / automation-api-quickstart

Control-M Automation API quickstart

Control-M Automation API quickstart

This repository contains code samples and how-to for Control-M Automation API.

• Download Workbench for Docker - the latest development Control-M environment.
• Download Automation API CLI (ctm-cli.tgz).
• Installation instructions.

Online Documentation

You can find the latest Control-M Automation API documentation, including a programming guide, on the project web page.

Contribution guide

To contribute, please follow these guidelines.

Files, folders and naming conventions

1. Every sample and its associated files must be contained in its own folder. Name this folder something that describes what your sample does. Usually this naming pattern looks like level-sample-purpose (e.g. 201-automate-corrective-flow). Numbering should start at 101. 100 is reserved for things that need to be at the top.
2. For consistent ordering create all folders in lowercase.
3. Include a README.md file that explains the sample. A good description helps other community members to understand your sample. The README.md uses Github Flavored Markdown for formatting text. If you want to add images to your README.md file, store the images in the images folder. Reference the images in the README.md with a relative path (e.g. ![alt text](images/sampleImage.png "Sample Image Text")). This ensures the link will reference the target repository if the source repository is forked.
4. The samples are split into two major categories, Control-M and Helix Control-M. Make sure that you upload the sample under the correct category.

automation-api-quickstart's Issues

Installation packages

Hi
It would be super useful to have the API installation and for that matter the client/agent installation for Control-M packaged up into an RPM and DEB file for easier installation on Red Hat and Ubuntu based distros. Having such packages means I can more easily automate configuration of Control-M API and client/agent in tools like Puppet.

Docker image wont build when using workbench VM

I tried to follow the tutorial here using the workbench appliance running in VirtualBox and the docker build step fails with a connection refused being returned by the workbench server.

--2019-10-07 18:48:37--  (try: 6)  https://localhost:8443/automation-api/ctm-cli.tgz
Connecting to localhost (localhost)|127.0.0.1|:8443... failed: Connection refused.
Connecting to localhost (localhost)|::1|:8443... failed: Cannot assign requested address.
Retrying.

MacOS: 10.14.6
Docker: 19.03.2
VirtualBox: 6.0.12
Workbench Image: 9.0.18.300-20190218.133426-1

invalid env syntax

test

running 301-statefulset-agent-to-run-k8s-jobs-using-ai-job /9.0.20.200 step 3, CCM hosts didn't add

Folloing 301, I have finish to generate a image for Control-M/Agent.
while running the step 3 : "kubectl apply -f stateful_ha.yaml"

persistentvolumeclaim/ha-agent-pvc unchanged
statefulset.apps/ha-statefulset-agent unchanged

The messages as below:

[controlm@km ~]$ kubectl get all

NAME READY STATUS RESTARTS AGE
pod/web-app01-7b6d698dc6-4ktxj 1/1 Running 0 134m
pod/web-app01-7b6d698dc6-6n6mj 1/1 Running 0 134m

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 443/TCP 3d
service/web-app01 NodePort 10.105.40.208 80:30469/TCP 3d

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/web-app01 2/2 2 2 3d

NAME DESIRED CURRENT READY AGE
replicaset.apps/web-app01-7b6d698dc6 2 2 2 3d

NAME READY AGE
statefulset.apps/ha-statefulset-agent 0/2 65m

Unable to create Docker image using the 301-statefulset-agent-to-run-k8s-jobs-using-ai-job/docker

I have been trying to create docker image using the DockerFile in 301-statefulset-agent-to-run-k8s-jobs-using-ai-job/docker repo, but it fails at below step:

-> install agent, setup will be done during statup
&& ctm provision image $AGENT_IMAGE_NAME && echo installation ended successfully \

---

Below are the logs:

debug: Locating java command
info: Located java at:"/bin/java"
info: downloading https://clm-pun-v1pngr:8443/automation-api/utils/control-m.services.provision-9.20.200.jar into /home/controlm/.ctm/control-m.services.provision-9.20.200.jar
info: 4MB/4MB precent: 100%
debug: Set proxy settings for java process: http: [undefined]; https: [undefined]; no_proxy: [undefined]
debug: starting command: "/bin/java" -jar /home/controlm/.ctm/control-m.services.provision-9.20.200.jar -image "Agent_20.Linux.json" -agent_tag "" -server https://clm-pun-v1pngr:8443/automation-api -action image -environment myenv -ctms "" -name "" -port "" -cert 0 -file ""
info: Making SSL trust all certificates and all hostnames
info: OnPrem provision delivery mode
info: is agent exist cli exception:os json is not one of the supported OS types : Linux, Solaris, AIX, Windows (rc=1)
info: error while verifying image existence (CtmCliException), os json is not one of the supported OS types : Linux, Solaris, AIX, Windows (rc=1)
error: failed to verify image existence in server. reason: os json is not one of the supported OS types : Linux, Solaris, AIX, Windows (rc=1)

Python code needs updating to Python 3

Firstly, thanks for the great repo and sample code.

Python 2 goes End of Life very soon now, 1.1.2020. Main issue I see is the use of raw_input() built-in function which has been replaced with input() in Python 3 as proposed by PEP 3111:
https://www.python.org/dev/peps/pep-3111/

I believe the majority of users will be using Python 3 now, or at least heading that way very soon so the code is more likely to fail going forward than say 2-3 years ago where Python 2.7 was still very common. Here's some more details about what has changed in Python 3:
https://docs.python.org/3.0/whatsnew/3.0.html

Another option is that you make the code backwards compatible - here's a simple workaround you can use so input() function resolves in both Python 2 and Python 3:

if hasattr(__builtins__, 'raw_input'):
      input=raw_input

I'm happy to create a PR if you would like help with this.

WaitForEvents type NoDate and AnyDate not working.

Hello, i'm trying to develop new flows using the automation api on ctm 9.0

What is causing me trouble is that i can't choose for the waitforevents objects with type of NoDate and AnyDate.

When i try to build the flow.json i always get error.
Instead with waitforevents object that use OrderDate or SpecificDate value, the waitforevents is working flawlessy.

Can you check and try why is not working?

Even the example that ctm api web page shows is not working:

"Wait1":
{
"Type": "WaitForEvents",
"Events": [
{"Event":"e1"},
{"Event":"e2"},
{"Event":"e3", "OrderDate":"AnyDate"} #this line will not work, and will not permit build/deploy, removing this line or changing the value like event e1 and e2 will make the deploy/run work.
]
}

How do we contribute?

Tried creating a branch to contribute to project, but keep getting 403-denied errors when trying to push it back. Looks like this repo is locked such that you can't create branches and have the submitted for review and inclusion.

Integration with Service Catalog/Service Broker

Hi,

It will be nice if your Automation-API could respect Open Service Broker API (https://www.openservicebrokerapi.org/) to have the capacity to incude Control-M directly in solution like CloudFoundry and Kubernetes.

I will see this following service :

• deploy a job
• undeploy a job

The bind methods in the Open Service Broker will be used to trigger a job

Cédric

Workbench has port errors

The OVA for Workbench as linked on the main page won't startup due to port errors. This was confirmed by support as well. This needs to be fixed ASAP!

Kubernetes v1.26 environment based on TKG v2.3

Hi,

I have a question.
The document states that it has been tested in the EKS V1.20 environment and that additional drivers must be installed when configured in EKS V1.23.
I want to know if Control-M 9.0.20.200 is supported without any problems in the TKG v2.3-based Kubernetes v1.26 environment.
And please let me know if there are any additional drivers that need to be installed in the TKG v2.3 or Kubernetes v1.26 environment.

Remote host ssh key

Is there a way to automate adding remote host using a predefined ssh key stored in the controlm server?

Currently I couldn't use the add remotehost api using the ssh key already in the ctm server. I have to manually create the remote host from the gui where i can select the key.

OVA Workbench default user/password

I have just started testing the last version of Control-M Workbench, and would like to test the these API scripts.
The main issue is that no where is it indicated what us the default user/passwd and to be able to use the API we need a valid user and password.

Does anyone know whats the default user and password?

Failover

I'm struggling to work out how the approach will allow for failover of the Control-M Server, as the communication is initiated by the Control-M agent as a persistent connection, how will this suddenly switchover on the secondary when the primary fails?

Dockerfile deprecated (DR5V3.9.0.20.220_Linux-x86_64.BIN + java + Centos)

Hi,

I look at the helix procedure for k8s and i have notice some drawback that could be resolve ( link of the readme procedure : https://github.com/controlm/automation-api-quickstart/tree/master/helix-control-m/301-statefulset-agent-to-run-k8s-jobs-using-ai-job )

Could you update your Dockerfile in order to :

• use the last binary ( old DR5V3.9.0.20.220_Linux-x86_64.BIN -> new DR5V3.9.0.21.000_Linux-x86_64.BIN). It can be perhaps more useful if you specify dynamically the binary
• add a compatible version of java for the previous binary (jdk-11.0.17) in order to avoid the following message :
  Cannot find Java. The Java major version must be 11. The minor version must be 11.0.12 or higher. Set the BMC_INST_JAVA_HOME environment variable to the Java home path and then rerun the installation. Refer to Control-M Documentation for more information.
• select another image than centos 7 because centos is deprecated since 2011. You could use ubuntu (or rockylinux which is quite similar to centos).
• Add into the README an exemple of ARG AAPI_END_POINT value and warn that the domain should contain -aapi for a Helix Tenant

decommission_controlm.sh: config server delete missing -e

I suppose that the environment for the ctm cli is not really explicitly needed, as there should be only one in the container, but if a workload created and activated another environment for some work (and misbehaved and did not reset), the delete will fail.

The line containing

ctm config server:remotehost::delete

should be

ctm config server:agent::delete $CTM_SERVER $ALIAS -e $CTM_ENV

I did not pull because there are a few instances of the script and wanted to ensure it was not by design.

Authorization Bearer is not mandatory in header control M 9.18

Hi guys.

In our environment production there are automation-api 9.0.18 and it's possible send request Rest API o Curl without header Authorization Bearer, but we migrated QA's environment to 9.0.20 and now it's mandatory. somebody knows, how could we disable header Authorization bearer in 9.0.20?

decommission_controlm.sh and run_register_controlm.sh fail if active jobs use hostgroup/agent alias

decommission_controlm.sh and run_register_controlm.sh fail if active jobs use hostgroup/agent alias. The api will not let the agent alias be deleted if it is the same as the previous container's agent alias and there is active jobs ordered out against the hostgroup and/or agent alias. This causes both register and de-register scripts to fail and makes automating re-building a container with the same hostgroup and/or agent alias impossible. Manual intervention is required when this happens which defeats the entire purpose of this type of automation.

201-call-restapi-using-curl

The cut for the token is set to f1 in the curl example code. This does not cut the token field in the new version of aapi.
token=$(echo ${login##*token":"} | cut -d """ -f 1)

When I set the cut to field 8 it resolves the issue.
token=$(echo ${login##*token":"} | cut -d """ -f 8)