jrasell / chemtrail Goto Github PK

Is your feature request related to a problem? Please describe.
Currently Chemtrail does not have internal method of exclusion, and it is up to the operator to ensure only one server is running at any one time. If more than one server is actively running on a Nomad cluster, each server will make its own decisions about scaling and enforce.

Describe the solution you'd like.
Chemtrail should include a method where multiple servers can run on the same cluster, but only one will actively run scaling operations. The standby server should have the ability to take over active operations if the active server fails.

Is your feature request related to a problem? Please describe.
In older versions of Nomad (commit f969298), the API struct fields used to gather available resources for a node do not exist. This causes Chemtrail to panic and fail.

Describe the solution you'd like.
Chemtrail should perform checks to ensure the required fields are usable, and if not, attempt to fall back to the previous Nomad fields which would contain the resource information.

relates #25

Is your feature request related to a problem? Please describe.
When implementing Chemtrail for the first time, or when introducing scaling to a new node class, it can be useful to have an action which merely performs a notification of scaling intent. This removes any risk and allows users to perform tuning based on results without interacting with actual providers.

Describe the solution you'd like.
The scaling policy check action param should be extended to allow use of a notify action. The provider section can then also include additional backends, such as 'log' which are designed for use with this action only. The policy validate function will need to be updated to accommodate this change.

Is your feature request related to a problem? Please describe.
Cooldown is a feature of autoscaling which can help ensure previous scaling activities have a chance to impact the load on an application before another scaling event is triggered. This is particularly important when performing scaling of hosts as their startup can take a number of minutes.

AWS - https://docs.aws.amazon.com/autoscaling/ec2/userguide/Cooldown.html
Google - https://cloud.google.com/compute/docs/autoscaler/
Microsoft - https://docs.microsoft.com/en-us/azure/azure-monitor/platform/autoscale-virtual-machine-scale-sets?toc=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fvirtual-machine-scale-sets%2FTOC.json&bc=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fbread%2Ftoc.json

Is your feature request related to a problem? Please describe.
Hello,
I was curious for the scaling policies if we could allow for AWS ASG discovery based off resource tags? Similar to what cluster-autoscaler does for AWS EKS, which we do not need to set a specific autoscaling name, it just queries tags on the resource and then pulls in the autoscaling groups.

Describe the solution you'd like.
In the documentation for the ProviderConfig you specify the asg-name, instead of using a static asg-name, have users to tag=[tag id] for chemtrail to query upon. This way as new autoscaling groups get added it dynamically gets added to chemtrail to manage. This also allows for different scaling policies based on tagged resources on how it gets controlled dynamically.

Describe alternatives you've considered.
Alternative, build a custom autoscaling binary/docker image.

Explain any additional use-cases.
No additional use-cases I can for-see currently as this is a specific purpose.

Early this year I joined HashiCorp to work directly on the Nomad ecosystem. This work has resulted in the creation of nomad-autoscaler which provides horizontal app and cluster scaling. The nomad-autoscaler is the officially supported autoscaler for Nomad and has a dedicated team working on it. It therefore makes sense to now archive this repository. Thanks to everyone who helped build this project.

Hi @jrasell

Do you think I can try chemtrail on my nomad production cluster ? I really need to scale my ASG accordingly with Nomad.
According to my really nice experience with Sherpa, I think I can trust chemtrail !

I'm just asking if there are any known issues or something I should know before trying it ?

Thanks,

Describe the bug
The policy read CLI command does not correctly format the ProviderConfig section as like the scale status command. The results in CLI output which includes map[] which is not user friendly.

Expected behavior
The output would look like ProviderConfig = key:value.

Environment:

chemtrail version v0.0.1+dev
	Date:   2019-12-30 16:57:13.6989 +0000 UTC
	Commit: 2297a43
	Branch: master
	State:  clean

Is your feature request related to a problem? Please describe.
The system command should be expanded to allow for detailing the metrics currently stored on the Chemtrail server. This brings more complete coverage and parity between API and CLI functionality.

Describe the bug
Even when the AWS ASG client provider is not enabled, logging indicates that it has been setup.

To reproduce
Run Chemtrail via chemtrail server --autoscaler-enabled --log-level=debug and witness the logging output which will include the following line:

3:56PM DBG successfully setup AWS AutoScaling provider

Expected behavior
The log line only appears when the AWS ASG provider is enabled.

Environment:

chemtrail version v0.0.1+dev
	Date:   2019-12-30 15:57:42.969907 +0000 UTC
	Commit: 376c10e
	Branch: master
	State:  clean