discoriver / massh Goto Github PK
View Code? Open in Web Editor NEWGo package for running Linux distributed shell commands via SSH.
License: GNU General Public License v3.0
Go package for running Linux distributed shell commands via SSH.
License: GNU General Public License v3.0
./main.go:14:32: not enough arguments in call to config.SetPublicKeyAuth
have (string)
want (string, string)
If a user wants to execute large set of shell commands. It would annoying to always create a Go program every single time and setting the instructions in an array.
Based on the inspiration of Ansibles. We can set a simple yaml or json file which consists of an array of strings of shell commands to be executed.
I am working a p2p network for executing custom scripts. Currently I am using Ansibles as a plugin system. But it's extremely cluttered and requires external dependencies. If I can implement the following issue. I can use your project as a plugin system to my project.
Project link: https://github.com/Akilan1999/p2p-rendering-computation
One of the use cases for Massh is querying log files on a distributed environment, to consolidate them into a single filtered log file.
The way I use this right now is to run a grep command on each remote server, which gathers output and sticks it all in a timestamp-sorted local file. The problem is that some historical logs are archived in tar.gz
format, which severely limits simple/easy access to the files within.
Simplified, the problem is that we need a way to prepare files to be worked on.
workingDirectory
, may improve interactivity within the application and improve overall readability.Most of my hesitation is around how much we're offsetting to shell scripts and how we maintain readability and efficiency within the Go program. Being able to debug and follow exactly what commands are doing is critical to building with massh. I'm also aware that we need to avoid the package scope getting out of control. I'm unsure what the best approach is here right now, but will continue to think.
While working on https://github.com/discoriver/omnivore, the idea of pausing an in-progress command came to mind. In the event that file output breaks, and we need to continue in-memory (or at least give the option to continue), I may want to pause hosts temporarily while we determine if we should continue in-memory, to stop any processing being performed if the user takes a while to respond to the message.
Current thoughts are that we can signal the process to be suspended and resume it, which is pretty trivial, but we should explore what is available to us.
I'm creating this to explore options at a later date. If anyone else has thoughts, please feel free to join the conversation.
When providing a nil value to config.Stream()
, instead of running sshCommandStream
, sshCommand
is run, and no error is reported.
This is a failure in the worker
function logic in session.go
;
// worker invokes sshCommand for each host in the channel
func worker(hosts <-chan string, results chan<- Result, job *Job, sshConf *ssh.ClientConfig, resChan chan Result) {
if resChan == nil {
for host := range hosts {
results <- sshCommand(host, job, sshConf)
}
} else {
for host := range hosts {
sshCommandStream(host, job, sshConf, resChan)
}
}
}
This was a lazy way to detect if we should be streaming.
Intermittently, running stream to capture a reasonable amount of text sometimes misses chunks of lines.
You can see an example of this here in "Go Test": https://github.com/DiscoRiver/massh/runs/3832011162?check_suite_focus=true
I was using the command cat /var/log/auth.log
in this test, and you'll see the following on line 48
;
do: pam_unix(sudo:session): session opened for user root by (uid=0)
This should be something like what is on line 44
;
Oct 7 20:45:48 fv-az224-778 sudo: pam_unix(sudo:session): session opened for user root by (uid=0)
There doesn't immediately seem to be a pattern, but more investigation is necessary into the concurrency model.
Currently, for package errors we're relying on Result.Error
, which is rather inelegant, but it's necessary that we don't return an error from from the Stream method, for easier use.
I'd like to re-explore logging from within the Stream method (this will also apply to Run, but it's trivial). Perhaps adding a Messages
field to give the user the option of looking at the internals, so they can then log it. We can't write anything out ourselves via the Massh package, but giving the user a value they can do what they like with is well within the scope.
Currently, command output is only returned after every host has finished executing;
for r := 0; r < len(c.Hosts); r++ {
res = append(res, <-results)
}
return res
After implementing this package into a production app, for an environment of around 200 servers, it became clear that this can cause an almost invisible bottleneck, especially if the worker pool is low compared to the number of hosts being touched. I wanted to refer to it as a perceived delay, but it can actually hang up the caller because it is waiting for every host to finish the work and return something.
There are two requirements I have to improve this;
run
(see unexported), does not account for Config.JobStack
, and therefore only returns a single job result from each host. Function needs to be updated so that the results channel has the correctly length when JobStack
is present. Similar to how runStream
functions.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.