Coder Social home page Coder Social logo

devopservices / clustershell Goto Github PK

View Code? Open in Web Editor NEW

This project forked from cea-hpc/clustershell

1.0 4.0 0.0 5.16 MB

Scalable cluster administration Python framework — Manage node sets, node groups and execute commands on cluster nodes in parallel.

Home Page: http://cea-hpc.github.com/clustershell

License: Other

Python 99.52% Shell 0.48%

clustershell's Introduction

 -----------------------------------------
 ClusterShell 1.6 Python Library and Tools
 -----------------------------------------

ClusterShell is an event-driven open source Python library, designed to run
local or distant commands in parallel on server farms or on large Linux
clusters. It will take care of common issues encountered on HPC clusters, such
as operating on groups of nodes, running distributed commands using optimized
execution algorithms, as well as gathering results and merging identical
outputs, or retrieving return codes. ClusterShell takes advantage of existing
remote shell facilities already installed on your systems, like SSH.

ClusterShell's primary goal is to improve the administration of high-
performance clusters by providing a lightweight but scalable Python API for
developers. It also provides clush, clubak and nodeset, three convenient
command-line tools that allow traditional shell scripts to benefit from some
of the library features.

 -------------------
 Requirements (v1.6)
 -------------------

 * GNU/Linux, *BSD, Mac OS X, etc.

 * OpenSSH (ssh/scp)

 * Python 2.x (x >= 4)

 -------
 License
 -------

ClusterShell is distributed under the CeCILL-C license, a French transposition
of the GNU LGPL, and is fully LGPL-compatible (see Licence_CeCILL-C_V1-en.txt).

 ------------
 Installation
 ------------

When possible, please use the RPM/deb package distribution:

    https://github.com/cea-hpc/clustershell/wiki/GetClusterShell

Otherwise in the source directory, use:

    # python setup.py install
    # cp -r conf /etc/clustershell

For installation on Mac OS X, please see:

    https://github.com/cea-hpc/clustershell/wiki/ClusterShellOnMacOSX

 ----------
 Test Suite
 ----------

Regression testing scripts are available in the 'tests' directory:

    $ cd tests
    $ nosetests -sv <Test.py>
    $ nosetests -sv --all-modules

You have to allow 'ssh localhost' with no warning for "remote" tests to run.

 --------------
 Documentation
 -------------

Local API documentation is available, just type:

    $ pydoc ClusterShell

Online API documentation (epydoc) is available here:

    http://packages.python.org/ClusterShell/

 --------------------------
 ClusterShell interactively
 --------------------------

Python 2.7.1 (r271:86832, Apr 12 2011, 16:15:16)
[GCC 4.6.0 20110331 (Red Hat 4.6.0-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ClusterShell.Task import task_self
>>> from ClusterShell.NodeSet import NodeSet
>>> task = task_self()
>>> task.run("/bin/uname -r", nodes="linux[4-6,32-39]")
<ClusterShell.Worker.Ssh.WorkerSsh object at 0x20a5e90>
>>> for buf, key in task.iter_buffers():
...     print NodeSet.fromlist(key), buf
... 
linux[32-39] 2.6.40.6-0.fc15.x86_64

linux[4-6] 2.6.32-71.el6.x86_64

 ------------------
 ClusterShell Tools
 ------------------

Powerful tools are provided: clush, nodeset and clubak.

* clush is a friendly and full-featured parallel shell (see: man clush).
  If in doubt, just check if your other parallel tools can do things like:
  # tar -czf - dir | clush -w node[10-44] tar -C /tmp -xzvf -

* nodeset is used to deal with your cluster nodeset, it can be bound to
  external groups (see: man nodeset and man groups.conf).

* clubak is a tool used to format output from clush/pdsh-like output
  (already included in clush with -b), see man clubak.

 -----
 Links
 -----

Main web site:

    http://clustershell.sourceforge.net
    or http://cea-hpc.github.com/clustershell/

Github source respository:

    https://github.com/cea-hpc/clustershell

Github Wiki:

    https://github.com/cea-hpc/clustershell/wiki

Github Issue tracking system:

    https://github.com/cea-hpc/clustershell/issues

Sourceforge.net project page:

    http://sourceforge.net/projects/clustershell

Python Package Index (PyPI) link:

    http://pypi.python.org/pypi/ClusterShell

ClusterShell was born along with Shine, a scalable Lustre FS admin tool:

    http://lustre-shine.sourceforge.net

 -------
 Authors
 -------

Stephane Thiell           <[email protected]>
Aurelien Degremont     <[email protected]>
Henri Doreau                 <[email protected]>

CEA/DAM 2010, 2011, 2012 - http://www-hpc.cea.fr

clustershell's People

Contributors

degremont avatar hdoreau avatar rezib avatar thiell avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.