edolnx / nephology-server-perl

Nephology

noun [nɪˈfɒlədʒɪ]

1. The branch of meteorology that deals with clouds.
2. A system for building and deploying clouds.

Nephology is a flexible bare-metal provisioning system. It is designed to take newly racked hardware from an unknown state all the way through hardware configuration and finally booting an operating system with an (optional) configuration management client installed. The idea is to take new hardware provisioning time down from hours to minutes.

Unlike other systems, Nepholoy does not dictate which OS or configuration management tools you need to install. You can even use Nephology to perform initial hardware configuration tasks, verify physical requirements, and then pass off to a third-party netboot installer (like XenServer or WDS).

Features

• Template based configuration
• Modular Design (bring your own external tools or use ones already built in)
• Scriptable client and server
• Basic hardware inventory gathering information for use in an external DCIM
• OS Independent
• Configuration Management system independent (including no CM)
• Configutation of BIOS settings (on supported platforms)
• Configuration of RAID controllers
• Configuration of IPMI/DRAC/iLO controllers
• Firmware management (of supported devices)
• Detection, reporting, and optional configuration of network connections
• Detection and enforcement of minimum hardware requirements:
  • CPU Core Count
  • Memory Size
  • Disk Count
  • NIC Count
  • many others
• Support for multiple operating systems
• Support for a rescue mode in order to resolve issues in the same environment as the installer

Requirements

Nephology is designed to run in a netboot environment, and requires that the machine (at least initially) be able to netboot. Once the OS has been installed, it is possible to switch the system to localboot for security reasons.

The Nephology server needs to run on a host capable of running a Perl script. We recommend using Ubuntu Linux with lighttpd. The following Perl modules are required:

• mojolicious (ubuntu package:libmojo-perl)
• YAML (ubuntu package: libyaml-perl)
• JSON (ubuntu package: libjson-perl)
• Rose::DB (ubuntu package: librose-db-perl)
• Rose::DB::Object (ubuntu package: librose-db-object-perl)
• The appropriate packages for Rose::DB to talk to your database of choice

For the actual netbooting of systems, we recommend using iPXE -- configuration details in the wiki.

Status

This project is under active development, and currently in use at DreamHost for deployment of our systems. We will be releasing the project with a BSD license in the near future. If you would like to help, please contact me. There is a wiki page which will help you get started with hacking on the code.

nephology-server-perl's Issues

if boot_mac doesn't exist, action to create it.

This is an optional thing that I would like to have.

If a machine doesn't exist in our database, we should have the automatic option of forking it to a a new controller and populating the DB with its credentials (or ghost credentials)

Validate existence before processing

We should validate a machine exists at all times before we try to do any type action against itself.

Wrap around Rose::DB::Object in a way to access the manager

Currently for each model we have to create a ::Manager. This is not ideal as we have some redundant files. This should include cleaning up:

• NodeStatus
• MapCasteRule
• Node

Create a README.md

I would like to have one, but it's a question of if its even necessary. It could simply be the same thing as what's in the wiki as well

Machine root passwords need to be encrypted

These are currently cleartext! We should be using a two-way hash, preferably using a module from a package already available in ubuntu precise.

This needs to happen in lib/NephologyServer/Install.pm when we're setting the password for the node.

Models Need to be updated

According to @edolnx the schema has changed. We should also probably provide the sql file for import, hmm?

run_list json objects returns caste_rule hash references

During install, nephology-client fails due to caste_rule id's being returned as hash references. These objects should be either a.) dereferenced on the server side or b.) not be hashed at all. Here's an example:

brettg@odin:~$ lynx 50.19.218.14:3000/install/60:eb:69:ed:9f:00

Which returns:

brettg@odin:~$ vim 60\:eb\:69\:ed\:9f\:00 

And this is the contents inside:

1 {"runlist":       ["CasteRule=HASH(0x437dcb8)","CasteRule=HASH(0x4410aa8)","CasteRule=HASH(0x4367658)","CasteRule=HASH(0x436d8f8)"],"version_required":2}

This should probably be served as a straight json blob or not nested hashes maybe?

Here's a sample from the client:

hash- or arrayref expected (not a simple scalar, use allow_nonref to allow this) at ./neph-client.pl line 57

Which refers to this:

 56     for my $reqhash (@{$nephology_commands->{'runlist'}}) {
 57         print "Got command: " . JSON->new->utf8->encode($reqhash) . "\n";
 58     }

update status_id should be done only if we don't fail on a return

We should only be able to update a status_id if we return 200.

Mon Jul 9 22:28:33 2012] [debug] GET /boot/00:00:00:11:DD (Lynx/2.8.8dev.5 libwww-FM/2.14 SSL-MM/1.4.1 GNUTLS/2.8.6).
[Mon Jul 9 22:28:33 2012] [debug] Dispatching "NephologyServer::Boot->lookup_machine".
[Mon Jul 9 22:28:33 2012] [debug] Template "boot/ubuntu-precise.txt.ep" not found.

At this point a template had not been created, but status_id was changed in Boot.pm line #50.

After that point we were unable to return anything because our node status was updated to the next release (for 1005 it was set to 1000) and caused a 404.

Better way to install dependencies

Currently from hacking wiki page here:

https://github.com/Thingee/nephology-server-perl/wiki/Hacking

You have to install the dependencies manually. It would be awesome if there was something comparable to a requirements file.

YAML Config using relative path

Relative to wherever you're running nephology-server from. This should be absolute using either File::Spec or a specific location we have in mind.