kendallgoto / ilo4_unlock Goto Github PK

(Carrying over discussion from #11)

In v2.78, the fan tool was substantially altered, with most of its feature code removed. In v2.79, it was fully removed (https://github.com/kendallgoto/ilo4_unlock/blob/main/research/2022-02-18-building-279.md).

As such, this repo currently maintains versions up to v2.77. This presents a choice for users - either risk potential security vulnerabilities, etc., patched upstream by using an out of date iLO version, or lose the ability to control the fan system of their blade.

Work is proposed to begin patching later versions, either by carrying the <=v2.77 versions' fan CLI system, create builds that directly modify baked curves, or create a system to dynamically modify these curves with a new interface. Based on user input, a few different areas are proposed that can be modified in future FW:

• Fan minimum speeds
• Disabling sensors
• Altering PID curves
• Disabling "failed sensor" speed-up (happens for non-certified PCIe devices, for example)

Although the fan CLI was fully removed, it is unlikely that these underlying systems outright do not exist in iLO 4 v2.78+ (although, I was skeptical that some of the iLO 4 fan control components were replaced by iLO 5 components in v2.79, I am not sure if that is the case).

Bringing patches to iLO 5 is important to continue to provide reusable hardware in non-enterprise settings as iLO 5 systems reach the end of their usable life in enterprise and fall into the reuse market.

Here's a summary of what I know:

• Airbus' lab have done lots of work in the past to open up iLO 5, including many of the same exploits used in iLO 4.
• Some iLO 5 versions have rollback restrictions that prevent arbitrary firmware from being deployed
• Reading/writing firmware directly into the iLO 5 EEPROM is relatively easy, ensuring a pathway for testing and recovery efficiently
• I am not sure if secure boot, etc., is enforced on this chip
• There seem to be some differences between Gen10 and Gen10v2 with the iLO system, despite using the same iLO5 FW versions.
• The fan CLI from iLO 4 is not present in the iLO 5 versions I have looked at.
• iLO 5 systems tend to run substantially quieter than iLO 4 systems, so the need for fan controls is reduced.

Currently all the python files have a #!/usr/bin/python shebang, which forces the Python version installed system wide to be used. This doesn't sound intentional as the README says to create a virtualenv to install requirements in. Using a shebang of #!/usr/bin/env python would allow using the actual Python version inside the virtualenv and ensure that everything happens inside.

Hi, I am on a DL360 G9, and I can NOT get the flash to run, no matter what I do. This is my system:
System: Host: pb-ProLiant-DL360-Gen9 Kernel: 6.5.0-21-generic x86_64 bits: 64 Console: pty pts/1 Distro: Ubuntu 22.04.4 LTS (Jammy Jellyfish) Machine: Type: Server Mobo: HP model: ProLiant DL360 Gen9 serial: //serial// UEFI: HP v: P89 date: 01/22/2018 CPU: Info: 6-core model: Intel Xeon E5-2620 v3 bits: 64 type: MT MCP cache: L2: 1.5 MiB Speed (MHz): avg: 1314 min/max: 1200/3200 cores: 1: 2400 2: 1200 3: 1200 4: 1200 5: 1200 6: 1200 7: 1200 8: 1200 9: 1200 10: 1375 11: 1200 12: 1200
I am attempting to flash the patched 2.77 firmware to lower my fan speeds due to unsupported Barracuda drives - my PWM is at 100 and it's too loud for an apartment setting.

I followed the instructions down to the T, but this is what I am met with:
`
Error opening /dev/mem... you're probably not root.

FLASH_iLO4 v1.18 for Linux (Mar 31 2016)
(C) Copyright 2002, 2016 Hewlett-Packard Enterprise Development Company, L.P.
Firmware image: ilo4_250.bin

Component XML file: CP027911.xml
CP027911.xml reports firmware version 2.50

**IMPORTANT: This operation should only be used if iLO 4 is non-responsive and
cannot be flashed via standard methods.
This operation will flash the firmware on the iLO 4 in this server
with version "2.50". Following condition(s) must be met:
1. iLO 4 Security Override Switch must be set.
Continue (y/N)? y
Trying to reset iLO 4 IOP
write32: failed to map physical address 92a8c040 to virtual address.
write32: failed to map physical address 92a8c040 to virtual address.
iLO 4 IOP IS in reset!
write32: failed to map physical address 92a8c040 to virtual address.
write16: failed to map physical address 000000ab to virtual address.
write32: failed to map physical address 000001ff to virtual address.
write16: failed to map physical address 0000020d to virtual address.
write16: failed to map physical address 0000020d to virtual address.
write16: failed to map physical address 000000ab to virtual address.
Waking up iLO 4 IOP
write32: failed to map physical address 92a8c040 to virtual address.

ERROR: iLO 4 security override switch isn't set. []
pcilib: sysfs_write: write failed: Operation not permitted
pcilib: sysfs_write: write failed: Operation not permitted
pcilib: sysfs_write: write failed: Operation not permitted
`

I have tried to run from an Ubuntu LiveCD, on a brand new Ubuntu install, logged into terminal as sudo, added my user to kmem group, re-doing the whole process from start to finish, no luck.

I have confirmed:

1. Linux is clean
2. Permissions are correct
3. The original build runs without issue
4. The security switch is flipped
5. hpilo is not running

Any ideas/pointers would be appreciated!

Hello, first thanks for this stuff! I could get it flashed correctly, its just that after i power off (reset is fine iirc) the machine, the fan command stops working, it just gives me couple of empty lines back. Does somebody know what is wrong here?

I spent a couple of hours debugging why i wasnt able to flash the modified firmware on a ubuntu live USB
Then I remembered that on this particular server (DL380G9) Secure Boot was enabled

After disabling it i was able to flash it.
Bare in mind that the securuty override was disabled and i validated that on iLO web interface, that there was a notice there and i could login with any random amalgamation of letters, also during post there was always a alert message that the protection was disabled.

Here's the errors i got on ubuntu:
write32: failed to map physical address xxx... to virtual address.
ERROR: iLO security override switch isn't set.

First of all thanks for all the effort you put into this project!

Flashing a DL580 Gen9 worked without issues.
DIP switch number 1 was correct.

I do have a quick little question tho:
How does one find out what specific sensor (e.g. the name) correlates to a certain sensor number?

To silence the server i disabled sensor 13 (which apparently was reporting nothing at all?! No prev_drive, no output or rather both 0 value) which brought the fans from 47% all the way down to 11%.

But i can see the HD controller getting way hotter now which is a little problem.
Therefore the interest to see what sensor that is.

Is it the same number on the iLO webpage "Temperatures" tab?

Best regards,
Dominik

I've tried everything I can think of including building python2.7 from source on debian but can't find a way to get python2+pip working any longer. The following happens as of Jan 24...

eric@vogon2:~/Downloads/ilo4_unlock$ sudo python get-pip.py
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/pip/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/pip/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/pip/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/pip/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/pip/
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)) - skipping
ERROR: Could not find a version that satisfies the requirement pip<21.0 (from versions: none)
ERROR: No matching distribution found for pip<21.0

is there a workaround or a way to port the scripts to python3 ?

I've tried to patch 2 servers (HP DL160 G8 & DL360 G9). Both servers gave me the same "Segmentation fault" error when running the "sudo ./flash_ilo4 --direct" command. I tried running it from Ubuntu Live CD 21.10 and the latest Ubuntu Live CD version.

I'm not sure if this could affect anything, but I ran the build.sh scripts from WSL - Windowos Subsystem for Linux (Ubuntu distro).
I got the BIN file properly patched, as far as I can tell.

Is there any other way to install the patched BIN?

Thank you!

Work is proposed to provide patches for iLO 3 FW. Based on my understanding, here is what's known:

• iLO 3 does not seem to have a fan control CLI similar to iLO 4. It may have a replacement interface.
• iLO 3 HW is mostly out of life, even in the homelab space. It is very expensive to run due to power costs, and iLO 4 HW costs mostly the same. Nonetheless, it is valuable to provide a quick fix for users stuck on iLO 3 systems.

iLO 3 FW can be decoded using a similar method to iLO 4. I wrote a modified extractor (https://gist.github.com/kendallgoto/530526755d527f44618ee3baf75e154e) that seemed to get good binaries for the main partitions. I haven't tested re-packing. I also am not sure what the installation method will be, if there is a similar CHIF upload method like iLO 4, etc.

It seems like the commands are working, but I don't get any output like fan info.

I'm on a ml350 gen9 with the patched ilo4 2.77

I just wanted to report that I successfully flashed the modded firmware on my Proliant ML110 Gen9 without any deviation from the provided guide.

I used a fresh install of Ubuntu 20.04 LTS to build the firmware and a Ventoy USB liveCD Ubuntu 20.04 LTS to flash it.

DIP switch on motherboard is number 1 as per official docs.