open-power / op-test Goto Github PK
View Code? Open in Web Editor NEWTesting Firmware for OpenPOWER systems
License: Apache License 2.0
Testing Firmware for OpenPOWER systems
License: Apache License 2.0
/home/ipjoga/op-test-framework/op-test --bmc-type SMC --bmc-ip 10.33.1.21 --bmc-username ADMIN --bmc-password ADMIN --bmc-usernameipmi ADMIN --bmc-passwordipmi ADMIN --host-ip 10.33.0.156 --host-user root --host-password passw0rd --ffdcdir test-reports/ --machine-state OS --host-pnor p9dsu03152018_IBM_imp_sign.pnor --pupdate ./pupdate --run testcases.BasicIPL.BootToOS EXECUTING
Traceback (most recent call last):
File "/home/ipjoga/op-test-framework/op-test", line 42, in
OpTestConfiguration.conf = OpTestConfiguration.OpTestConfiguration()
File "/home/ipjoga/op-test-framework/OpTestConfiguration.py", line 154, in init
for dir in (os.walk('addons').next()[1]):
StopIteration
I am executing the BVT tests targeted for a Open Power Firestone Hardware as below.
[root@lop827 bvt]# ./run-op-bvt --bmcip 9.40.193.47 --bmcuser sysadmin --bmcpwd superuser --usernameipmi ADMIN --passwordipmi admin --cfgfiledir ../ci/source/ --imagedir /tmp/ --imagename firestone_update.pnor --fverbose /tmp/new_firestone ./op-ci-basic-bvt.xml
ipmitool -H 9.40.193.47 -I lanplus -U ADMIN -P admin sdr elist |grep 'OCC Active'
ipmitool -H 9.40.193.47 -I lanplus -U ADMIN -P admin sdr elist |grep 'OCC Active'
ipmitool -H 9.40.193.47 -I lanplus -U ADMIN -P admin sdr elist |grep 'OCC Active'
ipmitool -H 9.40.193.47 -I lanplus -U ADMIN -P admin sdr elist |grep 'OCC Active'
IPL timeout
00:26:29 :: op-ci-basic: ERROR: command failed with exit code 256
00:26:29 :: op-ci-basic: Finished Test:
00:26:29 :: op-ci-basic: Starting Test:
00:26:29 :: op-ci-basic: Running command: op-ci-bmc-run "op_ci_bmc.ipmi_sel_check()"
ipmitool -H 9.40.193.47 -I lanplus -U ADMIN -P admin sel elist
00:26:45 :: op-ci-basic: OK: command finished successfully
00:26:45 :: op-ci-basic: Finished Test:
00:26:45 :: op-ci-basic: ERROR: Finished tests on 2015-November-04 with one or more testcase errors.
00:26:45 :: ERROR: "OP CI Basic BVT" (op-ci-basic.xml) finished with one or more errors
00:26:45 :: ERROR: Finished BVT tests on 2015-November-04 with 1 testcase failures.
All logs and FFDC saved under /tmp/root/bvt/2015-11-04-89279
As can be seen the tests fail because of the failure of the command:
ipmitool -H 9.40.193.47 -I lanplus -U ADMIN -P admin sdr elist |grep 'OCC Active'
As per my analysis, I have found out that the above command is failing because the
grep 'OCC Active' is failing to fetch the correct output from the sdr elist of the Firestone Hardware.
However, on a Habanero Hardware the output looks as follows:
[root@lop826 src]# ./ipmitool -H 9.47.71.252 -I lanplus -U ADMIN -P admin sdr elist |grep 'OCC Active'
OCC Active | 08h | ok | 210.0 | Device Enabled
There are 2 OCC chips on Firestone and 1 OCC chip on Habanero.
In the sdr elist entries there will be OCC 1 Active and OCC 2 Active mentioned for Firestones.
[root@lop826 src]# ./ipmitool -H 9.40.193.53 -I lanplus -U ADMIN -P admin sdr elist |grep '^OCC'
OCC 1 Active | 08h | ok | 210.0 | Device Enabled
OCC 2 Active | 09h | ok | 210.1 | Device Enabled
[root@lop826 src]# ./ipmitool -H 9.47.71.252 -I lanplus -U ADMIN -P admin sdr elist |grep '^OCC'
OCC Active | 08h | ok | 210.0 | Device Enabled
Please suggest me if I can submit a patch with the changes mentioned above to the source file op-test-framework/common/OpTestIPMI.py which has the code mentioned in the function ipl_wait_for_working_state().
Or else should we enable the commented line
# cmd = 'sdr elist |grep 'Host Status''
instead of the existing line
cmd = 'sdr elist |grep 'OCC Active''
Either way, we need to submit a patch and fix the same in-order to make the Build Verification Tests work on Firestone Hardware.
With #144 having been implemented, and --scratch-disk
being added earlier, we now have all the pieces in place to (at least optionally) take away the manual setup steps of having to install an OS on the Host along with packages needed for op-test
.
Currently, when you run op-test
, the default suite will run tests in petitboot shell ('skiroot') and then reboot into the default OS and run tests there ('host').
I think we should modify that so that IF --scratch-disk
is specified, after the skiroot suite, we then run the OS install test, boot off that OS and then run the Host suite.
When we put op-test
into automated systems, this then future proofs us from ever having to touch the test machines, as the OS will always end up being the version and state we want it to be. As a side effect, this also gets us (some) kernel backwards compatibility testing.
Even with mambo/qemu we should be able to run many tests, but with more complete ones such as simics, we should be able to run the full set of tests.
Test that petitboot translations are functional
Test that PCI slots have somewhat correct looking slot names.
Being able to run op-test on a simulator (such as mambo, etc) would be great for bringup purposes and testing specific hardware features. This would probably be quite similar to how OpTestQemu works currently.
Testing an OpenPOWER system can involve a myriad of things, and I don't think it's particularly clear what this repository presently does, or intends to do. It would be nice to have more documentation, even if just in the README, to cover things like:
It's awesome to see tests, I'm sure I'll be contributing at some point, but I think there needs to be more clarity around what's going on here.
GARD out each GARDable element (or do some random sampling), check we still boot, and that we report things correctly.
This would be quite useful for Hostboot testing too.
We currently assume IPMI, which is not the case (at least currently) for OpenBMC based systems.
There are couple of issues in executing the framework. As of now command line execution failed, when we gave inputs through command line, It is working only when we fill the inpust in op-ci-setup.xml.
Changes were made with the following issue to use the users environment version of python2
#252
However, this change was missed in op-test script.
git pull
Already up-to-date.
egrep -iHr "/usr/bin/python2" *
op-test:#!/usr/bin/python2
Test that we train PCIe cards as we expect to.
We should verify version strings of what we flash with what we expected to flash to ensure that we're actually running the code we expected to.
I've seen an OpenBMC system "successfully" flash BMC firmware but it not take, so this is a Real World issue.
It would be helpful if op-test framework can generate html test reports,
something in similar lines as below report,
sl.no|test name|status|time taken|detail(link to detailed result)
Some Sample report: http://tungwaiyip.info/software/sample_test_report.html
we expect a certain set of sensors to be present on a system. Currently, the sensors test makes no attempt to ensure the correct sensors exist.
we may want to add tests that:
@stewart-ibm
op-test-framework has all the required backend support to communicate with the openpower system, With the help of these libraries we can easily enable deployment of distros (RHEL, Ubuntu, Centos and Hostos).
This will help both internal and external teams to use the framework for distro deployment.
Advantages :
Run any test at all that confirms NVLink is functioning as expected on machines with NVLink and GPUs
Test the FSP resetReload testcases by below two methods.
Host initiated RR---> putmemproc 300000f8 0x00000000deadbeef
FSP initated RR ----> smgr resetReload
Once it finishes wait for FSP to come up and check below interafaces resumes back or not
ipmi
nvram
sensors
console
surveillance
fsp dumps
OpenBMC image flashing steps are similar as like PNOR image flashing on witherpsoon. And also we need to add for SMC systems.
Hi Saqib,
I am trying to execute the tests related to PNOR and BMC Firmware updation using the Web GUI of the BMC with the functionality coded in the OpTestWeb.py which in turn uses the functions in the source files in the directory op-test-framework/common/util/web/.
So, I have developed wrapper scripts in this framework to invoke the functions in OpTestWeb.py
I am getting the below error when I try to execute the tests.
[root@lop826 bvt]# ./run-op-bvt --bmcip 9.40.193.26 --bmcuser sysadmin --bmcpwd superuser --usernameipmi ADMIN --passwordipmi admin --lparip 9.40.193.27 --lparuser root --lparPasswd passw0rd --hpmimage /root/garrison_820.1618.20160414o.hpm --cfgfiledir ../ci/source/ --imagedir /tmp/ --imagename garrison.pnor --fverbose /tmp/new_garrison ./op-bmc-web-update-bvt.xml
01:47:25 :: Command line: ./run-op-bvt --bmcip 9.40.193.26 --bmcuser sysadmin --bmcpwd superuser --usernameipmi ADMIN --passwordipmi admin --lparip 9.40.193.27 --lparuser root --lparPasswd passw0rd --hpmimage /root/garrison_820.1618.20160414o.hpm --cfgfiledir ../ci/source/ --imagedir /tmp/ --imagename garrison.pnor --fverbose /tmp/new_garrison ./op-bmc-web-update-bvt.xml
26657| 26740|failsumm_log set to: /tmp/root/bvt/2016-04-20-26740/failsumm.log
01:47:25 :: Starting BVT (./op-bmc-web-update-bvt.xml) on 2016-04-20...
01:47:25 :: Log file: /root/run-op-bvt/logs/run-op-bvt.log
26657| 26740|>getXMLFilePath(./op-bmc-web-update-bvt.xml)
26657| 26740| checking for /root/op-test-framework/bvt/./op-bmc-web-update-bvt.xml
26657| 26740|<getXMLFilePath(./op-bmc-web-update-bvt.xml) returning /root/op-test-framework/bvt/./op-bmc-web-update-bvt.xml
26657| 26740|fullbvtxmlfile: /root/op-test-framework/bvt/./op-bmc-web-update-bvt.xml
26657| 26740|run-op-bvt
26657| 26740|xmlvars{machine}:
26657| 26740|bvtxmlfile: ./op-bmc-web-update-bvt.xml
26657| 26740|savedir: /tmp/root/bvt/2016-04-20-26740
26657| 26740|test_id: op-bmc-web-update test_state:
26657| 26740|>runTest(op-bmc-web-update, "OP System BMC and PNOR Firmware update BVT", , op-bmc-web-update.xml)
01:47:25 :: Starting "OP System BMC and PNOR Firmware update BVT" (op-bmc-web-update.xml)
26657| 26740| cmd: run-op-it --fverbose --logpfx op-bmc-web-update --testenv hw --bmcip 9.40.193.26 --bmcuser sysadmin --bmcpwd superuser --usernameipmi ADMIN --passwordipmi admin --lparip 9.40.193.27 --lparuser root --lparPasswd passw0rd --hpmimage /root/garrison_820.1618.20160414o.hpm --cfgfiledir ../ci/source/ --imagedir /tmp/ --imagename garrison.pnor op-bmc-web-update.xml
01:47:25 :: Starting op-bmc-web-update.xml on 2016-04-20...
01:47:25 :: Starting Test: 1
01:47:25 :: Running command: run-bvt-setup --bmcip 9.40.193.26 --bmcuser sysadmin --bmcpwd superuser --usernameipmi ADMIN --passwordipmi admin --cfgfiledir ../ci/source/ --ffdcdir %%ffdcdir%% --imagedir /tmp/ --imagename garrison.pnor --lparip 9.40.193.27 --lparuser root --lparPasswd passw0rd --hpmimage /root/garrison_820.1618.20160414o.hpm
01:47:25 :: OK: command finished successfully
01:47:25 :: Finished Test: 1
01:47:25 :: Starting Test: 2
01:47:25 :: Running command: op-ci-bmc-run "op_bmc_web_update.test_init()"
/root/op-test-framework/ci/source/op_ci_tools.cfg
01:47:25 :: OK: command finished successfully
01:47:25 :: Finished Test: 2
01:47:25 :: Starting Test: 3
01:47:25 :: Running command: op-ci-bmc-run "op_bmc_web_update.bmc_web_pnor_update_hpm()"
/root/op-test-framework/ci/source/op_ci_tools.cfg
ipmitool -H 9.40.193.26 -I lanplus -U ADMIN -P admin chassis power off
Host doesn't have selenium installed
01:48:20 :: ERROR: command failed with exit code 256
01:48:20 :: ERROR: terminating test
26657| 26740| cmd returned 256
01:48:20 :: ERROR: "OP System BMC and PNOR Firmware update BVT" (op-bmc-web-update.xml) finished with one or more errors
26657| 26740|<runTest(op-bmc-web-update, "OP System BMC and PNOR Firmware update BVT", , op-bmc-web-update.xml)
01:48:20 :: ERROR: Finished BVT tests on 2016-04-20 with 1 testcase failures.
All logs and FFDC saved under /tmp/root/bvt/2016-04-20-26740
26657| 26740|>run-op-bvt cleanUp(1)
All logs and FFDC saved under /tmp/root/bvt/2016-04-20-26740
26657| 26740|<run-op-bvt cleanUp(1)
It shows, the error "Host doesn't have selenium installed" which is the code from OpTestConstants.py
ERROR_SELENIUM_HEADLESS = "Host doesn't have selenium installed"
These values are in web_update_hpm function in the source file OpTestWeb.py
So, could you please let me know which all Selenium packages are to be installed on the HOST machines from which I trigger these tests and also the packages required for the target HOST machine.
[console-expect]#date
Mon Jul 16 05:54:28 EDT 2018
[console-expect]#which whoami && whoami
/usr/bin/whoami
root
[console-expect]#echo $?
0
OpTestSystem now running as root
[console-expect]#
[console-expect]#
[console-expect]#cat /proc/version | cut -d ' ' -f 3 | grep openpower; echo $?
1
[console-expect]#
[console-expect]#
[console-expect]#cat /proc/version | cut -d ' ' -f 3 | grep openpower; echo $?
1
OpTestSystem CURRENT DETECTED STATE: 0
OpTestSystem START STATE: 0 (target 1)
Test ran ./op-test -c machine.conf --run testcases.RunHostTest.RunHostTest
host_cmd_file
cat /proc/cmdline
ls -a
uname -r
[SOL Session operational. Use ~? for help]
[console-pexpect]#
[console-pexpect]#^[[H^[[J[console-pexpect]#PS1=[console-pexpect]\#
Shell prompt changed
[console-pexpect]#cat /proc/cmdline
root=UUID=75440ce6-d801-4fc5-b27d-cc3c11dafdc0 ro console=tty0 console=ttyS0,115200 quiet splash crashkernel=2G-4G:320M,4G-32G:512M,32G-64G:1024M,64G-128G:2048M,128G-:4096M@128M
[console-pexpect]#
[console-pexpect]#echo $?
0
# LAST COMMAND EXIT CODE 0 ('0\r\n')
[console-pexpect]#ls -a
. .jenkins .ssh
.. .lesshst .systemtap
01SV860_151_056.img linux-diag-ppc64-diag .targetcli
.bash_history .local .vim
.bashrc .profile .viminfo
.cache [email protected] .wget-hsts
.config .selected_editor .Xauthority
.gnupg sosreport-guest-20180312015742.tar.xz
[console-pexpect]#
[console-pexpect]#echo $?
0
# LAST COMMAND EXIT CODE 0 ('0\r\n')
[console-pexpect]#uname -r
4.15.0-21-generic
[console-pexpect]#
ERROR (4.939s)
======================================================================
ERROR [4.939s]: runTest (testcases.RunHostTest.RunHostTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/jenkins_data/sath/op-test-framework/testcases/RunHostTest.py", line 58, in runTest
con.run_command(line, timeout=self.host_cmd_timeout)
File "/home/jenkins_data/sath/op-test-framework/common/OpTestIPMI.py", line 276, in run_command
exitcode = int(console.before)
ValueError: invalid literal for int() with base 10: ''
----------------------------------------------------------------------
Ran 1 test in 4.939s
FAILED (errors=1)
Generating XML reports...
def host_list_gard_records(self, i_gard_dir):
try:
return self.ssh.run_command(BMC_CONST.SUDO_COMMAND + i_gard_dir + BMC_CONST.LIST_GARD_CMD, timeout=60)
except:
l_errmsg = "Can't clear gard records"
print l_errmsg
raise OpTestError(l_errmsg)
Currently all reports go into a test-reports dir local to the framework install. That may not always be a writeable space by all calling op-test (if installed on a shared LCB, etc..)
It seems to make sense to give the user an option to specify where the output should go. -o/--output for the cmdline?
Perhaps the output location could also be controllable via an env variable so the user doesn't have to put it on the cmdline each time - something like OP_TEST_OUTPUT.
Proposed condition flow
If nothing is set: keep current behavior
If OP_TEST_OUTPUT is given and no cmdline: use OP_TEST_OUTPUT
If cmdline is set, regardless of OP_TEST_OUTPUT set or not: use cmdline
The idea sound good? Any tweaks to switch/variable names? I'll make the changes, just wanted input before doing so.
Hi,
I have executed test_hwmon_driver() from op-test-framework/testcases/OpTestSensors.py on garrison system. Manually I am able to get output for "sudo yes | sensors-detect" but when i tried Checking sensors command functionality with different options... i saw this issue
root@g68L:/home/tyan# sudo sensors -f; echo $?
No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
1
can you please tell me what are the prerequisites to execute above command
bmc: g68.aus.stglabs.ibm.com
lpar: g68L.aus.stglabs.ibm.com
FW_VERSION=2.13.00022
FW_DATE=Apr 14 2016
FW_BUILDTIME=16:53:41 CDT
FW_DESC=8335-GTB PASS1 89a207b gar.820.160414a
FW_PRODUCTID=1
FW_RELEASEID=RR9
FW_CODEBASEVERSION=2.X
-bash-4.1$ ipmitool -H g68 -I lanplus -U ADMIN -P admin fru print 43
Product Manufacturer : 2c80
Product Name : 0b
Product Part Number : 18KSF1G72PDZ-1G6N1
Product Version : 314e
Product Serial : 0f5f0f9f
-bash-4.1$
I get the following error trying to do even a simple -h on 14.04:
albertj@opt-ub1404:~/phqf-install$ ./op-test -h
Traceback (most recent call last):
File "./op-test", line 40, in <module>
import OpTestConfiguration
File "/home/albertj/phqf-install/OpTestConfiguration.py", line 8, in <module>
from common.OpTestFSP import OpTestFSP
File "/home/albertj/phqf-install/common/OpTestFSP.py", line 40, in <module>
from OpTestASM import OpTestASM
File "/home/albertj/phqf-install/common/OpTestASM.py", line 47, in <module>
ssl._create_default_https_context = ssl._create_unverified_context
AttributeError: 'module' object has no attribute '_create_unverified_context'
According to the internet, this wasn't supported until python 2.7.9 and 14.04 is 2.7.6. https://stackoverflow.com/questions/28228214/ssl-module-object-has-no-attribute-sslcontext
If I comment the line out, everything in my scenarios runs fine - but I don't use the OpTestASM path. It doesn't seem like there is a easy work around on 2.7.6. If OpTestASM is not intended to be supported on 14.04, could we put this in a version check to only include on python 2.7.9 or later?
We should have several DTS sensors present for each CPU core.
We should verify that these sensors exist for each core as part of the sensors test.
If I install my public key on a BMC, I see errors like the following when using op-test to flash skiboot.
As you can see it uses -o PubkeyAuthentication=no
when connecting to the box to execute shell commands, but not in the rsync ssh transport, which causes it to get confused when it doesn't see a password prompt.
sshpass -p 0penBmc ssh -p 22 -l root wcapibmc4.aus.stglabs.ibm.com -o PubkeyAuthentication=no -q -o 'UserKnownHostsFile=/dev/null' -o 'StrictHostKeyChecking=no'
PS1=\[PEXPECT\]#
PS1=\[PEXPECT\]#
root@witherspoon:~# PS1=\[PEXPECT\]#
[PEXPECT]#systemctl restart mboxd.service
[PEXPECT]#echo $?
0
[PEXPECT]#rsync -P -v -e "ssh -k -o StrictHostKeyChecking=no" /home/ajd/code/skiboot/skiboot.lid.xz.stb [email protected]:/tmp
skiboot.lid.xz.stb
377,240 100% 359.10MB/s 0:00:00 (xfr#1, to-chk=0/1)
sent 2,252 bytes received 3,269 bytes 1,226.89 bytes/sec
total size is 377,240 speedup is 68.33
ERROR
======================================================================
ERROR: runTest (testcases.OpTestFlash.OpalLidsFLASH)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/ajd/code/op-test-framework/testcases/OpTestFlash.py", line 501, in runTest
self.cv_BMC.image_transfer(self.skiboot)
File "/home/ajd/code/op-test-framework/common/OpTestOpenBMC.py", line 1060, in image_transfer
self.bmc.image_transfer(i_imageName)
File "/home/ajd/code/op-test-framework/common/OpTestBMC.py", line 139, in image_transfer
rsync.expect('assword: ')
File "/usr/lib/python2.7/dist-packages/pexpect/spawnbase.py", line 321, in expect
timeout, searchwindowsize, async)
File "/usr/lib/python2.7/dist-packages/pexpect/spawnbase.py", line 345, in expect_list
return exp.expect_loop(timeout)
File "/usr/lib/python2.7/dist-packages/pexpect/expect.py", line 105, in expect_loop
return self.eof(e)
File "/usr/lib/python2.7/dist-packages/pexpect/expect.py", line 50, in eof
raise EOF(msg)
EOF: End Of File (EOF). Exception style platform.
<pexpect.pty_spawn.spawn object at 0x7f237736b490>
command: /usr/bin/rsync
args: ['/usr/bin/rsync', '-P', '-v', '-e', 'ssh -k -o StrictHostKeyChecking=no', '/home/ajd/code/skiboot/skiboot.lid.xz.stb', '[email protected]:/tmp']
buffer (last 100 chars): ''
before (last 100 chars): 'ent 2,252 bytes received 3,269 bytes 1,226.89 bytes/sec\r\ntotal size is 377,240 speedup is 68.33\r\n'
after: <class 'pexpect.exceptions.EOF'>
match: None
match_index: None
exitstatus: None
flag_eof: True
pid: 10625
child_fd: 7
closed: False
timeout: 30
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: <open file '<stdout>', mode 'w' at 0x7f2379d74150>
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
searcher: searcher_re:
0: re.compile("assword: ")
----------------------------------------------------------------------
Ran 4 tests in 15.997s
FAILED (errors=1, skipped=3)
DVT has some required packages to be installed on RHEL. You added several using this change:
4d4b78c
After doing a test automated install of Pegas 1.1 I noticed these packages were missing and should be included as part of the automated RHEL installation post processing for DVT. Thanks!
nano
sysstat
rpm-build
gcc-gfortran
hdparm
tk
tcsh
lsof
python-devel
dtc
libX11-devel
mesa-libGLU-devel
freeglut-devel
ntpdate
We currently just check the sysfs file, we should check ppc64_cpu --frequency, as it does an actual measurement.
Need adjust code to use pwd/cwd or some other mechanism to find it's required tools. Currently it needs ./ to be your PATH value to run.
Explanation of the Issue:
Execution of /op-test-framework/testcases/OpTestI2Cdriver.py failed and test case got hung when executing the command 'sudo i2cdetect -y 6; echo $?' in the test flow
BMC Info
cat /proc/ractrends/Helper/FwInfo
FW_VERSION=2.16.104497
FW_DATE=Oct 27 2016
FW_BUILDTIME=17:27:08 CDT
FW_DESC=8348 P2 SRC RR9 10272016
FW_PRODUCTID=1
FW_RELEASEID=RR9
FW_CODEBASEVERSION=2.X
HOST OS info:
cat /etc/os-release
NAME="Ubuntu"
VERSION="16.10 (Yakkety Yak)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.10"
VERSION_ID="16.10"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="http://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=yakkety
UBUNTU_CODENAME=yakkety
root@habcap10p1:/home/tyan#
Log info:
Attached opal, dmesg, syslogs collected from the HOST OS
dmesg_logs.txt
opalmsg_logs.txt
sys_logs.txt
Customers are coming to us to create Marvell RAID VD, so we have working test case implemented for Host OS (RHEL & Ubuntu) in avocado-misc framework. avocado-framework-tests/avocado-misc-tests#564
We would like create a same test case to work on petitboot shell.
Let us know your opinion on having this test case on op-test-framework...
Thank you,
Naveed
i.e. once open-power/skiboot#163 is fixed
At best, we can get people to run a single command to run a test suite to confirm conformance.
Currently, the default test suite is not nearly complete enough to ensure that.
Currently we have all dbus interfaces for REST API using /org/openbmc this will soon depricated. We need to change it to /xyz/openbmc_project/ . and check existing API is working fine or not.
Validate the properties against what OPAL actually reserved the regions like size and names.
OpenPower systems:
Homer Image 4MB
SLW Image 1MB
OCC Common Area 8MB
cat /sys/firmware/opal/msglog | grep -i size
[ 45.078419667,7] HOMER Image at 0x1ffd800000 size 4MB
[ 45.078428921,7] SLW Image at 0x1ffda00000 size 1MB
[ 45.078437864,7] OCC Common Area at 0x1fff800000 size 8MB
FSP systems:
SLW 1MB
OCC 8MB
cat /sys/firmware/opal/msglog | grep -i size
[ 42.301531454,7] SLW Image at 0x2ffe600000 size 1MB
[ 42.301544504,7] OCC Common Area at 0x2000800000 size 1MB
Resrved names property
/proc/device-tree# cat reserved-names
ibm,slw-imageibm,slw-imageibm,slw-imageibm,slw-imageibm,hbrt-code-imageibm,hbrt-target-imageibm,hbrt-vpd-imageibm,firmware-stacksibm,firmware-dataibm,firmware-heapibm,firmware-codeibm,firmware-allocs-memory@2800000000ibm,firmware-allocs-memory@2000000000ibm,firmware-allocs-memory@1000000000ibm,firmware-allocs-memory@0
/proc/device-tree# hexdump -C reserved-ranges
00000000 00 00 00 2f fe 60 00 00 00 00 00 00 00 10 00 00 |.../�`..........|
00000010 00 00 00 2f fe 20 00 00 00 00 00 00 00 10 00 00 |.../� ..........|
00000020 00 00 00 2f fd e0 00 00 00 00 00 00 00 10 00 00 |.../��..........|
00000030 00 00 00 2f fd a0 00 00 00 00 00 00 00 10 00 00 |.../��..........|
00000040 00 00 00 2f fd 51 00 00 00 00 00 00 00 19 00 00 |.../�Q..........|
00000050 00 00 00 2f fd 6a 00 00 00 00 00 00 00 06 00 00 |.../�j..........|
00000060 00 00 00 2f fd 70 00 00 00 00 00 00 00 10 00 00 |.../�p..........|
00000070 00 00 00 00 31 c0 00 00 00 00 00 00 02 3a 00 00 |....1�.......:..|
00000080 00 00 00 00 31 00 00 00 00 00 00 00 00 c0 00 00 |....1........�..|
00000090 00 00 00 00 30 30 00 00 00 00 00 00 00 d0 00 00 |....00.......�..|
000000a0 00 00 00 00 30 00 00 00 00 00 00 00 00 30 00 00 |....0........0..|
000000b0 00 00 00 28 00 00 00 00 00 00 00 00 00 cc 02 00 |...(.........�..|
000000c0 00 00 00 20 00 00 00 00 00 00 00 00 01 d8 02 00 |... .........�..|
000000d0 00 00 00 10 00 00 00 00 00 00 00 00 00 d8 02 00 |.............�..|
000000e0 00 00 00 00 39 c0 00 00 00 00 00 00 00 d8 02 00 |....9�.......�..|
000000f0
Actual Reserved regions:
[ 22.169629796,5] Reserved regions:
[ 22.169630777,5] 0x002ffe600000..002ffe6fffff : ibm,slw-image
[ 22.169634046,5] 0x002ffe200000..002ffe2fffff : ibm,slw-image
[ 22.169637262,5] 0x002ffde00000..002ffdefffff : ibm,slw-image
[ 22.169640328,5] 0x002ffda00000..002ffdafffff : ibm,slw-image
[ 22.169643433,5] 0x002ffd510000..002ffd69ffff : ibm,hbrt-code-image
[ 22.169647093,5] 0x002ffd6a0000..002ffd6fffff : ibm,hbrt-target-image
[ 22.169650970,5] 0x002ffd700000..002ffd7fffff : ibm,hbrt-vpd-image
[ 22.169654569,5] 0x000031c00000..000033f9ffff : ibm,firmware-stacks
[ 22.169658241,5] 0x000031000000..000031bfffff : ibm,firmware-data
[ 22.169661620,5] 0x000030300000..000030ffffff : ibm,firmware-heap
[ 22.169664945,5] 0x000030000000..0000302fffff : ibm,firmware-code
[ 22.169668365,5] 0x002800000000..002800cc01ff : ibm,firmware-allocs-memory@2800000000
[ 22.169673871,5] 0x002000000000..002001d801ff : ibm,firmware-allocs-memory@2000000000
[ 22.169679361,5] 0x001000000000..001000d801ff : ibm,firmware-allocs-memory@1000000000
[ 22.169684887,5] 0x000039c00000..00003a9801ff : ibm,firmware-allocs-memory@0
This way we could, under most normal circumstances, verify SSH keys if we wanted
Boot old kernels, test things still work.
This could be best accomplished by netbooting kernel+initramfs we serve directly from op-test.
Any plan to add support for Junit test results output?, which would be helpful for jenkins report
We have a few warnings in the device tree that dtc picks up:
Warning (reg_format): "reg" property in /ibm,pcie-slots/root-complex@0,4/switch-up@10b5,8725/down-port@c/builtin@10de,1db1 has invalid length (8 bytes) (#address-cells == 2, #size-cells == 1)
Warning (reg_format): "reg" property in /ibm,pcie-slots/root-complex@0,4/switch-up@10b5,8725/down-port@a/builtin@10de,1db1 has invalid length (8 bytes) (#address-cells == 2, #size-cells == 1)
Warning (reg_format): "reg" property in /ibm,pcie-slots/root-complex@0,4/switch-up@10b5,8725/down-port@b/builtin@10de,1db1 has invalid length (8 bytes) (#address-cells == 2, #size-cells == 1)
We could fix all the warnings then add a test for this
It would be helpful if we can take a optional param as disk and that can be used to boot
and run further tests, will be helpful incase of multiboot system and we can control where we want to run
tests which would help/save time.
there is a method in common/OpTestInstallUtil.py
- set_bootable_disk()
but that would require a reboot again, if we can have a handle in petitboot and have a mechanism to boot into specific disk would be more helpful.
Let me know incase if we have that already implemented. Thanks.
We get a bunch of failures from FWTS, we should investigate each of those so that we have 0 test failures.
use set_poweron_time tool to use test this feature.
======================================================================
ERROR [424.678s]: runTest (testcases.OpTestHMIHandling.HMI_TFMR_ERRORS)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/pridhiviraj/new_optests/op-test-framework/testcases/OpTestHMIHandling.py", line 483, in runTest
self._testHMIHandling(BMC_CONST.TFMR_ERRORS)
File "/home/pridhiviraj/new_optests/op-test-framework/testcases/OpTestHMIHandling.py", line 246, in _testHMIHandling
self._testTFMR_Errors(BMC_CONST.TFMR_SPURR_PARITY_ERROR)
File "/home/pridhiviraj/new_optests/op-test-framework/testcases/OpTestHMIHandling.py", line 410, in _testTFMR_Errors
console.run_command("dmesg -C")
File "/home/pridhiviraj/new_optests/op-test-framework/common/OpTestIPMI.py", line 312, in run_command
return self.util.run_command(self, command, timeout, retry)
File "/home/pridhiviraj/new_optests/op-test-framework/common/OpTestUtil.py", line 541, in run_command
raise cf
CommandFailed: Command 'dmesg -C' exited with -1.
Output:
['0']
======================================================================
ERROR [11.550s]: runTest (testcases.OpTestHMIHandling.PROC_RECOV_DONE)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/pridhiviraj/new_optests/op-test-framework/testcases/OpTestHMIHandling.py", line 496, in runTest
self._testHMIHandling(BMC_CONST.HMI_PROC_RECV_DONE)
File "/home/pridhiviraj/new_optests/op-test-framework/testcases/OpTestHMIHandling.py", line 205, in _testHMIHandling
l_con.run_command("uname -a")
File "/home/pridhiviraj/new_optests/op-test-framework/common/OpTestIPMI.py", line 312, in run_command
return self.util.run_command(self, command, timeout, retry)
File "/home/pridhiviraj/new_optests/op-test-framework/common/OpTestUtil.py", line 541, in run_command
raise cf
CommandFailed: Command 'uname -a' exited with -1.
Output:
['0']
======================================================================
ERROR [11.143s]: runTest (testcases.OpTestHMIHandling.PROC_RECV_ERROR_MASKED)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/pridhiviraj/new_optests/op-test-framework/testcases/OpTestHMIHandling.py", line 500, in runTest
self._testHMIHandling(BMC_CONST.HMI_PROC_RECV_ERROR_MASKED)
File "/home/pridhiviraj/new_optests/op-test-framework/testcases/OpTestHMIHandling.py", line 205, in _testHMIHandling
l_con.run_command("uname -a")
File "/home/pridhiviraj/new_optests/op-test-framework/common/OpTestIPMI.py", line 312, in run_command
return self.util.run_command(self, command, timeout, retry)
File "/home/pridhiviraj/new_optests/op-test-framework/common/OpTestUtil.py", line 541, in run_command
raise cf
CommandFailed: Command 'uname -a' exited with -1.
Output:
['0']
======================================================================
ERROR [11.147s]: runTest (testcases.OpTestHMIHandling.TOD_ERRORS)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/pridhiviraj/new_optests/op-test-framework/testcases/OpTestHMIHandling.py", line 487, in runTest
self._testHMIHandling(BMC_CONST.TOD_ERRORS)
File "/home/pridhiviraj/new_optests/op-test-framework/testcases/OpTestHMIHandling.py", line 205, in _testHMIHandling
l_con.run_command("uname -a")
File "/home/pridhiviraj/new_optests/op-test-framework/common/OpTestIPMI.py", line 312, in run_command
return self.util.run_command(self, command, timeout, retry)
File "/home/pridhiviraj/new_optests/op-test-framework/common/OpTestUtil.py", line 541, in run_command
raise cf
CommandFailed: Command 'uname -a' exited with -1.
Output:
['0']
Ignore WARNING: Test 4, cpu 0 has claimed frequency of 3800000, higher than max freq of 3268000
with fwts prior to version that includes this patch: https://lists.ubuntu.com/archives/fwts-devel/2018-April/010318.html
since we have infra in the FWTS test we should ignore this on older fwts without this ptach.
I grabed the latest code using git pull:
git pull
remote: Counting objects: 25, done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 25 (delta 15), reused 20 (delta 15), pack-reused 4
Unpacking objects: 100% (25/25), done.
From https://github.com/open-power/op-test-framework
2a64f01..792fa18 master -> origin/master
+ bad163f...85329d0 gh-pages -> origin/gh-pages (forced update)
Updating 2a64f01..792fa18
Fast-forward
common/OpTestHost.py | 2 +-
common/OpTestInstallUtil.py | 3 +++
osimages/rhel/rhel.ks | 18 ++++++++++++++++++
testcases/InstallRhel.py | 2 +-
testcases/InstallUbuntu.py | 2 +-
5 files changed, 24 insertions(+), 3 deletions(-)
I was able to successfully use the op-test tool to automatically install Pegas 1.1 onto my Tuleta system using command:
./op-test -c machine.conf --host-scratch-disk /dev/disk/by-id/scsi-35000cca01d0e4d60 --os-repo ftp://MY_ID:[email protected]/redhat/release_cds/RHEL-ALT-7.5-GA/Server/ppc64le/os/ --run testcases.InstallRhel.InstallRhel
Contents of machine.conf
[op-test]
bmc_type=FSP
bmc_ip=tul217fp.aus.stglabs.ibm.com
bmc_username=dev
bmc_password=FipSdev
bmc_passwordipmi=PASSW0RD
host_ip=9.41.165.230
host_user=root
host_password=f00tball
host_gateway=9.41.164.1
host_dns=9.3.1.200
host_submask=255.255.252.0
host_mac=6C:AE:8B:02:A8:78
Full install log output:
http://ausgsa.ibm.com/gsa/ausgsa/projects/a/autoipl/status/tul217p1.180622.102131.op-test.os_install.log
However, after the installation finished I logged into the system and noticed the prompt was:
[root@9 ~]#
This is because the hostname is set to the IP addresss
cat /etc/hostname
9.41.165.230
Our IPL DVT automation relies on the prompt of the LPAR to be set to the DNS Hostname. In this case it should be:
[root@tul217p1 ~]#
I was able to manually set this by modifying /etc/hostname to be:
cat /etc/hostname
tul217p1.aus.stglabs.ibm.com
Can op-test OS install scripts be updated to set the hostname value to the DNS hostname instead of the IP address?
I tried to work around this by using the DNS hostname in machine.conf file for the key host_ip:
host_ip=tul217p1.aus.stglabs.ibm.com
But that resulted in a failure as the tool expects that value to be an IP address.
# LAST COMMAND EXIT CODE 0 ('0\r\n')
[console-pexpect]#ifconfig enP3p9s0f0 tul217p1.aus.stglabs.ibm.com netmask 255.255.252.0
ifconfig: bad address 'tul217p1.aus.stglabs.ibm.com'
[console-pexpect]#echo $?
1
# LAST COMMAND EXIT CODE 1 ('1\r\n')
ERROR
======================================================================
ERROR: runTest (testcases.InstallRhel.InstallRhel)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/fspmount/dlshaw/tul217fp/op-test-framework/testcases/InstallRhel.py", line 65, in runTest
my_ip = OpIU.get_server_ip()
File "/fspmount/dlshaw/tul217fp/op-test-framework/common/OpTestInstallUtil.py", line 141, in get_server_ip
self.assign_ip_petitboot()
File "/fspmount/dlshaw/tul217fp/op-test-framework/common/OpTestInstallUtil.py", line 125, in assign_ip_petitboot
self.console.run_command(cmd)
File "/fspmount/dlshaw/tul217fp/op-test-framework/common/OpTestIPMI.py", line 331, in run_command
raise CommandFailed(command, res, exitcode)
CommandFailed: Command 'ifconfig enP3p9s0f0 tul217p1.aus.stglabs.ibm.com netmask 255.255.252.0' exited with 1.
Output:
["ifconfig: bad address 'tul217p1.aus.stglabs.ibm.com'"]
----------------------------------------------------------------------
Ran 1 test in 182.111s
FAILED (errors=1)
[console-pexpect]#
Full Log:
http://ausgsa.ibm.com/gsa/ausgsa/projects/a/autoipl/status/tul217p1.180622.101458.op-test.os_install.log
Thanks
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.