openmainframeproject / qclib Goto Github PK

                      qclib (Query Capacity Library)
                      ==============================

qclib provides a C API for extraction of system information for Linux on IBM Z.

For instance, it will provide the number of CPUs
  * on the machine (CEC, Central Electronic Complex) layer
  * on the PR/SM (Processor Resource/Systems Manager) layer, i.e. visible to
    LPARs, including LPAR groups
  * in z/VM hosts, guests and CPU pools
  * in KVM hosts and guests
This allows calculating the upper limit of CPU resources a highest level guest
can use.
E.g.: If an LPAR on a z13 provides 4 CPUs to a z/VM hypervisor, and the
hypervisor provides 8 virtual CPUs to a guest, qclib can be used to retrieve
all of these numbers, and it can be concluded that not more capacity than 4
CPUs can be used by the software running in the guest.

qclib uses various interfaces and provides this data through a common and
consistent API (Application Programming Interface), using information provided
by:
  * STSI (Store System Information) instruction - for more details, refer to
    the z/Architecture Principles of Operation (SA22-7832)
  * STHYI (Store Hypervisor Information) instruction as provided by a z/VM
    hypervisor - for more information, refer to z/VM: V6R3 CP Programming
    Service (SC24-6179), chapter 'Store Hypervisor Information (STHYI)
    Instruction'.
  * hypfs file system - for more information, refer to 'Device Drivers,
    Features, and Commands', chapter 'S/390 hypervisor file system'.
  * Firmware and other interfaces as made available through sysfs. For more
    information, refer to 'Device Drivers, Features, and Commands', chapter
    'Identifying the z Systems hardware'.

Please refer to:
   http://www.ibm.com/developerworks/linux/linux390/qclib.html
for the latest available version.


Usage
=====
See query_capacity.h for details on how to use the API, and qc_test.c for a
sample program.


Requirements
============
See query_capacity.h for details.


Build
=====
Use the following 'make' targets:
  * 'all' (default): Build static and dynamic libraries, as well as respective
               sample programs 'qc_test' (statically linked) and 'qc_test-sh'
               (dynamically linked).
  * 'test': Build and run the statically linked test program qc_test.
               Note: Requires a static version of glibc, which some
                     distributions do not install per default.
  * 'test-sh': Build and run the dynamically linked test program qc_test.


API Documentation
=================
All documentation is available in file query_capacity.h.
If you have doxygen 1.8.6 (or higher) installed, you will find the
documentation in subdirectory html, after using 'make doc'.


License
=======
See enclosed file LICENSE for details.


Bug Reports
===========
See section 'Code Contributions'.


Code Contributions
==================
Code contributions will not be accepted for qclib.
Therefore, please do not send DIFFs or code-snippets. Thank you!
If you want to report bugs or suggest enhancements, please contact:
   [email protected]
and put "[qclib]" as the first word in the subject line of your mail.
For bug reports, at a minimum attach a log file and a dump (see QC_DEBUG as
described in query_capacity.h or, yet better, use the qc_dump utility), and
describe the scenario in which you observed the bug, so that the problem can
be reproduced.
For enhancements, please describe the proposed change and its benefits.


Release History:
================

2.2.0 (2020-09-09)
    Changes:
    - Add new tools zname and zhypinfo

    Bug fixes:
    - KVM hosts in mixed mode LPARs indicated both, CPs and IFLs, while only
      CPs are actually used

2.1.0 (2020-04-20)
    Changes:
    - New attributes in support of secure boot in all final layers:
         * qc_has_secure
         * qc_secure
      Note: Changed src column indicator from 'o' to 'F' to summarize
            firmware-related data
    - Recognize IBM z15 midrange models

    Bug fixes:
    - qc_open(): Memory leaks on errors

2.0.1 (2020-01-07)
    Changes:
    - Retry up to three times when a live guest migration is detected

    Bug fixes:
    - Attribute qc_num_ziip_threads in CEC layer was not correctly set.
    - Do not account zIIPs in layer QC_LAYER_TYPE_ZOS_ZCX_SERVER for
      qc_num_cpu_* attributes.
    - qc_dump: Handle non-writable /tmp

2.0.0 (2019-11-11)
    Changes:
    - Add support for z/OS Container Extensions (zCX)
    - New attributes in layer CEC:
         * qc_type_name
         * qc_type_family
         * qc_lic_identifier
    - qc_test: Reworked output for subtle consistency improvements.
    - Replaced attribute qc_hardlimit_consumption with
      qc_limithard_consumption. Use CONFIG_V1_COMPATIBILITY for previous
      version.
    - Require CONFIG_DUMP_READING in query_capacity.h to allow running from a
      dump. Disabled by default.
    - Disabled v1 compatibility functionality per default. To re-enable,
      activate CONFIG_V1_COMPATIBILITY in query_capacity.h.

1.4.1 (2018-06-25)
    Bug fixes:
    - qc_dump: Don't abort the dump in case qc_test fails.
    - Attributes qc_cp_weight_capping and qc_ifl_weight_capping were set even
      when initial capping was not set in the LPAR's activation profile.

1.4.0 (2018-04-10)
    Changes:
    - Added SMT support by properly differentiating between cores and CPUs.
      I.e. switched from qc_num_cpu_* to qc_num_core_* attributes in layers
      CEC, LPAR, ZVM_HYPERVISOR and KVM_HYPERVISOR.
      NOTE: qc_num_cpu_* attributes remain to be valid in these cases to
            preserve backwards compatibility for now. This will be removed in
            one of the next releases! It is recommended to switch to the new
            attributes _now_ and test with CONFIG_V1_COMPATIBILITY disabled!
    - Added new attributes qc_num_threads_cp and qc_num_threads_ifl to layers
      CEC, LPAR and ZVM_HYPERVISOR.
    - Deprecated attribute qc_mobility_eligible (remains valid for now) and
      replaced with qc_mobility_enabled to match z/VM terminology. Likewise
      switched QC_LAYER_TYPE_ZVM_CPU_POOL to QC_LAYER_TYPE_ZVM_RESOURCE_POOL.
    - Moved build customization defines (e.g. CONFIG_V1_COMPATIBILITY) to
      query_capacity.h.
    - Don't build with textual hypfs per default anymore due to unrecoverable
      issues (see section 'Bug fixes'). Since all Linux distributions ship
      with debugfs (providing binary hypfs support), overriding textual hypfs,
      for years, this change will hardly ever be noticable. Enable define
      CONFIG_TEXTUAL_HYPFS in query_capacity.h to revert.
      Note that textual hypfs support will be removed in a future release.

    Bug fixes:
    - Added an exception to consistency check to ignore inconsistencies between
      textual hypfs and STHYI for attributes qc_num_cp_total and
      qc_num_ifl_total in the LPAR layer.
      Background: Textual hypfs cannot tell whether a core is configured or
                  not. It therefore reports all cores as configured, which can
                  be wrong.

1.3.1 (2018-01-18)
    Bug fixes:
    - Security: Fix PATH attack vulnerability when dumping (see QC_DEBUG=2)
    - STHYI: Add fallback for pre-glibc 2.16 (not using getauxval())
    - Handle mismatching STHYI and /proc/sysinfo layer counts
    - On LPAR, fix incomplete dump of binary hypfs when textual hypfs is mounted

1.3.0 (2017-10-27)
    Changes:
    - Added STHYI support in LPAR
    - Added new env variable QC_DEBUG_FILE (see qc_open())
      Note: Failure to open a file for logging is now treated as a fatal error
    - Added script qc_dump to collect debug data in a standardized manner
    - Added attributes qc_layer_uuid and qc_layer_extended_name to LPAR layer
    - /proc/sysinfo parsing: Switch from "KVM/Linux" to the less strict "KVM"
      to detect KVM systems
    - Detect unregistered and closed handles
    - Makefile: Compile SONAME into shared library

    Bug fixes:
    - STHYI: Properly support cc==3&&rc==4 as introduced in APAR VM65419
    - Logs: Fix month in timestamp (was off by 1)
    - qc_test: Fix flags for qc_layer_name in QC_LAYER_TYPE_ZVM_HYPERVISOR

1.2.0 (2016-06-10)
    Changes:
    - Removed source [S] for attributes qc_num_cpu_dedicated and
      qc_num_cpu_shared in LPAR layer for consistency
    - Retrieve qc_layer_name in CEC layer from OCF
    - Extended hypfs usage to provide more CP, IFL and CPU counts for
      CEC and LPAR layers, as well as capping information for LPAR group
      and LPAR layers
    - Added attributes for IFLs, CPs and CPUs for KVM hypervisor and guest
      layers
    - Added support for LPAR Groups
    - Added new attribute qc_prorated_core_time
    - Fill qc_num_cp_total and qc_num_ifl_total in LPAR layer from STHYI
    - Consistency checks: Detect inconsistent values across data sources
    - Documentation improvements

    Bug fixes:
    - Fixed crash when setting QC_USE_DUMP to an invalid directory and
      QC_DEBUG=1
    - Fixed reset of debug level when QC_CHECK_CONSISTENCY is invalid
    - Display all values in attribute qc_partition_char in case of multiple
    - Set qc_cp_dispatch_type in presence of CPs only

1.1.0 (2016-02-15)
    Changes:
    - Makefile: Added targets 'clean' and 'install'
    - qc_test: Support command line options
    - Performance improvements
    - Consistency checks: Turned into a run-time option. Disabled per
      default, enabled in qc_test

    Bug fixes:
    - Makefile: Fixed breakages, including when run in verbose mode
    - Fixed handling of hostnames with <8 characters in presence of hypfs
    - If no SSI cluster was present, attribute qc_cluster_name was set
      to an empty string instead of being left undefined
    - Consistency checks: Fixed wrong positive
    - Fixed source indicators in log

1.0.0 (2015-08-28)
    Changes:
    - Introduced new API, replacing the previous one
    - Renamed the following attributes for consistency:
        * qc_container_name became qc_layer_name
        * qc_entity_* became qc_layer_*
    - Introduced the following new attributes for a numeric representation
      of the respective string attributes:
        * qc_layer_type_num      (alternative to qc_layer_type)
        * qc_layer_category_num  (alternative to qc_layer_category)
        * qc_partition_char_num  (alternative to qc_partition_char)
        * qc_capping_num         (alternative to qc_capping)
    - Removed/renamed the following attributes, since they were duplicates of
      other layers' content:
        * In layer type QC_LAYER_TYPE_ZVM_GUEST: Removed qc_hyp_*, qc_pool_*
          qc_system_identifier, qc_cluster_name, qc_control_program_id,
          qc_adjustment, and qc_hardlimit_consumption
        * In layer type QC_LAYER_TYPE_ZVM_CPU_POOL: Removed qc_hyp_*,
          qc_system_identifier, qc_cluster_name, qc_hardlimit_consumption, and
          renamed qc_pool_* to qc_*
    - Added support for KVM hypervisor
    - Added logging facility
    - Added dump support: Capability to create and run on dumps
    - Added autodump support: Create dumps on errors only
    - Added doxygen support for API description in query_capacity.h
    - Added support for hypfs mounted at arbitrary locations
    - Added support for binary hypfs API (requires RHEL6.1 and SLES11.2 or
      higher)
    - Added detection of Live Guest Migration while retrieving data
    - Handled NULL pointer arguments in all API calls
    - Reported errors as such when occurring while searching for capacity
      information

    Bug fixes:
    - Handled file access errors
    - Enabled attributes that were incorrectly indicated as not present
    - Fixed qc_get_num_layers() to return the number of layers (as documented),
      not the highest index
    - Fixed race by reading /proc/sysinfo only once
    - Only set qc_ifl_dispatch_type in presence of IFLs (as intended)

0.9.2
    Bug fixes:
    - Fixed memory leaks

0.9.1
    Bug fixes:
    - Fixed crash with more than 1 layers of nested z/VM virtualization
    - Fixed crash on 1st layer z/VM systems with hypfs
    - Fix: Information from /proc/sysinfo was collected in wrong sequence
           with more than 1 layers of nested virtualization
    - Fixed left open file handles in hypfs parsing code.
    - Added consistency check for hypfs

0.9.0
    Initial version



Copyright IBM Corp. 2013, 2018