microsoft / hidtools

Human Interface Device (HID) Tools for Windows and Devices

Project

This repo is the central location of Microsoft HID Tools. Currently, the only tool available is Waratah. The underlying HidSpecification and HidEngine libaries are available via Nuget

https://www.nuget.org/packages/Microsoft.HidTools.HidSpecification

https://www.nuget.org/packages/Microsoft.HidTools.HidEngine

Or from the package manager console:

> Install-Package Microsoft.HidTools.HidEngine -Version 1.3.0

Waratah

What A Really Awesome Tool for Authoring HIDs

Waratah is a HID descriptor composition tool. It offers a high-level of abstraction, eliminates common errors (by design), and optimizes the descriptor to reduce byte size. It implements the HID 1.11 specification so developers don't have to.

It is expected to be used by device firmware authors during device bring-up.

See Wiki for more details

Overview

Waratah uses a TOML-like hierarchical language of sections and keys to represent a HID Report Descriptor (Note: There is currently no support for HID Physical Descriptors). This can then be compiled to to either a simple plain-text format, or a C++ header file suitable for ingestion into device firmware.

Waratah is NOT a direct dt.exe replacement. dt.exe permits the use of specialized items (e.g. Push/Pop) and non-optimal practices (e.g. ReportSize larger than LogicalMinimum/Maximum). There are also known bugs in dt.exe, that have not been replicated in Waratah. It is reasonable to think of Waratah as high-level compiler like C and dt.exe as an assembler. No further development of dt.exe is planned.

Features:

• Human-readable text for easy composition and meaningful source-control management.
• Inbuilt with all public Usages from HID Usage Table
• Composition of custom Vendor Usages.
• Inbuilt with all defined Units.
• Support for composition of new named Units.
• Comprehensive error messaging with line-level blame.
• C++ struct generation with context-aware variable-name generation.
• Optimistic descriptor optimization (redundant global items removed, and items combined).
• Report summary with itemized Id/Type/Size.
• Context-aware integer types bounds validation (ReportId, UsageId,...).
• Context-aware Usage type validation.
• Automatic generation of ReportIds.
• Automatic 8bit report alignment (padding inserted as needed).
• Automatic collection termination.
• Report Item size inferred from LogicalMin/Max (or vice-versa).
• Overspanning preventation, guaranteeing no item spans more than 4bytes (padding inserted as needed).
• Validate conditionally invalid Report Flags.
• Inline Usage transformation.
• Usage Range validation.

Example: Simple mouse with 3 buttons.

[[applicationCollection]]
usage = ['Generic Desktop', 'Mouse']

    [[applicationCollection.inputReport]]

        [[applicationCollection.inputReport.physicalCollection]]
        usage = ['Generic Desktop', 'Pointer']

            [[applicationCollection.inputReport.physicalCollection.variableItem]]
            usage = ['Generic Desktop', 'X']
            sizeInBits = 8
            logicalValueRange = 'maxSignedSizeRange'
            reportFlags = ['relative']

            [[applicationCollection.inputReport.physicalCollection.variableItem]]
            usage = ['Generic Desktop', 'Y']
            sizeInBits = 8
            logicalValueRange = 'maxSignedSizeRange'
            reportFlags = ['relative']

            [[applicationCollection.inputReport.physicalCollection.variableItem]]
            usageRange = ['Button', 'Button 1', 'Button 3']
            logicalValueRange = [0, 1]

Plain-text output

05-01....UsagePage(Generic Desktop[1])
09-02....UsageId(Mouse[2])
A1-01....Collection(Application)
85-01........ReportId(1)
09-01........UsageId(Pointer[1])
A1-00........Collection(Physical)
09-30............UsageId(X[48])
09-31............UsageId(Y[49])
15-80............LogicalMinimum(-128)
25-7F............LogicalMaximum(127)
95-02............ReportCount(2)
75-08............ReportSize(8)
81-06............Input(Data, Variable, Relative, NoWrap, Linear, PreferredState, NoNullPosition, BitField)
05-09............UsagePage(Button[9])
19-01............UsageIdMin(Button 1[1])
29-03............UsageIdMax(Button 3[3])
15-00............LogicalMinimum(0)
25-01............LogicalMaximum(1)
95-03............ReportCount(3)
75-01............ReportSize(1)
81-02............Input(Data, Variable, Absolute, NoWrap, Linear, PreferredState, NoNullPosition, BitField)
C0...........EndCollection()
95-01........ReportCount(1)
75-05........ReportSize(5)
81-03........Input(Constant, Variable, Absolute, NoWrap, Linear, PreferredState, NoNullPosition, BitField)
C0.......EndCollection()

C++ output

// AUTO-GENERATED by WaratahCmd.exe

#include <memory>

// HID Usage Tables: 1.3.0
// Descriptor size: 50 (bytes)
// +----------+-------+------------------+
// | ReportId | Kind  | ReportSizeInBits |
// +----------+-------+------------------+
// |        1 | Input |               24 |
// +----------+-------+------------------+
static const uint8_t hidReportDescriptor [] = 
{
    0x05, 0x01,    // UsagePage(Generic Desktop[1])
    0x09, 0x02,    // UsageId(Mouse[2])
    0xA1, 0x01,    // Collection(Application)
    0x85, 0x01,    //     ReportId(1)
    0x09, 0x01,    //     UsageId(Pointer[1])
    0xA1, 0x00,    //     Collection(Physical)
    0x09, 0x30,    //         UsageId(X[48])
    0x09, 0x31,    //         UsageId(Y[49])
    0x15, 0x80,    //         LogicalMinimum(-128)
    0x25, 0x7F,    //         LogicalMaximum(127)
    0x95, 0x02,    //         ReportCount(2)
    0x75, 0x08,    //         ReportSize(8)
    0x81, 0x06,    //         Input(Data, Variable, Relative, NoWrap, Linear, PreferredState, NoNullPosition, BitField)
    0x05, 0x09,    //         UsagePage(Button[9])
    0x19, 0x01,    //         UsageIdMin(Button 1[1])
    0x29, 0x03,    //         UsageIdMax(Button 3[3])
    0x15, 0x00,    //         LogicalMinimum(0)
    0x25, 0x01,    //         LogicalMaximum(1)
    0x95, 0x03,    //         ReportCount(3)
    0x75, 0x01,    //         ReportSize(1)
    0x81, 0x02,    //         Input(Data, Variable, Absolute, NoWrap, Linear, PreferredState, NoNullPosition, BitField)
    0xC0,          //     EndCollection()
    0x95, 0x01,    //     ReportCount(1)
    0x75, 0x05,    //     ReportSize(5)
    0x81, 0x03,    //     Input(Constant, Variable, Absolute, NoWrap, Linear, PreferredState, NoNullPosition, BitField)
    0xC0,          // EndCollection()
};

Legal

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

hidtools's Issues

Constant support

To be able to "tell the truth" better in the descriptor, and also to potentially increase performance, it would be very valuable to be able to set constants in the report.

Currently, the only time I see Kind.Constant being used is for padding. We should be able to assign Constant to selected fields such as fixed device attribute/feature reports, and in the TOML provide the fixed value to be assigned to the struct field in C++ generation.

The closest I can current get to communicating this intent in a variableItem is to set a logicalValueRange with the lower and upper bound equal, but this still produces Variable.

Interestingly, if I set LogicalMinimum(0), LogicalMaximum(0) then Waratah completely omits a ReportSize - seemingly assuming sizeInBits = 0, even though if I write the latter explicitly it's rejected. Caused by contiguous attribute optimization; bit size is still correctly 1

C compatibility

Output is very close to being C-compatible. So far the only thing I've encountered that departs from this is enum class. It would be very useful to offer a [[settings]] entry to degrade enum class to a plain C enum, as well as any other output that C would not support.

Vendor Pages / Usages

How can the tool be used to create descriptors which include vendor pages and usages? Specifying a usage or usages by number produces

Error @ 'usage' (L2) WAR764E75F5: Unknown usage [65315, 1].

Version: WaratahCmd 1.3.0.0
Example wara:

[[applicationCollection]]
usage = [65315,1]

    [[applicationCollection.inputReport]]
        [[applicationCollection.inputReport.variableItem]]
        usage = [65315, 2]
        logicalValueRange = [0, 1]

Invocation: PS> & .\WaratahCmd.exe --source=example.wara

HID Usage Tables V.1.3

Do you plan improve utility for HID Usage Tables V.1.3 (page 0x84/85)

Integration with hid-rdf

Hi there! I have discovered this library through https://usb.org/sites/default/files/hutrr109-soc_1.pdf , I wonder where else I could have found this earlier 😅
I have a complementary library to this one: hid-rdf
Its end goal is similar to that of Waratah, but it tries to achieve all of it using modern C++ (plus a hint of Python to generate usage pages code). Some nice things I have achieved with the hid-rdf library:

1. it allows creating HID report descriptor building blocks (by using template parameters) that are reusable in different compositions
2. it is a standalone C++ library that can serve for both host and device side HID report descriptor operations
3. it supports compile-time verification and attribute detection (max report lengths, max report ID, number of TLCs) of HID report descriptors

I wonder if it could be used by Waratah in some way. Let me know what you think.

Copyright violation by republishing the HID Usage Tables?

I'm not the copyright holder, so it's not really my business, but still wondering if it is intentional. This repository distributes hut1_3_0.pdf. The document's copyright notice says:

THIS SPECIFICATION MAY NOT BE REPUBLISHED EXTERNALLY OR OTHERWISE TO THE PUBLIC.

Does Microsoft have the permission to republish it here?

Where to find the correct naming for all the usages?

Hi,

In the examples I see naming as "Generic Desktop" and "Mouse". Where does this software get these keys and the associated values? I cannot find where the machine-readable hid usage tables this software is getting the data from are

Fails at first token

Error @ 'applicationCollection
usage = ['Generic Desktop', 'Gamepad']

    applicationCollection' (L0) WAR99F361C9: Invalid token.

I get this error in the last 4 versions trying any sample. Haven't found a version that works yet

Control padding for byte-alignment

The sample for a keyboard (see below) uses 7 bits for each regular button. Thus the array items are not byte-aligned. While it is correct and compact, it's very cumbersome to use, e.g. when programming a MCU as a USB HID device. How can the report size be increased to 8 bits to make life easier?

Similarly for the below mouse descriptor. It's similar to the provided sample, however the buttons are moved to the front of the report. Waratah automatically adds 5 bits of padding, but it's added at the end. Thus, the mouse x and y movements are not byte aligned. How can the padding be moved before x and y?

Keyboard descriptor

[[applicationCollection]]
usage = ['Generic Desktop', 'Keyboard']

    [[applicationCollection.inputReport]]

        # Regular Buttons
        [[applicationCollection.inputReport.arrayItem]]
        usageRange = ['Keyboard/Keypad', 'ErrorRollOver', 'Keyboard Application']
        count = 8

        # Special buttons
        [[applicationCollection.inputReport.variableItem]]
        usageRange = ['Keyboard/Keypad', 'Keyboard LeftControl', 'Keyboard Right GUI']
        logicalValueRange = [0, 1]

    # LEDS
    [[applicationCollection.outputReport]]

        [[applicationCollection.outputReport.variableItem]]
        usageRange = ['LED', 'Num Lock', 'Kana']
        logicalValueRange = [0, 1]

Mouse descriptor

[[applicationCollection]]
usage = ['Generic Desktop', 'Mouse']

    [[applicationCollection.inputReport]]

        [[applicationCollection.inputReport.physicalCollection]]
        usage = ['Generic Desktop', 'Pointer']


            [[applicationCollection.inputReport.physicalCollection.variableItem]]
            usageRange = ['Button', 'Button 1', 'Button 3']
            logicalValueRange = [0, 1]

            [[applicationCollection.inputReport.physicalCollection.variableItem]]
            usage = ['Generic Desktop', 'X']
            sizeInBits = 8
            logicalValueRange = 'maxSignedSizeRange'
            reportFlags = ['relative']

            [[applicationCollection.inputReport.physicalCollection.variableItem]]
            usage = ['Generic Desktop', 'Y']
            sizeInBits = 8
            logicalValueRange = 'maxSignedSizeRange'
            reportFlags = ['relative']

Member name control

We should introduce a [[settings]] entry to drop implied suffixes to struct members like Button. For backwards compatibility this could not be enabled by default, but would produce more sensible output for the many dozens of HID fields that are one-bit and do not in fact represent buttons.

waratah-v1.5.0 published binary gets tagged as a severe trojan threat by Defender - Trojan:Win32/Wacatac.H!ml

Not quite sure what exactly causes Windows Defender to tag it as such, but wasn't an issue in 1.4.0, just recently in 1.5.0. I can pull source locally and compile from that so not a major issue but it was a bit unexpected to have Defender quarantining the zip.

(if this is expected then all good, feel free to close this btw)

Please build for Linux and MacOS as well

Hi,

May I ask you if you could kindly build and release Linux and MacOS binaries as well?

Units encoding

I'm very sorry to spam this project with my question, but I could not find any pointer on the usb.org site where questions like these can be asked or any recommended forum where it can be discussed. I couldn't find any contact information on usb.org. My issue is that the HID specification v1.11 pages 36 - 38 is very vague, or even incomplete, in how units are actually encoded.

This 10 year old post sums it up pretty well: https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/e87d0db1-486e-42ae-bf95-d1ac5ffc0b02/unit-exponent-item-value-encoding-in-hid-report-descriptors

To make this question more related to this project: What did the Waratah tool developers use as a reference on how units are actually encoded? Is there any better and clearer document than HID v1.11 pages 36-38?

I think usb.org should make a new release of the specification, where the encoding is clarified.

`usages` crashes Waratah

This piece of TOML.:

            [[applicationCollection.inputReport.logicalCollection.arrayItem]]
            usages = ['Auxiliary Display', 'Stat Not Ready', 'Stat Ready', 'Err Not a loadable character']

gets past the parser, and then crashes with

Unable to cast object of type 'System.String' to type 'System.Object[]'.

This occurs on Waratah 1.3.0.0. Of course the above syntax is invalid, but ideally a better error message should be generated.

Wrong `Keypad Dash` usage name in HUT 1.3

Seems JSON attached to the current https://usb.org/document-library/hid-usage-tables-13 document have wrong button name:

        {
          "Id": 86,
          "Name": "Keyboard Dash",
          "Kinds": [
            "Sel"
          ]
        },

Keyboard Dash should be Keypad Dash according to the PDF text.