Coder Social home page Coder Social logo

microsoft / wdkmetadata Goto Github PK

View Code? Open in Web Editor NEW
83.0 7.0 5.0 44.03 MB

Tooling to generate metadata for Win32 APIs in the Windows Driver Kit (WDK).

License: Other

PowerShell 0.01% C++ 64.54% C 35.36% Assembly 0.03% HLSL 0.01% Objective-C 0.02% Rich Text Format 0.03% C# 0.01% BitBake 0.01%

wdkmetadata's Introduction

Overview

This repository produces metadata for the Windows Driver Kit (WDK) using the tooling provided by the win32metadata project. The metadata is published to NuGet.org as the Microsoft.Windows.WDK.Win32Metadata NuGet package. This mirrors the Microsoft.Windows.SDK.Win32Metadata package that it depends on.

DISCLAIMER: wdkmetadata is EXPERIMENTAL and intended to provide a place for the community to build out rich metadata for the WDK to enable driver development in language projections like the windows crate for Rust.

Principles

  • This repository is sponsored by Microsoft but will primarily be driven by outside collaborators and community contributions
  • This repository will use public tooling provided by Microsoft.Windows.WinmdGenerator wherever possible
  • This repository will aim to mirror win32metadata in layout and development experience to provide a consistent contributor experience across the WDK and the SDK
  • This repository will provide foundational metadata for native APIs that ship in the WDK and will not provide abstractions
  • This repository does not change the license of the original WDK and SDK headers used to generate the metadata. Any such artifacts checked into this repository support metadata production only and do not imply a change of their original licenses.

Repository Bootstrap

  • [DONE] Clone the win32metadata repo
  • [DONE] Remove win32metadata baggage (samples, scripts, sources, tests, etc.) and onboard to Microsoft.Windows.WinmdGenerator
  • [DONE] Rename generation/SDK to generation/WDK and rename Windows.Win32.winmd to Windows.Wdk.winmd
  • [DONE] Import WDK headers
    • NOTE: This repository needs both SDK headers and WDK headers since the WDK headers depend on the SDK headers to compile. Original SDK and WDK headers for Windows 11 Build 22000 were checked into IdlHeaders and then recompiled to RecompiledIdlHeaders using an adapted version of RecompileIdlFilesForScraping. Only WDK headers should be added to partitions in this repository.
  • [DONE] Create example API partition (https://github.com/microsoft/win32metadata/issues/509)
  • [DONE] Produce Microsoft.Windows.WDK.Win32Metadata NuGet package for Windows.Wdk.winmd
  • [DONE] Port over full/incremental build support from win32metadata
  • [DONE] Port over baseline diffing support from win32metadata
  • [DONE] Port over CI and release pipelines from win32metadata
  • [DONE] Port over header ingestion script from win32metadata and include WDK headers
  • [DONE] Port over automatic import library detection to minimize changes to libMappings.rsp
    • NOTE: When adding new headers or troubleshooting why APIs may not be emitted in the winmd, make sure that libMappings.rsp has entries for the APIs with a valid import library that includes the proper extension (e.g. dll, drv, exe, etc.).
  • Onboard outside collaborators and community contributors

wdkmetadata's People

Contributors

microsoft-github-operations[bot] avatar microsoftopensource avatar mikebattista avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

wdkmetadata's Issues

Need a build folder in the new WDK package

The SDK package contains a build folder that allows CsWin32 to consume the winmd via the nupkg directly. But the WDK package is missing that.

Here's what the SDK package has:

C:\.TOOLS\.NUGET\PACKAGES\MICROSOFT.WINDOWS.SDK.WIN32METADATA\52.0.65-PREVIEW\BUILD
│   Microsoft.Windows.SDK.Win32Metadata.props
│
├───net20
│       Microsoft.Windows.SDK.Win32Metadata.props
│
└───netstandard1.0
        Microsoft.Windows.SDK.Win32Metadata.props

Can the WDK package be enhanced to include that?

Add missing NDIS APIs

          Apparently `NdisRegisterProtocolDriver` or `PNDIS_BIND_PARAMETERS` (which are both gated behind `NDIS_SUPPORT_NDIS6`) aren't showing up in the metadata, which seems to be caused by the defines made in `ndis/version.h` not being visible inside of `ndis.h`.

There are a bunch of "ndis" headers other than just ndis.h so you'll have to tell me if I got the full closure of what's needed.

It seems like the only header that's missing is netpnp.h, though once NET_PNP_EVENT_NOTIFICATION (gated behind NDIS_SUPPORT_NDIS6) is scraped it'd be needed anyways.

Originally posted by @DropDemBits in #50 (comment)

WDK+SDK duplicated APIs

The following APIs appear in the WDK and SDK metadata files:

  • RtlConvertSidToUnicodeString
  • RtlCrc64

Invaid prototype of PDRIVER_UNLOAD and other callbacks

As earlier was stated here: microsoft/windows-rs#2612, there are no arguments in callack definitions like PDRIVER_UNLOAD in the Foundation module of WDK.

pub type PDRIVER_UNLOAD = ::core::option::Option<unsafe extern "system" fn() -> ()>;

But in C++ WDK we have this:

//
// Define driver unload routine type.
//
_Function_class_(DRIVER_UNLOAD)
_IRQL_requires_(PASSIVE_LEVEL)
_IRQL_requires_same_
typedef
VOID
DRIVER_UNLOAD (
    _In_ struct _DRIVER_OBJECT *DriverObject
    );

So, the Rust callback should be:

pub type PDRIVER_UNLOAD = ::core::option::Option<unsafe extern "system" fn(driver_object: *mut DRIVER_OBJECT) -> ()>;

ntdll functions

Some NTDLL Functions Are Currently Not Working

Tested Not Generating:
NtDuplicateObject
RtlZeroMemory
LdrGetProcedureAddress
LdrLoadDll
NtMapViewOfSection
NtUnmapViewOfSection
NtProtectVirtualMemory
RtlCreateUserThread
NtCreateThreadEx
NtQueueApcThread
NtOpenThread
NtSuspendProcess
NtResumeProcess

Expected behavior

These Should Be Generating,
Ive Generated Other Imports With 100% Successrate,
But The Above List Doesnt Generate Anything!

Repro steps

  1. NativeMethods.txt content:
//Working
NtOpenProcess
NtOpenFile
RtlGetVersion
NtCreateSection
NtQueryObject
NtQueryVirtualMemory
NtQuerySystemInformation
NtQueryInformationProcess
NtWaitForSingleObject
RtlInitUnicodeString
RtlUnicodeStringToAnsiString
//Not Working
NtDuplicateObject
RtlZeroMemory
LdrGetProcedureAddress
LdrLoadDll
NtMapViewOfSection
NtUnmapViewOfSection
NtProtectVirtualMemory
RtlCreateUserThread
NtCreateThreadEx
NtQueueApcThread
NtOpenThread
NtSuspendProcess
NtResumeProcess

Context

  • CsWin32 version: 0.1.42.13748
  • Runtime Version: v4.0.30319
  • Target Framework: .NETFramework,Version=v4.8

Support fltkernel.h

          Any update on this? I would love to use [`FltCreateCommunicationPort`](https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/fltkernel/nf-fltkernel-fltcreatecommunicationport) in a project.

Originally posted by @sn99 in #1 (comment)

Invalid handle types

Further testing revealed some invalid handle types that make certain APIs inexpressible. I'm not too familiar with the WDK but it seems that all or some of these underscore-prefixed structs should be handles:

image

For example: https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/nf-wdm-ioinitializeworkitem

Here the parameter is named PIO_WORKITEM and this typedef should be a pointer to an opaque structure defined by the driver. I would expect it to be defined something like this:

[NativeTypedef]
public struct PIO_WORKITEM
{
	public IntPtr Value;
}

And then the empty _IO_WORKITEM struct should be removed. I assume many of the other underscore-prefixed structs fall into the same category, but again I'm not that familiar with the WDK.

NtCreateFile CreateOptions enum

NtCreateFile's CreateOptions parameter should be typed as an enum with several existing constants gathered within it. The list of constants are in the docs, but as that's long, and I found a definition in System.Private.CoreLib, here it is for reference:

public enum CreateOptions : uint
{
	FILE_DIRECTORY_FILE = 1u,
	FILE_WRITE_THROUGH = 2u,
	FILE_SEQUENTIAL_ONLY = 4u,
	FILE_NO_INTERMEDIATE_BUFFERING = 8u,
	FILE_SYNCHRONOUS_IO_ALERT = 0x10u,
	FILE_SYNCHRONOUS_IO_NONALERT = 0x20u,
	FILE_NON_DIRECTORY_FILE = 0x40u,
	FILE_CREATE_TREE_CONNECTION = 0x80u,
	FILE_COMPLETE_IF_OPLOCKED = 0x100u,
	FILE_NO_EA_KNOWLEDGE = 0x200u,
	FILE_RANDOM_ACCESS = 0x800u,
	FILE_DELETE_ON_CLOSE = 0x1000u,
	FILE_OPEN_BY_FILE_ID = 0x2000u,
	FILE_OPEN_FOR_BACKUP_INTENT = 0x4000u,
	FILE_NO_COMPRESSION = 0x8000u,
	FILE_OPEN_REQUIRING_OPLOCK = 0x10000u,
	FILE_DISALLOW_EXCLUSIVE = 0x20000u,
	FILE_SESSION_AWARE = 0x40000u,
	FILE_RESERVE_OPFILTER = 0x100000u,
	FILE_OPEN_REPARSE_POINT = 0x200000u,
	FILE_OPEN_NO_RECALL = 0x400000u
}

Feature request: ZwUnmapViewOfSection is missing

Motivation

I am currently writing up a tool for my own software suite which uses process hollowing on Windows.
For that I need to do some WinAPI calls:

  • CreateProcessW
  • ZwUnmapViewOfSection
  • VirtualAllocEx
  • WriteProcessMemory
  • ResumeThread

I have searched to the best of my abilities, but I was not able to find ZwUnmapViewOfSection in the windows crate, not even with a slight typo or such.

Drawbacks

There is no drawback - it's just a missing function which would be added to get closer to the statement "lets you call any Windows API past, present, and future"

Rationale and alternatives

There isn't really any alternative other than not using Rust - I found another crate named "ntapi" which apparently provides this function, but lacks in all other points.
The impact if you don't do it: I would be very sad :'(

Additional context

Starting on page 43 is pseudo code showcasing the process:
https://www.blackhat.com/presentations/bh-usa-07/Harbour/Presentation/bh-usa-07-harbour.pdf

Documentation on the Microsoft website:
https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/nf-wdm-zwunmapviewofsection

Support Ndis.h

Currently trying to follow the ndisprot_kmdf example in my ndisprot-rs project, and ndis.h is particularly annoying to generate bindings for due to depending on msvc-specific behaviour (see the code for the workaround I do), so it'd be nice for wdkmetadata to support it instead. 😀

I technically generate bindings for NDIS 6.82, but am technically registering as an NDIS 6.0 protocol driver so generating metadata for the latest NDIS version would be fine with me.

Thanks!

MEMORY_BASIC_INFORMATION declared in WDK and SDK

Having the struct declared twice may be problematic to consumers and projections alike.
CsWin32 in particular happens to have a test for generating this struct and ran into the issue when two structs were generated.

But the two structs are not equivalent today. The declaration in the SDK has an extra PartitionId field in the middle of the list. So either there's a bug in one of them, or they are in fact two distinct structs.
If this is truly by design, I'll try to fix up CsWin32 to not fail.

Invalid nested types

Some types include invalid field types that I can't parse. They're considered nested types but the names are invalid since "@void*" is not a valid/existing nested type name.

.field public valuetype [Windows.Wdk.winmd]Windows.Wdk.System.SystemServices.IO_DRIVER_CREATE_CONTEXT/@void* pSystemMem

image

image

Missing WDF Device Functions in Metadata

Hello,

While utilizing CsWin32 to generate P/Invoke signatures, I've discovered that several essential WDF device functions are absent from the available metadata:

WdfDeviceSetPnpCapabilities
WdfDeviceSetPowerCapabilities
WdfDeviceAssignMofResourceName
WdfDeviceSetFailed
WdfDeviceSetDeviceState

The lack of these functions is constraining the capabilities of the generated projects, impacting the development of varied and dependable drivers.

Could you please review the metadata and consider incorporating these missing functions, along with any other absent ones, to address this limitation?

Investigate references from WDK metadata to per-architecture types like CONTEXT and SLIST_HEADER in the SDK metadata

The below were temporarily excluded with d4d6095#diff-a02962413f2cd0360613fa6a55753e08d3c32eebf64ecb27951a9a91f7aa4214 since the build couldn't resolve the types with multiple definitions per architecture.

_GENERAL_LOOKASIDE
_GENERAL_LOOKASIDE_POOL
_LOOKASIDE_LIST_EX
_NPAGED_LOOKASIDE_LIST
_PAGED_LOOKASIDE_LIST
ALLOCATE_FUNCTION_EX
ExDeleteLookasideListEx
ExDeleteNPagedLookasideList
ExDeletePagedLookasideList
ExFlushLookasideListEx
ExInitializeLookasideListEx
ExInitializeNPagedLookasideList
ExInitializePagedLookasideList
FirstEntrySList
FREE_FUNCTION_EX
InterlockedPushListSList
_NET_BUFFER
_NET_BUFFER_HEADER
_NET_BUFFER_LIST
_NET_BUFFER_LIST_HEADER

Example error message:

error CS0433: The type 'SLIST_HEADER' exists in both 'Windows.Win32.winmd, Version=47.0.26.58549, Culture=neutral, PublicKeyToken=null' and 'Windows.Win32.winmd, Version=47.0.26.58549, Culture=neutral, PublicKeyToken=null' [C:\wdkmetadata\generation\WDK\Windows.Wdk.proj]

Support d3dhal.h

This header was excluded from the SDK metadata since it covers kernel mode display driver scenarios.

The API "LsaFreeReturnBuffer" is ambiguous.

Actual behavior

Error : PInvoke007 - The API "LsaFreeReturnBuffer" is ambiguous. Please specify one of: or "Windows.Win32.Security.Authentication.Identity.LsaFreeReturnBuffer" or "Windows.Wdk.Storage.FileSystem.LsaFreeReturnBuffer".

New to all of this in c#. Was playing around with this library. I see 2 g.cs files, "Windows.Wdk.PInvoke.SECUR32.dll.g.cs" and "Windows.Win32.PInvoke.SECUR32.dll.g.cs" with the LsaFreeReturnBuffer function.

Repro steps

  1. NativeMethods.txt content:
    LsaFreeReturnBuffer

  2. Any of your own code that should be shared?
    Windows.Win32.PInvoke.LsaFreeReturnBuffer(temp);

Context

  • CsWin32 version: 0.3.49-beta
  • Win32Metadata version (if explicitly set by project):
  • Target Framework: .net 7.0
  • LangVersion (if explicitly set by project): [e.g. 9]

REPARSE_DATA_BUFFER is missing

in ntifs.h a structure called REPARSE_DATA_BUFFER exists. I would expect windows or windows_sys to provide a Rust definition for REPARSE_DATA_BUFFER.

Missing API `RtlGetVersion` from `NtDll`

Support IddCx.h

A bit of background for my request, (specifically, I'm using iddcx 1.9)

So, I went ahead and followed the official idd sample driver and manually created bindings to the entire WDK and Idd for my driver project. And I got a working umdf driver now,

To that end, the specific functions I used since I couldn't find them in the generated bindings (and please correct me if I am wrong anywhere), are:

Idd:
IddCxDeviceInitConfig, IddCxDeviceInitialize, IddCxAdapterInitAsync, IddCxMonitorCreate, IddCxMonitorArrival, IddCxSwapChainSetDevice, IddCxSwapChainReleaseAndAcquireBuffer, IddCxSwapChainFinishedProcessingFrame

Wdf:
WdfDriverCreate, WdfDeviceCreate, WdfDeviceInitSetPnpPowerEventCallbacks, WdfObjectGetTypedContextWorker (this is internal, but I needed to recreate the WDF_DECLARE_CONTEXT_TYPE!() macro), WdfObjectDelete

Also had to manually export FxDriverEntryUm entry point to get the ball rolling.

Thanks for all your guys effort! I would absolutely love to throw away all my bindings 😄

Kernel Headers

It would be fantastic to be able to use this crate for kernel mode development (e.g., drivers), but it seems so far the metadata is limited to user-mode bindings. Is Kernel mode development in scope for this metadata?

Invalid function imports

The latest build appears to include invalid function imports. Functions like:

[DllImport("CLFS", ExactSpelling = true, PreserveSig = false)]
public static extern void ClfsFinalize();

And also:

[DllImport("ksecdd", ExactSpelling = true, PreserveSig = false)]
public unsafe static extern SspiAsyncContext* SspiCreateAsyncContext();

To begin with, they lack a file extension so it's unclear how to link them.

Note that some of these may not even be DLLs to begin with (they may be .sys files) but I'm not sure how you determine that when generating the metadata.

Also, I though all library names should be lowercase but that doesn't appear to be enforced consistently across the Win32 and WDK metadata.

Feature request: No impelemtation CONTEXT_CONTROL, CONTEXT_FULL etc for windows::Win32::System::Diagnostics::Debug::CONTEXT

Motivation

There are some CONTEXT_* implementation in ntddk.h.

#define CONTEXT_i386    0x00010000L    // this assumes that i386 and
#define CONTEXT_i486    0x00010000L    // i486 have identical context records



#define CONTEXT_CONTROL         (CONTEXT_i386 | 0x00000001L) // SS:SP, CS:IP, FLAGS, BP
#define CONTEXT_INTEGER         (CONTEXT_i386 | 0x00000002L) // AX, BX, CX, DX, SI, DI
#define CONTEXT_SEGMENTS        (CONTEXT_i386 | 0x00000004L) // DS, ES, FS, GS
#define CONTEXT_FLOATING_POINT  (CONTEXT_i386 | 0x00000008L) // 387 state
#define CONTEXT_DEBUG_REGISTERS (CONTEXT_i386 | 0x00000010L) // DB 0-3,6,7
#define CONTEXT_EXTENDED_REGISTERS  (CONTEXT_i386 | 0x00000020L) // cpu specific extensions

#define CONTEXT_FULL (CONTEXT_CONTROL | CONTEXT_INTEGER |\
                      CONTEXT_SEGMENTS)

#define CONTEXT_ALL             (CONTEXT_CONTROL | CONTEXT_INTEGER | CONTEXT_SEGMENTS | \
                                 CONTEXT_FLOATING_POINT | CONTEXT_DEBUG_REGISTERS | \
                                 CONTEXT_EXTENDED_REGISTERS)

#define CONTEXT_XSTATE          (CONTEXT_i386 | 0x00000040L)

#define CONTEXT_EXCEPTION_ACTIVE    0x08000000L
#define CONTEXT_SERVICE_ACTIVE      0x10000000L
#define CONTEXT_EXCEPTION_REQUEST   0x40000000L
#define CONTEXT_EXCEPTION_REPORTING 0x80000000L

This will be great if there will be rust const implementation for convenience.

Drawbacks

No response

Rationale and alternatives

No response

Additional context

No response

NtCreateFile function enums

The NtCreateFile method parameters could use some enum love:

  • DesiredAccess parameter is currently declared as uint, but it ought to be as this FILE_ACCESS_FLAGS enum.
  • FileAttributes parameter should be declared as a FILE_ATTRIBUTES enum. Previously suggested to be FILE_FLAGS_AND_ATTRIBUTES but @KalleOlaviNiemitalo made a good comment in microsoft/win32metadata#1440 (comment) to consider.
  • CreateOptions (#1441)
  • Remove NtCreateFile from the SDK metadata as part of this change

See also discussion in microsoft/win32metadata#1440.

If NtCreateFile is in the WDK, it's already in the win32metadata, so... do we remove it, or fix the enums?

Support ntifs.h

          It would be nice to have access to `ntifs.h` headers like [NtReadFile](https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/nf-ntifs-ntreadfile) given that shims for NtCreateFile et. al already exist, but are basically impossible to work with within the `windows` crate.

Originally posted by @chyyran in #1 (comment)

FILE_DISPOSITION_* constants missing

FILE_DISPOSITION_DO_NOT_DELETE | 0x00000000
FILE_DISPOSITION_DELETE | 0x00000001
FILE_DISPOSITION_POSIX_SEMANTICS | 0x00000002
FILE_DISPOSITION_FORCE_IMAGE_SECTION_CHECK | 0x00000004
FILE_DISPOSITION_ON_CLOSE | 0x00000008
FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE | 0x00000010

FileDispositionInfoEx is defined, just not the constants for it.

Missing NTSTRSAFE constants

Would like to be present important values, for example, in the form of constants (or similar):
ntstrsafe.h
NTSTRSAFE_UNICODE_STRING_MAX_CCH
NTSTRSAFE_MAX_LENGTH
NTSTRSAFE_MAX_CCH
and other

Support vhf.h

          > If you have specific WDK headers or scenarios you'd like to see supported, please provide the details here to help us prioritize.

On my end I'd love to have access to vhf.h for creating software HID devices.

Originally posted by @Progdrasil in #1 (comment)

Feature request: Include KSProxy (ksproxy.h)

Motivation

It would allow access to the KS API which gives access to some low level device control which can be quite useful.

Drawbacks

More code to maintain, not sure how much is "automatic" once it's been implemented but I leave that up to a maintainer to evaluate:)

Rationale and alternatives

Some functions can be worked around by accessing alternatives,
say DeviceIoControl can be used instead of KsSynchronousDeviceControl etc.
But not sure if all functionality can have alternatives though.

Additional context

Thanks for your time:)

Feature request: NtLoadDriver support

Motivation

Hello,

I know you are not porting low level NTAPI to Rust in this project, but IMO this one is a bit special. The NtLoadDriver call is easy to call from C/C++, but is really hard not to mess in Rust due to the pointer to UNICODE_STRING as parameter.
Source: https://docs.rs/ntapi/latest/ntapi/ntioapi/fn.NtLoadDriver.html

Could you consider supporting this very specific NTAPI function in windows-rs ?
All the best !

Drawbacks

Main drawback:

  • If you port this, you will have to maintain an undocumented NTAPI function

Rationale and alternatives

  • Support another way to easily load drivers in Windows using Rust

Additional context

No response

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.