michaelgrafnetter / dsinternals

Directory Services Internals (DSInternals) PowerShell Module and Framework

Hi there 👋

dsinternals's Issues

The distinguished name specified for this replication operation is invalid

A few users have reported that they sometimes see an error with message "The distinguished name specified for this replication operation is invalid" instead of "Access denied" when using the Get-ADReplAccount cmdlet.

Add support for Visual Studio 2015

Migrate from Visual Studio 2013 to Visual Studio 2015. This is work in progress and a special branch, vs_2015, has been created for this purpose.

Test-PasswordQuality : Object reference not set to an instance of an object. At line:1 char:19

When I run a test using the -SkipDuplicatePasswordTest switch I always get this error

PS C:\Temp> $adReplica | Test-PasswordQuality -WeakPasswordHashes $p1 -SkipDuplicatePasswordTest
Test-PasswordQuality : Object reference not set to an instance of an object.
At line:1 char:14

The same command without that switch runs fine. Has this been encountered before? The -debug switch isn't telling me anything helpful

Can't import DSInternals

Failed to import DSInternals with error message below with Win7 (64bit)
Tried WMF5.0 and 5.1 and .Net 4.5.1 and 4.5.2 with no luck

PS C:\WINDOWS\system32> import-module DSInternals
Add-Type : Could not load file or assembly
'file:///C:\Windows\System32\WindowsPowerShell\v1.0\Modules\DSInternals\x86\DSInternals.Replication.Interop.dll' or
one of its dependencies. An attempt was made to load a program with an incorrect format.
At C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules\DSInternals\DSInternals.psm1:16 char:5
Add-Type -Path $interopAssemblyPath
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Add-Type], BadImageFormatException
+ FullyQualifiedErrorId : System.BadImageFormatException,Microsoft.PowerShell.Commands.AddTypeCommand

Better errors on ntds.dit from different ESENT version

The JET API cannot simply open DB files coming from a different OS version. Such attempts end with this error: "Secondary index is corrupt. The database must be defragmented or the affected index must be deleted. If the corrupt index is over Unicode text, a likely cause a sort-order change."

A better error message should be displayed, together with the DB version and the exact command (esentutl) that would "repair" the DB.

Test replication with optional features enabled

Could someone please test if the replication (Get-ADReplAccount) works with Recycle Bin and Privileged Access Management enabled? Mimikatz had a problem there.

Increase OrgIdHash iteration count

Newer versions of Azure AD Connect use 1000 PBKDF2 iterations instead of 100, which is good news. The ConvertTo-OrgIdHash cmdlet should be updated accordingly.

Get an attribute with Object(DS-DN) syntax

Hi,
I'm trying to get some extra attributes in result of Get-ADDBAccount -All.
attributes with String(Unicode) syntax added easily,
but I can't change source code to get attributes with Object(DS-DN) syntax like "Member".
what should I do?

Add support for Windows Server 2016 TP4 ntds.dit encryption schema

The Password Encryption Key List (pekList) encryption has changed since Windows Server 2016 TP4. Does anyone have any ideas?

feature request: password audit without password disclosure

For some job roles and policy environments, it would be preferable if the person performing the audit has no direct knowledge of the actual passwords used. It would be good for Test-PasswordQuality to have a flag/option to suppress displaying passwords to the screen and in any reports. In other words, instead of this output:

Passwords of these accounts have been found in the dictionary:
  adam                Pa$$w0rd
  peter               July2016

Historical passwords of these accounts have been found in the dictionary:
  april               Pa$$w0rd
  brad                Pa$$w0rd

... this output:

Passwords of these accounts have been found in the dictionary:
  adam
  peter

Historical passwords of these accounts have been found in the dictionary:
  april
  brad

If this is already an option, please consider this to be a documentation clarification request. :)

A few cmdlets do not work against ADAM / AD LDS

Databases from WS 2016 cannot be opened on non-DCs

When you try to open a ntds.dit file from Windows Server 2016 on Windows 10, you get error JET_errCallbackNotResolved, "A callback function could not be found", pointing to a missing ntdsai.dll file. This file is only present on DCs and LDS servers. Thanks to @ZilentJack for reporting this issue.

I will try to play with the JET_paramDisableCallbacks system parameter that might bypass this issue.

Cannot Import Module

For some reason, I can't install this on Server 2008 R2, x64 because of the above error.

Online SIDHistory synchronization

It would be very helpful to have ability of online SIDHistory synchronization. At least via legitimate "DsAddSidHistory" API.

False positives reported when smart card auth is enforced

The Test-PasswordQuality cmdlet is reporting accounts that require smart card authentication as missing AES keys. The thing is that such accounts are actually supposed to be missing these keys.

ntds.dit sometimes needs defragmentation on Windows Server 2012 R2

On Win Server 2012 R2, Windows PowerShell returns 6.3 as OS version, but Windows PowerShell ISE returns 6.2, which causes incorrect OS detection and the DSInternals module throws an exception.
This issue has something to do with Operating system version changes in Windows 8.1 and Windows Server 2012 R2.
As a quick fix, the OS version detection will be dropped and esent.dll version detection will be implemented in the future.

Add DER and PVK export to BackupKeys

Get-ADDBAccount : Parameter is not a hexadecimal string

Got this bug report through a different channel:

ntds.dit and SYSTEM is exported from a win2008R2 server using ntdsutil. Running the powershell module from a win2012R2.

Unblocked the module zip file before before installing.

PS C:> $key = Get-BootKey -SystemHivePath “C:\SYSTEM”
PS C:>
PS C:> $key
2bc5ae2c28662f04b23a33008c743be8
PS C:>
PS C:> Get-ADDBAccount -All -DBPath “C:\ntds.dit” -BootKey $key
Get-ADDBAccount : Parameter is not a hexadecimal string.
At line:1 char:1

• Get-ADDBAccount -All -DBPath “C:\ntds.dit” -BootKey $key
• CategoryInfo : OpenError: (:) [Get-ADDBAccount], ArgumentException
• FullyQualifiedErrorId : DBContextError,DSInternals.PowerShell.Commands.GetADDBAccountCommand

Upgrade to the latest Managed Esent library

Replication does not work on x86 since v 2.9

This symptom only occurs on 32-bit Windows, with message "The Array Bounds Are Invalid" from IDL_DRSBind.

Importing the powershell modules

Hello Michael

I am using Powershell 3.0. I have an issue importing the powershel modules.

Here are my steps before doing 'import-module :

• downloading the repository DSInternals as a ZIP
• copying DSInternals-master.zip\DSInternals-master\Src\DSInternals.powershell into C:\windows\system32\WindowsPowerShell\v1.0\Modules

Refactor schema loading to follow ATTRTYP documentation

https://msdn.microsoft.com/en-us/library/cc223224.aspx

Add support for KDS Root Key extraction from ntds.dit files

Add PlatyPS-based documentation

Cannot set percent because PercentComplete cannot be greater than 100.

The Get-ADReplAccount cmdlet sometimes throws this error: Cannot set percent because PercentComplete cannot be greater than 100. Parameter name: value Actual value was 101.

This is caused by the fact that AD's object count estimate is sometimes lower than the actual one.

Add support for replication progress reporting

Import Module in c# DLL

I am trying to import the DSInternals module for use in my C# dll.

when using the powershell, everything works just fine but this code does not seem to load the module.

Any idea on what am doing wrong?

            InitialSessionState init = InitialSessionState.CreateDefault();
            init.ImportPSModule(new string[] { @"D:\\DSInternals\\dsinternals.psd1" });
            Runspace runspace = RunspaceFactory.CreateRunspace(init);
            runspace.Open();
            PowerShell ps = PowerShell.Create();
            ps.Runspace = runspace;
            ps.Commands.AddCommand("Get-ADReplAccount");

            foreach (PSObject result in ps.Invoke())
            {
                Console.WriteLine(result); //this always returns null
            }

Add support for Windows Server 2003 database files

Current code relies on objectClass attribute to be index, which was not true for Windows Server 2003 DCs. We have to move to objectCategory instead.

Replication cmdlets do not work against Windows Server 2003

The replication cmdlets throw error “The distinguished name specified for this replication operation is invalid.” when executed against Windows Server 2003.

EsentLogFileSizeMismatchDatabasesConsistentException gets thrown sometimes

Exception: databases have been recovered, but the log file size used during recovery does not match JET_paramLogFileSize

Workaround: Defragment the DB or delete logs.

2008 R2 module cannot be imported because its manifest contains one or more members that are not valid

I've tested this on multiple 2008R2 machines. When attempting to run "Import-Module DSInternals" in powershell, the following text in the output below is returned stating. Method of installation was downloading zip, and extracting it to the Modules directory as instructed in 2a of the Readme.txt

Multiple tests of 2012R2 do not have this problem, and comparing the 2008/2012 directories have the same files and sizes.

Here is the error output below.

Import-Module : The 'C:\Windows\system32\WindowsPowerShell\v1.0\Modules\dsinternals\dsinternals.psd1' module cannot be
imported because its manifest contains one or more members that are not valid. The valid manifest members are ('ModuleT
oProcess', 'NestedModules', 'GUID', 'Author', 'CompanyName', 'Copyright', 'ModuleVersion', 'Description', 'PowerShellVe
rsion', 'PowerShellHostName', 'PowerShellHostVersion', 'CLRVersion', 'DotNetFrameworkVersion', 'ProcessorArchitecture',
'RequiredModules', 'TypesToProcess', 'FormatsToProcess', 'ScriptsToProcess', 'PrivateData', 'RequiredAssemblies', 'Mod
uleList', 'FileList', 'FunctionsToExport', 'VariablesToExport', 'AliasesToExport', 'CmdletsToExport'). Remove the membe
rs that are not valid ('RootModule'), then try to import the module again.
At line:1 char:14

• import-module <<<< dsinternals
  • CategoryInfo : InvalidData: (C:\Windows\syst...sinternals.psd1:String) [Import-Module], InvalidOperatio
    nException
  • FullyQualifiedErrorId : Modules_InvalidManifestMember,Microsoft.PowerShell.Commands.ImportModuleCommand

Error using Get-ADReplAccount: An item with the same key has already been added.

Hello
I use this function to synchronize password hashes from a old to a new domain. Works fine but on 6 of 1440 user accounts I get an error:

Commandline:
Get-ADReplAccount -SamAccountName U123456 -Domain CONTOSO -Server SRV01234 -Protocol TCP

Result:

Get-ADReplAccount : An item with the same key has already been added.
At line:1 char:1
+ Get-ADReplAccount -SamAccountName U123456 -Domain CONTOSO -Server SRV01234 -Prot ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Get-ADReplAccount], ArgumentException
    + FullyQualifiedErrorId : System.ArgumentException,DSInternals.PowerShell.Commands.GetADReplAccountCommand

I tried many things to solve this:

• Changed the password
• Check the security on the object and removed and re enabled inheritance
• Move the account to a different OU
• Used on both domain controllers
• Used different admin account

No luck so far. Any tips ?

Database cmdlets run indefinitely on some ntds.dit files

@ZilentJack has found this strange bug and helped me with debugging: On some ntds.dit files, the cmdlets run indefinitely. The root cause is that Microsoft's TableDefinition.Indices enumerator endlessly cycles through the first 3 indices. While I was not able to fix it, I have implemented a workaround. I will commit this change soon.

Get-ADDBAccount : Invalid file path

Hello,

For some reasons Get-ADDBAccount doesn't load the ntds.file:

PS C:\Users\Administrator\Downloads\DSInternals_v2.14\DSInternals>
PS C:\Users\Administrator\Downloads\DSInternals_v2.14\DSInternals> echo $key
5c4445b6782e70f9a0be268ba2c401ee
PS C:\Users\Administrator\Downloads\DSInternals_v2.14\DSInternals> dir C:\ntds.dit

    Directory: C:\

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
-a---         5/16/2016  12:42 PM   16793600 ntds.dit
PS C:\Users\Administrator\Downloads\DSInternals_v2.14\DSInternals> Get-ADDBAccount -All -DBPath 'C:\ntds.dit' -BootKey $key
Get-ADDBAccount : Invalid file path
At line:1 char:1
+ Get-ADDBAccount -All -DBPath 'C:\ntds.dit' -BootKey $key
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OpenError: (:) [Get-ADDBAccount], EsentInvalidPathException
    + FullyQualifiedErrorId : DBContextError,DSInternals.PowerShell.Commands.GetADDBAccountCommand

PS C:\Users\Administrator\Downloads\DSInternals_v2.14\DSInternals>

Any suggestions?

Thanks

"Input string was not in a correct format."

System Hive and ntds.dit exported from Windows Server 2003 Standard Edition with SP1 (English)

Get-ADDBAccount : Cannot bind parameter 'BootKey'. Cannot convert value "13d20976d63ea5e836036ec8bc68d6eb" to type
"System.Byte". Error: "Input string was not in a correct format."
At line:1 char:66

• Get-ADDBAccount -All -DBPath 'D:\ntds.dit' -BootKey $key

•                                                              ~~~~
  

  • CategoryInfo : InvalidArgument: (:) [Get-ADDBAccount], ParameterBindingException
  • FullyQualifiedErrorId : CannotConvertArgumentNoMessage,DSInternals.PowerShell.Commands.GetADDBAccountCommand

PS C:\Windows\system32> $key
13d20976d63ea5e836036ec8bc68d6eb

Implement Pester tests

Add support for SPNs

query by upn rather than samaccount, guid or sid

I'd like to be able to search users by upn for DirectoryReplicationClient.GetAccount.

schema without samaccounttype

Hi Michael. I have a problem I hope you can help with.
My directory schema not contain samaccounttype or samaccountname so i try using -ObjectGuid and surprise :

Get-ADDBAccount -ObjectGuid xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxxx -DBPath ‘C:\prj\adamntds.dit’ -BootKey $key
Get-ADDBAccount : Directory schema does not contain attribute ‘sAMAccountType’.
Au caractère Ligne:1 : 1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Get-ADDBAccount], SchemaAttributeNotFoundException
+ FullyQualifiedErrorId : DSInternals.Common.Exceptions.SchemaAttributeNotFoundException,DSInternals.PowerShell.Co
mmands.GetADDBAccountCommand

Best regards

Get-ADReplAccount fails against renamed domains

Get-ADReplAccount (and I presume any other commands that rely on the MS-DRSR protocol) appears to have a problem working against domains that have previously gone through a rename process. The following error is presented.

Get-ADReplAccount : The directory service cannot perform the requested operation because the servers involved are of
different replication epochs (which is usually related to a domain rename that is in progress)
At line:1 char:1
+ Get-ADReplAccount -SamAccountName XXXXXX -domain XXXX -server XXXXX ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Get-ADReplAccount], Win32Exception
    + FullyQualifiedErrorId : System.ComponentModel.Win32Exception,DSInternals.PowerShell.Commands.GetADReplAccountCom
   mand

Despite the error message indicating the problem might be related to a rename in progress, this particular domain was renamed successfully about 5 years ago and is not currently still being renamed. The value of msDS-RelicationEpoch on the nTDSDSA object for all DCs is currently 1. If I temporarily clear the value on the DC I'm targetting, the command works as expected (though that DC obviously has broken replication until I set it back).

I'm guessing the code just always uses 0 and perhaps it should query the current value on the target DC first and use that. If I get some time, I may try submitting a PR for this.

P.S. Love this project. Amazing work.

Add support for Visual Studio 2017

Distribute the libraries as a NuGet package

The DSInternals project has been designed to be modular and the helper libraries might be useful in other projects. To make it easier for other developers to use them, these libraries could be distributed in the form of a NuGet package:

• DSInternals.Replication
• DSInternals.DataStore
• DSInternals.SAM

Get-ADReplAccount : Object reference not set to an instance of an object.

It would seem that I have run into another error on some user accounts. When I run this command on specific users:
Get-ADReplAccount -SamAccountName TestUser1 -Domain Domain -Server Server

I get the following error:

Get-ADReplAccount : Object reference not set to an instance of an object.
At line:1 char:1
+ Get-ADReplAccount -SamAccountName TestUser1 -Domain Domain -Server Server ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Get-ADReplAccount], NullReferenceException
    + FullyQualifiedErrorId : System.NullReferenceException,DSInternals.PowerShell.Commands.GetADReplAccountCommand

Here is the stack trace:

   at DSInternals.Common.Data.DSAccount.LoadRoamedCredentials(DirectoryObject dsObject)
   at DSInternals.Common.Data.DSAccount..ctor(DirectoryObject dsObject, DirectorySecretDecryptor pek)
   at DSInternals.Replication.DirectoryReplicationClient.GetAccount(Guid objectGuid)
   at DSInternals.PowerShell.Commands.GetADReplAccountCommand.ReturnSingleAccount()
   at System.Management.Automation.CommandProcessor.ProcessRecord()

I have a few users that have this issue. I am also getting the error if I use -All parameter and it kills the command.

Test ntds.dit from a RODC

Message from Eugen:

I have a question about RODC’s NTDS.dit file. It seems that it is been built differently as the NTDS on writable DC.

So, my purpose was to demonstrate to my collegues in lab, that it is impossible to stolen non-cached user passwords from the RODC. I tried to read pwd hashes from NTDS file extracted from a RODC. I’ve pre-populated my RODC by some user passwords, but $key = Get-BootKey -SystemHivePath ‘d:\SHARE\SYSTEM’
Get-ADDBAccount -all -DBPath ‘d:\share\ntds.dit’ -BootKey $key -Verbose

does not generate any output. The ADUC snap-in says some password are replicated to the RODC. I pushed the replication of those passwords from repadmin too. When I specify a NTDS file from writable DC in the same domain, it shows me NT hashes of all accounts.

Have tried 2012 R2 and 2016 domains. What may be a reason?

Add support for incremental replication

Add support for incremental replication using a cookie to the DSInternals.Replication library. Although the replication cookie is used internally, it is not exposed to the user of the library.

Get-ADDBAccount : Could not load file or assembly Esent.Isam. Strong name validation failed.

Bug report from andres:

Get-ADDBAccount : Could not load file or assembly ‘Esent.Isam, Version=1.9.3.2, Culture=neutral,
PublicKeyToken=af7e77ba04a3c166’ or one of its dependencies. Strong name validation failed. (Exception from HRESULT:
0x8013141A)
En línea: 1 Carácter: 1

• Get-ADDBAccount -all -DBPath ‘.\ChiliMango\pass\Active Directory\ntds.dit’ -Boot …
• CategoryInfo : OpenError: (:) [Get-ADDBAccount], FileLoadException
• FullyQualifiedErrorId : DBContextError,DSInternals.PowerShell.Commands.GetADDBAccountCommand

Apparently, I have forgotten to configure one of the assemblies to get signed.

Replication does not work for Protected Users

It has been reported that the Get-ADReplAccount cmdlet does not work when it is executed by users who are members of the Protected Users group. I will have to verify that.

Provide a better error message if an assembly is blocked

Database cmdlets only work on Windows 8+ / Windows Server 2012+

The database cmdlets only work on Windows 8+ / Windows Server 2012+, because there is a bug in the underlying libraries on older systems. I am currently trying to bypass it.

Get-BootKey : Object reference not set to an instance of an object.

Get-BootKey : Object reference not set to an instance of an object.
At /Users/dixon_r/Downloads/DSInternals_v2/fetchhashes.ps1:5 char:8

• $key = Get-BootKey -SystemHivePath 'SYSTEM'

•    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  

  • CategoryInfo : OpenError: (:) [Get-BootKey], NullReferenceException
  • FullyQualifiedErrorId : GetBootKey_OtherError,DSInternals.PowerShell.Commands.GetBootKeyCommand

I feel I'm doing something wimple and wrong.

any ideas?