benjaminkim / dokanx

user-mode filesystem framework for Windows

🔭 하는 일

직장인 소개팅 커피한잔을 개발하고 있습니다.

블로그

• 프로그래밍 에세이 를 씁니다. K리그 프로그래머
• 전업 투자자 에세이 도 씁니다.

책

• 건물주의 기쁨과 슬픔

dokanx's Issues

"The directory name is invalid. " For VFS implemetation

Command prompt always throwing error for the directories and not able to perfome CD command on Windows 8.1...

dokanx.dll requires ProcMonDebugOutputx64(Win32).dll to run

After commit 49d9016 dokanx.dll requires presence of ProcMonDebugOutputx64.dll or ProcMonDebugOutputWin32.dll (depending on platform). Could you please either add reference about this dll into installation manual or (if possible) make it optional in dokanx.dll.
Thanks!

Dokany: Dokan fork synced with DokanX

Hi,
Following #22 we created Dokany repository.
It's a Dokan fork synced with DokanX fixes.
We don't have the ambition to maintain an independent Dokan fork (community is too small). The purpose of Dokany is to provide DokanX changes but:

• with clean git history
• with all dokan related projects (.net wrapper, ...)
• with dokan legacy compatibility (no file/lib rename, no function signature changes if not required by bug fixes, no status code change, no too heavy/specific logger)
• using latest WDK8.1 & Visual Studio 2013

All other changes will only be synced according to DokanX changes.

The only additional bug fix not present in DokanX (because it had to be fixed for one of our internal project) is: dokan-dev/dokany@2d41002
Please apply this fix on DokanX too.

GitHub releases are signed using our company code/driver signing certificate ; including x86 and x64 drivers.

Threading issue using "Run as Administrator"

I am able to execute programs as administrator (elevated) on Windows 8 using either mirrorfs.exe or the original Dokan mirror.exe as long as I set the maximum number of threads to 1 (/t 1).

Once I increase the thread count (e.g. /t 0), executing setups (or using "Run as administrator" from the context menu) seems to fail every 2 to 3 starts (but this varies greatly). This behavior also seems to vary depending on the application I try to run.

---

Example:

• I mount my D:\ drive as X:\ using mirrorfs.exe (mounted as a removable drive)
• Then I browse to the Downloads folder which is located there.
• I double click on the DokanInstall_0.6.0.exe. This is just an example, this happens for every installer eventually. For the Dokan installer I can reproduce the behavior after 2 to 4 starts.
• I confirm the UAC prompt.
• If the setup is started successfully (i.e. Dokan displaying a compatibilty warning for Windows 8) I retry.

After a few tries Windows will refuse to start the application (or report it as crashed) either before or after the UAC prompt.

dokan_net

Any plans to fork dokan_net as well?

MirrorFs doesn't interact with FileSystemChangeNotification.

Supporting Visual Studio 2013

virtual-fs implementation

virtualfs concept is more valuable than mirrorfs for the sample code.

Fully support 64bit executable and dll.

dokanx.sys support 64bit driver. But other user mode executable and libraries can't be built under 64bit environment yet.

Driver signing

Eventually signed drivers are going to be needed:

Here's a potential workaround for a while: https://github.com/koush/UniversalAdbDriver/blob/master/UniveralAdbDriverInstaller/Program.cs

Basically, create a certificate that's distributed with an installer. Before the driver is installed, the certificate is added to the root store.

Low level API

Is it possible to make some kind of low level API to allow integration of parts of DokanMain into some larger (custom?) main loop? Could you make one?

I've already found that such a thing can be done with FUSE for linux (with low level API). FUSE just returns file descriptor (chan descriptor) that can be used inside poll (http://hg.qmsk.net/evfuse.hg/file/0b92d553400a/src/evfuse.c).

P.S.
perhaps you should place a link at
http://sourceforge.net/apps/mediawiki/fuse/index.php?title=OperatingSystems#Windows

Adding sshfs project.

I made a feature/sshfs branch to try maintaining sshfs. Although I am not sure I can maintain this code, I will try it.
Please give me your opinion or pull request if you are interested in this project.

Dokanx project release configuration

I'm trying to build dokanx driver in release (and release_win_7) configuration.
The problem is that dokanx project in release configuration builds into static library only. I'm a little bit confused, because in debug configuration it is build into dll and .lib for linking with FS application (and I expected the same approach in release configuraiton).
Could you please explain the difference (and what sould I do to use release dokanx) or is it a solution configuraiton error?

Process Monitor can not see the irps on dokanx volume.

Process Monitor can see filesystem drivers irps and prints them on his viewer.
It can be NTFS or FAT32 or even CIFS(samba), except Irps on dokanx volume.
This is very bad because we can't use Process Monitor as a debugger.
I don't know why yet. I guess dokan and dokanx have a same problem, and [this code] is the point to dig this problem.
If anybody know about this, please let me know.

mirrorfs.cpp DOKAN_OPTIONS malloc'ed, but dokanOperations pointer checked for result

PDOKAN_OPTIONS dokanOptions = (PDOKAN_OPTIONS)malloc(sizeof(DOKAN_OPTIONS));
if (dokanOperations == nullptr)

should be replaced with

PDOKAN_OPTIONS dokanOptions = (PDOKAN_OPTIONS)malloc(sizeof(DOKAN_OPTIONS));
if (dokanOptions == nullptr)

(this is probably copy-paste issue)

Sharing a DokanX drive

If I try and share a dokan drive it seems to share fine but on accessing from command line, exporer or anywhere else I get "Device not ready"

C:>dir \millermini\dokan
The device is not ready.

Which is different than it not existing at all.
C:>dir \millermini\dracula
The network name cannot be found.

Any idea if this is something fundamental in windows or if there's a workaround?

Dokanx version

There is a minor request to provide dokanx appropriate version.
It's 0.6.1 now and several small releases already, yet dll's version is still 0.6.0.
Kinda confusing =)
And I'm thinking about some minor build version 0.6.1.#buildnumber maybe?

Second callback: Ioctl error 6

I'm currently trying to build a java wrapper for dokan(x) with BridJ, but the second callback always fails with ERROR_INVALID_HANDLE. This is the debug output:

[4472] Dokan: debug mode on
[4472] device opened
[4472] mounted: D:\ -> \DokanRedirector{d6cc17c5-1710-4085-bce7-964f1e9f5de9}
[4472] ###Create 0000
[4472]    CreateDisposition 108X
[4472] CreateFile status = 0
[4472] Ioctl failed with code 6

What could be the cause of this problem? Locked resources? I would appreciate any hints. I'm a Java developer and really not into drivers / windows / C.

[UPDATE]
This seems to be a per-thread problem. Increasing the number of threads to 3 yields the following:

[5348] Dokan: debug mode on
[5348] device opened
[5348] mounted: D:\ -> \DokanRedirector{d6cc17c5-1709-4085-bce7-964f1e9f5de9}
[5348] ###Create 0000
[5348]    CreateDisposition 108X
[5348] CreateFile status = -2
[5348] Ioctl failed with code 6
[5348] ###Create 0001
[5348]    CreateDisposition 108X
[5348] CreateFile status = -2
[5348] Ioctl failed with code 6
[5348] ###Create 0002
[5348]    CreateDisposition 108X
[5348] CreateFile status = -2
[5348] Ioctl failed with code 6

Support Logging to Process Monitor

Making better logger is most important part to maintain dokanx.
To be better logger, it can print to not only DebugView but also Process Monitor. Because Process Monitor catches and prints FileSystem Drivers' Irps, it can be much nicer logger viewer to trace bugs. Imagine if all of them(applications' irps input/output, driver's log, your filesystem application's log) are synchronized and printed to Process Monitor Output.

(Though, Process Monitor can't print dokanx.sys' Irps. I don't know why. But I guess this could be related with. It is an another problem to solve.)

• Process Monitor Logger for Native User-mode Application.
• Process Monitor Logger for Managed User-mode Application.
• Process Monitor Logger for filesystem driver.

How to build DOKANX

Hi:

I downloaded the latest version of your software, and I downloaded the WinDDK 7.1.0. I started the environment from the WinDDK command line, added the variables that you mentioned (DOKANX_PATH, win7base.) I ran "build /wcbg" in the dokanx directory, but it didn't build. Does your build depend on VS 12? Can it be built with just the WinDDK? If it doesn't require VS 12, can you post build directions? If it does require VS 12, will it work with the free version?

Thanks,
Brent

Can't build projects on DokanX volume, represented by the mirrorfs sample

On a Win7 machine:

1. Install DokanX
2. Install wdk7600
3. Start mirrorfs and make it serve volume M:
4. Copy any of the samples under wdk's \src directory to M:. I personally copied fastfat
5. Open WDK's Win7 buildfre environment, navigate to the m:\fastfat\win7 and try to build it.
6. Build fails:

"
Linking Executable - mp\mp\objfre_win7_x86\i386\fastfat.sys
1>errors in directory m:\fastfat\fastfat\win7\mp
1>link : error LNK1218: warning treated as error; no output file generated
BUILD: Finish time: Fri Jan 30 23:59:41 2015
BUILD: Done

38 files compiled
1 executable built - 1 Warning - 1 Error
"

The wrn file has the following content:

1>warnings in directory m:\fastfat\fastfat\win7\mp
1>m:\fastfat\fastfat\win7\mp\write.obj : warning LNK4099: PDB 'vc90.pdb' was not found with
'm:\fastfat\fastfat\win7\mp\objfre_win7_x86\i386\write.obj' or at
'm:\fastfat\fastfat\win7\mp\objfre_win7_x86\i386\vc90.pdb'; linking object as if no debug info

Here is the link to the DbgView output:
https://gist.githubusercontent.com/voyvoda/d39be21e24bbfa29b4a5/raw/dokanx_build_fastfat.txt

Flush not happening after Write operation

I've been unable to get FlushFileBuffer method to invoke. Is there a special requirement or criteria that needs to qualify for FlushFileBuffer to work?
As a workaround, I've been managing flush operation within the CloseFile method when needed.

Problem using DokanX at Windows 7 64 bits enviroment

Hello,

I followed the steps for correctily building the driver, and everythings was okay, but when I go to OSR Driver Loader, WLH>AMD64, I can sucessfully Register Service, but I can't Start Service, so I tried mannually at command prompt as administrator, using: net start dokanx and I got an 1275 system error. So, I unregistered the service, reboot the machine and I Disabled the Driver Enforcement, and tried the procedure again and I got the same error. Is there something wrong ? What should be the correct path ?

Thanks in advance.

Improve documentation.(README.md)

• How to registering/starting dokanx driver.
• How to mount a volume.
• How to test their file system working well.

Don't use GetVersionEx function.

Because GetVersionEx has been deprecated, the compiler warns it.
There are alternative functions.
http://msdn.microsoft.com/en-us/library/windows/desktop/dn424972(v=vs.85).aspx

Which versions of Windows are you planning to support?

e.g. the way files are deleted seems to have changed in Windows 8 so this would need more code to handle it.

Future of Dokanx

Hello,

I am happy to see that someone made a fork of Dokan.
Unfortunately, I am also pretty scare to move from Dokan to Dokanx.

The name/description of the commit are not clear.
I cannot see the changed you made and specialy because you changed the indentation.
The commit "Fixed a lot of bugs." have 8,006 additions and 1,710 deletions and I really would like to know what have changed in this otherwise that the remove of "\n" in DDbgPrint.

On the develop you moved to WDK 8.1 does that mean Windows XP is no longer compatible ? Probably you should keep the compatiblity for WDK 7.1 ?
Does the issue "DeleteFile method not being called" (https://code.google.com/p/dokan/issues/detail?id=269&q=windows%208) have been fixed on Dokanx ?

Sorry to say it but the develop branch look like a garbage with this pull #19 There is debug/build/folder/binary everywhere.

I really would like to contribute to the project Dokanx and specially because you are really the only one who have keep the Dokan logic and let it open source !!!

The best that you can do to make the people like me to come and contribut, is to revert from dokan source and remake all your changed with clean commits to leting us understand your change.
I think this is really important for a project of such complexity.

In case you agree with me, I would be very happy to help you !

Thank again for taking time to read it :)

Implementing dokannp.dll

To mound volume as Network Drive like Windows network drive(SMB) does, it needs network provider dll.(Usually named by np.dll)

Creating files >= 4MB are getting corrupted

I am running into a strange issue and so far not been able to figure out where the issue might be.
We've put a DokanX wrapper around our proprietary API to expose access to embedded files as a Windows File System.
Almost everything is functional. We can create, modify, and delete files. Issue happens if we try to write files of size 4MB or greater. The data is getting corrupted somehow right at the 4MB mark.
I've tried flush on close as well as flush after write approaches with similar results.
Has anyone come across something like this before?

Any guarantees of file <-> thread affinity or read/write chunk size?

Is it guaranteed that (consequent) operations on the same file (in virtual FS) will be performed by the same worker thread?
Or is it guaranteed that reading/writing requests will be aligned (e. g. at 16-bytes boundary)?

My case is encrypted file system. Underlying cipher operates on 16-byte blocks, not single byte. Thus, to write 16 bytes at offset 8, I must first read bytes [0..7] and [17..31], and then write bytes [0..31].
The worst possible case is overwriting existing file, writing bytes one by one, making the following event sequence possible:

1. Worker thread A polls request to write single byte at offset 7
2. Worker thread A reads other bytes of 16-byte block (i. e. bytes [0..6] and [8..15]) from underlying file system
3. Context is switched
4. Worker thead B polls request to write single byte at offset 8
5. Worker thread B reads other bytes of the 16-byte block
6. Worker thread B encrypts 16-byte block and writes it back to disk
7. Context is switched
8. Worker thread A encrypts 16-byte block and writes it back to disk
   Result: 8th byte is lost (old value is stored)

about installation

please add some instruction regarding x64 built and installation of driver i tried many time but every time a i am getting error like "Driver installation failed.."

Unable to access Dokan drive as non admin

https://code.google.com/p/dokan/issues/detail?id=243

I don't know what is the problem and I am not sure this is even a problem. But I will do some investigation.
This issue could be closed without the patch. If there is someone to explain this exactly, I would love to hear. Please comment.

run test cases for the mirrorfs

Any particular reason why Dokanx.dll exclude exports (.lib) when built as 64bit?

Hi,
I was running into issues when attempting to compile the user mode libraries and executables under as 64 bit projects. Upon researching the code, I came across the block of code at the bottom in dokan.h which disable exporting symbols. Commenting the outer #ifndef allows building all the modules in 64 bit and the application (MirrorFS) continues to work as expected as it does in 32 bit.
Any particular reason why exports are excluded in 64 bit?

Thanks.

ifndef _M_X64

#ifdef _EXPORTING
#define DOKANAPI __declspec(dllimport) __stdcall
#else
#define DOKANAPI __declspec(dllexport) __stdcall
#endif

else

#define DOKANAPI

endif

Installation

Hi,

I have few questions.

1. I manage to build DokanX, but didn't manage to find the way to install the dokanx.sys
   What is the way to install the driver?
   I think that in Dokan it was done automatically by the installer, right?
2. I am running Win7 64 bit, do I need to build the x64 version of the driver, or the x86?
3. I need to be able to dev my FS for Dokan in both x86 and x64 builds, what do I need to config in order to have a x86\x64 DLLs and driver? do the only change is to change the target to x64 in visual studio?

Thanks in advance,

Shaul

Files greater than 4Gb can't be read (copied)

Well, I found out that files that are greater than ~4Gb can't be read from or written to drive mounted with dokanx.
I'll investigate this issue, but there is a possibility that something is wrong with my local build..
Anyway, I'll let you know if I find something about this issue.

BSODs after DokanMounter service restart

Hi everyone.
Here is the thing (on Win8.1 Pro, x64, latest available dokanx build):

1. Install dokanx.
2. Run "mirrorfs /t 1 /r c: /l z" (I tested on 1-thread fs-app, and BSOD logs where collected for 1-thread app, but I had also multi-threaded mirrorfs caused BSOD as well).
3. Restart DokanMounter service.
4. Kill mirrorfs.
   After some time a BSOD will occur. If it didn't, run mirrorfs again.

I tried to detect what the hell happened several times and BSODs where different. Here are BSODs full descriptions (I didn't figure how to attach a txt file here, so external link, sorry). Here is the list of most "popular" BSODs:

• UNEXPECTED_KERNEL_MODE_TRAP
• SYSTEM_SERVICE_EXCEPTION
• PAGE_FAULT_IN_NONPAGED_AREA
• BAD_POOL_CALLER
• UNEXPECTED_KERNEL_MODE_TRAP

Most of them point out to the memory problem, attempt to delete what was already deleted or attempt get access to what it shouldn't, etc., yet no one points onto dokanx directly.

Frankly speaking, I'm not that good at driver development neither at BSOD analyze, so I tried to analyze what led to such behavior. And here is what I found out:

1. When mirrorfs (or any other fs-app) calls DokanMain function, dokanx_mount.exe receives a DOKAN_CONTROL_MOUNT event, it inserts mount entry into it's entry list (mouter.cpp):

if (DokanControlMount(Control->MountPoint, Control->DeviceName)) {
    Control->Status = DOKAN_CONTROL_SUCCESS;
    InsertMountEntry(Control);
} else {
    Control->Status = DOKAN_CONTROL_FAIL;
}

And when fs-app detaches, dokanx_mount.exe receives DOKAN_CONTROL_UNMOUNT event, tries to find mount record and umount mount point:

mountEntry = FindMountEntry(Control);
if (mountEntry == NULL) {
    if (Control->Option == DOKAN_CONTROL_OPTION_FORCE_UNMOUNT &&
        DokanControlUnmount(Control->MountPoint)) {
            Control->Status = DOKAN_CONTROL_SUCCESS;
            break;
    }
    Control->Status = DOKAN_CONTROL_FAIL;
    break;
}
if (DokanControlUnmount(mountEntry->MountControl.MountPoint)) {
    Control->Status = DOKAN_CONTROL_SUCCESS;
    if (wcslen(Control->DeviceName) == 0) {
        wcscpy_s(Control->DeviceName, sizeof(Control->DeviceName) / sizeof(WCHAR),
            mountEntry->MountControl.DeviceName);
    }
    RemoveMountEntry(mountEntry);
} else {
    mountEntry->MountControl.Status = DOKAN_CONTROL_FAIL;
    Control->Status = DOKAN_CONTROL_FAIL;
}

In normal case the record is found and DokanControlUnmount is called and DefineDosDevice(DDD_REMOVE_DEFINITION, drive, NULL) is executed. And previously mounted volume is detached. Everyone's happy. In normal case. But when DokanMounter service is being restarted it looses mount entry list and when DOKAN_CONTROL_UNMOUNT event comes, dokanx_mount.exe doesn't know what to do with such mount entry (unless DOKAN_CONTROL_OPTION_FORCE_UNMOUNT option is specified, but it can be specified only if unmount was performed via dokanx_control utility).
The point is that after DokanMounter service is being restarted, unmounting fails (unless force unmount) and after some time something memory-incorrect happens in kernel and BSOD occurs.

Again, I have very few knowledge about drivers and it would take lots of time for me to discover what and why happening in kernel mode, so if anyone willing to dig up the truth - you are welcome.
For now I have some solution-like-thoughts :

1. Why do we need DOKAN_CONTROL_OPTION_FORCE_UNMOUNT? If we remove check for that option and will try to unmount everything that being requested, what problems will it cause? Something like this:

mountEntry = FindMountEntry(Control);
if (mountEntry == NULL) {
    if (DokanControlUnmount(Control->MountPoint)) {
            Control->Status = DOKAN_CONTROL_SUCCESS;
            break;
    }
    Control->Status = DOKAN_CONTROL_FAIL;
    break;
}
if (DokanControlUnmount(mountEntry->MountControl.MountPoint)) {
    Control->Status = DOKAN_CONTROL_SUCCESS;
    if (wcslen(Control->DeviceName) == 0) {
        wcscpy_s(Control->DeviceName, sizeof(Control->DeviceName) / sizeof(WCHAR),
            mountEntry->MountControl.DeviceName);
    }
    RemoveMountEntry(mountEntry);
} else {
    mountEntry->MountControl.Status = DOKAN_CONTROL_FAIL;
    Control->Status = DOKAN_CONTROL_FAIL;
}

I tried this approach and everything went fine (no BSODs) except for BOOL SendReleaseIRP(LPCWSTR DeviceName) function fails because of GetRawDeviceName fails, because there were no entry list and therefore required entry wasn't found. The device wasn't release as I understood, how bad is that?
2. Another approach - is to create some kind of mount entry list serialization/deserialization for DokanMounter service, so when it is being shut down, it would dump this list to somewhere and when it comes back alive it would know all about mount entries. This approach is kinda row yet, it should take into account situation when fs-app detaches when service is still down, or when new fs-app being attached. Maybe combine those two approaches...

That is it. Thanks for reading. I would like you guys to talk about these matter. Please fill free to correct me, give advices, any kind of thoughts. For now I'll use first approach, but I thinks it is not fully correct and I'm not going to pull it yet.

Code comments language

Could you please translate all comments written in korean into english?
Thanks.