mhtvssfrphde / prefetch Goto Github PK
View Code? Open in Web Editor NEWLoad file into ram for fast first access.
License: MIT License
Load file into ram for fast first access.
License: MIT License
I personally don't use RescanInterval, so it's not covered by "user test".
Just discovered it has a wrong default value, but not trigger any bug.
After #26, the RescanInterval will change from optional to necessary.
I will ignore this config related things until that.
Since there is going to add pause, why not pause automatically if a certain process is running?
By learn video and document, seems there were no something called "perfect pause at any time".
The implement is use cross thread variable, and tons of if statement to check the variable is changed or not.
I personally don't like to see I do one thing a time, but do 150 times to check on a variable.
When I say a pause, I went to gaming or performance matter task.
After I return, I probably don't want to resume from where I left.
Only god know what's in the RAM now. I will continue to prefetch from the start point.
The only case I pause is I don't want to rescan, keep scan result in RAM.
Or I just simply close the program, later start again is also a solution.
So the solution should be done on thread entry, I remember there is a "skip" variable.
That's a chance. The variable has an initial value on creating a thread instance.
I will get the initial value from global, instead always use "false".
Once I set global variable to "true", all new created read thread will inherit "true".
When they are fired, will return immediately and do nothing.
The thread pool runner will fire all the thread anyway.
It will change to only check global pause status once before fire any thread.
If thread pool already running, for example 47/1055,
all 1055 thread will fire, and counting on "skip" variable.
The reason mentioned before, I don't like to see every time I do something,
but first I checked a suddenly appeared variable.
I have PrefetchInterval
, the one currently is use sleep on loop thread.
After send resume command, the rest of interval time should discard and do prefetch now.
So instead of sleep on the loop thread, the sleep will change to a new thread,
and loop thread will try to join the sleep thread.
In this case, when it comes to resume, I terminate the sleep thread,
prefetch will start immediately.
#include "samename.h"
#include "B.h"
#include "samename.h"
I find this was really confused in some case.
They mixed with other potential bug, and can cause project failed to compile,
takes many hours to find why.
Rename the file to samename_a.h
and samename_b.h
.
// Delete excluded file thread
readThreadPool->waitForDone();
auto dbg_PendingDeleteThread = &ReadThread::pendingDeleteThread;
for (int i = 0; i < ReadThread::pendingDeleteThread.size(); ++i)
{
auto threadPointer = ReadThread::pendingDeleteThread[i];
readThreadQueue.removeOne(threadPointer);
delete threadPointer;
}
ReadThread::pendingDeleteThread.clear();
In Qt Creator debugging, the local and expression window will not show,
and the debugger will not get a remote variable value.
For example, a static variable on another class is accessible, and its value is matter for debugging.
However, without this unused pointer variable, mouse hover on it shows nothing.
With this unused pointer variable, mouse hover can show the value.
https://rotadev.com/console-output-in-a-qt-gui-app-dev
Windows does not really support dual mode applications.
https://www.faqcode4u.com/faq/263030/hide-console-of-windows-application
A C# example shows use platform specified API to switch current console:
[DllImport("Kernel32.dll")]
private static extern IntPtr GetConsoleWindow();
[DllImport("User32.dll")]
private static extern bool ShowWindow(IntPtr hWnd, int cmdShow);
static void Main(string[] args)
{ Console.WriteLine("Press any key to hide me.");
Console.ReadKey();
IntPtr hWnd = GetConsoleWindow();
if (hWnd != IntPtr.Zero) {
ShowWindow(hWnd, 0);//hide
Thread.Sleep(5000);//5s
ShowWindow(hWnd, 1);//show } Console.ReadKey();
The skip variable is instance variable, and do snapshot before consume thread pool.
Thus, each file thread require another if condition on static pause variable.
Perhaps use a thread to consume thread pool, when paused, terminate it like sleep thread.
Need more investigation on how to properly run a Qt thread.
For example, currently I don't have a thread worker, but call start method directly.
Now, all thread are create and save to the thread pool
if a file is excluded, the thread will run and return immediately.
Some cpu is wasted during the process
Try to delete them after excluded.
I need a dynamically size list instead of a fixed size array.
I believe someone could simply create a Task Scheduler item to do such thing on startup?
StdIn display widget have no size limit set right not, after long time run, generated output can fill the RAM.
I feel like a century has passed, but Idle, Time: 13.35 Sec
.
Confirm if created and use on different thread.
Prepare
9ce8a55lrelease
to compile translate fileprefetch.exe
Standard versionprefetch_verbose.exe
Standard +Logprefetch_performance.exe
Standard -Translatepack\translate
By read these link:
https://docs.microsoft.com/en-us/windows/win32/memory/file-mapping
https://stackoverflow.com/questions/31349539/how-file-mapping-works
Here is the guess about why kernel cache will not override mapped file (virtual machine disk file):
Virtual machine programs sure not to load the entire 120 GB into system RAM,
they use address via file mapping, and when they do I/O,
Windows handle the disk access and RAM cache.
When a process does CreateFileMappingW
or something, they say:
"Hey, I'm important."
During the entire system does I/O, the one created file mapping has higher priority.
Unless explicit clear standby list, they won't shrink their size in RAM until closed.
A test can give more details (doing the test, not complete yet):
In this version, command are hard coded string everywhere.
Change to const variable can prevent grammar related bug.
Firefox store its extension data along with other website data under
Profiles\default\storage\default
with folder named moz-extension+++*
.
However, the parent folder storage
is excluded because
it's wasting of time to prefetch rare access website data.
I need to prefetch extension data, so any file path contains moz...
will have higher priority than under excluded folder.
Open other exe to do specified things like clear standby list.
Similar to startup item, but run on user click event.
With this, many features no need to integrated to software.
Provide alternative to replace Windows startup manager,
to make sure these program start in cached environment.
Hello @mhtvsSFrpHdE ,
EDIT: sorry, now I have seen this link:
https://github.com/mhtvsSFrpHdE/prefetch/wiki/Frequently-asked-questions
I have discovered recently some tools like Primocache and Ultra RAMDisk, and this repo too.
What do you think about these other alternatives pro/cons vs this prefetch repo?
My intention is to speed up a bit my Visual Studio workflow.
Some VS files are located on c:/program files, and usually the project files are in another location, and the framework/libs you are using somewhere else...
Then, the RAM disk way I am not sure that could help here,
because the files are not in the same folder,
or maybe we don't want to include many folders that could make too bigger for the available RAM memory.
What do you think about a good approach to start think the workflow?
Thanks for the repo!
Regards
Seems request safe quit read loop, this part can utilize pause variable and to trigger an event after read loop is done.
Type command moo
, draw the apt moo, text 扣1佛祖跟你一起moo
,
then type command 1
, draw Gautama Buddha says moo
I have a static variable in class, if add declaration in cpp, the cpp report Multiple definitions
.
if remove declaration in cpp, other cpp report Undefined references
.
The syntax is absolute correct, QString
and std::wstring
not work, but int
work.
In 2.3,
unused file thread are removed after first prefetch.
I notice that the thread overhead is almost zero,
but Windows take vast of time to iterate through the file system and list all file.
This is much slower than expected.
Instead of scan all 26968 files, use a config array as cache?
Well, try NOT to stick a wood branch into a spinning bike wheel. Especially if you are riding it.
When MainWindow from background to foreground, there may be two or more blank line in console.
If any text update triggered, blank line will gone.
It seems background update won't trigger certain event.
If hide emit something()
This can bring better forward compatible,
for example, less breaking change, version 1 config can upgrade to version 15 with no pain.
Run startup items...
printed multiple times 7447a9cWhen deploy multiple program instance, copy all program files to new folder is required
because now program always read prefetch.ini
.
By command argument, same program instance can start with different ini file.
Speed up cpptools only parse project file
VSCode not support exclude a folder but include subfolder
microsoft/vscode#869
lang en_US
command
Language
, TranslateFolder
Steam client just downloaded a game "the witcher 2",
after that, m/prefetch
report Time: 43.58 Sec
every time,
even if available RAM is 24.9 GB and cached is 24.8 GB.
That shows, if cached is full, Windows won't free unused cache for new cache content.
The result is low performance. This explained why my Firefox always read from disk:
when Firefox requested cache, the cached space is full.
By use sysinternals RAMMap\File Summary
, there are pack0.dzip
, en0.w2speech
cached in standby list, pack0.dzip
is 9.79 GB, en0.w2speech
is 1.01 GB.
The computer download and update games whole night.
Until I awake and discover Firefox is slow, I never open that game before.
That shows Windows is extremely stupid to judge what are important cache to keep.
It may just a "first come, first served, until full" simple design.
New cache won't free old cache, this only happens if "in use memory" take that space.
Same situation also happen on running virtual machine,
single vmdk file can take 2~9 GB in stand by list.
Use sysinternals CacheSet
to change Working set minimum to 1024 KB,
and set maximum to 204800 KB,
open a VM still left a 2.37 GB cache in standby list.
In this case, the other thing can try is once discovered standby list space
is not enough to hold all cache content, empty it before start prefetch.
I found this: https://gist.github.com/bitshifter/c87aa396446bbebeab29
it seems there is no API to selective delete cache from standby list, and only empty all?
A test shows explorer does pass shortcut to program, but on 22H2, explorer have different rule to solve relative path:
Command "explorer.exe Firefox.lnk"
Use unique lnk file name, or add full path to lnk like this:
"explorer.exe C:\Folder1\Firefox.lnk"
In this version, exit is just "terminate everything",
the program first run into undefined status, then exit via a crash from unknown reason.
Currently, the build system for example
are draft quality, not contain any comment, unmaintainable: discovered by myself after 5 months
Result: Two script merge into one 317a10a
For example, StdOut print 10 line while minimized to tray,
after show, in a period of time, there may only 3 line on console text view until a new StdOut print comes.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.