Buffer issues, Lines get cut in the middle about textractor

[Artikash]

Procrastinated to the point I forgot about this. Thx for reminder haha.

https://github.com/Artikash/NextHooker/blob/7a059cc2e0cc317e7cd19b3ca29c718ea15add9b/GUI/host/textthread.cc#L30

[Artikash]

Fixed in 3.2.0 (you can change the split/flush delay in NextHooker.ini)

[Kuraperunat]

I tried 3.2.0 and didn't manage to notice any change when changing in the flush delay outside the default value being lower. It also seems NextHooker resets the .ini file value to 0 after closing it. Maybe a bug somewhere?

[Artikash]

Yeah, dumb typo. Fixed in 3.2.1.

[Kuraperunat]

I gave 3.2.1 a try and while the delay value now seems to affect the flush timing, the initial issue isn't resolved. I believe this is due to the fact that the game spams random numbers constantly, making the buffer split at random times when it keeps hitting the 400 limit.

I fixed this by setting the .ini delay value to 0 and modifying the clipboard extension as follows:

	if (sentenceInfo["current select"] == 1) {

		// This line below should happen before the raw input from the hook hits the buffer!
		sentence = std::regex_replace(sentence, std::wregex(LR"([:\d\.])"), L"");
		if (sentence.size() == 0) { sentence = L""; return true; } // false spams the gui

		static std::wstring prevString = L"";
		static long long prevTime = 0;

		auto currTime = std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::system_clock::now().time_since_epoch()).count();

		if (currTime - prevTime < 180 && prevString.size() < 200)
			sentence = std::wstring(prevString + sentence);

		HGLOBAL hMem = GlobalAlloc(GMEM_MOVEABLE, (sentence.size() + 2) * sizeof(wchar_t));
		memcpy(GlobalLock(hMem), sentence.c_str(), (sentence.size() + 2) * sizeof(wchar_t));
		GlobalUnlock(hMem);
		OpenClipboard(0);
		EmptyClipboard();
		SetClipboardData(CF_UNICODETEXT, hMem);
		CloseClipboard();

		prevString = std::wstring(sentence);
		prevTime = currTime;

		return true;
	}

	return false;

However this is still not the correct way to deal with this, as this causes excessive clipboard copies with the sentences appearing gradually over time like this:

「部屋はもう
「部屋はもうとってるから
「部屋はもうとってるから、　いつでも
「部屋はもうとってるから、　いつでも好きなときに
「部屋はもうとってるから、　いつでも好きなときに使ってくれて
「部屋はもうとってるから、　いつでも好きなときに使ってくれていいよ

The correct way would be to filter the garbage out before it hits the buffer, so it won't cause it to hit the 400 limit all the time. This doesn't seem to be possible with extensions right now and would likely need a new callback for this "pre-filtering" action.

[Artikash]

You'll be able to specify the max size for the sentence buffer in 3.2.2, does that work?

[Kuraperunat]

I doubt it, as the continuous spam will just prevent the buffer from being flushed as it keeps thinking there's more text coming all the time. Therefore increasing the sentence buffer size should simply make it flush less frequently, assuming I've understood the buffer implementation here correctly.

I haven't taken a closer look at the source yet though so let me double check here, is the buffer flush time:

A. a static time interval at which the buffer flushes, or
B. does it represent the maximum time since the last change in the buffer before a flush?

I'm fairly sure B. is the correct way to implement the buffering for these kind of text hookers along with some sort of possibility to filter the input before it even hits the said buffer. I've authored a text hooker / speedhack tool for RPGMaker MV games myself since 2016 and have found this approach to be the most robust for most games.

If I have any extra time these next weeks I can try to take a look at the source and possibly make a PR to exemplify further what I'm after.

[Artikash]

It does B. The buffer gets flushed iff the buffer size is > Max_Buffer_Size OR the last write to the buffer was > Flush_Delay ms ago AND the buffer isn't empty.

[Artikash]

Please try again with 3.2.2 and Flush_Delay 250 and Max_Buffer_Size 100000.

[Kuraperunat]

I tried 3.2.2 with the parameters you pasted and as expected the buffer flushes extremely (like every 10s+) infrequently now. This happens because the constant spam (that I'd like to filter out) doesn't let the timer reach 250ms, and the large buffer size makes it almost never flush. If the spam could be filtered by extensions before it hits the buffer it wouldn't confuse the 250ms timer and the sentences would be split as they should, even if the buffer size was large.

[Artikash]

Finally managed to get off my ass and find/test this game. Now I see what the issue is.
Using hook code /HW4*0@1f150:rgss301.dll works perfectly for me.
Still, you're right. A way to filter before it's added to buffer would be good. Will implement...someday...

[Artikash]

Actually nvm, there's no need. What you can do is Max_Buffer_Size 2000 and use the extension you posted at the start.