danielga / luapack

A system for Garry's Mod that provides Lua files to clients in a more efficient way. WIP

LuaPack

A Garry's Mod addon that attempts to speed up Lua files downloading by compressing and packing them.

Info

garrysmod/lua/send.txt must be renamed to garrysmod/lua/_send.txt

garrysmod/lua/includes/init.lua must be renamed to garrysmod/lua/includes/_init.lua

This addon requires the gmsv_luapack_internal module.

luapack's Issues

Reduce binary modules dependencies

luaiox is a dangerous module which opens up a nest of very nasty stuff.
crypt is nice and all but is very bulky and has way too many features we don't use.
With an internal module we could group all the required features together like CRC32 (GMod already has this but I don't trust its performance and how we're forced to push all data at once, which can be quite big on Lua files), SHA-1, file writing and required detours for AddCSLuaFile.

Improve pack file structure

Include a version number on each pack file so the appropriate protocol is called on the decoder.
Handle files that are changed rather frequently or some similar system (based on signatures?).

Module not found

Hello

After successful compilation (windows) gmod says ' module not found ' ( sv_core.lua:25)
but gmsv_luapack_internal_win32 is in lua/bin

Is it still working in 2018?

Improve pack file creation times

Currently, the pack files on Metastruct takes between 8 to 20 seconds (how they vary so much, I don't know) but usually take around 10 seconds. They are compressed and packed. Maybe don't compress in Lua at all and use an internal binary module to do this with threads?

Lua files not being sent to client

client error that lead to init.lua not loading and crashing the game

server

said file

could be the size limit thing but im not sure

gm_crypt dependency ??

There seems to be a dependency to gm_crypt but it there isn't any binaries for linux.
I'm a stupid man, is it possible to have a binary for gm_crypt for linux ??

Automatic disposal of old pack files

Possibly use date checking or just remove all pack files that are not pertinent to the current pack.

Not working server

I did everything according to the instructions

Some Lua files are not run successfully

Hello again :)
Transferring the issue from: danielga/gmsv_luapack_internal#7
Something is wrong with mounting files from the pack. I tried to debug it a bit, looks like some files are mounting but init fails and ents / vgui, everything is broken.

Server:

Client:

Pack successfully downloaded.

Broken init:

Captured on windows, server files are clean.

Doesn't work for me 🤔

luapack to addons folders and gmsv_luapack_internal.dll to garrysmod/lua/bin right?

workshop issue

when your server forces clients to download map from workshop, you get kicked for missing map because it skips the workshop loading thing and starts to search for the map in fastdl

debug.getregistry fix inside includes/init.lua

i'm sure you knew but rubat destroyed debug.getregistry, and appended a temporary fix to includes/init.lua

Here's my version of the fix, and below that is his version.

local _R

OldFindMetaTable = OldFindMetaTable or FindMetaTable
function FindMetaTable(name)
	return name and _R[name] or nil
end

do
	_R = setmetatable({
		[1] = function() end,
		[2] = _G,
		[3] = {},
	}, {
		__index = function(obj, key)
			local tab = key and oldFindMetaTable(key)
			if rawget(obj, key) then
				return rawget(obj, key)
			elseif tab then
				obj[key] = tab
				return tab
			else
				return nil
			end
		end
	})
end

function debug.getregistry()
	return _R
end

-- Temporary hack
local meta = {}
function meta.__index(self, key)
	return FindMetaTable(key)
end

local metas = {}
function meta.__newindex(self, key, value)
	metas[ key ] = value
end

debug.getregistry = function()
	local tbl = {}
	setmetatable(tbl, meta)

	return tbl
end

local oldFindMetaTable = FindMetaTable
FindMetaTable = function( name )
	local f = oldFindMetaTable( name )
	if ( f ) then return f end

	return metas[ name ]
end

"include" does not return a value

luapack/lua/luapack/cl_overrides.lua

Lines 80 to 97 in 7d602d0

	function include(filepath)
	local time = SysTime()
	local obj = GetFileFromPathStack(filepath)
	if obj ~= nil then
	CompileString(obj:GetContents(), obj:GetFullPath())()
	luapack.AddTime(SysTime() - time)
	return
	end
	luapack.DebugMsg("Couldn't include Lua file from luapack, proceeding with normal include", filepath)
	luapack.include(filepath)
	luapack.AddTime(SysTime() - time)
	end

---

Wiki

https://wiki.facepunch.com/gmod/Global.include

---

Expected

The result of the script execution (vararg)

---

Now

Just compiles the code, and does not return the compilation result. There is a chance to break scripts that are dependent on the result of execution. For example:

local BaseClass = include('mod_core/classes/base_class.lua')

Engine Error

keep trying to install this to my sandbox server and this keeps poping up other than that it seems to be working fine besides crashing upon join, after i remove the luapack stuff the server works fine off of the workshop dl.

Failed to open pack file

[ERROR] addons/luapack/lua/luapack/cl_core.lua:71: failed to open pack file 'download/data/luapack/D4AE9D00F4AE58093E883DEE61928D7A696F2B13.dat' for reading
  1. error - [C]:-1
   2. BuildFileList - addons/luapack/lua/luapack/cl_core.lua:71
    3. unknown - addons/luapack/lua/luapack/cl_core.lua:95
     4. include - [C]:-1
      5. unknown - addons/luapack/lua/includes/init.lua:11

I got this error upon join. It created the file on the server, but it isn't in the mentioned folder on the client. There's no console error before this one.

Failed client init

Hello

I'm fighting with this sice first time opened an issue for luapack :/
Everything is set correctly for sure.

When I connect to server, everytime init fails:

[LuaPack] Found the current pack file hash ('313C9C317B2A3E9E49AEDA1B2754D5A8309A5D2A')!
[LuaPack] Starting Lua file list build of 'download/data/luapack/313C9C317B2A3E9E49AEDA1B2754D5A8309A5D2A.dat'!

[ERROR] lua/luapack/cl_directory.lua:177: attempt to get length of local 'list' (a nil value)

1. GetIterator - lua/luapack/cl_directory.lua:177
2. GetSingle - lua/luapack/cl_directory.lua:156
   3. AddFile - lua/luapack/cl_directory.lua:73
   4. BuildFileList - lua/luapack/cl_core.lua:83
   5. unknown - lua/luapack/cl_core.lua:95
   6. include - [C]:-1
   7. unknown - lua/includes/init.lua:9

Couldn't include file 'derma\init.lua' (File not found) ()
Couldn't include file 'includes\modules\notification.lua' (File not found) (@startup (line 1))

-----> and spam of errors here, everything no value, client init totaly dead.

I have no more ideas, tried everything.

🌵

Server and client files are clean, no serverside errors.

Issue when used along side Server Secure (I believe could be related to finding a file)

Error Recieved:
`[ERROR] gamemodes/darkrp/gamemode/init.lua:16: attempt to index global 'serversecure' (a nil value)

1. v - gamemodes/darkrp/gamemode/init.lua:16
2. unknown - addons/plib_v2/lua/plib/libraries/hook.lua:25
   `

Other then that issue showing up server starts up perfectly with no other lua error except for when a GM hook tries to use a function inside serversecure. Other modules such as tmysql work perfectly fine however serverside.

On joining the server tho I have several files that are trying to be included in various places that are showing up as 'file not found'

My guess is some how not being able to properly recognize where the files are located?

Other then this I don't know what other information to give besides that I have verified that the send.txt is renamed to _sent.txt, the default lua/includes/init.lua is renamed to _init.lua and the serversecure.lua is in lua/includes/modules/ where it should be.

I hope this information is somewhat useful.

p.s. Thank you for the work you've done with these amazing modules