thisismypassport / shrinko8 Goto Github PK
View Code? Open in Web Editor NEWShrink (minify) Pico-8 carts, as well as other tools (e..g linting, format conversion)
License: MIT License
Shrink (minify) Pico-8 carts, as well as other tools (e..g linting, format conversion)
License: MIT License
I am getting this error.
input chars: 462515 705.75%
input tokens: 100740 1229.74%
tokens: 100740 1229.74%
chars: 462515 705.75%
compressed: 124640 798.16%
Traceback (most recent call last):
File "shrinko8.py", line 210, in <module>
File "shrinko8.py", line 195, in main
File "pico_compress.py", line 25, in write_compressed_size
File "pico_compress.py", line 463, in compress_code
File "utils.py", line 983, in u16
struct.error: ushort format requires 0 <= number <= 0xffff
[4184] Failed to execute script 'shrinko8' due to unhandled exception!
As you can see there's a quite big input here. I moved from PICO8 to PICOLOVE and use shrinko8 to compile all the Lua/p8 scripts into one, minified .p8 file. If possible, I would like to get some more info on this error: on what's going on, if it is related to my source.
And yes, a great tool, thx for creating it.
reported here: https://discord.com/channels/215267007245975552/1131687395088142357/1131687395088142357
minimal reproduction:
echo 'local x=?"hi"' | shrinko8 - - --minify
Compilation errors:
-:1:10: unknown expression
(note that echo 'local x=print"hi"' | shrinko8 - - --minify
works as expected, producing local l=print"hi"
)
This command:
echo 'print(--[[member]]"apple,banana,~coconut")' | python3 shrinko8.py - - --minify --format lua
Produces this output:
print"a,p,~coconut"
But I expect this output:
print"a,p,~c"
Is there a way to add more delimiters here, besides just commas?
This snippet:
print("test"..(@16).."str")
Minifies to this:
print("test"..@16.."str")
Which doesn't compile under pico-8 0.2.5g.
An option to export the sprite sheet would be nice.
My apologies if this is not a bug, I have no idea how the __meta:*__
tag works or what it does, and can't find any documentation on it other than it being briefly mentioned in the 0.2.4 changelog.
Shrinko8 appends this line to the end of the cart when minified:
__meta:title__
>8
Those lines' presence in the cart causes Pico-8 to print this message to stdout in a Linux shell (not stdout, though I'm not sure if Pico-8 would differentiate between the two).
codo_free fail 21 0
It seems like some kind of warning message?
Anyway, thanks for making this great new tool!
When 1.p8 has no __label__
section, 2.p8 will not have one either:
shrinko8 1.p8 2.p8 --label label.png
Workaround: shrinko8 1.p8 temp.p8.png && shrinko8 temp.p8.png 2.p8
I tried minifying an already-somewhat-hand-compressed cart to see what I missed. shrinko8 seems to have deleted spaces between a series of assignments in such a way that =0b=
appeared as a substring.
Before (w/o _min
) / after (w/ _min
) .p8 files:
not-working script:
#!/usr/bin/env bash
# usage:
# ./compress.sh # build + minify
# ./compress.sh 1 # build only
MINIFY="--minify"
if [[ -n "$1" ]]; then
echo "skipping minification..."
MINIFY=""
fi
/w/shrinko8/shrinko8.py game.p8 tower.p8.png --lint --count "$MINIFY" --script compress_combine.py
when I ./compress.sh
, shrinko8 runs normally. when I ./compress.sh 1
, it shows the usage string and says shrinko8.py: error: unrecognized arguments:
, which is unexpected (I expect it to run normally)
I assume the problem is because the command ends up having two spaces in it, which confuses the argument parser, but I haven't looked into it.
My shrinko8 commit hash is 8a4f6e0
workaround:
#!/usr/bin/env bash
# usage:
# ./compress.sh # build + minify
# ./compress.sh 1 # build only
MINIFY="--minify"
if [[ -n "$1" ]]; then
echo "skipping minification..."
MINIFY="--count" # hacky
fi
/w/shrinko8/shrinko8.py game.p8 tower.p8.png --lint --count "$MINIFY" --script compress_combine.py
Pico-8, unlike standard lua, supports nested comments like "--[[ bla [[ bla ]] bla ]]".
It seems like the sort of thing that's part of the language rather than a mistake, so I should probably support it.
(Noticed by Heracleum on discord)
Am I misunderstanding the readme? I can't find a combination of flags that makes something like a,b = circfill, rectfill
appear in the output:
--preserve: !circfill, !rectfill
circfill, rectfill = --[[preserve]]circfill, --[[preserve]]rectfill
circfill(10,10,20); circfill(90,90,30)
rectfill(0,0,100,100); rectfill(20,20,40,40)
--preserve: !rnd
rnd = --[[preserve]]rnd
rnd(12)
shrinko8 --minify test.lua - -f lua --no-minify-spaces
circfill, rectfill = circfill, rectfill
circfill(10,10,20) circfill(90,90,30)
rectfill(0,0,100,100) rectfill(20,20,40,40)
a = rnd
a(12)
This doesn't actually affect me, its just a bug(?) I noticed recently.
Notice that it does work for rnd
, likely because rnd is (correctly) missing from this list?
I want to add per-function token and character counts, pre and post minification, to https://github.com/sparr/pico8lib. My initial idea was to use a script to parse the functions out of a source file, then feed them separately to shrinko8 -c
, parse the output, prepend it as a comment to the function, then reassemble the source file. However, it occurs to me that a lot of the necessary functionality to do this better already exists in this project.
My proposal is a mode for shrinko8 that does all of this directly. Parse a cart, identify all the functions, count their tokens and characters, minify them, count those tokens and characters, insert a comment into the original cart above the function (possibly before existing comments, and overwriting a previous comment from this process) and emit the annotated original cart.
I'm willing to do most of the work to make this happen, but I find the code here a bit daunting. I've made it to process_code
where it calls tokenize
then parse
, and I can look through Node
s under root
, but I am not sure how to get back from a Node
to tokens that count_tokens
wants, or how to turn a Node
back into a cart after I've modified some of its grandchildren. Generally moving back and forth between the source, tokens, and nodes, in a way that keeps them mapped to each other for modifying one based on the other, is a mystery to me so far. I would appreciate any pointers you can share before I get started on this. Even if this isn't a welcome feature for the main repo, I'd still like to pursue it as a fork or patch so I can use it myself.
This cart, when minified, will crash:
function --[[preserve]]obj_new()
return {}
end
function _init()
obj=obj_new()
end
The minified code (with line breaks) is:
function obj_new()
return{}
end
function _init()
n=u()
end
Strangely, if I use either the global
or member
annotations instead it works fine. It also works fine with no annotation.
Is this maybe the wrong way to preserve the names of these functions?
Shrinko8 brings the uncompressed code of DAGGER WULF down to 1009 bytes using Balanced - Aggressive Minification 😲
However, pasting the compressed code into Pico8 0.2.5g reports 1021 bytes
Is shrinko8 (at least the web version) miscounting UTF-8 characters ?
If that is of any help, PXAVIZ does report 1021 bytes
for y=0,62do
for x=0,79do
-- draw sprites
-- 1 = grass
-- 2 = palm tree
-- 3 = rocks
-- 4-7 = amulet pieces
-- 8 = spider?
-- 9 = player
-- 10 = hut
sset(x+8,y%7,"0x"..(
"0000000003bbbb00dd505d50000000000000000000a0aaaaaaaa0a0000288820000ccd0000a9a900000000003bbb3bb0dd50dd5000000000aaaa000000a0a0a0aaaa0a00028888200ccccd000aaa9a9000000000b32bb3b0555055500000aaaaa00aa00000a0000aaaaa0a000288822000a4acc0aaa999a900000100b042b3b000000550000aa0000a00a00000a00aaa0000aa0088822220000aa0000a4004a0001001000022b0b05dd50000000a00a00aa0a00000aa0000aaaaa0002828208000a88a00a400004a0001000004422000dd550d50000a00aaaaa0a000000aaaaaa000000088808080000880000400004000000000044420005555055000aa00a0a0a00a000000000000000000020080000044040004000040"
)[x+1+y%7*80])
-- pick room tile out of
-- 9 prefab rooms
i=
("222022222002222200002000000020000022020002222022200002222000002222020200002000022202202000222202222220222220002220000020020200200020020222222000022222022220000022000002000000020000022000002222022200000222200222022000000000200000020022222200002222000000222220200000000220220000000022200220000022000002202220020200000000220002002000222202000000222202222000002202220200200002022202200000222202222220222200020220222000020000202022220000022220000")
[(x\7+y\7*2)%9*49+x%7+y%7*7+1]
-- wall the whole map
if(x*y<1or x==62or y==62)i=2
-- draw map
if(x<=62)mset(x,y,i+rnd(2))
end
end
-- health
ph=0
-- beasts table
beast = {}
-- move entity if possible
function clear(d)
yy=z.y+sin(d/4)
xx=z.x+cos(d/4)
if(mget(xx,yy)\2!=1)z.x,z.y,z.m=xx,yy,true
end
::_::
-- graphic mode 3 - 64x64
poke(24364,3)
-- wave dagger
b❎=btnp(❎)
-- wait frame and cls
?"⁶1⁶c"
-- when no health
-- intro screen
if(ph<1)then
-- reset count of amulet pieces
-- reset hit
pp=0hit=0
srand()
-- reset amulet pieces
mset(3,3,4)
mset(62-3,3,5)
mset(62-3,62-3,7)
mset(3,62-3,6)
-- reset hut
mset(31,31,10)
-- draw grass and reset beasts
for x=0,62do
p={x=32;y=32}
beast[x]={x=rnd(60)\1;y=rnd(60)\1,h=1}
spr(1,x,rnd(124))spr(4,32-8,32,2,1)spr(6,32-8,32+8-1,2,1)end
-- draw title and co ♪
?"⁶w⁶td𝘢𝘨𝘨𝘦𝘳\n`w𝘶𝘭𝘧",8,8,8
?"@𝘱01 #𝘱𝘪𝘤𝘰1𝘬",8,58,5
?"❎ to start⁷szdagerwf",10,50,t()*62
-- if dagger -> start game
if(b❎)ph=5
else
-- if some health
-- game loop
-- reduce hit intensity
hit*=.9
-- draw the move
map(0,0,4-p.x\7*56-hit\1*sin(rnd()),4-p.y\7*56,64,64)
z=p
z.m=b❎
-- play dagger sounds ♪
if(b❎)?"⁷ce"
-- play hit sound ♪
if(hit>1)?"⁷i6c"
-- move the player
if(btnp(➡️))clear(0)
if(btnp(⬆️))clear(1)
if(btnp(⬅️))clear(2)
if(btnp(⬇️))clear(3)
-- draw the player
spr(9,4-p.x\7*56+z.x*8,4-p.y\7*56+z.y*8,1,1,t()%2>1)
i=mget(z.x,z.y)
-- pick up a piece of amulet? ♪
if(i\4==1)pp+=1mset(z.x,z.y,0)?"⁷szc-g"
-- reached the hut with
-- full amulet! ⌂♪
if(i*pp==40)?"well done⁷szdagerwf",14,19,10
-- beast loop
for x=0,62do
z=beast[x]
-- if it's alive
-- draw it
-- if the player moved
-- move randomly
-- if it touches the player
-- die if the player used the dagger
-- else hit the player
if(z.h>0) spr(8,4-p.x\7*56+z.x*8,4-p.y\7*56+z.y*8,1,1,(x+t())%2>1) if(p.m) clear(rnd(4)\1) if(p.x==z.x and p.y==z.y) if(b❎) z.h=0 else hit=2 ph-=1
end
-- draw health and
-- number of pieces of amulet
?ph.."웃 ◆"..pp,1,2,0
?ph.."웃 ◆"..pp,0,1,0
?ph.."웃 ◆"..pp,2,1,0
?ph.."웃 ◆"..pp,1,1,t()*62
end
goto _
Hey! I don't know if this is really an "issue" per se, it's more of a curiosity that I thought you might be interested in. I found a large cart and idly wondered how well shrinko8 would compress it, but it seemed to get caught in an infinite loop! but when I added some debugging print
s, it wasn't actually stuck, it was just very slow:
> time shrinko8 '#amiquest-4' --bbs --count --no-count-tokenize
chars: 58092 89%
compressed: 15450 98.94%
I timed this a few times and got 10 or 11 seconds each time:
10.77s user 0.04s system 98% cpu 11.002 total
11.24s user 0.04s system 97% cpu 11.563 total
10.94s user 0.05s system 98% cpu 11.203 total
In comparison, a cart of mine only takes 2 seconds to compress; it also is very full of stringified sprite/map data.
> time shrinko8 '#thetower-0' --bbs --count --no-count-tokenize
chars: 27851 42%
compressed: 12498 80%
I timed it a few times:
1.15s user 0.03s system 81% cpu 1.439 total
1.11s user 0.04s system 89% cpu 1.290 total
1.17s user 0.03s system 89% cpu 1.347 total
Maybe this is just a case of "yep, that's expected with the compression algorithm". (I tried amiquest with --fast-compression and it dropped down to 2 seconds as well) But I did think shrinko8 was stuck in an infinite loop, possibly a progress bar of some kind might be nice?
Here's a bug that seems to require the bizarre combination of flags --focus-compressed
and --no-minify-spaces
. I ran across it while debugging -- that's why I had --no-minify-spaces
if(x==1)?1
if x==0 then
?2
?3
end
shrinko8 test.lua - -f lua --minify --focus-compressed --no-minify-spaces
if i==1then?1 end if i==0 then
?2
?3
end
(the output is invalid pico8 code; ?1 end if
needs a newline in there somewhere)
Is there a way to create a "tiny" rom with this tool? the compressor works better than pico8's default, but if I can't get that output as a tiny rom (like you can get in pico8 by typing export -t out.p8.rom
) then, for example, the better-compressed result can't be entered into sizecoding contests.
The only way I know how to export ROMs from shrinko8 is with shrinko8 game.p8 out.rom
, but out.rom is always a 32K file.
for reference, see the "PICO1K Tools" section of the 0.2.5 release notes or the "tiny cartridges" section of the 0.2.4 release notes. (I don't imagine they're too helpful to you but maybe export -t @clip
would be useful while debugging)
This cart runs fine in pico8; it prints 9 numbers with some colors:
-- test2.lua
function _draw()
cls()
for --[[preserve]] n=1,3 do
for i=1,3 do
print(i+1,n+1)
end
end
end
(btw be careful with that annotation -- the spaces are required)
But minifying breaks things -- the minified cart is upset because o+1
is nil arithmetic:
shrinko8 --minify -f lua --no-minify-lines --no-minify-spaces test2.lua out.lua
-- out.lua
function _draw()
cls()
for n=1,3 do
for n=1,3 do
print(n+1,o+1)
end
end
end
another similarly bugged example:
> echo 'function foo(--[[preserve]] x) return x+1 end' | shrinko8 --minify -f lua - -
function n(x)return n+1end
(I would expect function n(x)return x+1end
instead)
I initially tried to preserve these vars from the commandline (--preserve 'n'
) but it didn't work. However, local vars declared like local n=3
were correctly preserved. Maybe this is related?
My shrinko8 is the commit tagged v1.1.2
("avoid bxBX...")
Compress fails with:
ValueError: '■' is not in list
or:
---> 11 write_cart("carts/new_cart.p8.rom", new_cart, CartFormat.rom)
File [c:\Users\Frederic\Source\Repos\hexen\.venv\lib\site-packages\pico_cart.py:711](file:///C:/Users/Frederic/Source/Repos/hexen/.venv/lib/site-packages/pico_cart.py:711), in write_cart(path, cart, format, **opts)
709 file_write(path, write_cart_to_image(cart, **opts))
710 elif format == CartFormat.rom:
--> 711 file_write(path, write_cart_to_rom(cart, **opts))
712 elif format == CartFormat.tiny_rom:
713 file_write(path, write_cart_to_tiny_rom(cart, **opts))
File [c:\Users\Frederic\Source\Repos\hexen\.venv\lib\site-packages\pico_cart.py:101](file:///C:/Users/Frederic/Source/Repos/hexen/.venv/lib/site-packages/pico_cart.py:101), in write_cart_to_rom(cart, with_trailer, keep_compression, **opts)
99 w.bytes(cart.code_rom)
100 else:
--> 101 compress_code(w, cart.code, **opts)
103 if with_trailer:
104 w.setpos(k_cart_size)
...
475 w.u16(size & 0xffff) # only throw under fail_on_error above
477 else:
--> 478 w.bytes(bytes(ord(c) for c in code))
ValueError: bytes must be in range(0, 256)
Repro code:
from pico_cart import write_cart, CartFormat, Cart
new_cart = Cart()
new_cart.rom[0x0000:0x2000] = bytearray(0x2000) # zero it out
new_cart.set_code("""-- my game
-- @freds72
local title_gfx={bytes="\0\0\0\0\0\0\0\0\0¹\0\0\0■\0\0■▮\0■■■■■■■■"}
function _init()
end
""")
write_cart("carts/new_cart.p8.rom", new_cart, CartFormat.rom)
note: the binary string has been copied from Pico (eg. valid)
(We've talked about this a bit before, but I've found a nice clear use-case for this, and figured I'd make an issue to hold discussion.)
This cart is a simplified version of my pico1k jam entry:
-- gitignored/test.lua
x,lvl=8,1
::_::
menuitem(1,"level "..lvl,function(b)
lvl+=b\2%2-b%2
menuitem(nil_,"level "..lvl)
end)
b=btnp()
x+=b\2%2-b%2
flip()cls()
spr(0,x*8,64)
goto _
Notice that b\2%2-b%2
is repeated. It would be great for it to be repeated in the output as well, but unfortunately it uses different variable names (e\2%2-e%2
and n\2%2-n%2
).
> shrinko8 gitignored/test.lua out.lua --minify --focus-compressed --no-minify-lines --count
tokens: 60 1%
chars: 145 0%
compressed: 124 1%
-- out.lua
e,l=8,1
::e::
menuitem(1,"level "..l,function(e)
l+=e\2%2-e%2
menuitem(o,"level "..l)
end)
n=btnp()
e+=n\2%2-n%2
flip()cls()
spr(0,e*8,64)
goto e
If I manually change the varnames, I can get it down to 120 compressed, but I want the renaming to happen automatically every time I build my game. I don't want shrinko8 to be smarter and do it automatically -- I just want more control over this somehow.
How can I get the out.lua to use the same variable names in the e\2%2-e%2
/n\2%2-n%2
lines?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.