Comments (9)
I've previously written a similar multipurpose tool in C++. It did work fast, and it could have worked even faster, but I couldn't help questioning myself whether I actually needed that kind of performance or not. If I were to write it again now, I'd probably go with Python (I'm not familiar with Ruby) or try out Go. Python would gather more contributors, if any. Go would run considerably faster.
How slow is quite slow? Unpacking of an archive is rarely done twice, so that shouldn't be an issue. If the packing process is not unreasonably slow, then the shift may not be worth it.
PS: Rust has just hit v1.0 alpha, which is reassuring in terms of stability.
from arc_unpacker.
Hmm... when I implement stuff such as LZSS compressor in Ruby (which uses bit-level arithmetic), it can take up to 40-50 minutes to convert all the graphic files, while the C-powered version crunches everything down in about 2 minutes (yay for unsafe type casting). That's why I keep implementing compressors in C, while implementing everything else in Ruby.
That is to be expected, though: C instructions such as >>
and pointer arithmetic translates almost directly into machine code such as shr
and lea
, while Ruby has to emulate everything in its VM.
I'd go with either Go or Rust. Go seems promising with regard to short compilation times. Although I was aware Rust was going to hit alpha soon, I wasn't aware that they have bold plans to release 1.0 final in, like, just a few months.
from arc_unpacker.
By the way, I'm considering withdrawing the support for compressing/packing.
The reason I keep implementing packers is that they make unit testing really easy: assert stuff == unpack(pack(stuff))
. But the truth is that:
- I should provide some hand-crafted minimal binary files in the unit tests, otherwise packer and unpacker can just return
stuff
and it will go unnoticed by the unit test. - It actually makes little sense to add packing support:
- What's the point of being able to repack files for novels that are already translated?
- Translating untranslated novels needs to be provided with much more than a simple repacker such as this one. So instead of spending time on trying to satisfy everyone in advance, I probably should focus on adding unpack support for more games.
from arc_unpacker.
40-50 minutes? It definitely extends beyond being unreasonable, then. As a wise woman once said, ain't nobody got time for that.
I think it boils down to what your intentions are, and how the tool is supposed to be used. Having a nice and clean codebase is quite helpful when another developer wants to extend the functionality or fix a bug. As long as you don't use a relatively unknown language such as OCaml, it should be fine.
That said, even though most people can figure out how to set up a development environment and to use the command line, non-developers will always prefer having a simple executable file in hand, preferably with a GUI (e.g. AnimED, Crass, ExtractData). This is also true for developers, actually. I don't mind this when I'm working on a translation project, but when I just want to quickly extract the contents of an eroge, I'd rather drag-and-drop the archive on a window and be done with it.
from arc_unpacker.
40-50 minutes if I use Ruby, though. I do the critical stuff in C, so it's sort of acceptable. Regarding the purpose of the tool: frankly, most of archives I support so far can be extracted using other tools, so I guess it boils down to this:
This, and personally, I consider GUI to be a total bloat most of the time. Converting some files is definitely one of these cases. Majority of the tools out there does #include <windows.h>
... why? arc_unpacker
supports drag'n'drop even though it's CLI. It's only dependency is rmagick, which is probably going to go away after I switch languages.
Like I said in the ticket, I'll give it some more time, and when I feel up to the task, I'll try out Rust and Go. They should allow me to:
- avoid the need of having a "dual-language" repository to keep things fast enough
- allow users with no development environment to run the project (Windows package manager when?)
- control the code cleanliness myself
- keep it CLI and portable
Dropping packing support seems reasonable, since (I think) every translation project needs its own hacker anyway. Giving the source code to him allows him to build his own tools and set up any environment he wants, and reversing unpacking from looking at the source code shouldn't be too difficult.
from arc_unpacker.
I finally completed the research, and here are my thoughts from the standpoint of this project:
- Scripting languages are slow and cannot be compiled to standalone .exe, thus making target audience even smaller than it already is.
- Go's main advantage lies in compile speeds and parallel processing. Go's toolchain, workspace management and requirement to set up
$GOLANG
are a deal-breaker to me. - Rust has super weird syntax. I could get over it, but there's another huge disadvantage - compiling hello world results in 3.5 mb exe, which I find totally unacceptable. It might improve in the future, but we're talking about here and now.
- D, Ada and others: too exotic.
- C++. Bloatware.
- C. Yeah... C.
I checked out how developing in vanilla C feels like. After winning an epic fight with necessary evil that is makefile, I found the development in C to be... kind of calming.
- It's as close to machine as it gets before digging into assembler.
- Standard library is minimal, which means there is no temptation to link against bloatware.
- Memory footprint is minimal.
- So is the executable size.
- Performance is as good as the implementation thanks to nonexistent overhead.
make
andgcc
are available on virtually every platform out there.make
andgcc
are more reasonable dependencies than forcing users to download Ruby/Rust/whatever new shiny tool.- As opposed to C++, the differences in standard library implementations are minimal (example reasonable problem with C and example unreasonable problem with C++).
I'll go with C. Results will be committed into c
branch.
from arc_unpacker.
I agree with most of the points, but I'd argue that C++ brings more to the table with no practical cost. You can always pick and choose which features of C++ to use, and continue coding in C where you see fit. As a result, you can write less code and spend less time on dealing with pointers and stuff. I don't have a strong opinion on the matter though, and you should totally use C if you enjoy it more.
from arc_unpacker.
I learned you were right the hard way.
At first, everything went smooth: I had total control over the program, no stuff was happening under the hood, etc. Memory footprint and executable sizes were minimal.
Then I wanted my program not to SIGSEGV when things went bad (e.g. archive was corrupted). Since C doesn't have exception model, I need to check all the function's return values ALWAYS. Not only is this tiresome, it's also totally counterproductive because it makes refactoring more difficult. The only alternative is to use longjmp
, or a nice wrapper for longjmp
such as e4c
that introduces try-catch-finally
keywords to C. This, however, means I need to be extra careful about my malloc
s and put them inside finally
blocks, otherwise I'll leak memory on exceptions (and won't even know about this). So my code still needs to be very verbose, just in another way.
Now C++ suffered the same problems... until C++11 introduced smart pointers. This should allow me to write almost assert
-free code, which sounds great.
from arc_unpacker.
Finally done.
from arc_unpacker.
Related Issues (20)
- Game Request: Memories Off -Innocent Fille-
- Game Request: Wagamama High Spec
- Game Request: Making*Lovers HOT 1
- Add "If You Love Me, Then Say So!" to supported game list
- 'recognition finished with errors (Bad value for option "--plugin".' HOT 1
- help :( with vnmaker
- G-Senjou No Maou (decrypting chunk issue)
- Are there plans for a new release? HOT 1
- Game Request: Oppai Gakuen Marching Band Bu!
- Game Request: Kyonyuu Kazoku Saimin
- Add "Baku Ane ~Otouto Shibocchau zo!" to supported game HOT 1
- boost : : filesystem: :path codecvt to wstring: error
- Front wing's (ISLAND STEAM GAME) HOT 1
- Game Request: Koi x Shin Ai Kanojo
- Game Request: [Silky's] Gakuen Saimin Reido -Sakki made, Daikirai Datta Hazu na no ni-
- Game request: The Runaway Girl And Me - recognition finished with errors (Premature end of file)
- Convert from tlg to png?
- Game request: Dolphin Divers
- Unpack Tsumamigui3 use CLI:--dec=alice-soft/afa report error: recognition finished with errors (Invalid byte sequence) HOT 1
- [Valkyria][140926]Gachinko! B* Club HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from arc_unpacker.