nine-lives-later / go-xdelta Goto Github PK
View Code? Open in Web Editor NEWXdelta for Go
License: Other
Xdelta for Go
License: Other
I'm trying to use this on Linux and MacOS, but I see there's only a build script for Windows. Has it been tested on other platforms? Are there any plans to support Linux/MacOS?
If not, could you give pointers on how to add support?
Thanks.
Hi, I am on Mac OS and apparently the library can't find the xdelta3.h file, I see the the submodule is included in the repository, any fix?
Hi,
I have been using bsdiff for a while, however it indexes the patch locations, which either takes a long time for larger files, or (which I've been doing) when I store the index so that I can easily retrieve it rather than generating it every time, It is absolutely massive on disk (~10X the size of the original file.
I am looking for a faster and quicker way to create diffs/patches as for my application they need to happen in (user) real time. I also don't want to have to store an index if I don't have to.
This library therefore looks very interesting. At first glance it also seems much faster. I have put a gist of my demo application using this library, here.
The issue I'm asking about then is the final size of the patch. My patches, using the above code, create patches that are basically identical to the size of the "to file", which in effect is no more efficient than creating copies of the file.
Looking at the original C implementation that I think this is based off, there is a compression option.
However it still seems like I am using it incorrectly, as I would have thought the patch would be just the difference between the original and the final anyway.
From my understanding and how I am using it in that gist is:
Am I doing something wrong here? The only thing I note is that you define a context in your full test, and I just use the context.Background()
Nice work, and thanks, any help working out why my patches are 100% the size of the resulting image would be great!
I found a case where using go-xdelta
is significantly slower than using the xdelta3 executable, to the point of making it unusable. I'd like to share it with you in case you find it useful. I tried to see if I can figure out why myself, but unfortunately I'm not familiar enough with xdelta to do so.
Just because it fit my use case, I downloaded electron 2.0.17 and electron 5.0.12 and tried to patch the electron executable alone, from one version to the other. I generated the vcdiff using the xdelta3 executable, and then tried to apply it both with that and using a simple go program using go-xdelta:
package main
import (
"context"
"fmt"
xd "github.com/konsorten/go-xdelta"
"os"
"path/filepath"
)
func main() {
sourceFile := os.Args[1]
filename := filepath.Base(sourceFile)
diffFileArg := os.Args[2]
patchOutputFileArg := os.Args[3]
sourceFileReaderSeeker, err := os.OpenFile(sourceFile, os.O_RDWR|os.O_CREATE, 0755)
if err != nil {
fmt.Printf("Error: %v\n", err)
return
}
patchOutputFileWriter, err := os.OpenFile(patchOutputFileArg, os.O_RDWR|os.O_CREATE, 0755)
if err != nil {
fmt.Printf("Error: %v\n", err)
return
}
diffFileReader, err := os.Open(diffFileArg)
if err != nil {
fmt.Printf("Error: %v\n", err)
return
}
var decoderOptions = xd.DecoderOptions{
FileID: filename,
FromFile: sourceFileReaderSeeker,
ToFile: patchOutputFileWriter,
PatchFile: diffFileReader,
}
decoder, err := xd.NewDecoder(decoderOptions)
if err != nil {
fmt.Printf("Error: %v\n", err)
return
}
defer decoder.Close()
fmt.Printf("Processing %v\n", filename)
err = decoder.Process(context.TODO())
if err != nil {
fmt.Printf("Error: %v\n", err)
return
}
fmt.Printf("Processing of %v ended\n", filename)
}
I timed both with the linux command line utility time
, and xdelta3 took 0,7 seconds, while my program with go-xdelta
took 13 minutes and 5 seconds.
At first I thought this was because the files were large (from ~80MB to ~110MB), but then I tried the same with the debian amd64 netinst and i386 netinst CD images, which clock at around 300-400MB, and the results were very different: xdelta3 again took around 0,5 seconds to complete the patch, while my program above took 11 seconds. Still a lot slower, but not as prohibitively so.
I wonder if it relates to the fact that the electron executables were executables, although since xdelta3 performs very well, I expect it's not an issue with the library, but probably with this wrapper.
Anyway, this makes the wrapper not usable for me at the moment, but thank you very much for producing it anyway, hope this gets figured out and fixed at some point! Let me know if you need help reproducing it.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.