Comments (10)
We ran into this as well i.e., LZString.compressToUTF16 over XHR not working in Safari. Weirdly it does work over WebSocket.
Our solution was the same as @ktsaou: switch to https://github.com/nodeca/pako.
from lz-string.
As with many of the transports and storage mechanisms, you may be getting an escape of some particular character happening. This then breaks the binary aspect of the compressed data.
If you want to transmit the binary (though it is UTF-16), you will need to escape it appropriately for your requirement and then unescape it from within your service.
from lz-string.
I imagined that the problem may be in some sort of character escaping. Is there a list of "known" character that are escaped by Safari (and maybe some other browsers), when using UTF-16?
from lz-string.
The compressToUTF16 is a hack because old JS engines didn't know any other data type apart from number and booleans. It was never meant to get out of the browser. Plus, depending on your encoding when sending the string out of the browser, you may be wasting lots of space.
Can you post the code you use to send the data? Specifically, which content-type and/or encoding are you setting for your XHR?
I created specifically a compressToEncodedURIComponent which can almost maximize bandwidth on a application/x-www-form-urlencoded transmission. It'll go faster and this one is not a hack, it's pure ASCII.
from lz-string.
Thank you for the reply,
I'm using a plain-old-XMLHttpRequest to send the data, forcing text/plain;charset=UTF-16 encoding (but I noticed that almost every browser rewrites it as text/plain;charset=UTF-8).
Something like:
function _sendData(address, payload) {
var xhr = new XMLHttpRequest();
xhr.open("POST", address, true);
xhr.overrideMimeType("text/plain;charset=UTF-16");
xhr.send(payload);
}
Regarding the compressToEncodedURIComponent function, I did several benchmarks and noticed that the space occupied by the resulting string is almost the same as compressToBase64, is it correct?
For example, here's a simple benchmark:
Original: 281718 bytes (281.718 kB)
UTF-16 compressed: 48835 bytes (48.835 kB)
Binary encoding: 91564 bytes (91.564 kB)
Invalid UTF-16 encoding: 45782 bytes (45.782 kB)
Base 64 encoding: 122088 bytes (122.088 kB)
URI-encoded: 122085 bytes (122.085 kB)
(I get the size reading the resultString.length property)
from lz-string.
compressToEncodedURIComponent is a variation over compressToBase64, you are correct. But if you send it in UTF-8 (or plain ASCII or ISO-LATIN-1, ...), each character will take 8 bits. If you send the output of compressToUTF16, each character will use 16 bits, and if for some reason the browser defaults to UTF-8 charset, they will take on average much more than 16 bits and you will waste a lot of bandwidth.
from lz-string.
I noticed that on some browsers, and when they send the content as UTF-8 instead of UTF-16, the space saved thanks to UTF-16 is gone.
Regarding LZString, wouldn't it be better to have a compressToUTF8 function, in order to save a bit more space in comparison to Base64/URIencoded strings?
from lz-string.
base64 is using 6 bits out of 8 and the best I could do with UTF-8 would be to use 7 bits out of 8. Maybe there's something to be gained here. The downside is that none of the existing ports of lz-string in other languages support this encoding.
In the meantime, I suggest you use the compressToEncodedURIComponent variety. It's not that bad. It'll consume 14% more space.
from lz-string.
Actually, I read the UTF-8 spec wrong. I cannot use 7 out of 8 bits, but just 6. So we're back to base64.
from lz-string.
Have you been able to figure something out on this issue ?
from lz-string.
Related Issues (20)
- Newbie question: How would I go about downloading / uploading a file? HOT 2
- Does `compressToUint8Array()` output smaller than `compress()` ? HOT 1
- Quotes in the compressed string HOT 3
- Implement IRI Encoding HOT 1
- Bug: can not decompress for some compressed string HOT 4
- Crash with large strings when running on Hermes (React Native)
- Is there a way to allow custom dictionaries? HOT 9
- How to realize that only specified characters are allowed to appear in the compression result? HOT 1
- LZ-String port to current typescript HOT 9
- Tag for v1.5.0 HOT 2
- Version 2 TODO list HOT 15
- Feature request: compress to source-compatible character set HOT 1
- New port for a linux executable HOT 1
- Can utf-8 codes be avoided in the output? HOT 10
- `decompressFromBase64` can return `null` HOT 7
- Docs: JSPerf.com links are no longer valid
- Docs: A Complete Demo for various compression/decompression HOT 3
- Docs: TSDoc HOT 3
- License Inconsistencies HOT 5
- Community Question: Breaking Changes and Version 2! HOT 10
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lz-string.