compress compressable filetypes, dont compress everything else about l1-node

[guanzo]

Slack convo with lidel

me

we want nginx to gzip CAR files that contain compressible files like text, and not gzip CAR files that contain already compressed files like images/video/audio.

generally u make this decision based on the mime type, but for CAR files the mime type is always application/vnd.ipld.car

is there some convention to specify the content type of the file inside the CAR file?

lidel

no, adding content type as UnixFS metadata was discussed in the past but never spec'd nor implemented.

right now, the only place where we deal with content-type are HTTP gateways – they will sniff the content type based on magic bytes from the beginning of each file – go-ipfs does this: https://github.com/ipfs/go-ipfs/blob/9bd346e25004df4fd7927ab151097f975fb433c2/core/corehttp/gateway_handler_unixfs_file.go#L46L78

You could do something similar – get first N bytes of the file and make decision based on sniffed content type.

[DiegoRBaquero]

Per top results in google, it makes sense to compress everything rather than not. Bandwidth is more expensive than CPU and we can later optimize by peeking into and compressing only the ones with actual gains.

[gruns]

Per top results in google, it makes sense to compress everything rather than not.

link pls 🙂. really curious to compare decomp speeds on slow devices. ie how much page render latency would it add to decomp every image needlessly on slow devices (eg feature phones)

otherwise, https://github.com/ipfs/go-ipfs/blob/9bd346e25004df4fd7927ab151097f975fb433c2/core/corehttp/gateway_handler_unixfs_file.go#L46L78 is the 'right' first approach to determining whether to enable comp or not. which, zooming out sucks: the mimetype(s) should be included in the car file metadata. this is a step backwards; there's a reason mimetypes exist :-/

in the future we could have the l2s, or the SPs, parse the car file and add the appropriate mimetype(s) for use upstream by the l1s and/or l2s

[DiegoRBaquero]

https://serverfault.com/questions/253074/what-is-the-best-nginx-compression-gzip-level
https://sudonull.com/post/172479-GZip-and-nginx-performance-impact
http://paulbuchheit.blogspot.com/2009/04/make-your-site-faster-and-cheaper-to.html