Comments (3)
Hey, thanks for the report. I think this breakage is due to how we've tried to support gzip-compressed files that do not properly set Content-Encoding
in the past by looking at other potential indicators. In hindsight, assuming all application/octet-stream
content was gzipped was definitely wrong, and now we're in a position where "correct" behavior is breaking the integration. Oops :/
I put together #23 which should fix this issue. The plan is to break some use cases while providing a backwards-compatible flag to set as a workaround, while doing things correctly going forward. Will get it reviewed, but let me know if you see any issues with this approach.
from agentless-integrations-for-aws.
Very impressed with the time it took to turn around!
I agree with the approach and think it looks basically fine. I am on the Honeycomb free trial so in order to get started quickly I've written my own Lambda (in Node.js). I've had success so far reading various log types with the following logic:
module.exports = async function s3FetchLogFile (Bucket, Key) {
const params = { Bucket, Key }
const response = await s3.getObject(params).promise()
if (response.ContentType === 'application/x-gzip' || response.ContentEncoding === 'gzip') {
return unzipBuffer(response.Body)
} else {
return response.Body.toString('utf8')
}
}
Not sure if your Go version follows the same pattern, but this has successfully been auto-unzipping files with no false positives. Hope this helps.
from agentless-integrations-for-aws.
Closing as I believe this was addressed as part of the v2.0.1 release.
from agentless-integrations-for-aws.
Related Issues (20)
- patch openSSL CVE
- Add support for configuring integrations using environment variables HOT 1
- Add support for mysql audit logs HOT 2
- Build ARM binary as part of publish workflow HOT 2
- Document the regexps now that they're embedded in Go code HOT 1
- Allow masking values based on regex
- Support provided.al2 runtime in place of deprecated Go1.x
- s3-handler: Add NLB logs support
- s3-handler: support SNS as an alternative to SQS
- Deploy lambdas to us-gov-west-1 and us-gov-east-1 HOT 2
- Add DEBUG mode
- Add support for CloudTrail logs HOT 1
- Add CloudFront Access Logs Support HOT 1
- De-dupe docs HOT 1
- Allow adding fields in cloudwatch-handler HOT 2
- Allow API token to be pulled from SSM Secure String HOT 4
- cloudwatch-handler currently errors on START/END/REPORT lines; issue seems to be fixed in publisher. Is this intentional? HOT 3
- s3-handler doesn't provide enough configuration for ALB logs to be correctly imported HOT 4
- "Event dropped due to sampling and response body" HOT 6
- Verify gzip handling HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from agentless-integrations-for-aws.