charcoal-se / halflife Goto Github PK
View Code? Open in Web Editor NEWMetasmoke back-end analysis client
License: Apache License 2.0
Metasmoke back-end analysis client
License: Apache License 2.0
The internal architecture is rather messy, and should be rearchitected.
Must:
Should:
The restored "x/y over timespan" calculation was wrong, it counts x and y per individual feedback, not per post!
Regex queries have been disabled in Metasmoke so the client should stop attempting to make them.
Ref: https://chat.stackexchange.com/transcript/message/48992963
I created a metasmoke ticket, might be a duplicate? Charcoal-SE/metasmoke#596
The logic is not accurate, and should use a different approach.
A month ago I had to restart Halflife a number of times after the month had rolled over, and now I'm seeing the same thing again.
In brief, it seems to eat up all the disk space, and require a number of restarts before the space is properly reclaimed.
This could be a weird artifact of the Docker deployment model and/or how it works on the EC2 instance where I'm running this. Probably the deployment model should be reworked altogether.
When two or more posts are reported in rapid succession, the second and subsequent posts are sometimes missed by Halflife.
Searching for tagged/skipped
shows many such incidents.
I originally thought this was a problem with Metasmoke, but in fact, it's probably a bug in Halflife. It should pick up the websocket again quickly after a message is received in order to keep on listening properly.
Regex search over the Metasmoke API was turned off for stability and performance reasons. (See #6.)
I have a fix in the works for domains, but URL tails (mostly useful for updating drugs brands) doesn't look like it's going to be easy.
The SO chat interface abbreviates long URLs. This is not visible in the chat log so it's not really meaningful to link to examples here, but Jake noted as much with a copy/paste here:
https://chat.stackexchange.com/transcript/65945?m=44952582#44952582
He suggests adding a line break or something.
Frequently when a post is reported, you would like to see very quickly the first results from the analysis: is the domain name blacklisted? did the URL tail match any keywords? and only then proceed to report the more-detailed analysis.
The chat interface throttles sending to one message per second. Some messages could be collected into multi-line chat messages (but then you cannot use formatting -- no bold, italics, links etc) to make reports appear quicker.
Ultimately, I'm thinking the back-end queries should be done using some sort of asynch framework so that multiple queries could be pending at the same time and you don't have to wait for the queries to execute serially before you can get the result from the one you actually care about for this particular post.
https://metasmoke.erwaysoftware.com/post/164468 contains an entire Bachelor's degree PDF on a low connection which took forever to download and caused Halflife to crash with Memory error:
https://chat.stackexchange.com/transcript/message/49448449#49448449
https://stackoverflow.com/questions/22346158/python-requests-how-to-limit-received-size-transfer-rate-and-or-total-time has some ideas for how to abort a download when it keeps going for too long.
sevenmentor.com
in particular is notoriously switching IP addresses like others change underwear. It would be nice if Halflife could periodically check their IP address, and perhaps even automatically add the new address to the IP watchlist when it changes.
(My practice has been to keep the old addresses on the watch list still. If we find that this is problematic, maybe also eventually figure out a way to remove old ones.)
The bot stopped posting messages at 07:30 UTC this morning.
The last message is https://chat.stackexchange.com/transcript/message/44783748#44783748
It's not entirely clear what caused this, but Art of Code suggests switching to the v2 API.
https://chat.stackexchange.com/transcript/message/44785359#44785359
https://chat.stackexchange.com/transcript/message/61810642#61810642 shows a bit of a train wreck in the URL parsing logic.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.