apache / incubator-flagon

Apache Flagon is a suite of comprehensive, thin-client behavioral logging tools

Apache Flagon Docker Containers and Website

Specific build instructions located in docker, kubernetes, release and site folders.

incubator-flagon's Issues

How to deploy in a a load balancing environment?

Moving question over from incubator-flagon-useralejs:

We have a proxy server that redirects user requests to different servers for load balancing. On the proxy side we have our UI application. Some analytics do need to track information about the host. That being said what is the advice wrt:

Capturing every user interaction with the application would eventually end up creating large volume of data relatively quickly. Is there a way to purge data that is “x” days old or some retention/deletion policy of some sort?
Is there a need to have a cluster of ELK or is one sufficient?

Update Test new asf.yaml

to moderate github alert distribution

[RFC] Consolidation of Apache Flagon Projects into a Monorepo Structure

TL;DR

Problem

The Apache Flagon project, encompassing its child projects, faces operational fragmentation resulting from the polyrepo approach used for its version controlling. This fragmentation imposes challenges in maintaining project consistency, ensuring community health, and attracting as well as retaining contributors.

Solution

I believe migrating to a monorepo approach would vastly lower technical debt by simplifying the project structure. Over time this will pay dividends which will help us increase our velocity and hopefully spur more community growth.

Details

I've circulated an email to [email protected] requesting feedback on this RFC in Google Docs itself. I highly encourage participation from the community.

update kibana configs and dockerfiles

update kibana configs and dockerfiles

Split 'Flagon' repo into 2 to make for easier site builds

Create new repo and rename flagon-website. Maintain site assets in different repo than integrations.

See: https://lists.apache.org/thread/lmvpfg4nvlswb6z0j71hq5byc3jmyod8

Update Website Publishing through asf.yaml

How to publish via .asf.yaml:

Publishing via .asf.yaml is described at: https://s.apache.org/asfyamlpublishing
You can also see an example .asf.yaml with publishing and staging profiles for our own infra web site at: https://github.com/apache/infrastructure-website/blob/asf-site/.asf.yaml

Add Nix [ shell ] config for developing in this repo

See https://nix.dev/

etc

update metricbeats configs and dockerfile

update metricbeats configs and dockerfile

Update TLS Cert

Navigating to http://flagon.incubator.apache.org/ indicates that our TLS cert needs updating:

Add Flagon Logo to ASF logo repository

Add Flagon Logos to ASF logo repository

New Test Issue

test

Update contributing docs

At least for web, maybe also add to README or link from it?

Add bot to close stale issues

There are a growing number of issues which are stale either because they've seen no action or they have been overcome by events (OBE). I recommend adding a bot to automatically track, flag, and eventually close stale issues to help keep this clean:

https://github.com/marketplace/actions/close-stale-issues

We would need to ensure this is Apache friendly, first. We could look to other Apache libraries to see what's used there.

Create Example Dashboard

Superset was called out as an integration people are exploring -- it seems prudent to have an issue for tracking that work?

Is this the correct repo, though? Or should this be in a different flagon repo?

update logstash configs and dockerfiles

update logstash configs and dockerfiles

Update Site UserALE.js API text with new callback info

Update Site UserALE.js API text with new callback info

Question about embedding training as part of user analytics

Use case: User on boarding, training etc are also part of User Analytics. Is there anything out there that has been integrated with UserALE.js to support use cases such as guided tour, training, etc?

More verbose description at top level README

I mean, c'mon.

Update Docker Compose file

current compose files are antiquated and break on versions newer than 6.8

Update Docker ELK build instructions

Current instructions are outdated relying on virtual box or hyper-v, not internal docker vm managers.

Update instructions for ELK Stack

Update instructions for new ELK Stack

Refer to https://github.com/deviantony/docker-elk

update kibana dashboards as needed

update kibana dashboards as needed

Remove Docs for UserALE py and UserALE (not js) and STOUT

Also remove all refs on style sheets

Remove "incubating"

Now that we've graduated, we should update the website and remove references to us being in incubation. Noting that the web address itself needs an update: http://flagon.incubator.apache.org/

Roadmap for Flagon

We are planning to incorporate Flagon in our application. However without knowing the roadmap kind of tough to make a decision.

Is there any timeline when the incubator project will be moved to the next level?

Also are there plans to leverage Apache Superset and other projects to replace the visualizations currently used in Flagon?

Ruby versioning issue in Site build

It looks like there is an issue in site dependencies that coerce Ruby 2.5 to run which breaks build. May be issue in RUBY_VERSION assignment. See below:

`
=> ERROR [ 4/12] RUN bundle install 8.4s

[ 4/12] RUN bundle install:
#0 4.336 Fetching gem metadata from https://rubygems.org/............
#0 8.006 Fetching gem metadata from https://rubygems.org/.
#0 8.039 Resolving dependencies...
#0 8.232 public_suffix-5.0.0 requires ruby version >= 2.6, which is incompatible with the
#0 8.232 current version, ruby 2.5.1p57

---

Dockerfile:7

5 | COPY Gemfile* /tmp/
6 | WORKDIR /tmp
7 | >>> RUN bundle install
8 |
9 | ENV app /app

ERROR: failed to solve: process "/bin/sh -c bundle install" did not complete successfully: exit code: 5
(base) jpoore@MacBook-Pro-2 site %
`
Active version of Ruby I'm running (via rbenv):

(base) jpoore@MacBook-Pro-2 site % ruby -v ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin22]
Confirmed that 2.5 isn't even installed on computer...

Explore other visualization support

Tableau, Graphana, Apache Superset are targets that users have expressed interest in. Would be good to expand examples beyond kibana alone.

Updated es configs and dockerfiles

Updated es configs and dockerfiles