openappsec / openappsec Goto Github PK

open-appsec is a machine learning security engine that preemptively and automatically prevents threats against Web Application & APIs. This repo include the main code and logic.

Home Page: https://openappsec.io

License: Apache License 2.0

CMake 1.05% C 2.06% C++ 88.35% Mustache 3.13% Smarty 0.05% Dockerfile 0.01% Shell 4.68% Roff 0.02% Python 0.65%

api-security application-security kubernetes nginx web-application-firewall appsec devsecops kong rate-limiting security-tools

openappsec's People

Stargazers

Watchers

Forkers

abhihub j4ckzh0u byteskater chkp-tob nahum-p theguinz mokhtas sumodgeorge ramkumareinstein upanyasa-llp onemanteam123 segun76 mynameisjimmy zouxiaodong tobey123 gmh5225 zanonluc rmaues cleber-rocha itdabid jzj bakaralisec syruichi thangmai-cleeksy syinwu daniynwa winstona 5l1v3r1 andyoulovexy chkp-anank monkburger pedronet4k epaphrasmakoko ralaam dieti80 bmbeverst freemanpolys benjja ch-rigu pouriajalilian parantheses haszayan muharihar daudln

openappsec's Issues

Blocking my web admin access

We are running nginx stable 1.24.0 and I have the openappsec agent connected into the SAAS interface.
Within the web interface, I've setup a Profile and and managing the agent policy through the interface.
The agent was installed on the 14 Sep
I've setup an asset with the site url's, added trusted sources for our two main public IP's + our dev's IP and it's reached graduate learning level.
When ever I flip the asset from Learn/Detect to protect, the next time our dev tries to access the admin back end and edit things, it's blocking him with a Cross Site Scripting/ SQL injection incident type.
I'm then right clicking on the event and creating an exception but I'm on double figures now for the exceptions. Exception examples are below....

Source Identifier : xxx Trusted IP xxx
URI : /xxx/admin/dashboard/
Parameter Name : s_sq..old
Paramter Value : .*

Source Identifier : xxx Trusted IP xxx
URI : /xxx/admin/import/
Parameter Name : s_sq..old
Paramter Value : .*

How can I stop my dev getting blocked?

When installing openappsec on Kubernetes, the service name was too long.

When installing openappsec on Kubernetes, the service name was too long. For example, k3s needs to create a daemonset resource when creating SVC, as the name is too long to create a daemonset.

ngx_module_1.18.0-6ubuntu14.3.tar.gz has no content

Hello,

I would like to give this a try on Ubuntu 22.04 (Jammy).
I have NGINX version 1.18.0-6ubuntu14.3 installed which is listed on the supported versions list.

It seems both the -auto flag and the -download flag of the open-appsec-nginx-install fetches an empty ngx_module_1.18.0-6ubuntu14.3.tar.gz.

/tmp/open-appsec# ls -ahl
total 22M
drwxr-xr-x  4 root root 4.0K Dec 27 08:03 .
drwxrwxrwt 13 root root 4.0K Dec 27 08:05 ..
drwxr-xr-x  2 root root 4.0K Dec 14 15:16 ngx_module_1.18.0-6ubuntu14.3
-rw-r--r--  1 root root  133 Dec 27 08:02 ngx_module_1.18.0-6ubuntu14.3.tar.gz
drwxr-xr-x  2 root root 4.0K Dec 13 14:19 openappsec
-rw-r--r--  1 root root  22M Dec 27 08:02 openappsec.tar.gz

/tmp/open-appsec# ll ngx_module_1.18.0-6ubuntu14.3
total 8
drwxr-xr-x 2 root root 4096 Dec 14 15:16 ./
drwxr-xr-x 4 root root 4096 Dec 27 08:03 ../

Is there a timeline available where the nginx module is available?
Im looking forward giving this product a try!

NGINX 1.22

Hi guys all right? I would like to know if there is any prediction for me to be able to install the sec app with nginx 1.22?

Thank you for understanding

Unable to register the agent to FOG

Hi Guys,

I compiled the agent successfully with the codes and tried registering the agent to the FOG however I am consistently getting error.

root@appsec:/opt/build_out# ./install-cp-nano-agent.sh --install --token cp-xx-xxxx.xxxxxxxxx.xxxxxxxxx
Check Point Nano Agent Version 1.0.0 Install Package
Verifying archive integrity... 100% All good.
Uncompressing... 100%
Fog address='https://inext-agents.cloud.ngen.checkpoint.com'
/usr/bin/which
/usr/sbin/ldconfig

Starting installation of open-appsec Nano Agent [Sun 27 Nov 2022 03:22:16 PM UTC]
Creating env details file
Copying cp-nano-agent binary file to folder: /etc/cp/orchestration/cp-nano-orchestration
Installing the watchdog
Start cp-nano-agent service
Note: in order for the agent to remain active and effective it must connect to the Fog/Cloud at least every 45 days
open-appsec Nano Agent installation completed successfully
Registering open-appsec Nano Agent to Fog..
open-appsec Nano Agent registration failed. Failed to register to Fog: https://inext-agents.cloud.ngen.checkpoint.com

Istio ingress gateway support

We use istio as ingress gateway. The openappsec support for istio would be appreciated.

Traefik Ingress support

Hey, we're wondering if integration with Traefik Ingress is on the radar? That would be nice!

Why open-appsec is blocking in learn-detect mode ?

I am trying out open-appsec on k8s with Kong Ingress controller.

Name: openappsec-75c272
ID: eq6YP84yRHRTO0Ly

Deployed using the below command

helm install open-appsec-k8s-kong-latest.tgz \
--name-template=kong \
--set appsec.mode=standalone \
--set appsec.persistence.enabled=false \
-f values.yaml

And applied the below policy

apiVersion: openappsec.io/v1beta1
kind: Policy
metadata:
    name: open-appsec-best-practice-policy
spec:
    default:
        mode: detect-learn
        practices: [appsec-best-practice]
        triggers: [appsec-log-trigger]
        custom-response: 403-forbidden
        source-identifiers: ""
        trusted-sources: ""
        exceptions: [open-appsec-kong-exception]

Added the below exception:

apiVersion: openappsec.io/v1beta1
kind: Exception
metadata:
  name: open-appsec-kong-exception
spec:
- action: accept
  comment: "Kong config push"
  sourceIp:
  - 127.0.0.1
  url:
  - "/config"

BUT in the appsec container logs, I am seeing internal Kong update is blocked ? The securityAction says "Prevent" even though the mode is set to detect-learn. What am I missing ? I even added an exception to no avail.

"eventData": {
        "logIndex": 4550,
        "eventReferenceId": "8f1587f0-bf16-4efc-869c-dab397c5ac2a",
        "assetId": "Any",
        "assetName": "Any",
        "eventConfidence": "High",
        "sourceIP": "127.0.0.1",
        "httpSourceId": "127.0.0.1",
        "sourcePort": 48614,
        "httpHostName": "localhost:8444",
        "httpMethod": "POST",
        "httpUriPath": "/config",
        "httpUriQuery": "check_hash=1&flatten_errors=1",
        "ruleId": "Any",
        "securityAction": "Prevent",
        "waapOverride": "None",
        "practiceType": "Threat Prevention",
        "practiceSubType": "Web Application",
        "ruleName": "Any",

Kong ingress controller log entry:

time="2023-10-20T09:26:48Z" level=error msg="could not update kong admin" error="performing update for https://localhost:8444 failed: failed posting new config to /config: got status code 403" subsystem=dataplane-synchronizer

ORCHESTRATION_INSTALLATION_SCRIPT="install-cp-nano-agent.sh" files not found

where a file install-cp-nano-agent.sh

I have downloaded Appsec Advanced Model.

followed the steps in the README file, but during the docker build process, the intsall-cp-agent.sh file was not found

docker logs >>

Grafana dashboard

This enhancement is about designing and developing a Grafana dashboard for open-appsec.
One of the main data sources can be events - https://docs.openappsec.io/references/events-logs-schema

Apache Support

Hi,

Are you guys working on support for Apache http server? I understand your current focus is in nginx and supporting nvoy in the future, however, I do not see any indication that apache might be down the road map.

Tracking local learning

Hi,
I'm demoing open appsec for a local-only replacement of Modsecurity. Reading the documentation, part of the process to move from detect to prevent is to look at the "learning level". Is there a CLI command that allows viewing this locally without using the SaaS web panel?

systemd log cluttered with garbage

Hi,
Last time I tried to check the status on nano_agent installed on Debian 11 nginx, then I found that I can only see those error messages:

Oct 14 15:31:23 waf1 cp-nano-watchdog[2336437]: sh: 1: cpprod_util: not found
Oct 14 15:31:23 waf1 cp-nano-watchdog[2336438]: sh: 1: cpprod_util: not found
Oct 14 15:31:53 waf1 cp-nano-watchdog[2337247]: sh: 1: clish: not found
Oct 14 15:31:53 waf1 cp-nano-watchdog[2337251]: sh: 1: cpprod_util: not found
Oct 14 15:31:53 waf1 cp-nano-watchdog[2337252]: sh: 1: cpprod_util: not found
Oct 14 15:32:24 waf1 cp-nano-watchdog[2338207]: sh: 1: clish: not found
Oct 14 15:32:24 waf1 cp-nano-watchdog[2338211]: sh: 1: cpprod_util: not found
Oct 14 15:32:24 waf1 cp-nano-watchdog[2338212]: sh: 1: cpprod_util: not found

Is there any chance those can be suppressed ? Or maybe the watchdog could be modified accordingly ?

haproxy support ?

Hi,

is their any plan to add support for haproxy? (through spoe protocol for example)

https://www.haproxy.com/fr/blog/extending-haproxy-with-the-stream-processing-offload-engine/

Upload of large files blocked on detect-learn

I am trying to upload some large files to my web foto gallery. All configuration is default, when I try to define specific rules (with higher upload limit), then it also fails. What I am getting is 413 errors

I thought that detect-learn shouldn't block anything. Is my assumption wrong ?

When I baypass openappsec on my load-balancer and send the traffic directly to backend server, the it all starts to work again.

Example log:

{"eventTime": "2023-10-14T13:40:21.275","eventName": "Web Request","eventSeverity": "Info","eventPriority": "Low","eventType": "Event Driven","eventLevel": "Log","eventLogLevel": "info","eventAudience": "Security","eventAudienceTeam": "","eventFrequency": 0,"eventTags": ["Threat Prevention","Web Application & API Protection"],"eventSource": {"agentId": "495390e2-2692-4331-9ffc-b7d4908e9315","eventTraceId": "","eventSpanId": "","issuingEngineVersion": "1.1.0-open-source","serviceName": "HTTP Transaction Handler","serviceId": "2","assetId": "Any","assetName": "Any"},"eventData": {"logIndex": 75,"eventReferenceId": "bc9be452-35b1-4c89-9ff1-dc6dd03e48c0","assetId": "Any","assetName": "Any","sourceIP": "192.168.31.45","httpSourceId": "192.168.31.45","sourcePort": 47764,"httpHostName": "example.com","httpMethod": "GET","ruleId": "Any","securityAction": "Detect","waapOverride": "None","practiceType": "Threat Prevention","practiceSubType": "Web Application","ruleName": "Any","practiceId": "36dec53c-3703-4357-8505-7993b6c390fd","practiceName": "local_policy/webapp-default-practice","waapIncidentType": "","matchedSample": "","matchedLocation": "","matchedParameter": "","waapFoundIndicators": "","matchedIndicators": "","learnedIndicators": "","waapUserReputationScore": 444,"waapUserReputation": "Normal","waapUriFalsePositiveScore": 0,"waapKeywordsScore": 0,"waapFinalScore": 0,"waapCalculatedThreatLevel": 0}}
{"eventTime": "2023-10-14T13:44:58.865","eventName": "Web Request","eventSeverity": "Info","eventPriority": "Low","eventType": "Event Driven","eventLevel": "Log","eventLogLevel": "info","eventAudience": "Security","eventAudienceTeam": "","eventFrequency": 0,"eventTags": ["Threat Prevention","Web Application & API Protection"],"eventSource": {"agentId": "495390e2-2692-4331-9ffc-b7d4908e9315","eventTraceId": "","eventSpanId": "","issuingEngineVersion": "1.1.0-open-source","serviceName": "HTTP Transaction Handler","serviceId": "2","assetId": "Any","assetName": "Any"},"eventData": {"logIndex": 84,"eventReferenceId": "ff9f3267-4732-4aea-83c7-aa830e69cdd0","assetId": "Any","assetName": "Any","sourceIP": "192.168.31.45","httpSourceId": "192.168.31.45","sourcePort": 39264,"httpHostName": "example.com","httpMethod": "GET","ruleId": "Any","securityAction": "Detect","waapOverride": "None","practiceType": "Threat Prevention","practiceSubType": "Web Application","ruleName": "Any","practiceId": "36dec53c-3703-4357-8505-7993b6c390fd","practiceName": "local_policy/webapp-default-practice","waapIncidentType": "","matchedSample": "","matchedLocation": "","matchedParameter": "","waapFoundIndicators": "","matchedIndicators": "","learnedIndicators": "","waapUserReputationScore": 444,"waapUserReputation": "Normal","waapUriFalsePositiveScore": 0,"waapKeywordsScore": 0,"waapFinalScore": 0,"waapCalculatedThreatLevel": 0}}

Installer errors on Ubuntu 22.04

Hi,

when trying to install on Ubuntu 22.04 the installer gives an error:

ubuntu@vm-server:~$ sudo ./open-appsec-install --auto
open-appsec for NGINX and Kong Installer v1.2245.1
For release notes and known limitations check:
https://docs.openappsec.io/release-notes
Searching local NGINX…
NGINX version found: 1.18.0-6ubuntu14.4
Downloading open-appsec NGINX attachment... stored in '/tmp/open-appsec'
Unsupported NGINX version, for supported platforms, OS and NGINX versions please see docs.openappsec.io
Downloading open-appsec agent... stored in '/tmp/open-appsec'
Unsupported OS, for supported platforms, OS and NGINX versions please see docs.openappsec.io

trying to manually download https://downloads.openappsec.io/packages/agent/aarch64/ubuntu/jammy/openappsec-jammy.tar.gz gives me an access denied error.

OpenAppSec NGinx Ingress is not detecting any traffic

Hi,

First of all, I'd like to thank the team for providing what seems an awesome tool, and a suitable solution for all those who wants an opensource kube-native and prod ready WAF solution.

Problem description

By following the howto videos and the tutorial section of the docs, I just can't get to have the NGinx Ingress installation to work.

Environment

I'm on a managed K8S cluster (provided by OVH), which runs the 1.25.9-2 version of Kubernetes.

I'm deploying an application called REDCap on that cluster, and am just trying to test the WAF by testing SQL injection (by hand like in the howto video, or via Zed Attack Proxy).

I ensured that my application was reachable as normal with the vanilla NGinx Ingress Controller, then installed OpenAppSec in default detect-learn mode using the recommended installation script, with the "Ingress duplication" method, and redirected my DNS entry towards the OpenAppSec Ingress IP.

FYI, the Ingress does the TLS termination with certificates, set as recommended by Kubernetes/NGinx. Nothing fancy, those are official signed certificates by a recognized CA, and the application is accessible via HTTPS without issues, just as it should be.

Issue description

I can access my application by its URL, but by looking at the logs as recommended in the documentation, it seems that the open-appsec container only detects local calls to the Ingress Controller made by its probe, and not the calls made to my application :

❯ kubectl -n appsec logs -f deployments/open-appsec-open-appsec-k8s-nginx-ingress-controller -c open-appsec
{
    "eventTime": "2023-08-10T07:00:13.079",
    "eventName": "Web Request",
    "eventSeverity": "Info",
    "eventPriority": "Low",
    "eventType": "Event Driven",
    "eventLevel": "Log",
    "eventLogLevel": "info",
    "eventAudience": "Security",
    "eventAudienceTeam": "",
    "eventFrequency": 0,
    "eventTags": [
        "Threat Prevention",
        "Web Application & API Protection"
    ],
    "eventSource": {
        "agentId": "9ea71274-32b4-4815-a23c-240ac97d2df4",
        "eventTraceId": "",
        "eventSpanId": "",
        "issuingEngineVersion": "1.0.0-open-source",
        "serviceName": "HTTP Transaction Handler",
        "serviceId": "1",
        "k8sClusterId": "871275fe-a843-486b-b739-b3e6ef09f51a",
        "assetId": "Any",
        "assetName": "Any"
    },
    "eventData": {
        "logIndex": 7154,
        "eventReferenceId": "408a0a5d-7db7-4eab-9b7b-66bcdad5ba02",
        "assetId": "Any",
        "assetName": "Any",
        "sourceIP": "127.0.0.1",
        "httpSourceId": "127.0.0.1",
        "sourcePort": 44202,
        "httpHostName": "127.0.0.1:10246",
        "httpMethod": "GET",
        "httpUriPath": "/is-dynamic-lb-initialized",
        "httpUriQuery": "",
        "ruleId": "Any",
        "securityAction": "Detect",
        "waapOverride": "None",
        "practiceType": "Threat Prevention",
        "practiceSubType": "Web Application",
        "ruleName": "Any",
        "practiceId": "fd1bea85-70e4-4d91-a58e-c675c3eb1a63",
        "practiceName": "open-appsec-best-practice-policy/appsec-best-practice",
        "waapIncidentType": "",
        "matchedSample": "",
        "matchedLocation": "",
        "matchedParameter": "",
        "waapFoundIndicators": "",
        "matchedIndicators": "",
        "learnedIndicators": "",
        "waapUserReputationScore": 298,
        "waapUserReputation": "Low",
        "waapUriFalsePositiveScore": 0,
        "waapKeywordsScore": 0,
        "waapFinalScore": 0,
        "waapCalculatedThreatLevel": 0
    }
}
{
    "eventTime": "2023-08-10T07:00:13.084",
    "eventName": "Web Request",
    "eventSeverity": "Info",
    "eventPriority": "Low",
    "eventType": "Event Driven",
    "eventLevel": "Log",
    "eventLogLevel": "info",
    "eventAudience": "Security",
    "eventAudienceTeam": "",
    "eventFrequency": 0,
    "eventTags": [
        "Threat Prevention",
        "Web Application & API Protection"
    ],
    "eventSource": {
        "agentId": "9ea71274-32b4-4815-a23c-240ac97d2df4",
        "eventTraceId": "",
        "eventSpanId": "",
        "issuingEngineVersion": "1.0.0-open-source",
        "serviceName": "HTTP Transaction Handler",
        "serviceId": "1",
        "k8sClusterId": "871275fe-a843-486b-b739-b3e6ef09f51a",
        "assetId": "Any",
        "assetName": "Any"
    },
    "eventData": {
        "logIndex": 7155,
        "eventReferenceId": "0df12102-9df2-4ee2-856c-6918cc1a4ae8",
        "assetId": "Any",
        "assetName": "Any",
        "sourceIP": "127.0.0.1",
        "httpSourceId": "127.0.0.1",
        "sourcePort": 44210,
        "httpHostName": "127.0.0.1:10246",
        "httpMethod": "GET",
        "httpUriPath": "/is-dynamic-lb-initialized",
        "httpUriQuery": "",
        "ruleId": "Any",
        "securityAction": "Detect",
        "waapOverride": "None",
        "practiceType": "Threat Prevention",
        "practiceSubType": "Web Application",
        "ruleName": "Any",
        "practiceId": "fd1bea85-70e4-4d91-a58e-c675c3eb1a63",
        "practiceName": "open-appsec-best-practice-policy/appsec-best-practice",
        "waapIncidentType": "",
        "matchedSample": "",
        "matchedLocation": "",
        "matchedParameter": "",
        "waapFoundIndicators": "",
        "matchedIndicators": "",
        "learnedIndicators": "",
        "waapUserReputationScore": 298,
        "waapUserReputation": "Low",
        "waapUriFalsePositiveScore": 0,
        "waapKeywordsScore": 0,
        "waapFinalScore": 0,
        "waapCalculatedThreatLevel": 0
    }
}

[and it goes on and on like this]

But, when I look at the logs of the NGinx controller container (the one in the appsec namespace), I can see that the calls to my application are logged as expected :

❯ kubectl -n appsec logs -f deployments/open-appsec-open-appsec-k8s-nginx-ingress-controller -c controller

10.2.0.0 - - [15/Aug/2023:15:41:13 +0000] "GET / HTTP/2.0" 200 8646 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/116.0" 332 0.132 [[REDACTED]-externe-qual-[REDACTED]-ext-qual-httpd-svc-80] [] 10.2.3.198:80 8666 0.132 200 7812e1225ea8aa7ec004c7c12f398e66
|2023-08-15T15:41:13.263: is_ngx_cp_attachment_disabled@ngx_http_cp_attachment_module.c:243 [uid 5 | pid 319] | Reconfiguring the local NGINX attachment state
10.2.0.0 - - [15/Aug/2023:15:41:13 +0000] "GET /[REDACTED]_v13.1.13/Resources/webpack/css/bundle.css?1692112200 HTTP/2.0" 304 0 "https://[REDACTED]/" "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/116.0" 217 0.026 [[REDACTED]-externe-qual-[REDACTED]-ext-qual-httpd-svc-80] [] 10.2.3.198:80 0 0.026 304 bad0a742b21817b312416d7ac79f9ea9
10.2.0.0 - - [15/Aug/2023:15:41:13 +0000] "GET /[REDACTED]_v13.1.13/Resources/webpack/css/fontawesome/css/all.min.css?1692112200 HTTP/2.0" 304 0 "https://[REDACTED]/" "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/116.0" 89 0.065 [[REDACTED]-externe-qual-[REDACTED]-ext-qual-httpd-svc-80] [] 10.2.3.198:80 0 0.065 304 3f260df420bc2d2fa5ce8322c14f3711
10.2.0.0 - - [15/Aug/2023:15:41:13 +0000] "GET /[REDACTED]_v13.1.13/Resources/css/messenger.css?1692112199 HTTP/2.0" 304 0 "https://[REDACTED]/" "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/116.0" 98 0.065 [[REDACTED]-externe-qual-[REDACTED]-ext-qual-httpd-svc-80] [] 10.2.3.198:80 0 0.065 304 e7203b47148cbb45a78c33adc781dc08

Finally, I can see that the annotation and Ingress class on the Ingres resource have been configured correctly by the install script :

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:

...

  annotations:
    meta.helm.sh/release-name: [REDACTED]
    meta.helm.sh/release-namespace: [REDACTED]
    nginx.ingress.kubernetes.io/affinity: cookie
    nginx.ingress.kubernetes.io/client-body-timeout: "3600"
    nginx.ingress.kubernetes.io/client-header-timeout: "3600"
    nginx.ingress.kubernetes.io/client_max_body_size: 5000m
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: 5000m
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/session-cookie-expires: "86400"
    nginx.ingress.kubernetes.io/session-cookie-max-age: "86400"
    openappsec.io/policy: open-appsec-best-practice-policy

...

spec:
  ingressClassName: appsec-nginx
...

Analysis

I have the impression that the solution has been correctly setup by the script, and the traffic is going to the right Ingress Controller, but for some reasons, the traffic just cannot be analyzed by the open-appsec controller.

Any thoughts?

Thanks! :)

Http Transaction Handler temporarily unavailable

open-appsec in Linux Nginx deployment doesn't detect any traffic

Problem description

I'm trying to install open-appsec in a a Linux vm and deploying it using nginx. I've followed the online documentation (also the video tutorial) but it seems that for some reason the traffic is not detected/inspected (similar to #45).

Environment

Linux distribution: Ubuntu 22.04.2 LTS (Jammy Jellyfish)
nginx version: nginx/1.22.1 (it's listed as supported here)

Installation steps

I already have nginx running, working in proxy_pass mode for a local application. I can access the application through the reverse proxy without any problem.

I've run the open-appsec-install --auto --prevent and it looks it successfully installed:

luca@luca-virtual-machine:~$ sudo ./open-appsec-install --auto --prevent
open-appsec for NGINX and Kong Installer v1.2245.1
For release notes and known limitations check:
https://docs.openappsec.io/release-notes
Searching local NGINX…
NGINX version found: 1.22.1-1-jammy
Downloading open-appsec NGINX attachment... stored in '/tmp/open-appsec'
Downloading open-appsec agent... stored in '/tmp/open-appsec'
Add your email to receive important security updates and so you can approach us with technical questions (enter IGNORE to ignore):
IGNORE
Installing open-appsec for NGINX...
Updating NGINX server configuration...
Starting open-appsec installation...
Setting mode to prevent-learn...
Successfully installed open-appsec for NGINX and Kong...

All the required services are running:

luca@luca-virtual-machine:~$ sudo open-appsec-ctl -s
---- open-appsec Nano Agent ----
Version: 1.0.0-open-source
Status: Running
Management mode: Local management
Policy files: 
    /etc/cp/conf/local_policy.yaml
Policy load status: Success
Last policy update: 2023-08-18T09:29:48.793731

---- open-appsec Orchestration Nano Service ----
Type: Public, Version: 1.0.0-open-source, Created at: 2023-08-18T03:06:14+0300
Status: Running

---- open-appsec Attachment Registrator Nano Service ----
Type: Public, Version: 1.0.0-open-source, Created at: 2023-08-18T03:06:14+0300
Status: Running

---- open-appsec Http Transaction Handler Nano Service ----
Type: Public, Version: 1.0.0-open-source, Created at: 2023-08-18T03:06:14+0300
Registered Instances: 4
Status: Running

Below the current policy:

policies:
  default:
    triggers:
    - appsec-default-log-trigger
    mode: prevent-learn
    practices:
    - webapp-default-practice
    custom-response: appsec-default-web-user-response
  specific-rules: []

practices:
  - name: webapp-default-practice
    openapi-schema-validation:
      configmap: []
      override-mode: prevent-learn
    snort-signatures:
      configmap: []
      override-mode: prevent-learn
    web-attacks:
      max-body-size-kb: 1000000
      max-header-size-bytes: 102400
      max-object-depth: 40
      max-url-size-bytes: 32768
      minimum-confidence: critical
      override-mode: prevent-learn
      protections:
        csrf-protection: prevent-learn
        error-disclosure: prevent-learn
        non-valid-http-methods: true
        open-redirect: prevent-learn
    anti-bot:
      injected-URIs: []
      validated-URIs: []
      override-mode: prevent-learn

log-triggers:
  - name: appsec-default-log-trigger
    access-control-logging:
      allow-events: false
      drop-events: true
    additional-suspicious-events-logging:
      enabled: true
      minimum-severity: high
      response-body: false
    appsec-logging:
      all-web-requests: false
      detect-events: true
      prevent-events: true
    extended-logging:
      http-headers: false
      request-body: false
      url-path: false
      url-query: false
    log-destination:
      cloud: false
      stdout:
        format: json

custom-responses:
  - name: appsec-default-web-user-response
    mode: response-code-only
    http-response-code: 403

Below the log file output:

{"eventTime": "2023-08-17T12:32:27.826","eventName": "Web AppSec Policy Loaded Successfully","eventSeverity": "Info","eventPriority": "Low","eventType": "Event Driven","eventLevel": "Log","eventLogLevel": "info","eventAudience": "Security","eventAudienceTeam": "","eventFrequency": 0,"eventTags": ["Threat Prevention"],"eventSource": {"agentId": "689f04c1-0086-4e34-bfcb-61a615e2a089","eventTraceId": "","eventSpanId": "","issuingEngineVersion": "1.0.0-open-source","serviceName": "HTTP Transaction Handler","serviceId": "4"},"eventData": {"logIndex": 4}}
{"eventTime": "2023-08-17T14:11:31.460","eventName": "Web AppSec Policy Loaded Successfully","eventSeverity": "Info","eventPriority": "Low","eventType": "Event Driven","eventLevel": "Log","eventLogLevel": "info","eventAudience": "Security","eventAudienceTeam": "","eventFrequency": 0,"eventTags": ["Threat Prevention"],"eventSource": {"agentId": "9b21d1d9-281e-418e-9b1c-24bf1f8d7ab2","eventTraceId": "","eventSpanId": "","issuingEngineVersion": "","serviceName": "HTTP Transaction Handler","serviceId": "4"},"eventData": {"logIndex": 1}}
{"eventTime": "2023-08-17T14:11:34.336","eventName": "Check Point Nano-service started","eventSeverity": "Info","eventPriority": "Medium","eventType": "Event Driven","eventLevel": "Log","eventLogLevel": "info","eventAudience": "Security","eventAudienceTeam": "","eventFrequency": 0,"eventTags": ["Informational"],"eventSource": {"agentId": "9b21d1d9-281e-418e-9b1c-24bf1f8d7ab2","eventTraceId": "","eventSpanId": "","issuingEngineVersion": "1.0.0-open-source","serviceName": "HTTP Transaction Handler","serviceId": "4"},"eventData": {"logIndex": 2,"serviceName": "HTTP Transaction Handler"}}
{"eventTime": "2023-08-17T14:14:56.983","eventName": "Web AppSec Policy Loaded Successfully","eventSeverity": "Info","eventPriority": "Low","eventType": "Event Driven","eventLevel": "Log","eventLogLevel": "info","eventAudience": "Security","eventAudienceTeam": "","eventFrequency": 0,"eventTags": ["Threat Prevention"],"eventSource": {"agentId": "9b21d1d9-281e-418e-9b1c-24bf1f8d7ab2","eventTraceId": "","eventSpanId": "","issuingEngineVersion": "","serviceName": "HTTP Transaction Handler","serviceId": "4"},"eventData": {"logIndex": 1}}
{"eventTime": "2023-08-17T14:14:59.185","eventName": "Check Point Nano-service started","eventSeverity": "Info","eventPriority": "Medium","eventType": "Event Driven","eventLevel": "Log","eventLogLevel": "info","eventAudience": "Security","eventAudienceTeam": "","eventFrequency": 0,"eventTags": ["Informational"],"eventSource": {"agentId": "9b21d1d9-281e-418e-9b1c-24bf1f8d7ab2","eventTraceId": "","eventSpanId": "","issuingEngineVersion": "1.0.0-open-source","serviceName": "HTTP Transaction Handler","serviceId": "4"},"eventData": {"logIndex": 2,"serviceName": "HTTP Transaction Handler"}}
{"eventTime": "2023-08-17T15:18:07.208","eventName": "Web AppSec Policy Loaded Successfully","eventSeverity": "Info","eventPriority": "Low","eventType": "Event Driven","eventLevel": "Log","eventLogLevel": "info","eventAudience": "Security","eventAudienceTeam": "","eventFrequency": 0,"eventTags": ["Threat Prevention"],"eventSource": {"agentId": "9b21d1d9-281e-418e-9b1c-24bf1f8d7ab2","eventTraceId": "","eventSpanId": "","issuingEngineVersion": "","serviceName": "HTTP Transaction Handler","serviceId": "4"},"eventData": {"logIndex": 1}}
{"eventTime": "2023-08-17T15:18:10.331","eventName": "Check Point Nano-service started","eventSeverity": "Info","eventPriority": "Medium","eventType": "Event Driven","eventLevel": "Log","eventLogLevel": "info","eventAudience": "Security","eventAudienceTeam": "","eventFrequency": 0,"eventTags": ["Informational"],"eventSource": {"agentId": "9b21d1d9-281e-418e-9b1c-24bf1f8d7ab2","eventTraceId": "","eventSpanId": "","issuingEngineVersion": "1.0.0-open-source","serviceName": "HTTP Transaction Handler","serviceId": "4"},"eventData": {"logIndex": 2,"serviceName": "HTTP Transaction Handler"}}
{"eventTime": "2023-08-18T07:36:52.478","eventName": "Web AppSec Policy Loaded Successfully","eventSeverity": "Info","eventPriority": "Low","eventType": "Event Driven","eventLevel": "Log","eventLogLevel": "info","eventAudience": "Security","eventAudienceTeam": "","eventFrequency": 0,"eventTags": ["Threat Prevention"],"eventSource": {"agentId": "b3705d29-7eb7-4fed-90a6-9ffee996c1d0","eventTraceId": "","eventSpanId": "","issuingEngineVersion": "","serviceName": "HTTP Transaction Handler","serviceId": "4"},"eventData": {"logIndex": 1}}
{"eventTime": "2023-08-18T07:36:56.667","eventName": "Check Point Nano-service started","eventSeverity": "Info","eventPriority": "Medium","eventType": "Event Driven","eventLevel": "Log","eventLogLevel": "info","eventAudience": "Security","eventAudienceTeam": "","eventFrequency": 0,"eventTags": ["Informational"],"eventSource": {"agentId": "b3705d29-7eb7-4fed-90a6-9ffee996c1d0","eventTraceId": "","eventSpanId": "","issuingEngineVersion": "1.0.0-open-source","serviceName": "HTTP Transaction Handler","serviceId": "4"},"eventData": {"logIndex": 2,"serviceName": "HTTP Transaction Handler"}}

Expected behaviour

I've tried some classic injection (i.e. SQL injection and similar) but it seems that thet are not detected nor prevented.
Also, to check if the traffic was actually inspected by open-appsec I've set the all-web-requests to true
(and applied the policy again). However, I can't see any log related to HTTP requests (using open-appsec-ctl -vl). So I presume that the traffic is not inspected at all.

Stable version

Dear Team,

I was looking for a open source WAF solution for a K8's cluster and your project looks very interesting to me. Can you please share a timeline for a stable release, if possible? Thank you!

Regards,
Arun Prasad

Pre-built binaries for Debian stable Bookworm

Hello,

Do you have any plans to make a pre-built version compatible with the stable version of debian (12 bookworm) ?

Thanks for your help.
Regards,

cp-nano-http-transaction-handler.log no such file error

After a fresh installation on an Ubuntu Server 22.04 LTS , trying to view the logs for a sanity check comes up with the following error:

> open-appsec-ctl --view-logs

/var/log/nano_agent/cp-nano-http-transaction-handler.log? : No such file or directory

Autoinstaller not detecting a supported version

Hi there!

I'm trying to install open-appsec using the auto installer method ./open-appsec-install --auto. My server is currently running NGINX version 1.18.0-6ubuntu14.3 on Ubuntu Jammy 22.04 LTS. As stated in the supported pre-compiled versions list the script should find the correct binaries and install them but I keep getting no supported versions found.

# ./open-appsec-install --auto
open-appsec for NGINX and Kong Installer v1.2245.1
For release notes and known limitations check:
https://docs.openappsec.io/release-notes
Searching local NGINX…
NGINX version found: 
Downloading open-appsec NGINX attachment... stored in '/tmp/open-appsec'
Unsupported NGINX version, for supported platforms, OS and NGINX versions please see docs.openappsec.io
Downloading open-appsec agent... stored in '/tmp/open-appsec'

Is this a bug? Also, I can't compile from source code and I expect the script to work as I meet all the requirements.

Thanks.

OpenAppSec is blocking sane URL without any possibility for correction

Hi guys,

This issue is more of a demand of assistance, because I'm maybe missing something.

One of the URL of my application is blocked by OpenAppSec ; I have a 403 return code, but nothing is displaying in the logs of the agent.
As soon as I switch from prevent to learn mode, the page is accessible again. I tried to stay in learning mode a few days in the hope that the model would have learned that the URL is not dangerous, but unfortunately, as soon as I switch back to prevent mode, the page is blocked again.

Here are some details about my installation :

OpenAppSec version : 1.0.1, running along an NGinx Ingress Controller in a Kubernetes cluster (installed via the HelmChart as indicated in the documentation).
Blocked URL : POST method to update the configuration of the application. The pattern of the URL is https://[APP-HOST]/[APP-VERSION]/ControlCenter/security_settings.php
Model used : production model downloaded from OpenAppSec Portal
Configuration & CRDs : everything is in that gist

Don't hesitate if you need anything else.

Thank you :)

Openappsec coredumping

OS : RH 9.2
Nginx : nginx -V
nginx version: nginx/1.20.1 ( nginx-1.20.1-13.el9.x86_64 )

Used automated and manual install .
Selinux on AND off .. same result below

cp-nano-watchdog[34058]: /etc/cp/watchdog/cp-nano-watchdog: line 859: 34911 Aborted (core dumped) LD_LIBRARY_PATH=/usr/lib/cpnano/ /etc/cp/orchestration/cp-nano-orchestration --orchestration-mode=hybrid_mode Sep 01 22:15:38 zabbix cp-nano-watchdog[34058]: /etc/cp/watchdog/cp-nano-watchdog: line 859: 34924 Aborted (core dumped) LD_LIBRARY_PATH=/usr/lib/cpnano/ /etc/cp/attachmentRegistrator/cp-nano-attachment-registrator Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35031]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [preloadComponents@components_list_impl.h:118 Sep 01 22:15:38 zabbix cp-nano-watchdog[35044]: [[email protected]:587 | ---] ignoring an illegal configuration argument. Argument: /etc/cp/orchestration/cp-nano-orchestration
| ---] Preloading component: Environment
| ---] Preloading component: DebugIS
| ---] Preloading component: Version
| ---] Preloading component: Buffer
| ---] Preloading component: ShellCmd
| ---] Preloading component: GenericMetric
| ---] Preloading component: ConfigComponent
| ---] Preloading component: InstanceAwareness
| ---] Preloading component: IntelligenceComponentV2
| ---] Preloading component: AgentDetails
| ---] Preloading component: LoggingComp
| ---] Preloading component: TimeProxyComponent
| ---] Preloading component: MainloopComponent
| ---] Preloading component: SignalHandler
| ---] Preloading component: RestServer
| ---] Preloading component: Encryptor
| ---] Preloading component: SocketIS
| ---] Preloading component: ProtoMessageComp
| ---] Preloading component: CPUCalculator
| ---] Preloading component: CPUManager
| ---] Preloading component: MemoryCalculator
| ---] Preloading component: MessagingBuffer
| ---] Preloading component: TenantManager
| ---] Preloading component: GenericRulebase
| ---] Preloading component: AttachmentRegistrator
96846656968606f61286565">[email protected]:587 | ---] ignoring an illegal configuration argument. Argument: /etc/cp/attachmentRegistrator/cp-nano-attachment-registrator

Agent Container libcurl missing

Hi,

I used the docker container ghcr.io/openappsec/agent:latest (V 1.0.1)

when I run it, I see an error after the Fog address:

Attaching to open-appsec-agent

open-appsec-agent  | Check Point Nano Agent Version 1.0.1 Install Package
open-appsec-agent  | Verifying archive integrity... All good.
open-appsec-agent  | Uncompressing...
open-appsec-agent  | Fog address='https://inext-agents.cloud.ngen.checkpoint.com'
open-appsec-agent  |   100%    100%  install: cannot stat 'lib/libcurl.so': No such file or directory

and I have problems - I assume the install script was not able to correctly set the Fog address - as in the logfile /var/log/nano_agent/cp-nano-http-transaction-handler.dbg1 I see the following:

| ###] IP address was not found for the given host name. Host: i2-agents.cloud.ngen.checkpoint.com
| ###] Failed to establish connection to the Fog: Failed to establish new connection with: i2-agents.cloud.ngen.checkpoint.com:443
| ###] Failed to connect to the Fog, Address: https://i2-agents.cloud.ngen.checkpoint.com/

this address indeed does not resolve via DNS... ( i get NXDOMAIN)

I assume this is due to the error when trying to set the fog server, because of the missing lib/libcurl.so in the container...

thanks for any help.

label:"help wanted" I need to use Declarative policy hence have few doubts about it

Hi Team,

So if I want to manage policy using declarative statements.

I need to edit local_policy.yaml file?
Then writing down statements as mentioned in local_policy.yaml is sufficient?
Because when I see policy is being managed by opeappsec.io portal; I see policy is created in json file which is policy.json
Do I need to write that as well?
Or can I use the existing policy from one instance to another?

Can you please help me on understanding policies when management is through declarative and through portal?

Is ML model already part of default open-appsec-agent?

Hi Team,

As per documentation if I need to use ML modul I need to download the open-appsec-advanced-model.tgz and follow the README file. Which exactly I did and the file is not clear with a pre-build agent downloaded from portal instead of compiling it.
How do I use with my installation?

docker ml model learning data location

Hi, I'm using open appsec with docker and is connected to the web management portal. I wanted to know how does docker restart or re-running the open-appsec agent docker container affect the learning data ?
Where is the learning data for the ML model stored ? Will docker restart affect it ?

Unable to see HTTP transactions - Install with Docker (Centrally Managed)

Hello,

I've managed to install containers through the following official documentation

Some technical informations are changed (domain, application name and port used).

versions

$ date
dim. 15 oct. 2023 12:02:09 CEST
ghcr.io/openappsec/agent latest
ghcr.io/openappsec/nginx-attachment latest

compose file

`version: '3'
services:
  pod-openappsec-agent:
    container_name: open-appsec-agent
    image: ghcr.io/openappsec/agent:latest
    hostname: open-appsec-agent
    domainname: ***
    restart: unless-stopped
    ipc: host
    network: host
    privileged: true
    command: "/cp-nano-agent --token ***************"
    volumes:
      - <local dir>/agent/conf:/etc/cp/conf:rw
      - <local dir>/agent/data:/etc/cp/data:rw
      - <local dir>/agent/log:/var/log/nano_agent:rw
  pod-nginx:
    container_name: open-appsec-nginx
    image: ghcr.io/openappsec/nginx-attachment:latest
    hostname: open-appsec-nginx
    domainname: ***
    restart: unless-stopped
    ipc: host
    pid: host
    network: host
    privileged: true
    ports:
      - 14443:443
    volumes:
      - <local dir>/nginx/conf:/etc/nginx:rw
      - <local dir>/nginx/log:/var/log/nginx:rw

run

podman-compose -f compose.yml up -d

['podman', '--version', '']
using podman version: 4.3.1
** excluding:  set()
['podman', 'network', 'exists', 'containers_default']
podman run --name=open-appsec-agent -d --label io.podman.compose.config-hash=123 --label io.podman.compose.project=containers --label io.podman.compose.version=0.0.1 --label com.docker.compose.project=containers --label com.docker.compose.project.working_dir=/opt/containers --label com.docker.compose.project.config_files=<local dir>/compose.yml --label com.docker.compose.container-number=1 --label com.docker.compose.service=pod-openappsec-agent -v <local dir>/agent/conf:/etc/cp/conf:rw -v <local dir>/agent/data:/etc/cp/data:rw -v <local dir>/agent/log:/var/log/nano_agent:rw --net containers_default --network-alias pod-openappsec-agent --hostname open-appsec-agent --privileged --restart unless-stopped ghcr.io/openappsec/agent:latest /cp-nano-agent --token *******
exit code: 0
['podman', 'network', 'exists', 'containers_default']
podman run --name=open-appsec-nginx -d --label io.podman.compose.config-hash=123 --label io.podman.compose.project=containers --label io.podman.compose.version=0.0.1 --label com.docker.compose.project=containers --label com.docker.compose.project.working_dir=/opt/containers --label com.docker.compose.project.config_files=<local dir>/compose.yml --label com.docker.compose.container-number=1 --label com.docker.compose.service=pod-nginx -v <local dir>/nginx/conf:/etc/nginx:rw -v <local dir>/nginx/log:/var/log/nginx:rw -net containers_default --network-alias pod-nginx -p 11443:443 --hostname open-appsec-nginx --privileged --restart unless-stopped ghcr.io/openappsec/nginx-attachment:latest
exit code: 0

Problem

On the SaaS web gui the agent is successfully registered.

After adding a first asset linked to the policy I do not see any trafic

Diagnostic

I'm able to reach my application behing the NGinx (reverse-proxy)

location / {
    proxy_pass http://your-web-application;
    # Additional proxy settings if needed
}

NGinx cp_attachment module still in conf

conf/nginx.conf

load_module /usr/lib/nginx/modules/ngx_cp_attachment_module.so;

user  nginx;
worker_processes  auto;

error_log  /var/log/nginx/error.log notice;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}

It seems that the containers are not correctly setup regarding http transaction, maybe a clue ?

$ podman exec -it open-appsec-agent /usr/sbin/cpnano -vl
less: can't open '/var/log/nano_agent/cp-nano-http-transaction-handler.log?': No such file or directory

$ podman exec -it open-appsec-agent /usr/sbin/cpnano -s
---- open-appsec Nano Agent ----
Version: 1.1.0-open-source
Status: Running
Management mode: Cloud management
Policy files:
    /etc/cp/conf/local_policy.yaml
Policy load status: Success
Last policy update: 2023-10-15T10:19:39.607845

---- open-appsec Orchestration Nano Service ----
Type: Public, Version: 1.1.0-open-source, Created at: 2023-10-05T17:46:48+0000
Status: Running

---- open-appsec Attachment Registrator Nano Service ----
Type: Public, Version: 1.1.0-open-source, Created at: 2023-10-05T17:46:48+0000
Status: Running

---- open-appsec Http Transaction Handler Nano Service ----
Type: Public, Version: 1.1.0-open-source, Created at: 2023-10-05T17:46:48+0000
Status: Ready


For release notes and known limitations check: https://docs.openappsec.io/release-notes
For troubleshooting and support: https://openappsec.io/support

cp-nano-agent-install.log

Copy cp-agent-info tool
Copy cp-nano-package-list
cp: -r not specified; omitting directory 'EULA.txt'
cp: -r not specified; omitting directory 'Licenses-for-Third-Party-Components.txt'

Starting installation of open-appsec Nano Agent [Sun Oct 15 09:37:07 UTC 2023]
rm: cannot remove '/etc/cp/conf': Resource busy
rm: cannot remove '/etc/cp/data': Resource busy
Creating env details file
Building the default policy json
Copying cp-nano-agent binary file to folder: /etc/cp/orchestration/cp-nano-orchestration
Copy source md5:
7824ac827120aa147f4711b786b43436  ./bin/orchestration_comp
Copy destination md5:
destination '/etc/cp/orchestration/cp-nano-orchestration' does not exist.
Destination md5, after the copy:
7824ac827120aa147f4711b786b43436  /etc/cp/orchestration/cp-nano-orchestration
Copy source md5:
d1765d79c93cbb0d7ab20386206a0b9f  open-appsec-cloud-mgmt
Copy destination md5:
destination '/etc/cp/scripts/open-appsec-cloud-mgmt' does not exist.
Destination md5, after the copy:
d1765d79c93cbb0d7ab20386206a0b9f  /etc/cp/scripts/open-appsec-cloud-mgmt
Copy source md5:
db703b597d494bd2397b0119e1d26a5f  open-appsec-cloud-mgmt-k8s
Copy destination md5:
destination '/etc/cp/scripts/open-appsec-cloud-mgmt-k8s' does not exist.
Destination md5, after the copy:
db703b597d494bd2397b0119e1d26a5f  /etc/cp/scripts/open-appsec-cloud-mgmt-k8s
Copy source md5:
fbaa4b08b74902e802d87f54e7fbdd02  open-appsec-ctl.sh
Copy destination md5:
destination '/etc/cp/scripts/open-appsec-ctl.sh' does not exist.
Destination md5, after the copy:
fbaa4b08b74902e802d87f54e7fbdd02  /etc/cp/scripts/open-appsec-ctl.sh
Copy source md5:
cb54f53385c4a0089ea1bcdd16cf1604  ./scripts/cp-nano-makefile-generator.sh
Copy destination md5:
destination '/etc/cp/scripts/cp-nano-makefile-generator.sh' does not exist.
Destination md5, after the copy:
cb54f53385c4a0089ea1bcdd16cf1604  /etc/cp/scripts/cp-nano-makefile-generator.sh
Saving authentication token to file
Copy source md5:
59a3d4a3243c60933a9ac8a1039a0256  certificate/ngen.body.crt
Copy destination md5:
destination '/etc/cp/certs/fog.pem' does not exist.
Destination md5, after the copy:
59a3d4a3243c60933a9ac8a1039a0256  /etc/cp/certs/fog.pem
chmod: cannot access '/etc/cp/conf/cp-nano-orchestration-conf.json': No such file or directory
Copy source md5:
a6d809c0b02126b719e9890bd22afc78  configuration/cp-nano-orchestration-conf.json
Copy destination md5:
destination '/etc/cp/conf/cp-nano-orchestration-conf.json' does not exist.
Destination md5, after the copy:
a6d809c0b02126b719e9890bd22afc78  /etc/cp/conf/cp-nano-orchestration-conf.json
Copy source md5:
681e4ec40932318bb4cf782ffcb18ba3  configuration/cp-nano-orchestration-debug-conf.json
Copy destination md5:
destination '/etc/cp/conf/cp-nano-orchestration-debug-conf.json' does not exist.
Destination md5, after the copy:
681e4ec40932318bb4cf782ffcb18ba3  /etc/cp/conf/cp-nano-orchestration-debug-conf.json
Installing the watchdog
Copy source md5:
2cc8c5c4697ebd53ed7e5e6289b0cec8  watchdog/watchdog
Copy destination md5:
destination '/etc/cp/watchdog/cp-nano-watchdog' does not exist.
Destination md5, after the copy:
2cc8c5c4697ebd53ed7e5e6289b0cec8  /etc/cp/watchdog/cp-nano-watchdog
Copy source md5:
4e4f26afb12a89baa97ce831d20f7fd2  watchdog/wait-for-networking-inspection-modules.sh
Copy destination md5:
destination '/etc/cp/watchdog/wait-for-networking-inspection-modules.sh' does not exist.
Destination md5, after the copy:
4e4f26afb12a89baa97ce831d20f7fd2  /etc/cp/watchdog/wait-for-networking-inspection-modules.sh
Install cp-nano-agent service file
Start cp-nano-agent service
./orchestration_package.sh: line 395: service: not found
Note: in order for the agent to remain active and effective it must connect to the Fog/Cloud at least every 45 days
open-appsec Orchestration Nano Service installation completed successfully
Copy cp-agent-info tool
Copy cp-nano-package-list
cp: -r not specified; omitting directory 'EULA.txt'
cp: -r not specified; omitting directory 'Licenses-for-Third-Party-Components.txt'

Starting upgrading of open-appsec Nano Agent [Sun Oct 15 09:40:49 UTC 2023]
Installing the watchdog
Copy source md5:
2cc8c5c4697ebd53ed7e5e6289b0cec8  watchdog/watchdog
Copy destination md5:
destination '/etc/cp/watchdog/cp-nano-watchdog' does not exist.
Destination md5, after the copy:
2cc8c5c4697ebd53ed7e5e6289b0cec8  /etc/cp/watchdog/cp-nano-watchdog
Copy source md5:
4e4f26afb12a89baa97ce831d20f7fd2  watchdog/wait-for-networking-inspection-modules.sh
Copy destination md5:
destination '/etc/cp/watchdog/wait-for-networking-inspection-modules.sh' does not exist.
Destination md5, after the copy:
4e4f26afb12a89baa97ce831d20f7fd2  /etc/cp/watchdog/wait-for-networking-inspection-modules.sh
Install cp-nano-agent service file
Restart cp-nano-agent service
Upgrade to latest
Copying cp-nano-agent binary file to folder: /etc/cp/orchestration/cp-nano-orchestration
Copy source md5:
7824ac827120aa147f4711b786b43436  ./bin/orchestration_comp
Copy destination md5:
destination '/etc/cp/orchestration/cp-nano-orchestration' does not exist.
Destination md5, after the copy:
7824ac827120aa147f4711b786b43436  /etc/cp/orchestration/cp-nano-orchestration
Copy source md5:
d1765d79c93cbb0d7ab20386206a0b9f  open-appsec-cloud-mgmt
Copy destination md5:
destination '/etc/cp/scripts/open-appsec-cloud-mgmt' does not exist.
Destination md5, after the copy:
d1765d79c93cbb0d7ab20386206a0b9f  /etc/cp/scripts/open-appsec-cloud-mgmt
Copy source md5:
db703b597d494bd2397b0119e1d26a5f  open-appsec-cloud-mgmt-k8s
Copy destination md5:
destination '/etc/cp/scripts/open-appsec-cloud-mgmt-k8s' does not exist.
Destination md5, after the copy:
db703b597d494bd2397b0119e1d26a5f  /etc/cp/scripts/open-appsec-cloud-mgmt-k8s
Copy source md5:
fbaa4b08b74902e802d87f54e7fbdd02  open-appsec-ctl.sh
Copy destination md5:
destination '/etc/cp/scripts/open-appsec-ctl.sh' does not exist.
Destination md5, after the copy:
fbaa4b08b74902e802d87f54e7fbdd02  /etc/cp/scripts/open-appsec-ctl.sh
Copy source md5:
cb54f53385c4a0089ea1bcdd16cf1604  ./scripts/cp-nano-makefile-generator.sh
Copy destination md5:
destination '/etc/cp/scripts/cp-nano-makefile-generator.sh' does not exist.
Destination md5, after the copy:
cb54f53385c4a0089ea1bcdd16cf1604  /etc/cp/scripts/cp-nano-makefile-generator.sh
Upgrade completed successfully
Copy cp-agent-info tool
Copy cp-nano-package-list
cp: -r not specified; omitting directory 'EULA.txt'
cp: -r not specified; omitting directory 'Licenses-for-Third-Party-Components.txt'

cp-nano-http-transaction-handler-install.log

Starting installation of Check Point HTTP Transaction Handler service [Sun Oct 15 10:19:16 UTC 2023]

cp: cannot stat 'conf/cp-nano-ips-protections.json': No such file or directory
chmod: cannot access '/etc/cp/conf/data/cp-nano-ips-protections.data': No such file or directory
cp: cannot stat 'resources/cp-ab.js': No such file or directory
cp: cannot stat 'resources/cp-csrf.js': No such file or directory
chmod: cannot access '/etc/cp/conf/waap/cp-ab.js': No such file or directory
chmod: cannot access '/etc/cp/conf/waap/cp-csrf.js': No such file or directory
Requested to restart service '/etc/cp/HttpTransactionHandler/cp-nano-http-transaction-handler', but it is not registered
Installation completed successfully.

Both containers can ping each other. No sure how to validate the communication and how it works between NGinx+module container and the agent container.

Even after trying all troubleshooting tips given in the documtation and tests like http://myappli.mydomain.tld:14443?shell_cmd=cat/etc/passwd I can not find where is the problem with my installation.

Any help will be more than welcome !

Difficulties switching to non-default practice using specific-rules policy

Hi,
I am struggling with switching to non-default practice.
As per your documentation I should be using "policies --> specific-rules", however when I look at logs, then the only practice which is triggered is the webapp-default-practice (see log below and statement -->"practiceName": "local_policy/webapp-default-practice")

Is it some problem with configuration, or is openappsec not taking the configuration in ? I fail to see any error here, yet the custom policy seems not to be applied.
By the way what I edit the policy with open-appsec-ctl -ep, then apply it with open-appsec-ctl -ap

Here is the configuration I use:

policies:
  default:
    triggers:
    - appsec-default-log-trigger
    mode: detect-learn
    practices:
    - webapp-default-practice
    custom-response: appsec-default-web-user-response
  specific-rules:
  - host: foto.example.com
    triggers:
    - appsec-default-log-trigger
    mode: detect-learn
    practices:
    - immich
    custom-response: appsec-default-web-user-response

practices:
  - name: webapp-default-practice
    openapi-schema-validation:
      configmap: []
      override-mode: detect-learn
    snort-signatures:
      configmap: []
      override-mode: detect-learn
    web-attacks:
      max-body-size-kb: 200000000
      max-header-size-bytes: 102400
      max-object-depth: 40
      max-url-size-bytes: 32768
      minimum-confidence: critical
      override-mode: detect-learn
      protections:
        csrf-protection: detect-learn
        error-disclosure: detect-learn
        non-valid-http-methods: true
        open-redirect: detect-learn
    anti-bot:
      injected-URIs: []
      validated-URIs: []
      override-mode: detect-learn
  - name: immich
    openapi-schema-validation:
      configmap: []
      override-mode: detect-learn
    snort-signatures:
      configmap:
      override-mode: detect-learn
    web-attacks:
      max-body-size-kb: 200000000
      max-header-size-bytes: 102400
      max-object-depth: 40
      max-url-size-bytes: 32768
      minimum-confidence: critical
      override-mode: detect-learn
      protections:
        csrf-protection: detect-learn
        error-disclosure: detect-learn
        non-valid-http-methods: true
        open-redirect: detect-learn
    anti-bot:
      injected-URIs: []
      validated-URIs: []
      override-mode: detect-learn

log-triggers:
  - name: appsec-default-log-trigger
    access-control-logging:
      allow-events: false
      drop-events: true
    additional-suspicious-events-logging:
      enabled: true
      minimum-severity: high
      response-body: false
    appsec-logging:
      all-web-requests: false
      detect-events: true
      prevent-events: true
    extended-logging:
      http-headers: false
      request-body: false
      url-path: false
      url-query: false
    log-destination:
      cloud: true
      stdout:
        format: json

custom-responses:
  - name: appsec-default-web-user-response
    mode: response-code-only
    http-response-code: 403

Example log:

{"eventTime": "2023-10-16T18:50:48.787","eventName": "Web Request","eventSeverity": "Info","eventPriority": "Low","eventType": "Event Driven","eventLevel": "Log","eventLogLevel": "info","eventAudience": "Security","eventAudienceTeam": "","eventFrequency": 0,"eventTags": ["Threat Prevention","Web Application & API Protection"],"eventSource": {"agentId": "495390e2-2692-4331-9ffc-b7d4908e9315","eventTraceId": "","eventSpanId": "","issuingEngineVersion": "1.1.0-open-source","serviceName": "HTTP Transaction Handler","serviceId": "2","assetId": "Any","assetName": "Any"},"eventData": {"logIndex": 199,"eventReferenceId": "7eeba4a1-1d0d-4c67-8fff-41f709a80cf7","assetId": "Any","assetName": "Any","sourceIP": "192.168.31.45","httpSourceId": "192.168.31.45","sourcePort": 51428,"httpHostName": "foto.example.com","httpMethod": "GET","ruleId": "Any","securityAction": "Detect","waapOverride": "None","practiceType": "Threat Prevention","practiceSubType": "Web Application","ruleName": "Any","practiceId": "5e34885f-de5b-421a-8aea-14a1cceb82f0","practiceName": "local_policy/webapp-default-practice","waapIncidentType": "","matchedSample": "","matchedLocation": "","matchedParameter": "","waapFoundIndicators": "","matchedIndicators": "","learnedIndicators": "","waapUserReputationScore": 438,"waapUserReputation": "Normal","waapUriFalsePositiveScore": 0,"waapKeywordsScore": 0,"waapFinalScore": 0,"waapCalculatedThreatLevel": 0}}

Is ML model already built with default agent installation with Nginx?

Hi Team,

I have nginx 1.23.2 and installed openappsec agent from my.openappsec.io and wondering if default installation supports ML algorithm by default? Or it does support only with compilation with git clone?

About technical documentation

Hi,

Thank you for open source. I am very interested in the technology.
Is there any technical documentation on training and scoring algorithms related to this? The code looks quite difficult to understand.
Another question is, how is the basic model trained? Is there any related information available?
I have read the technical white paper on the website, and the related algorithms are not described very clearly.

Traefik Support

This is a super interesting project! From the documentation I can see there are already a number of other integrations planned.

I would love to see a traefik plugin for this, is this something that would potentially be supported in the future or is it something the community should look to provide?

[enhancement] HTTP Flood identification

Good afternoon team,

I notice that many features can perform "open appsec," but only one feature, named rate limit, can protect against DDOS HTTP Flood attacks. Over time, this type of attack has grown.

For example, Cloudflare can detect and mitigate these attacks, as shown in this mitigation example:

https://pbs.twimg.com/media/FnvwRBVaUAESTpS?format=png&name=small.

To protect against this type of attack, I propose an idea: when one IP exceeds the rate limit 4 times in less than 2 minutes, it should be blocked to protect the backend. I now believe this is the best approach we can take.

I would like to attempt to create a pull request with this feature, but I think I lack knowledge about the project and require many hours of programming. Please share your perspective and let me know if you find this interesting or need more details.

2023_01_02 branch seems incomplete

Hello all,

The Jan 2th update seems incomplete and not yet fully pushed. For example the new ML model file seems missing from the repo.
When will this be done?

Thanks,
input-io

AWS NLB NGINX Ingress Support

Hi,

I've noticed that when using AWS with NGINX Ingress that the service load balancer is created with an AWS Classic ELB which means that I cannot retrieve the true source IP from pods. If you refer to the setup instructions for AWS (https://kubernetes.github.io/ingress-nginx/deploy/#aws), there is a provided YAML to set this up with AWS NLB but the file will need modifications to work with open-appsec. I would appreciate it if you could address this issue. Thanks.

Can't install on Debian 11 + nginx 1.18.0-6.1+deb11u3

Hi,
I am trying to install openappsec on Debian 11, Nginx version is 1.18.0-6.1+deb11u3 however the script fails on me stating that there is no support for it. However in the support page it says it is supported.
This is what I am getting:

root@waf1:~# ./open-appsec-install --auto --no-email
open-appsec for NGINX and Kong Installer v1.2245.1
Searching local NGINX…
NGINX version found: 1.18.0-6.1+deb11u3
Downloading open-appsec NGINX attachment... stored in '/tmp/open-appsec'
Unsupported NGINX version, for supported platforms, OS and NGINX versions please see docs.openappsec.io
Downloading open-appsec agent... stored in '/tmp/open-appsec'
Unsupported OS, for supported platforms, OS and NGINX versions please see docs.openappsec.io

root@waf1:~# apt list nginx
Listing... Done
nginx/stable,stable-security,now 1.18.0-6.1+deb11u3 all [installed]

root@waf1:~# cat /etc/issue
Debian GNU/Linux 11 \n \l

Is it me that fails to execute the script correctly or is it something wrong with the script ? Please help.

ARM64 Support

Hi,

I'm looking to use your application but I run exclusively ARM64. I've tried to run it in Kubernetes and I get the following error:
exec /cp-nano-agent: exec format error

I'm very familiar with this sort of error and immediately realise it's because an AMD64 image is being used rather than ARM64. I'm looking into a way to compile this myself but am unsure on how to do this.

LiteSpeed / OpenLiteSpeed compatibility

Hi,
This can be a good to add compatibility with LiteSpeed and OpenLiteSpeed.

Have a nice day.

Checkpoint Appsec vs openappsec

Hi,

i just wandering any different between cp appsec and openappsec?
even though checkpoint product.

Forbidden User: Cluster Role

Hello Open-AppSec Team,

With Helm Chart 4.1.4 as well as Kubernetes 1.26.3, the following issues exist on the controller:
W0517 11:35:16.512778 7 reflector.go:424] k8s.io/[email protected]/tools/cache/reflector.go:169: failed to list *v1.EndpointSlice: endpointslices.discovery.k8s.io is forbidden: User "system:serviceaccount:appsec:open-appsec-open-appsec-k8s-nginx-ingress" cannot list resource "endpointslices" in API group "discovery.k8s.io" at the cluster scope E0517 11:35:16.512860 7 reflector.go:140] k8s.io/[email protected]/tools/cache/reflector.go:169: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: endpointslices.discovery.k8s.io is forbidden: User "system:serviceaccount:appsec:open-appsec-open-appsec-k8s-nginx-ingress" cannot list resource "endpointslices" in API group "discovery.k8s.io" at the cluster scope E0517 11:35:18.271030 7 leaderelection.go:330] error retrieving resource lock appsec/ingress-controller-leader: leases.coordination.k8s.io "ingress-controller-leader" is forbidden: User "system:serviceaccount:appsec:open-appsec-open-appsec-k8s-nginx-ingress" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "appsec"

This error can be solved with adding two more records to ClusterRole "open-appsec-open-appsec-k8s-nginx-ingress"

Solution itself I found here: bitnami/charts#11192 (comment)

Attachment configuration resets indefinitely on Kubernetes

Hi,

I followed the installation guide for Kubernetes on a test server with microk8s installed. The installation went well and the ingress controller is still functional, but it seems there's some issue with the attachment configuration.

The output of kubectl logs open-appsec-open-appsec-k8s-nginx-ingress-controller-0 -n appsec shows the same "Web AppSec Policy Loaded Successfully" event recurring every ~35 seconds:

{"eventTime": "2023-02-01T13:53:53.238","eventName": "Web AppSec Policy Loaded Successfully","eventSeverity": "Low","eventPriority": "Low","eventType": "Event Driven","eventLevel": "Log","eventLogLevel": "info","eventAudience": "Security","eventAudienceTeam": "","eventFrequency": 0,"eventTags": ["Threat Prevention"],"eventSource": {"agentId": "Unknown","eventTraceId": "","eventSpanId": "","issuingEngineVersion": "1.0.0-open-source","serviceName": "HTTP Transaction Handler","serviceId": "1"},"eventData": {"logIndex": 261}}
{"eventTime": "2023-02-01T13:54:27.368","eventName": "Web AppSec Policy Loaded Successfully","eventSeverity": "Low","eventPriority": "Low","eventType": "Event Driven","eventLevel": "Log","eventLogLevel": "info","eventAudience": "Security","eventAudienceTeam": "","eventFrequency": 0,"eventTags": ["Threat Prevention"],"eventSource": {"agentId": "Unknown","eventTraceId": "","eventSpanId": "","issuingEngineVersion": "1.0.0-open-source","serviceName": "HTTP Transaction Handler","serviceId": "1"},"eventData": {"logIndex": 262}}
{"eventTime": "2023-02-01T13:55:02.030","eventName": "Web AppSec Policy Loaded Successfully","eventSeverity": "Low","eventPriority": "Low","eventType": "Event Driven","eventLevel": "Log","eventLogLevel": "info","eventAudience": "Security","eventAudienceTeam": "","eventFrequency": 0,"eventTags": ["Threat Prevention"],"eventSource": {"agentId": "Unknown","eventTraceId": "","eventSpanId": "","issuingEngineVersion": "1.0.0-open-source","serviceName": "HTTP Transaction Handler","serviceId": "1"},"eventData": {"logIndex": 263}}

No other events are recorded, even while sending traffic to the ingress controller.

I also checked the controller logs and it seems that the attachment is being reconfigured/reset indefinitely. The log sequence below appears every ~30 seconds.

|2023-02-01T13:52:13.151: is_ngx_cp_attachment_disabled@ngx_http_cp_attachment_module.c:243 [uid 1 | pid 739] | Reconfiguring the local NGINX attachment state
|2023-02-01T13:52:13.152: is_static_resource_request@ngx_cp_static_content.c:229 [uid 1 | pid 739] <session id 712> | Cannot determine whether request is for a static resource: static resources' table is not initialized
|2023-02-01T13:52:23.141: is_static_resource_request@ngx_cp_static_content.c:229 [uid 1 | pid 739] <session id 713> | Cannot determine whether request is for a static resource: static resources' table is not initialized
|2023-02-01T13:52:23.143: is_static_resource_request@ngx_cp_static_content.c:229 [uid 1 | pid 739] <session id 714> | Cannot determine whether request is for a static resource: static resources' table is not initialized
|2023-02-01T13:52:33.141: is_static_resource_request@ngx_cp_static_content.c:229 [uid 1 | pid 739] <session id 715> | Cannot determine whether request is for a static resource: static resources' table is not initialized
|2023-02-01T13:52:33.142: is_static_resource_request@ngx_cp_static_content.c:229 [uid 2 | pid 740] <session id 493> | Cannot determine whether request is for a static resource: static resources' table is not initialized
|2023-02-01T13:52:33.145: reset_attachment_config@ngx_cp_utils.c:971 [uid 2 | pid 740] <session id 493> | Resetting attachment configuration
|2023-02-01T13:52:33.145: init_general_config@ngx_cp_utils.c:921 [uid 2 | pid 740] <session id 493> | Successfully loaded configuration. inspection mode: 0, debug level: 2, failure mode: fail-open, fail mode timeout: 50 msec, failure wait mode: fail-open, fail mode wait timeout: 150 msec, sessions per minute limit verdict: Accpet, max sessions per minute: 0, req max processing time: 3000 msec, res max processing time: 3000 msec, registration thread timeout: 100 msec, req header thread timeout: 100 msec, req body thread timeout: 150 msec, res header thread timeout: 100 msec, res body thread timeout: 150 msec, wait thread timeout: 150 msec, static resources path: /dev/shm/static_resources, num of nginx ipc elements: 200, keep alive interval msec: 150000 msec

My ingress.yaml:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    openappsec.io/policy: open-appsec-best-practice-policy
    nginx.ingress.kubernetes.io/service-upstream: "true"
  labels:
    app: nginx-ingress-microk8s
  name: nginx-ingress-microk8s
  namespace: default
spec:
  ingressClassName: appsec-nginx
  defaultBackend:
    service:
      name: service1
      port:
        number: 80

And the sample service it communicates with:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: service1
spec:
  replicas: 2
  selector:
    matchLabels:
      app: service1
  template:
    metadata:
      labels:
        app: service1
    spec:
      containers:
        - name: service1
          image: httpd
          ports:
            - name: http
              containerPort: 80
      nodeSelector:
        kubernetes.io/os: linux
---
apiVersion: v1
kind: Service
metadata:
  name: service1
spec:
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: 80
  selector:
    app: service1
  type: ClusterIP

I'm using the default open-appsec-policy.yaml that comes with the installation.

Any ideas what went wrong?

Thanks.

Pod termination takes too long

Hello Open-Appsec Team,

A few months ago I have already placed the problem around restarts from the pods with the product owner of Open-Appsec. According to a feedback, a restart or deletion of a pod should be relatively fast by now. Unfortunately, it is still very slow with the version 4.1.4 and sometimes you prefer to kill the pod itself. The problem itself is still due to a health check of the controller.

Thanks for looking at the problem.

Scrub ip addresses

Hi,

We are looking to use open-appsec in our kubernetes cluster instead of Ingress Nginx. However, we have a requirement from GDPR that we should limit the processing of ip addresses. In order to do that we have included ipscrub in a custom nginx docker image as a module. Would such a thing also be possible with open-appsec?

Thanks!

So writing a simple local_policy.yaml is sufficient in declarative management?

Hi Team,

I am testing the declarative way managment of policy and main intention is to run the nano_agent in an isolated mode. In this case if this is installed in download and install mode. Wondering how it would fetch IPS policies or Antivirus or other security protection? Or even if the policy is offline nano_agent will talk to fog and fetch the updates?

libngen_core.so

Seems like I'm missing this library (libngen_core.so) and not finding any information about it on the web or the provided documents. Any help with pointing me in the right direction? Running Nginx 1.25.2 on RHEL 9.2.

Question regarding Python 2/3 compatibility

While working on a .spec file for EL8 I've stumbled upon across issues with Python compatibility.

The graphqlparser (external/graphqlparser/CMakeLists.txt:10) seems to require Python 3 while nodes/packaging.cmake seems to expect > Python 2, because when building with Python 2, cmake aborts with message "string sub-command STRIP requires two arguments."

Can I just patch external/graphqlparser/CMakeLists.txt to look for Python 3 interpreter?

Edit
Ended up using Python 2 and fixed the STRIP line in nodes/packaging.cmake by wrapping quotation marks around the ${PACKAGE_VERSION}" var.

However, is it planned to migrate to Python 3, because Python 2 is obviously deprecated?!

proxy-wasm support

Lot of reverse proxy servers now has support wasm. Making openappsec as wasm filter will help in integrating with many proxies at once.

table is not initialized

Hello Open-AppSec Team,

With Helm version 4.1.4 I have the following errors in the controller and no Ingress works.
|2023-05-17T11:50:54.369: is_static_resource_request@ngx_cp_static_content.c:229 [uid 2 | pid 26] <session id 783> | Cannot determine whether request is for a static resource: static resources' table is not initialized

I'm not sure if it has something to do with the error, why the individual Ingress do not work but according to the logs at least nothing arrives at the controller.

The exact same configuration with an old DEV Open-AppSec version everything works fine.

openappsec / openappsec Goto Github PK

openappsec's People

Stargazers

Watchers

Forkers

openappsec's Issues

Problem description

Environment

Issue description

Analysis

Problem description

Environment

Installation steps

Expected behaviour

versions

compose file

run

Problem

Diagnostic

Recommend Projects

Recommend Topics

Recommend Org