jmpsec / osctrl Goto Github PK
View Code? Open in Web Editor NEWFast and efficient osquery management
Home Page: https://osctrl.net
License: MIT License
Fast and efficient osquery management
Home Page: https://osctrl.net
License: MIT License
Regardless the logging system that we have configured, it would be a great enhancement to keep the last 24 hours of status logs for each node locally, so it would be displayed in the admin UI. There would be two main components to add/modify:
db
.The 24 hours mark could be configurable and also would be interesting to be able to chose whether to discard data as it expires, or archive it somewhere else.
Currently there is only a docker-compose.yml
to run osctrl
in a docker environment. It would be good to have an actual deploy for k8s, so it can be used within kubernetes.
Using kompose the docker-compose.yml
can be converted directly, so I will start exploring that option first.
The code to enroll new clients automatically makes the UUID
uppercase. In certain environments, for example docker, the UUID
is extracted and submitted as lowercase, so the function getNodeByUUID
can not find the node when trying to update the metadata or log status/result logs. This is the error seen in the logs:
osquery-ubuntu18 | I0710 17:30:03.995153 26 tls.cpp:253] TLS/HTTPS POST request to URI: https://osctrl-nginx/dev/log
osctrl-nginx | 172.25.0.3 - - [10/Jul/2020:17:30:04 +0000] "POST /dev/log HTTP/1.1" 200 22 "-" "osquery/4.4.0" "-"
osctrl-tls | dispatch.go:15: error updating metadata getNodeByUUID record not found
osctrl-tls | dispatch.go:31: error refreshing last result getNodeByUUID record not found
Every time the UUID
is looked up it will be uppercase'd so consistency will be achieved and this will not affect metadata updates or logging ability.
The utility provision.sh
allows to deploy all components of osctrl
in a standalone linux system. It would be useful to expand that to cloud providers and deploy using terraform.
First just in EC2, same as a standard deployment but further improvements could be deploy each component separately, for example if using AWS:
osctrl-tls
, osctrl-admin
and osctrl-api
-> lambda functions.osctrl-db
-> RDSosctrl-nginx
-> EBS/ABSUsing osctrl
v0.1.5
, using auth = "none"
in the service configuration (admin.json
) for osctrl-admin
, when attempting to browse the admin UI, server throws a panic and browser shows a nginx 502 Bad Gateway error.
1 - Deploy locally, using vagrant up
for example,
2 - Change auth = "db"
for auth = "none"
in /opt/osctrl/config/admin.json
3 - Run make install
to rebuild and deploy.
4 - Crash, panic and nginx 502 Bad Gateway. Also this stacktrace:
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: server.go:3012: http: panic serving 127.0.0.1:41128: interface conversion: interface {} is nil, not main.contextValue
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: goroutine 690 [running]:
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: net/http.(*conn).serve.func1(0xc000083cc0)
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: /usr/local/go/src/net/http/server.go:1769 +0x139
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: panic(0xa3a900, 0xc000341980)
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: /usr/local/go/src/runtime/panic.go:522 +0x1b5
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: main.environmentHandler(0xbcf240, 0xc0001bc700, 0xc000176400)
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: /vagrant/cmd/admin/handlers-get.go:121 +0xec6
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: net/http.HandlerFunc.ServeHTTP(0xb01fd0, 0xbcf240, 0xc0001bc700, 0xc000176400)
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: /usr/local/go/src/net/http/server.go:1995 +0x44
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: main.handlerAuthCheck.func1(0xbcf240, 0xc0001bc700, 0xc000176400)
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: /vagrant/cmd/admin/auth.go:17 +0xc2
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: net/http.HandlerFunc.ServeHTTP(0xc0003c3dc0, 0xbcf240, 0xc0001bc700, 0xc000176400)
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: /usr/local/go/src/net/http/server.go:1995 +0x44
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: github.com/gorilla/mux.(*Router).ServeHTTP(0xc0000de9c0, 0xbcf240, 0xc0001bc700, 0xc000176200)
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: /home/vagrant/go/pkg/mod/github.com/gorilla/[email protected]/mux.go:212 +0xe3
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: net/http.serverHandler.ServeHTTP(0xc0004c3c70, 0xbcf240, 0xc0001bc700, 0xc000176200)
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: /usr/local/go/src/net/http/server.go:2774 +0xa8
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: net/http.(*conn).serve(0xc000083cc0, 0xbd0b40, 0xc0001790c0)
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: /usr/local/go/src/net/http/server.go:1878 +0x851
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: created by net/http.(*Server).Serve
Aug 27 22:56:12 osctrl-Dev osctrl-admin[21166]: /usr/local/go/src/net/http/server.go:2884 +0x2f4
App security can be measured in each enrolled node by extracting information with queries. With a query pack this data can be aggregated and used for metrics.
A malicious XSS payload can be injected in the environment name, inside the function statsRefresh
:
https://github.com/jmpsec/osctrl/blob/master/admin/static/js/stats.js
Although exploitation would be hard, is better to follow best practices and consider this as a dangerous vulnerability.
Ability to display data security estimation per enrolled node within a dashboard in osctrl-admin
Data security can be measured in each enrolled node by extracting information with queries. With a query pack this data can be aggregated and used for metrics.
This may be superfluous with #19 coming down the pipeline. Nevertheless, including a configurable link to the actual logging system with the query name included would be convenient. This could be achieved with simple variable substitution in a URL (http://logs.example/search?q=name:${query_name}
).
Stretch goal: include some absolute time variables based around the time that the query was initially kicked off so that we can easily get to older queries. Graylog uses ISO 8601 in the URL, so the url template would be: https://graylog.example/streams/000000000000000/search?rangetype=absolute&from=${date_start}&to=${date_plus_1hour}&q=name:${query_name}
According to https://osctrl.net/components/backend/ it wouldn't be much work to support backends other than postgres. I'd love to see mysql support as that's what my database admins support.
Currently the permission to run on-demand queries is granted with the scope of all-or-none environments. It would be a good enhancement in RBAC to be able to isolate which environments to query.
Display enrolled date in the main table per environment, the same way as Last Seen
is displayed.
Currently osctrl does not have a CONTRIBUTING.md
file that outlines details for contributions. Let's create one!
All osctrl-admin
users should be able to change their own password and edit other fields such as fullname or email. The admins of osctrl-admin
can edit the fullname and the email but not change password so that would be also be a good addition.
Just for context this is already possible but only using osctrl-cli
.
Currently an existing environment only allows to edit the hostname using osctrl-cli
, it would be better if that can be done from the admin UI.
In the admin UI, viewing the nodes table:
1 - Input some text to filter nodes
2 - Use the select all elements of the table
3 - Apply an action to selected nodes (delete, query...)
Only the nodes filtered should get that action applied.
All the nodes of the table get the action applied.
With tags created by environment and platform, we can carve files based on tags and simplify how those carves are delivered.
Add the ability to create query packs within osctrl-admin
. Those packs can be added to tags so they will be delivered to specific enrolled nodes.
Ability to display app security estimation per enrolled node within a dashboard in osctrl-admin
When enrolling a new OSX node (platform darwin
) using the quick-enroll oneliner, if osqueryd is already running, the process is not stopped and there is the following error:
[+] Starting com.facebook.osqueryd
/Library/LaunchDaemons/com.facebook.osqueryd.plist: service already loaded
This seems because the current script uses unload
instead of stop
&& unload
for the launchd service.
Thanks @ewaher for finding this!
When using osctrl-cli
to make a change, for example, updating the hostname for an environment, the command runs but there is no confirmation of the success or failure of the operation. It can be a bit confusing.
OS security can be measured in each enrolled node by extracting information with queries. With a query pack this data can be aggregated and used for metrics.
Using osctrl
v0.1.4
, when running an on-demand query or carving a file that will have as target all environments or platforms, the target isn't set and there are no targets selected.
1 - Login to the admin console
2 - Go to Run Query
or Carve File
3 - Select all environments
or all platforms
as target, and submit it with any query or file.
Expected result: The on-demand query or carve will run for all environments or platforms.
Actual result: No target is selected, therefore the query or carve fails to get delivered to any target.
One of the most common use cases of osctrl is to forward logs to SIEM (Splunk, Graylog, ELK) solutions or long term storage such as Kafka or s3. Since it allows to store a local copy of those logs in the backend it may be helpful having a process that expires or deletes those entries when they are older than a specific threshold: 1 Day, 1 week, 1 month...
With tags created by environment and platform, we can run queries based on tags and simplify how those queries are delivered.
When using osctrl-cli
to update the hostname for a given environment, the flags are not updated so they will reflect the old hostname. Flags need to be generated again.
Thanks @ewaher for spotting it!
There is one admin setting called inactive_hours
that defines how nodes will be classified as active or inactive. For some environments it may be a good idea to show machines as inactive after a short period of time, and the other way around.
Currently the setting applies to the admin component only, it would be great if it can be applied by environment.
What happens:
Node Status logs and Result logs do not display until refresh counter triggers
Repro:
Using admin console, click into an environment, select a node.
Click Status Log or Result Logs tab - observe no data appears.
Wait 60 sec (default timer) and tables will fill.
I would love to see some docs on API endpoints. At a minimum a list of the urls, a list of parameters and a summary of the responses would be great. Right now it's pretty difficult to try and use the api as I don't write go, so digging into the code for the api is 😬
I 'm looking for documentation to install osctrl on ubuntu directly [not using docker]
make docker_all fails with:
writing new private key to 'docker/certs/osctrl.key'
docker/certs/osctrl.key: No such file or directory
140048378546064:error:02001002:system library:fopen:No such file or directory:bss_file.c:402:fopen('docker/certs/osctrl.key','w')
140048378546064:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:404:
make: *** [docker_all] Error 1
Attempting to manually start:
docker-compose up
...
ERROR: Cannot locate specified Dockerfile: docker/tls/Dockerfile
docker-compose.yml is now located inside the docker directory, it seems the path to the Dockerfile should be just tls/Dockerfile.
After editing paths, docker-compose up fails with:
ERROR: Service 'osctrl-tls' failed to build: COPY failed: stat /var/lib/docker/tmp/docker-builder567593261/cmd/tls/scripts: no such file or directory
This appears to be because it's looking for the ~/osctrl/cmd/tls/scripts directory that is one directory up from the docker-compose.yml.
Currently only one logger is allowed when shipping logs collected by osctrl-tls
. It can be Splunk or Graylog, but only one, although db logger is always enabled, to keep a small window of logs in the backend (postgres).
It would be a good feature to enable the forwarding to multiple loggers, so logs can be replicated easily as they arrive.
OSQuery is per definition the ideal tool to gather information from a host. maybe write an extention to allow this information to be used to manage the host as well. this makes OSCTRL in league with tools like SCCM, DesktopCentral, Pulseway etc.
For Windows this means the ability to execute powershell commands and/or modify the registry. for Linux this means modifying config files and/or execute bash commands.
Ability to display OS security estimation per enrolled node within a dashboard in osctrl-admin
The handler function that gets called after a user logs in always redirects users to the system-wide defaultEnv on the Osctrl server.
Unfortunately, if the user does not have permissions to view/access the defaultEnv, they are sent to a blank page. This happens both if a user has access to no environments or any number of other, non-defaultEnvs.
If they do not have access to any environments, they should have an error or message printed.
If they have access to $n other environments (none of which are the Osctrl server's defaultEnv), they should be redirected to https://IP_ADDR:8443/environment/ENV_NAME_THEY_CAN_ACCESS/active so they can view their environments.
Hello,
I am using the latest master branch for deploying osctrl. I am deploying using the dockerize.sh script and face the following issues on deployment.
I find that osctrl-api container is in an exited state. After the script runs I see the following.
$ docker ps -a | head
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
540aa20eaa4d osctrl_osquery-ubuntu18 "/bin/sh wait.sh" About an hour ago Up About an hour osquery-ubuntu18
1d14cad30249 nginx:1.13.5 "nginx -g 'daemon of…" About an hour ago Up About an hour 0.0.0.0:443->443/tcp, 80/tcp, 0.0.0.0:8443->8443/tcp osctrl-nginx
d40d37391fbb osctrl_osctrl-tls "/osctrl-tls/bin/osc…" About an hour ago Up About an hour 0.0.0.0:9000->9000/tcp osctrl-tls
11298d44696b osctrl_osctrl-api "/osctrl-api/bin/osc…" About an hour ago Exited (1) About an hour ago osctrl-api
b09756739c1a osctrl_osctrl-admin "/bin/sh /osctrl-adm…" About an hour ago Exited (1) About an hour ago osctrl-admin
85e04607fd8d postgres:10-alpine "docker-entrypoint.s…" About an hour ago Up About an hour 0.0.0.0:5432->5432/tcp osctrl-db
I would like to add that this issue I face with osctrl-admin as well.
osctrl-admin | Created admin user
osctrl-admin | main.go:159: Loading config/admin.json
osctrl-admin | jwt.go:18: Loading config/jwt.json
osctrl-admin | main.go:228: Error loading config/jwt.json - While parsing config: unexpected end of JSON input
osctrl-admin exited with code 1
I find osctrl-api and osctrl-admin in the exited state after the script runs.
I also find these messages in the log.
osctrl-admin | 2020/05/07 01:51:01 Failed to load DB - dial tcp 172.18.0.2:5432: connect: connection refused
osctrl-db | creating subdirectories ... ok
osctrl-db | selecting default max_connections ... 100
osctrl-db | selecting default shared_buffers ... 128MB
osctrl-db | selecting default timezone ... UTC
osctrl-db | selecting dynamic shared memory implementation ... posix
osctrl-db | creating configuration files ... ok
osctrl-db | running bootstrap script ... ok
osctrl-db | performing post-bootstrap initialization ... sh: locale: not found
osctrl-db | 2020-05-07 01:50:52.870 UTC [29] WARNING: no usable system locales were found
osctrl-db | ok
osctrl-db | syncing data to disk ...
osctrl-db | WARNING: enabling "trust" authentication for local connections
osctrl-db | You can change this by editing pg_hba.conf or using the option -A, or
osctrl-db | --auth-local and --auth-host, the next time you run initdb.
osctrl-db | ok
osctrl-db |
osctrl-db | Success. You can now start the database server using:
osctrl-db |
osctrl-db | pg_ctl -D /var/lib/postgresql/data -l logfile start
osctrl-db |
osctrl-db | waiting for server to start....2020-05-07 01:51:01.424 UTC [34] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
osctrl-db | 2020-05-07 01:51:01.568 UTC [35] LOG: database system was shut down at 2020-05-07 01:50:57 UTC
osctrl-db | 2020-05-07 01:51:01.601 UTC [34] LOG: database system is ready to accept connections
osctrl-db | done
osctrl-db | server started
osctrl-db | CREATE DATABASE
osctrl-db |
osctrl-db |
osctrl-db | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
osctrl-db |
osctrl-db | waiting for server to shut down....2020-05-07 01:51:02.910 UTC [34] LOG: received fast shutdown request
osctrl-db | 2020-05-07 01:51:03.016 UTC [34] LOG: aborting any active transactions
osctrl-db | 2020-05-07 01:51:03.017 UTC [34] LOG: worker process: logical replication launcher (PID 41) exited with exit code 1
osctrl-db | 2020-05-07 01:51:03.017 UTC [36] LOG: shutting down
osctrl-db | 2020-05-07 01:51:03.513 UTC [34] LOG: database system is shut down
osctrl-db | done
osctrl-db | server stopped
osctrl-db |
osctrl-db | PostgreSQL init process complete; ready for start up.
osctrl-db |
osctrl-db | 2020-05-07 01:51:03.893 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
osctrl-db | 2020-05-07 01:51:03.893 UTC [1] LOG: listening on IPv6 address "::", port 5432
osctrl-db | 2020-05-07 01:51:04.002 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
osctrl-db | 2020-05-07 01:51:04.330 UTC [45] LOG: database system was shut down at 2020-05-07 01:51:03 UTC
osctrl-db | 2020-05-07 01:51:04.360 UTC [46] FATAL: the database system is starting up
osctrl-db | 2020-05-07 01:51:04.456 UTC [1] LOG: database system is ready to accept connections
osctrl-admin | Postgres is unavailable - Waiting...
Please help me with fixing this issue in deployment.
Thank you,
Avinash.
Using osctrl
v0.1.8
, using logging = "graylog"
in the service configuration (tls.json
) for osctrl-tls
, using Graylog's HTTP GELF receiver, messages that are group in JSON dictionaries are not accepted.
1 - Set logging to graylog, logging = "graylog"
in tls.json
2 - Create graylog HTTP GELF input.
3 - Generate logs
GELF seems to expect individual log statements in JSON {GELF_OBJ}
. The osctrl graylog plugin is grouping logs together, and the final document being sent to graylog is [{GELF_OBJ},{GELF_OBJ},{GELF_OBJ},...]
, which evidently is not cool. This was validated via test GELF messages and tcpdump to view the raw texts to graylog - it dumps them silently otherwise.
To resolve this, I removed the grouping components of the plugin, and send a request for each log.
I haven't used Graylog much, so maybe I've missed something, but the message won't even make it to an index - it gets dropped.
An unprivileged user can see all environments in osctrl
. When trying to browse to one of them an error is thrown because of the lack of permissions but still the environments names are leaked.
When a new node is enrolled, tags are applied based on environment (the tags are created when an environment is created). Tags should be created based on platform
when a node is enrolled. Perhaps even more granular tags.
When visiting Manage Users
in the admin UI, some fields are not editable, such as email or full name. It would be nice to allow the edition of those fields, same as the settings are edited.
Ability to carve files should be added to osctrl-api
to increase the automation capabilities.
Hi Javier,
I have deployed osctrl using, make docker_all. I have created a new environment (and copied the certificates of the dev environment) using the admin interface. I was attempting to enroll an Ubuntu 18.04.3 LTS using the quick-enroll method. On running the enrollment link using curl, agent (enroll.sh) does not download and install. I have attached the logs generated by each container as well as the curl on the remote machine. I would like to add that the system is on the local network and accessible as the logs show. Please help me with this.
osctrl-enroll-file-download-fail-issue.txt
When adding a non-admin user to osctrl-admin
, and granting access to a specific environment. If the default environment (setting for admin is default_env
) is not the granted one, then when the user logs in, there will be a blank page.
Default environment should fall back to the one that the user has permissions to see.
Pretty easy, just need to switch the base box and change the provider configs a little bit.
# -*- mode: ruby -*-
# vi: set ft=ruby :
VAGRANTFILE_API_VERSION = "2"
IP_ADDRESS = "10.10.10.5"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.box = "hashicorp/bionic64"
config.vm.network "private_network", ip: IP_ADDRESS
# If we want to enroll nodes in the same network
#config.vm.network "forwarded_port", guest: 443, host: 443
config.vm.hostname = "osctrl-dev"
config.ssh.shell = "bash -c 'BASH_ENV=/etc/profile exec bash'"
config.vm.provision "shell" do |s|
s.path = "deploy/provision.sh"
s.args = [
"--nginx", "--postgres", "-E", "--metrics", "--all-hostname",
IP_ADDRESS, "--password", "admin"
]
privileged = false
end
["vmware_desktop", "virtualbox", "hyperv"].each do |provider|
config.vm.provider provider do |v, override|
v.memory = "1024"
v.cpus = 1
end
config.vm.provider "virtualbox" do |v|
v.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
end
config.vm.provider :vmware_desktop do |vmware|
vmware.vmx["ethernet0.pcislotnumber"] = "32"
end
end
end
So, we are looking for better tools then we currently have (Wazuh, works nice.. but its yugh in my opinion) and then i ran into osquery.. Seems nice :) doesnt seem yugh like Wazuh :)
Now i wonder, could osctrl be be used for a small MSSP with multi tenants and alerting? (in some form, or another)
in the end we would also like reporting.. but first thing's first ;-)
Currently osctrl
only support environments and platforms to group nodes. Whereas environments can be created and nodes are enrolled to them, platform comes from the enrolled node and can not be changed. There is a need to group machines so different configuration can be distributed, or query packs, and also on-demand queries / file carves can be run against.
Create tags so enrolled nodes can be associated. One node can have multiple tags.
Insecure server validation when establishing TLS connections may lead to MITM:
https://github.com/jmpsec/osctrl/blob/master/utils/http-utils.go
InsecureSkipVerify
should not be used.
Currently osctrl-tls
updates the metadata for each node when requests are received, before sending logs to the configured logger (splunk, graylog...). This may interfere in the ability of osctrl-tls
to deliver those logs no matter what.
Failures should be logged and metrics emitted but logs must always be delivered.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.