Coder Social home page Coder Social logo

my-links's Introduction

  • 98% OSS
  • 1% free-as-in-beer closed source software
  • 1% in-browser tools
  • Please don't submit to Reddit, HN, or post this on Twitter. Share, but share with close friends!

Table of Contents


Tools by Category

  • X Asset Management
  • X AWS and Azure Tools
  • X Backups
  • X Cloud File Sync and Sharing
  • X Collaboration Tools
  • X Containers
  • X Data Visualization and Dashboards
  • X Distributed Systems Tools
  • X Editors
  • X Git Tools
  • X Graphics Stuff
  • X High Availability Clustering Tools
  • X HPC Tools
  • X Infrastructure as Code Tools
  • X Kubernetes
  • X Live CD Tools
  • X Logging
  • X Metrics and Time Series Data
  • X Monitoring and Alerting
  • X Networking Tools
  • X Network Performance Analysis Tools
  • X Orchestration
  • X Package Patch and Repository Tools
  • X Performance Analysis Tools
  • X Provisioning Tools
  • X Python Tools and Resources
  • X Python Programming Tutorials
  • X RDBMS and SQL Tools
  • X RDBMS Performance Analysis Tools
  • X Regular Expressions
  • X Secrets Management
  • X Security Tools
  • X Shell Scripting and Tools
  • X Software Development Tools
  • X SSH Tools
  • X SSL Tools
  • X Storage Tools
  • X Storage Performance Analysis Tools
  • X Terminal Tools and SSH Clients
  • X Tracing and Profiling
  • X Two Factor Authentication
  • X Virtualization and SDN
  • X VMware Tools
  • X VPNs and Tunnels
  • X Web and HTTP Tools
  • X Web and HTTP Performance Analysis Tools
  • X Misc Tools of Note
  • X Learning Resources

News

A mix of industry and technical, traditional and next generation, dev and ops


Asset Management

Asset management, inventories, asset discovery, CMDB, and IPAM - see also Infrastructure as Code Tools

  • Collins - Infrastructure source of truth, created by Tumblr -- Scala
  • iTop - IT Service Management (ITSM), asset tracking, and ITIL -- PHP
  • Fusion Inventory - Multi-lingual, can be paired with GLPI for a killer solution -- perl
  • Genesis - Hardware discovery, by Tumblr, can report to Collins -- Ruby
  • GestioIP - IP address management (IPAM), web based, supports discovery -- perl
  • GLPI - Also provides license management, software auditing, and ticketing -- PHP
  • ITDB (IT ITems DataBase) - Includes purchase order management, floor plans, and ISO20000-like features -- PHP
  • NetBox - IPAM and DCIM by Digital Ocean -- Python
  • Netdisco - Web-based network management and discovery tool, written in perl, uses SNMP -- perl
  • NIPAP - Next-generation IPAM -- Python
  • OCS-NG (OCS Inventory NG) - Automated inventory, deployment system, can sync with GLPI -- perl
  • openDCIM - Data center infrastructure management -- PHP
  • Open-AudIT - Track and report assets and configurations, supports Windows too -- PHP
  • phpIPAM - IPAM -- PHP
  • pynetbox - Python API client library for NetBox -- python pip
  • RackTables - Data center asset management, being updated again! -- PHP
  • racktables-contribs - RackTables user-contributed plugins -- PHP
  • Ralph - DCIM and CMDB, supports auto-discovery -- Python pip
  • Snipe-IT - Uses Bootstrap, web based, supports mobile -- PHP

AWS and Azure Tools

And some GCP / Google Cloud

  • Action Hero - Uses an AWS SDK feature known as Client Side Monitoring to help you create least privilege IAM Policies for AWS -- Golang
  • asecurecloud - A free library of 400+ customizable AWS security configurations and best practices (CF, Terraform, and AWS CLI) -- collection
  • Audit Email Deletion in Office 365 - Exports deleted emails audit records to CSV, requires mailbox auditing to be enabled -- PowerShell
  • AutoSpotting - A tool implementing an automated bidding algorithm against the Amazon AWS EC2 spot market -- Golang
  • awacs (Amazon Web Access Control Subsystem) - Allows for easier creation of AWS Access Policy Language JSON by writing Python code to describe the AWS policies -- Python pip
  • aws-runas - A friendly way to do AWS STS AssumeRole operations so you can perform AWS API actions using a particular set of permissions -- Golang
  • aws-gate - Connect to instances by other means (e.g. DNS, IP, tag, instance name, autoscaling group) -- Python pip
  • aws-shell (formerly Supercharged AWS CLI (SAWS)) - The best CLI for interacting with AWS -- Python pip
  • aws-ssm-tree - Provides a tree visualization of the parameters hierarchy from AWS System Manager Parameter Store -- Python pip
  • aws-sso-tool - CLI to Smooth out the rough edges of AWS SSO -- Python pip
  • aws-vault - A vault for securely storing and accessing AWS credentials in development environments -- Golang
  • apilogs - Easy logging and debugging for Amazon API Gateway and AWS Lambda Serverless APIs -- Python pip
  • A giant list of Google Cloud resources - From Google -- collection
  • AWSConsoleRecorder - Records actions made in the AWS Management Console and outputs the equivalent CLI/SDK commands and CloudFormation/Terraform templates -- Chrome JavaScript
  • awslogs - AWS CloudWatch logs for Humans, for easier parsing and readability and searching -- Python pip
  • awspec - rspec for AWS resources -- Ruby gem
  • AWSSupport-SetupIPMonitoringFromVPC - SSM Automation document that launches a Monitor Instance in the specified subnet. The Monitor Instance pushes subnet network telemetry data to CloudWatch Logs -- article
  • AWS Amplify - Front end JS suite that provides a templated foundation for cloud-centric apps including authn, analytics, API, push notifications, Graph QL, and more -- JavaScript
  • AWS CloudFormation Guard - Provides developers with a simple-to-use, yet powerful and expressive domain-specific language (DSL) to define policies and enables developers to validate JSON or YAML formatted structured data with those policies -- Rust
  • AWS Controllers for Kubernetes (ACK) - Lets you define and use AWS service resources directly from Kubernetes -- Golang
  • AWS Config Rules Repository - Community driven -- collection
  • AWS Connect Manager - Open-source Windows GUI tool to simplify connecting to Amazon AWS EC2 instances -- various lang
  • AWS Copilot - OSS CLI to build, release, and operate apps for ECS and Fargate -- Golang
  • AWS Encryption CLI - CLI for KMS -- Python
  • AWS Extend Switch Roles - Extend your AWS IAM switching roles by Chrome extension or Firefox add-on -- JavaScript
  • AWS IAM Authenticator for Kubernetes - A tool for using AWS IAM credentials to authenticate to a Kubernetes cluster -- Golang
  • AWS Lambda Power Tuning - A state machine powered by AWS Step Functions that helps you optimize your Lambda functions for cost and/or performance in a data-driven way -- JavaScript
  • AWS Official Terraform Modules - A mix of official and community supported, the official'ist repo -- collection
  • AWS Perspective - Official tool for discovering and visualizing your AWS infra, turn off the Neptune DB when not in use in order to save money -- in-browser
  • AWS Quick Start - Automated gold-standard deployments on AWS, by AWS -- various lang
  • AWS SAM Local - CLI tool for local development and testing of Lambda applications -- Golang
  • AWS Samples - Over 2k code samples for all AWS services -- collection
  • AWS Secrets Manager and Configuration Provider (ASCP) - Plugin for the industry-standard Kubernetes Secrets Store Container Storage Interface (CSI) Driver used for providing secrets to applications operating on EKS -- Golang
  • AWS Secure Environment Accelerator - Designed to help deploy and operate secure multi-account, multi-region AWS environments on an ongoing basis -- various lang
  • AWS Toolbox - AWS scripts, tools, and snippits -- collection
  • AWS Toolkit for Visual Studio Code - Give it a go -- various lang
  • AWS Tools for PowerShell - Use an automation language instead of a programming langue -- various lang
  • Azucar - Multi-threaded plugin-based tool to help you assess the security of your Azure Cloud environment -- PowerShell
  • AzCopy - CLI to copy data to and from containers and file shares in Azure Storage accounts -- Golang
  • AzGovViz - Azure Governance Visualizer PowerShell script that captures Azure Governance related information -- PowerShell
  • AzureHunter - Cloud forensics Powershell module to run threat hunting playbooks on data from Azure and O365 -- PowerShell
  • Azure Arc - Multi-cloud + on-prem control plane to centralize hybrid management, magical -- cloud
  • Azure Cloud Guardrails - Rapidly cherry-pick cloud security guardrails by generating Terraform files that create Azure Policy Initiatives, by Salesforce -- Python pip
  • Azure Functions Core Tools - Develop and test your functions on your local computer from the command prompt or terminal -- various lang
  • Azure Governance Visualizer - Granular insights on your technical Azure Governance implementation -- PowerShell
  • Azure Kubernetes Service (AKS) Baseline Cluster - Reference implementation demonstrates the recommended starting (baseline) infrastructure architecture for a general purpose AKS cluster, by Microsoft -- various lang
  • Azure Official Terraform Modules - By Microsoft -- collection
  • Azure Policy Samples - Samples of Azure Policies that can be used as reference for creating and assigning policies -- collection
  • Azure Portal Desktop App - Faster than a browser, full Cloud Shell, fast search -- various lang
  • Azure Quickstart Templates - 1000+ ARM templates, community-driven -- collection
  • Azure Resource Manager (ARM) Tools for Visual Studio Code - Language support, resource snippets, and resource auto-completion -- various lang
  • Azure Resource Manager (ARM) Viewer for Visual Studio Code - Graphical preview of ARM templates -- various lang
  • Azure Sentinel ATT&CK - Tools to rapidly deploy a threat hunting capability on Azure Sentinel that leverages Sysmon and MITRE ATT&CK --various lang
  • Azure Sentinel KQL - Examples, breakdowns, and code snippits -- collection
  • Azure Service Operator - Helps you provision Azure resources and connect your applications to them from within Kubernetes and kubectl -- Golang
  • Azure StigRepo - Automate and customize configurations that are compliant with Security Technical Implementation Guides (STIGs) owned and released by the Defense Information Systems Agency (DISA) -- PowerShell
  • Azure Tenant Security (AzTS) - Used to obtain visibility to cloud subscriptions and resource configuration across multiple subscriptions in an enterprise environment, official tool by Microsoft -- various lang
  • Azure VM Comparison - Find and compare Azure Virtual Machines specs and pricing on one page across different tiers, payment types, and regions -- in-browser
  • BloodHound - Easily identify highly complex attack paths that would otherwise be impossible to quickly identify. Defenders can use BloodHound to identify and eliminate those same attack paths -- various lang
  • boto3 - The AWS SDK for Python 3 -- Python
  • boto3_type_annotations - Adds code completion in IDEs such as PyCharm -- Python
  • botostubs - boto3 code assistance for any API in any IDE, always up to date -- Python
  • Brainboard - Visually build & manage cloud infrastructures, outputs Terraform code from the diagram, paid, worth it -- SaaS
  • CDK-dia - Automated diagrams for AWS CDK infrastructure -- various lang
  • Chalice - Microframework for writing and testing serverless apps in Python -- Python
  • Chamber - Parameter Store + IAM for secrets including at rest protection, audit trail, and access control policies, by Segment -- Golang
  • Chrome AWS SAML Token Expiry Reminder (CASTER) - Automatically re-logs into AWS via ADFS before credentials expire -- Chrome
  • CloudBerry Explorer - Windows client for accessing AWS S3 buckets -- closed source Windows
  • CloudCraft - Create professional AWS architecture diagrams -- in-browser
  • CloudFormation Checklist - A list of all elements you need to have / to test before launching your infra to production -- collection
  • CloudFormation Designer - GUI for creating CloudFormation templates, very slick -- in-browser
  • CloudMapper - Generates network diagrams of Amazon Web Services (AWS) environments and displays them via your browser, by Duo Security -- Python
  • Cloudockit - Automatically generates diagrams and technical documentation, visualize and document your cloud and on-premises environments, in just a few clicks -- not free but very good
  • CloudSploit - Scans for risks, multi-cloud -- various lang
  • CloudSploit Scans - Scan AWS accounts for security risks -- JavaScript
  • CloudTracker - Find over-privileged IAM users and roles by comparing CloudTrail logs with current IAM policies. -- Python
  • Cloud Custodian - Rules engine for managing public cloud accounts and resources via policies, by Capital One -- various lang
  • Cloud Security Suite (cs-suite) - One stop tool for auditing the security posture of AWS/GCP/Azure infrastructure -- Python pip
  • Cloud Spanner Emulator - For Google Cloud / GCP, official -- various lang
  • ClusterCloner - Reads the Kubernetes clusters in one location (optionally filtering by labels) and clones them into another (or just outputs JSON as a dry run), to/from AWS, GCP, and Azure -- Golang
  • Complete AWS IAM Reference - Unofficial but concise -- collection
  • credstash - Secrets management using AWS KMS -- Python pip
  • CrowdStrike Reporting Tool for Azure (CRT) - Quickly and easily review excessive permissions in their Azure AD environments to help determine configuration weaknesses and provide advice to mitigate this risk -- PowerShell
  • curvy-auditd - Repository of Azure Sentinel detections/parser for Linux auditd -- collection
  • Diagrams - Diagrams lets you draw the cloud system architecture in Python code, it was born for prototyping a new system architecture without any design tools, diagram as code -- Python
  • Disposible Cloud Environment (DCE) - Temporary, limited Amazon Web Services (AWS) accounts. Accounts can be “leased” for a period of time or up to a pre-determined budget amount. When the period of time is reached or the maximum budgeted amount is exceeded, the lease is expired -- Golang
  • eb_deploy - Elastic Beanstalk blue-green deployment automation -- Ruby gem
  • EB CLI - CLI for Elastic Beanstalk -- Python
  • ec2-price-check - Gives a quick price check for an instance type -- shell
  • ec2instances.info - Open source and up to date instance price comparison tool -- Python
  • ec2.shop - ec2 price checker, supports curl -- various lang
  • eksctl - CLI tool for creating and managing clusters on AWS EKS -- Golang
  • EKS Boilerplate - IaC boilerplate in mostly Terraform -- various lang
  • EKS Distro - Use me for multi-cloud EKS -- various lang
  • Falco - CNCF incubated, multi-cloud security rule language to monitor for suspicious activity in clouds and k8s -- neat
  • Fargate - CLI for AWS Fargate, unofficial -- Golang
  • Force MFA - Allows users to manage their own passwords and MFA devices but nothing else unless they authenticate with MFA, also makes API calls require MFA -- policy
  • Former2 - Generate CloudFormation / Terraform / Troposphere templates from your existing AWS resources via self-hosted web GUI -- JavaScript
  • GCPinstances - Easy Google Compute instance comparison -- in-browser
  • gcpviz - Visualization tool that takes input from Cloud Asset Inventory -- Golang
  • GKE Autopilot - Think an opinionated version of GKE, somewhat similar to AWS Fargate -- neat
  • Google Cloud Official Terraform Modules - Included as part of the Cloud Foundation Toolkit -- various lang
  • Google Cloud Samples - Searchable by language and product -- collection
  • haproxy-autoscale - Wrapper for haproxy that handles auto-scaling EC2 instances -- Python
  • Hawk - Powershell Based tool for gathering information related to O365 intrusions and potential breaches -- PowerShell
  • iamlive - Generate a basic IAM policy from AWS client-side monitoring (CSM) -- Golang
  • IAM Policy Simulator - Built-in tool where you can test and troubleshoot identity-based policies, IAM permissions boundaries, Organizations service control policies (SCPs), and resource-based policies -- in-browser
  • kappa - CLI tool that makes it easier to deploy, update, and test Lambda functions -- Python
  • kinesis-scaling-tools - Tools to make Kinesis shards scale like ASGs -- Java
  • Lambda Powertools - Suite of utilities for AWS Lambda functions to ease adopting best practices such as tracing, structured logging, custom metrics, and more, by Amazon -- Python pip
  • Little Bigtable - Emulator for Google Bigtable w/ persistence in sqlite3 -- Golang
  • localstack - A fully functional local AWS cloud stack for offline dev and test -- Python
  • MicroBurst - A PowerShell toolkit for attacking Azure -- Powershell
  • Microsoft Security Tools by Zolderio - Tools to secure O365 and Azure, by Zolder -- various lang
  • Mitre ATT&CK Mappings for Amazon GuardDuty - For AWS -- various lang
  • Moto - Library that allows your Python tests to easily mock out the boto library -- Python pip
  • My Arsenal of AWS Security Tools - List of open source tools for AWS security: defensive, offensive, auditing, DFIR, etc -- collection
  • onelogin-python-aws-assume-role - Assume an AWS Role and get temporary credentials using Onelogin -- Python pip
  • ParallelCluster - AWS supported Open Source cluster management tool to deploy and manage HPC clusters in the AWS cloud -- Python
  • PlantUML - Formal diagramming software, supports C4, Mindmap, Archimate, and more -- Java
  • Prowler - For AWS account security assessment and hardening, based on aws-cli commands -- shell
  • refunc - Run AWS Lambda on Kubernetes, a Lambda-compatable API -- Golang
  • s3cmd - Backup to AWS via the command line -- Python
  • s4cmd - s3cmd with additional features -- Python
  • Scout Suite - Multi-cloud security-auditing tool, which enables security posture assessment of cloud environments. Using the APIs exposed by cloud providers, Scout Suite gathers configuration data for manual inspection and highlights risk areas -- various lang
  • Security Stack Mappings - Maps Azure and AWS product security controls to MITRE ATT&CK -- collection
  • Sentinel ATT&CK - Leverages Sysmon and MITRE ATT&CK on Azure Sentinel -- various lang
  • Serverless (formerly JAWS) - Javascript + AWS stack, the entire backend is Lambda functions, server-free -- JavaScript
  • Serverless by Design - In-browser or self-hosted GUI for making flow charts for serverless apps -- various lang
  • Serverless Stack Toolkit (SST) - extension of AWS CDK that includes a live Lambda dev environment and more -- JavaScript
  • Service Bus Explorer - Interact with Azure Service Bus for testing, debugging, etc -- Windows
  • Session Manager Plugin for AWS CLI - Used to start and end sessions that connect you to your managed instances -- Python
  • Sneaker - Store secrets on S3 using Amazon KMS -- Golang
  • ssh2ec2 - SSH into EC2 instances by tag name and/or other metadata filters -- Python pip
  • SSM Helpers - Interactive shell with an instance via AWS Systems Manager Session Manager and more -- Golang
  • StackMaster - Provides a ton of info pre-CloudFormation run so that you know exactly what will change -- Ruby gem
  • StarCluster - Toolkit for using AWS for high performance computing (HPC), by MIT -- Python
  • Steampipe - Discover, inventory, and query cloud resources using a SQL syntax -- Golang
  • Steampipe Mod - AWS Compliance - Run individual configuration, compliance and security controls or full compliance benchmarks for AWS Foundational Security Best Practices, CIS, GDPR, HIPAA, NIST 800-53, NIST CSF, PCI DSS, RBI Cyber Security Framework across all your AWS accounts -- Golang
  • Steampipe Mod - Azure Compliance - Run individual configuration, compliance and security controls or full CIS, HIPAA HITRUST and NIST compliance benchmarks across all your Azure subscriptions -- Golang
  • Steampipe Mod - Kubernetes Compliance - Run individual controls or full compliance benchmarks for NSA and CISA Kubernetes Hardening Guidance across all of your Kubernetes clusters -- Golang
  • Steampipe Mod - GCP Compliance Mod - Run individual configuration, compliance and security controls or full compliance benchmarks for CIS, Forseti Security and CFT Scorecard for all your GCP projects -- Golang
  • StormSpotter - Creates an “attack graph” of the resources in an Azure subscription. It enables red teams and pentesters to visualize the attack surface and pivot opportunities within a tenant -- Python
  • Stout - Easy way to reliably upload a static website to S3, capable of configuring CloudFront and Route 53 -- Golang
  • Superwerker - Open-source solution that lets you quickly set up an AWS Cloud environment following best practices for security and efficiency, built by AWS Advanced Partners -- various lang
  • taskcat - Tool that tests AWS CloudFormation templates. It deploys your AWS CloudFormation template in multiple AWS Regions and generates a report with a pass/fail grade for each region, by AWS -- Python
  • TerraCognita - Imports your current cloud infrastructure to Terraform configuration (HCL) or/and to a Terraform State -- various lang
  • Terraformer - Generate Terraform files from existing infrastructure (reverse Terraform), supports AWS and GCP, by Google -- Golang
  • Terraforming - Export all AWS resources into Terraform -- Ruby gem
  • Terragrunt Reference Architecture (AWS) - Nice, looking forward to their GCP one -- various lang
  • Tools for Amazon Web Services - Amazon's portal for their official tools -- various lang
  • troposphere - Python library to create CloudFormation descriptions -- Python pip
  • Zappa - Build and deploy serverless, event-driven Python applications -- Python

Backups

Traditional backup software - see also Cloud File Sync and Sharing

  • Amanda - The classic -- C perl
  • Backupninja - Centralize way to configure and schedule many different backup utilities -- shell
  • BURP (BackUp and Restore Program) - Reduces network traffic and the amount of space required by using librsync -- C
  • BackupPC - Dedupe and a web GUI for restores -- perl
  • Backup - Gem for backup and restore, supports tons of platforms and notifiers (RDBMS, S3, Dropbox, rsync, Hipchat, Zabbix) -- Ruby gem
  • Bacula - Open source backup tool, lots of downloads so I guess it's good I don't know -- C
  • Back in Time - Similar to TimeMachine, simple GUI backup program -- Python
  • Bareos (Backup Archiving REcovery Open Sourced) - Fork of Bacula with additional features -- C
  • borgmatic - A simple wrapper script for the Borg backup software that creates and prunes backups-- Python
  • bup - Uses the git packfile format, supports global dedupe, can use "par2" redundancy -- Python
  • DAR (Disk ARchive) - Better than tar, focuses on disks instead of tapes -- C++
  • ddrecover - This should be the first data recovery tool you use -- C
  • Deja Dup - GUI for duplicity -- C
  • Duplicati - Supports encryption and dedupe -- Mono
  • Duplicity - Encrypted bandwidth-efficient backup using the rsync algorithm -- Python
  • Elkarbackup - Comes as a ready to use VM, supports Linux and Windows -- PHP
  • Fpart - Packs file systems into "partitions" so you can do multi-threaded or multi-node rsyncs -- C
  • FSArchiver - Save the contents of a file system to a compressed archive, if one of the checksums doesn't match the file is lost, not the whole backup -- C
  • Grsync - GUI for rsync -- Windows OS X C
  • Mondo Rescue - Disaster recovery, supports tapes, disks, network and CD/DVD as backup media, multiple filesystems, LVM, software and hardware RAID -- C
  • rdiff-backup - Combines the best features of a mirror and an incremental backup in a bandwidth efficient manner -- Windows OS X Python
  • Redo Backup and Recovery - Simple bare metal backup and restore, live CD -- Windows
  • Relax and Recover (REAR) - BMR, simple, integrates with commercial backup solutions -- shell
  • rsnapshot - Uses rsync and hard links, can keep multiple full backups available while using very little disk space -- OS X perl shell
  • SafeKeep - Superb project, uses LVM snapshots -- Python
  • SnapRAID - Backup program that also stores RAID parity information -- C
  • Unison - Multi-OS file sync tool, syncs from both sides, no master -- OS X
  • UrBackup - Supports Windows, has a web interface -- Windows C++

Cloud File Sync and Sharing

Sync files to and share from public or private file stores (think Dropbox, Google Drive, etc) - see also Backups

  • Drive - Push or pull files to Google Drive via the command line -- Golang
  • git-annex Assistant - Sync folder(s) to any device (NAS, mobile, thumb, cloud, etc) via git -- C OS X mobile
  • lsyncd - Watches a local directory trees event monitor interface (inotify or fsevents) and kicks off an rsync when things change -- Lua
  • Mackup - Sync your Mac application settings to various cloud services or git -- Python OS X
  • Nextcloud - Fork of and replacement for OwnCloud, a self-hosted Dropbox -- various lang Windows OS X
  • Pydio - Formerly AjaXplorer, AGPL license -- PHP
  • rclone - Probably the best cloner, supports almost any source and dest -- Golang
  • Seafile - Also offers a paid professional edition with more features, supports most platforms -- C
  • SparkleShare - Uses git under the hood, neat -- Windows OS X
  • Syncany - Supports tons of different protocols for the transfer (SCP, FTP, Samba, S3, etc) -- Gradle
  • Syncthing - Uses an ID rather than an IP address, share your ID with friends and go -- Golang
  • Tahoe-LAFS - Free and open decentralized cloud storage system -- Python

Collaboration Tools

ChatOps, code review, groupware, webmail, code sharing, and more - see also Dashboards and Data Visualization and Graphics Stuff and Editors and Git Tools and Software Development Tools

  • Citadel - Messaging, collaboration tools, and groupware - an all-in-one package -- C
  • Codeshare - In-browser screenshare to teach coding, group code, or as an interview whiteboard -- in-browser
  • dev-setup - Automated setup scripts for laptop tools like Sublime Text, AWS, Spark, Android dev, and more -- collection
  • Etherpad - Enter, save, and share text/code in a web browser -- JavaScript
  • Fosscord - OSS self-hostable Discord compatible chat, voice and video platform -- various lang
  • FreeIPA - Identity, policy, and audit suite, think Active Directory for Linux (LDAP, CA, x509, DNS, Kerberos) -- various lang
  • FreeMind - OSS mind mapping software, great for brainstorming -- Java Windows OS X
  • gcalcli - CLI for Google Calendar -- Python pip
  • gmvault - Export/backup and restore your Gmail account -- Python
  • Got Your Back - Gmail backups over HTTPS -- Python
  • Haste / hastebin - Open source pastebin alternative for sharing code, can be installed locally / on-site -- JavaScript
  • Hubot - Chat bot that can do deploys, look up images, integrate with Google Maps, and tons of other stuff -- CoffeeScript
  • Isso - A commenting server similar to Disqus -- JavaScript
  • JXplorer - Cross platform LDAP browser and editor -- Java
  • Kanboard - Simple Kanban board -- PHP
  • Kolab - Unified communication and collaboration system -- PHP
  • LDAP Account Manager (LAM) - Full featured LDAP management GUI, can manage almost anything -- PHP
  • Mattermost - OSS Slack alternative -- Golang JavaScript
  • Mailtrain - Self hosted news letter e-mail app, similar to Mailchimp -- JavaScript
  • OpenProject - Web-based project management system built on Ruby on Rails -- Ruby
  • osTicket - Routes inquiries created via email, web-forms and phone calls into a web-based customer support platform -- PHP
  • OTRS - Open source help desk software -- perl
  • OwnTracks - Self-hosted location tracking you can share - use for diaries, work orders, etc -- various lang mobile
  • Pandoc - Convert files from one markup format to another, supports a ton of formats -- Haskell
  • Phabricator - Suite of web-based software dev collaboration tools, and all-in-one project management tool -- PHP
  • PrivateBin - Pastebin where the server has zero knowledge of pasted data, data is encrypted/decrypted in the browser using 256 bits AES -- various lang
  • QueryClips - Pastebin for Postgres or my mySQL SQL query sharing -- in-browser
  • RainLoop - Simple, modern & fast web-based email client -- PHP
  • Redmine - Project management webapp -- Ruby
  • Request Tracker - Bug tracking, help desk ticketing, customer service, workflow processes, change management and more -- perl
  • Review Board - Code review tool for multiple SCM systems -- Python
  • Rocket.Chat - OSS Slack clone built with Meteor.js -- JavaScript
  • Roundcube - Browser-based multilingual IMAP client -- PHP
  • Scribus - Open source desktop publishing (layout, typesetting, etc), Adobe InDesign alternative -- C++ Windows OS X
  • SOGo - Groupware that integrates with Microsoft, Android, and Apple products -- Objective-C OS X
  • Sovereign - Set of Ansible playbooks to deploy a suite of self-hosted apps (mail, colab, calendar, file sync, and more) -- Python
  • Synergy - Share a single keyboard and mouse with multiple physical computers, only the old version is free now -- closed source Windows OS X
  • Taiga - Project management web application with agile/ scrum in mind -- Python CoffeeScript
  • TermRecord - Record and playback terminal sessions, outputs self-contained HTML -- Python
  • VisioCafe - The largest collection of free Visio stencils -- collection
  • WeKan - OSS Trello-like kanban board -- JavaScript
  • YOURLS (Your Own URL Shortener) - Lets you run your own URL shortener a'la TinyURL or bit.ly -- PHP
  • Zulip - Group chat with chat threads, by Dropbox -- various lang mobile Windows OS X

Containers

Linux containers, container orchestration, networking, Docker, OpenShift, and related tools - see also Distributed Systems Tools and Virtualization and SDN and Web and HTTP Tools

  • Alpine Linux - Super minimal BusyBox based Linux distro, perfect for hosting containers -- various lang
  • Anchore - A centralized service for inspection, analysis and certification of container images -- Golang
  • appscale - Open source implementation of Google App Engine -- Python
  • Buildah - A low-level interface to core-utils, build container images with the scripting language of your choice without using Dockerfiles, compare to Podman -- Golang
  • cadvisor - Analyzes resource usage and performance characteristics of running containers -- Golang
  • cert-manager - Automate the management and issuance of TLS certificates from various issuing sources -- Golang
  • Cilium - Transparently secure layer 7 services, communicate based on identity groups, load balancing, eBPF-level for performance and instrumentation & more -- Golang
  • Clear Linux - New name for Clear Containers, attempts combine the security advantages of VMs with the deployment advantages of containers -- various lang
  • ClusterCloner - Reads the Kubernetes clusters in one location (optionally filtering by labels) and clones them into another (or just outputs JSON as a dry run), to/from AWS, GCP, and Azure -- Golang
  • CodeReady Containers - Run OpenShift 4.x locally on your laptop, RECOMMENDED -- various lang
  • CodeReady Single Node Cluster (SNC) - Script to create an OpenShift 4.x single node cluster on Linux using KVM -- shell
  • crane - Docker orchestration, similar to Docker Compose -- Golang
  • ctop - ncurses top-like UI for containers -- Golang
  • Dex - A federated OpenID Connect provider -- Golang
  • distroless - "Distroless" images contain only your application and its runtime dependencies and nothing else, by Google -- various lang
  • docker-debug - Attach a new "debug container" to existing namespaces so you don't have to include debug tools in the app containers -- Golang
  • docker-gc - Docker garbage collection of containers and images -- shell
  • DockerSlim - Uses static and dynamic analysis to create skinny image variants of your fat images -- Golang
  • dockerviz - Great tool for analyzing images -- Golang
  • Docker Bench - Checks for dozens of common best-practices around deploying Docker containers in production -- shell
  • Docker CE - Community Edition, use me to replace Docker Desktop, note thew new build procedure -- various lang
  • Docker Compose - Define and run multi-container apps with Docker, previously known as fig, official -- Python
  • Docker Compose Profiles - An easy way to switch between container configurations -- Python
  • Docker Desktop - Notable because it works without admin / root, Mac and Windows -- Golang
  • Docker Distribution - AKA Docker Registry 2.0 - pack, ship, store, and deliver containers -- Golang
  • Docker Hub - Official Docker images for many projects -- various lang
  • Docker Toolbox - Docker Client, Machine, Compose, Kitematic, VirtualBox, and the boot2docker VM in a single package, official -- various lang
  • dockerfile-security - Open Policy Agent (OPA) rules for dockerfiles that can be integrated into your pipeline -- collection
  • Dockit - Jump into a container image of your choosing, taking all the files from the current directory with you -- shell
  • Dokku - Docker powered mini-Heroku (PaaS) in around 100 lines of bash -- shell
  • dumb-init - Minimal init system for containers, by Yelp -- C
  • Fedora CoreOS - The best of CoreOS + Fedora Atomic Host, upstream to RHEL CoreOS, successor to now sunset RHEL / Fedora Atomic & Container Linux -- various lang
  • Flatcar Container Linux - Immutable Linux distribution for containers, the modern choice -- various lang
  • gvisor - User-space kernel, can be used to sandbox containers, by Google -- Golang
  • img - Standalone, daemon-less, unprivileged Dockerfile and OCI compatible container image builder -- Golang
  • Jenkins Docker Slaves Plugin - Aka Dockins, execute a Jenkins job inside one or more containers, supports most job types -- Java
  • jib - Build container images for your Java applications, by Google -- Java
  • Kata Containers - A mix of Clear Containers / Clear Linux and Hyper's runV -- frankencontainers various lang
  • Kitematic - GUI Docker management on Mac & Windows (laptops), official -- JavaScript Windows OS X
  • Kraken P2P Docker registry capable of distributing TBs of data in seconds using a slightly modified BitTorrent protocol -- Golang
  • Lazydocker - Terminal UI for both docker and docker-compose -- Golang
  • LinuxKit - Toolkit for building custom minimal, immutable Linux distributions -- Golang
  • Logspout - Log router for Docker containers -- Golang
  • MicroShift - Repackages OpenShift core components into a single binary that weighs in at a relatively tiny 160MB executable, by Red Hat -- neat
  • Minishift - OpenShift in a VM for testing / learning -- Golang
  • ngnix-proxy - Nginx proxy for Docker containers using docker-gen, recommened -- Python
  • Nomad - Highly scalable application, process, and container orchestraction, multi-cloud support -- Golang
  • OKD - The upstream for Red Hat's OpenShift 3.x, previously OpenShift Origin -- Golang
  • OKD Install - Set of file that installs OKD 3.x (OpenShift's upstream, formerly OpenShift Origin) on a single CentOS VM for testing & dev -- shell
  • OpenShift 4 Bare Metal Install - User Provisioned Infrastructure (UPI) - For testing and learning, requires access to RedHat OpenShift Cluster Manager -- various lang
  • OpenShift Container Platform Install Demo - Install an OpenShift 4.x demo system on Linux, Windows, or Mac, by Red Hat -- various lang
  • OpenShift Virtualization Hands-on Lab - Script that builds out an OpenShift UPI installation on a single baremetal machine where all of the masters and workers are virtualised -- shell
  • Panamax - Containerized app creator with an open-source app marketplace hosted in GitHub -- various lang
  • Permission Manager - Excellent solution for standalone or onprem isolated clusters -- Golang
  • pipework - SDN for Linux Containers -- various lang
  • Popeye - Scans the live cluster for dead or unused resources such as ports mismatches, metrics utilization, probes, container images, RBAC rules, naked resources, etc -- Golang
  • Portainer - Web interface for Docker aka us-for-docker -- JavaScript
  • pulumi - HOT create and deploy cloud programs that use containers, serverless functions, hosted services, and infrastructure, on any cloud, supports most languages -- various lang
  • Pulumi Cloud Framework - Multi-cloud support via a single API -- various lang
  • Rancher - Provides a complete platform for operating Docker in production -- various lang
  • Rancher Desktop - Replacement for Docker Desktop -- various lang
  • Registrator - Service registry bridge for Docker, supports Consul, etcd -- Golang
  • swarm-viz - Docker Swarm visualizer -- JavaScript
  • Visual Studio Code Remote - Containers - Lets you use a Docker container as a full-featured development environment. It allows you to open any folder inside (or mounted into) a container and take advantage of Visual Studio Code's full feature set -- neat
  • Watchtower - Monitors your running Docker containers and restart them when a new image is available -- Golang
  • Weave - Virtual network that connects Docker containers deployed across multiple hosts -- Golang

Dashboards and Data Visualization

Dashboards for monitoring, alerting, metrics, data visualization tools, and status boards - see also Metrics and Time Series Data and Logging and Monitoring and Alerting

  • Bigdesk - Live charts and statistics for Elasticsearch cluster -- JavaScript
  • Cachet - Create beautiful, responsive status pages -- PHP
  • Cacti - Web-based network monitoring and graphing tool designed as a front-end to RRDtool -- PHP
  • Dashkiosk - An excellent, simple dashboard that supports multiple screens -- JavaScript
  • Facette - Time series data visualization and graphing software -- Golang
  • Flame Graphs - Stack trace visualizer by Brendan Gregg -- perl
  • Gource - Software version control visualization tool -- C++
  • Graphene - Graphite dashboard in D3 and Backbone -- JavaScript
  • Hygieia - Visualize near real-time status of the entire delivery pipeline, by Capital One -- Java
  • ksar - Creates pretty graphs from sar output -- Java
  • logstalgia - Web site access log visualization tool, aka Apache Pong -- C++
  • Loki - Like Prometheus but for logs -- Golang
  • lsofgraph - lsof output into to Graphviz -- Lua
  • Grafana - Modern dashboard for Graphite -- JavaScript
  • grafana-statusmap - Grafana status panel -- JavaScript
  • Mozaik - Create beautiful dashboards using Node/React/D3 -- JavaScript
  • MRTG (Multi Router Traffic Grapher) - Still being updated -- perl
  • Nagdash - Dashboard / NOC screen for Nagios -- PHP
  • NagVis - Visualization suite for Nagios -- PHP
  • Nagiosgraph - Another visualization tool for Nagios data -- perl
  • Network Weathermap - Network visualization tool, create a "weather map" just like big ISPs use, not dead yet -- PHP
  • OpenSearch Dashboards - Derived from Kibana 7.10.2, for use with OpenSearch, by Amazon -- JavaScript
  • pdash - web dashboard for linux using data mainly served by psutil -- Python pip
  • PNP4Nagios - Analyzes performance data provided by plugins and stores them automatically into RRD-databases -- PHP
  • promviz - Visualize the traffic of your clusters in realtime from Prometheus data -- Golang
  • redash - Web application that allows to easily query an existing database, share the dataset and visualize it in different ways -- various lang
  • Seyren - Alerting dashboard for Graphite -- Java
  • Smashing - Successor to Dashing -- Ruby
  • Staytus - Complete solution for publishing the latest info about issues with your web applications, networks or services -- Ruby
  • Tessera - Graphite dashboard in Python -- Python pip
  • Thruk - Web interface for Nagios, Icinga, Shinken and Naemon, can create SLA reports, has a mobile client -- JavaScript
  • vnstati - Creates PNG images using vnStat data -- built-in

Distributed Systems Tools

DCOSes, microservices, service discovery, schedulers, and related tools for dynamic, warehouse-scale computing - see also Containers and HPC Tools

  • Akkio - Data placement service that determines how and when to move information in order to optimize retrieval speed for people across the globe, using the minimum required number of copies -- various lang
  • Avro - Data serialization system with backwards compatible schemas -- Java
  • Celery - Async task/job queue based on distributed message passing -- Python
  • Chaperone - End-to-end Kafka auditing (data loss, latency, message duplication, etc), by Uber -- Java
  • confd - Manage local application configuration files using templates and data from etcd or consul -- Golang
  • Crossplane - Multicloud control plane, manage 3rd party services as if they were K8s resources -- Golang
  • Cruise Control - Fully automate the dynamic workload rebalance and self-healing of a Kafka cluster, by Linkedin -- Java
  • Cruise Control UI - Also by Linkedin -- JavaScript
  • consul - Service discovery and configuration via DNS or HTTP, great for auto-scaling -- Golang
  • consul Tools - Official, includes consul-template and others -- various lang
  • Dapr - Serverless and event-driven runtime that makes it easy for developers to build resilient, stateless and stateful microservices that run on the cloud and edge in any language, think of it as as microservices framework -- various lang
  • DoctorKafka - Kafka cluster auto healing and workload auto-balancing -- Java
  • etcd - Distributed, consistent key-value store for shared configuration and service discovery -- Golang
  • fabio - Zero-conf load balancing HTTP(S) router for deploying microservices managed by Consul, by eBay -- Golang
  • Flink - Next-generation true stream processing platform for real-time analytics -- Java
  • glb-director - Stateless Layer 4 load balancer servers capable of line rate packet processing in bare metal datacenter environments -- Golang
  • GraphQL - Alternative to REST, allows clients to define the structure of the data, subscribing to data flows, and more -- various lang
  • groupcache - A replacement for memcached by the same guy -- Golang
  • Hystrix - Circuit breaker library to stop cascading failures, by Netflix -- Java
  • Ignite - General-purpose in-memory platform for in-memory computing use cases -- Java
  • jespen - A framework for distributed systems verification, with fault injection -- Clojure
  • JVM Profiler - Distributed profiler to collect JVM performance and resource usage metrics and serve them for further analysis, by Uber -- Java
  • kafdrop - Web UI for viewing Kafka topics and browsing consumer groups -- Java
  • Kafka - Stream processing platform (logs, IoT metrics, anything) -- Java
  • kafkacat - Generic CLI producer and consumer -- C
  • katran - C++ library and eBPF program to build high-performance layer 4 load balancing forwarding plane, uses XDP from the kernel to provide an in-kernel facility for fast packet's processing -- C++
  • keto - OSS implementation of Zanzibar: Google's Consistent, Global Authorization System -- Golang
  • Kong - Microservice abstraction layer (aka API Gateway or Service Mesh), great for creating API endpoints -- Lua
  • LogDevice - A distributed data store for logs, by Facebook -- C++
  • Mantl - Complete microservices infrastructure built using OSS tools by Cisco -- various lang yowza
  • MaxScale - General purpose DB query proxy, router, and load balancer, by MariaDB -- C
  • mcrouter - memcached protocol router for scaling memcached, by Facebook -- C++
  • Mitogen - Python library for writing distributed self-replicating programs like magic -- Python
  • mrjob - Lets you write MapReduce and Spark jobs in Python 2.7/3.4+ and run them on several platforms (AWS, GCP) -- Python pip
  • NATS - Pub / sub -- Golang
  • ngx_http_auth_request_module - Client authorization based on the result of a subrequest, great for microservices -- C
  • nsq - Realtime distributed messaging platform / message queue -- Golang
  • OpenStack - Private cloud -- Python
  • Pinpoint - Application Performance Monitoring (APM) for distributed systems, based on Dapper -- Java
  • Plumber - Read and write messages to Kafka, RabbitMQ, Google Cloud PubSub, and more -- Golang
  • redis-cell - Redis module that provides rate limiting in Redis as a single command using GCRA -- C
  • Redisson - Distributed and scalable Java data structures on top of Redis -- Java
  • Redpanda - Kafka compatible event streaming platform no Zookeeper, no JVM, and no code changes required -- C++
  • Riemann - Aggregates events from your servers and applications with a powerful stream processing language, for distributed systems, similar to Borgmon -- Clojure
  • Serf - Decentralized solution for service discovery and orchestration -- Golang
  • Spark - Near real-time analytics processing platform, succeeded by Flink (real streaming vs Spark's microbatches) -- various lang
  • Spring Cloud Config - Allows Java Spring to read config info from service discovery or similar source -- Java
  • twemproxy - Proxy for memcached and redis to reduce connections and allow for sharding -- C
  • uReplicator - Improved Kafka MirrorMaker by Uber -- Java
  • Zookeeper - Distributed configuration service, synchronization service, and naming registry -- Java

Editors

IDEs, text & source editors, vim plugins, and similar tools - see also Graphics Stuff and Collaboration Tools

  • 010 Editor - Professional hex editor that supports binary templates for easy reading, scripting, and more -- closed source
  • activate-power-mode - Activate POWER MODE and write code in style, an Atom plugin -- CoffeeScript
  • Atom - Superb text editor, created by GitHub -- CoffeeScript
  • Atom Vim Mode - vi/vim style controls for Atom -- CoffeeScript
  • Atom Linters - A collection of lint tools for the Atom editor -- various lang
  • Brackets - Modern editor that understands and focuses on web design, by Adobe -- JavaScript
  • LargeFile - vim plugin that automatically disables certain things so you can edit large (multi-gig) files faster -- vim
  • Light Table - Next-generation editor that gives you instant feedback -- Clojure
  • MacDown - Markdown editor and live preview for Mac -- Objective-C OS X
  • MacVim - Has far more features than the vim that's included with the OS -- C
  • Nuclide - Collection of packages for Atom to provide IDE-like functionality for a variety of programming languages and technologies, by Facebook -- JavaScript
  • Notepad++ - Killer GPL'ed text editor for Windows -- C++
  • percol - Interactive grep (search) tool -- Python
  • Powerline - Provides various statues on the bottom of your session, flexible -- Python
  • Textmate - GUI text editor for OS X -- C++
  • UltiSnips - The ultimate snippet solution for vim -- Python
  • vim-json - A better JSON plugin for vim -- vim
  • vim-snippets - snipMate & UltiSnip snippets -- vim
  • Vimium - Chrome extension that provides vi/vim style shortcuts for navigation and control -- CoffeeScript Chrome
  • Visual Studio Code - Open Source - Open source version of Microsoft's product -- JavaScript
  • Vundle - Popular plug-in manager for vim -- vim
  • wasavi - vim/vi controls in browser text areas -- JavaScript Firefox Chrome
  • wxHexEditor - Free hex editor, disk editor, and big file editor for Linux, Windows and Mac OS X -- C
  • YouCompleteMe - Fuzzy-search code completion engine for vim -- Python

Git Tools

Tools for interacting with git and GitHub - see also Software Development Tools

  • BFG Repo-Cleaner - Scrub large blobs and sensitive data from git history -- Scala
  • blackbox - Safely store secrets in Git, by Stack Exchange -- shell
  • Completion - Shell tab completion for git branch names -- shell
  • Gerrit - Web based code review and repo management for Git -- Java
  • git-fat - Like git-media but without the Ruby dependencies -- Python
  • GitKraken - Probably the best multi-platform git GUI -- closed source Windows OS X Linux
  • GitHub Branch Source Plugin - Jenkins plugin that you should use instead of the GPRBP -- C
  • GitHub Pull Request Builder Plugin (gprbp) - Jenkins plugin that allows certain comment strings to kick off builds or take other actions -- Java
  • GitLab - Kinda like an open source GitHub, has both a community and paid version -- Ruby
  • GitLab CI - CI that integrates with GitLab -- Ruby
  • gitsome - Supercharged CLI with GitHub integration -- Python
  • GitUp - Maybe the best git GUI -- Objective-C
  • gitwatch - Automatically commit changes when specified files or directories are modified -- shell
  • Git Interfaces, Frontends, and Tools - Massive list on the official kernel.org wiki -- various lang
  • Gogs - Painless, self-hosted Git service written in -- Golang
  • hub - Official CLI for GitHub -- Golang
  • myrepos - Flexible tool for managing many repos -- perl
  • pre-commit Framework - A framework for managing and maintaining multi-language pre-commit hooks -- Python
  • Signing - Sign commits and/or tags using GPG keys to verify the identity of the commiter -- built-in

Graphics Stuff

Not everything happens in the browser or on the command line - see also Collaboration Tools and Data Visualization and Dashboards

  • Archimate - Open source cross platform tool for enterprise architects -- Java
  • Avidemux - Simple, all-in-one, GUI video editor and converter -- C++ Windows OS X
  • Blender - 3D graphics software for animated films, visual effects, art, 3D printed models, and more -- C Python Windows OS X
  • blockdiag - Generate simple block/sequence/activity/network diagrams from text files -- Python pip
  • Darktable - Photography workflow application and RAW developer, Adobe Lightroom replacement -- C OS X
  • draw.io - Free online flow chart maker / Visio alternative, can be self-hosted -- in-browser
  • drawio-desktop - draw.io in Electron -- JavaScript
  • GlyphSearch - Search for icons from Font Awesome, Glyphicons, IcoMoon, Ionicons, and Octicons -- collection
  • GIMP (GNU Image Manipulation Program) - Open source Adobe Photoshop replacement -- C
  • Graphviz - Graph visualization and flow chart software -- wacky license
  • Greenshot - The best Windows screenshot tool -- C# Windows
  • Inkscape - Open source vector image editor, Adobe Illustrator replacement -- C C++ Windows Mac
  • LICEcap - Capture an area of your desktop and save it to a GIF -- C Windows OS X
  • mac2imgur - Auto-upload screenshots to Imgur -- Swift OS X
  • maim - The most powerful and flexible Linux desktop screenshot tool -- various lang
  • Media Player Classic - Home Cinema (MPC-HC) - Open source media player for Windows -- C++ Windows
  • OpenShot - Video editing software, 2.0 will support other additional platforms -- Python Windows OS X
  • Origami - Interactive UI design prototyping without programming, by Facebook -- various lang
  • Pencil - Open source GUI prototyping and mockup tool, supports all platforms -- Windows OS X
  • PlantUML - Easily create beautiful UML Diagrams from simple textual description -- Java
  • ScreenToGif - Record a selected area of your screen, edit and save it as a GIF or video -- C++
  • ShareX - One of the best screenshot/screencast capture and sharing tools for -- Windows OS X
  • Skitch - Feature-rich screenshot editing, sharing, and annotation tool Mac/OS X -- closed source
  • VLC Media Player - Media (music, video, etc) player and streaming server -- C Windows OS X
  • yEd Graph Editor - Flowcharts and UML diagrams -- closed source

High Availability Clustering Tools

HA clustering tools including storage replication, failover, VIPs, and more - see also Distributed Systems Tools and Containers

  • Corosync - HA framework and cluster engine -- C
  • csync2 - General purpose cluster file sync tool -- C
  • DRBD (Distributed Replicated Block Device) - Mirror block devices to a remote system aka replication -- C
  • Ganeti - Wrappers around existing tools to make it easy to create HA clusters, by Google -- Python
  • HAproxy - Open source software load balancer -- C
  • haproxyctl - Wrapper to talk to the HAProxy socket, as well as regular init (start stop restart) shit -- Ruby
  • keepalived - Load balancing and high availability -- C
  • huptime - Zero downtime restarts of unmodified programs -- C
  • Linux-HA - Building blocks for high availability systems -- wiki-and-collection
  • LVS (Linux Virtual Server) - Linux-based load balancer, also includes the IPVS kernel module -- C
  • Multibinder - Simple Ruby daemon that makes true zero downtime reloads simple, by Github -- Ruby
  • Pacemaker - HA resource manager -- C
  • rcron - cron redundancy and failover, ensures a job will only run on the "active" machine -- lost-to-the-internet C
  • rmanager - Resource group manager daemon for cluster services -- built-in
  • Seesaw - Load balancer based on Linux Virtual Server (LVS), by Google -- Golang
  • Traefik - Modern HTTP reverse proxy and load balancer, supports many backends -- Golang
  • UCARP - VIP management using the CARP protocol -- C

HPC Tools

High performance computing for simulations, supercomputing, shared memory systems, and grid computing - see also Distributed Systems Tools and Containers


Infrastructure as Code Tools

OS and cloud-level configuration and compliance management - see also Asset Management and Orchestration and Package Patch and Repository Tools

  • Ansible - CM and orchestration, also can do provisioning -- Python
  • ansible-hardening - For STIG compliance -- Python
  • ansible-runner - Provides a stable and consistent interface abstraction to Ansible so you can embed Ansible into other systems such as CI/CD, Jenkins, or other automated tooling -- Python pip
  • Ansible-Terraform Workspace - Dockerized development environment with Ansible, Terraform, and lots of other stuff installed, so that you don't need to do it yourself SO HOT 10 OUT OF 10 -- various lang
  • ansible-trace - Visualise where time is spent in your Ansible playbooks (what tasks and what hosts) -- Python
  • Ansible Galaxy - Community site for finding, reusing, and sharing Ansible content -- various lang
  • ara (ARA Records Ansible) - Provides reporting by saving detailed and granular results of ansible and ansible-playbook commands -- Python pip
  • asecurecloud - A free library of 400+ customizable AWS security configurations and best practices (CF, Terraform, and AWS CLI) -- collection
  • Atlantis - A unified workflow for collaborating on Terraform through GitHub and GitLab, by Hootsuite -- Golang
  • Automatic Server Hardening - Linux hardening cookbooks/manifests/playbooks for Puppet, Chef, and Ansible -- various lang
  • AWS Official Terraform Modules - A mix of official and community supported, the official'ist repo -- collection
  • awx - Upstream to Ansible Tower - REST API, task engine, etc -- Python
  • Azure Official Terraform Modules - By Microsoft -- collection
  • Azure Resource Manager (ARM) Tools for Visual Studio Code - Language support, resource snippets, and resource auto-completion -- various lang
  • Azure Resource Manager (ARM) Viewer for Visual Studio Code - Graphical preview of ARM templates -- various lang
  • Blueprint - Reverse engineer a server configuration -- Python pip
  • Boxen - Mac / OS X configuration management -- Ruby gem OS X
  • Cloud Custodian - Rules engine for managing public cloud accounts and resources via policies, by Capital One -- various lang
  • Former2 - Generate CloudFormation / Terraform / Troposphere templates from your existing AWS resources via self-hosted web GUI -- JavaScript
  • Gaia - Terraform UI for your Terraform modules, and self-service infrastructure -- JavaScript
  • GKE Demo - Demonstration of complete, fully-featured CI/CD and cloud automation for microservices, done with GCP/GKE -- various lang
  • Google Cloud Official Terraform Modules - Included as part of the Cloud Foundation Toolkit -- various lang
  • Infracost - Shows cloud cost estimates for infrastructure-as-code projects such as Terraform -- Golang
  • InSpec - Chef-centric IaC testing framework for compliance, security, and testing with a rspec-like syntax -- Ruby
  • Jenkins Ansible Tower Plugin - Run Ansible Tower jobs as a build step -- Java
  • KICS (Keeping Infrastructure as Code Secure) - Supports many many platforms, by Checkmarx -- Golang
  • kitchen-terraform - Test Kitchen plugins for testing Terraform configurations with InSpec -- Ruby gem
  • KOPS - Simple IaC system for for creating, managing, upgrading, and maintaining Kubernetes EC2-based clusters on AWS -- Golang
  • Molecule - Used for testing Ansible roles locally, supports Docker and/or Vagrant -- Python
  • Oxidized - RANCID replacement, supports many platforms (Cisco, Brocade, Juniper, Citrix, etc) -- Ruby gem
  • python-terraform - Python wrapper for Terraform -- Python pip
  • RANCID - (Really Awesome New Cisco confIg Differ) - Pulls and saves network device configs and saves them into a CVS, now supports git -- C
  • Reclass - External node classifier for most CM systems, allows for a tagging system that's a layer above the CM tool -- Python
  • Salt / Saltstack - Orchestration, server provisioning, and configuration management -- Python
  • Steampipe - Discover, inventory, and query cloud resources using a SQL syntax -- Golang
  • Steampipe Mod - AWS Compliance - Run individual configuration, compliance and security controls or full compliance benchmarks for AWS Foundational Security Best Practices, CIS, GDPR, HIPAA, NIST 800-53, NIST CSF, PCI DSS, RBI Cyber Security Framework across all your AWS accounts -- Golang
  • Steampipe Mod - AWS Compliance - Run individual configuration, compliance and security controls or full compliance benchmarks for AWS Foundational Security Best Practices, CIS, GDPR, HIPAA, NIST 800-53, NIST CSF, PCI DSS, RBI Cyber Security Framework across all your AWS accounts -- Golang
  • Steampipe Mod - Azure Compliance - Run individual configuration, compliance and security controls or full CIS, HIPAA HITRUST and NIST compliance benchmarks across all your Azure subscriptions -- Golang
  • Steampipe Mod - Kubernetes Compliance - Run individual controls or full compliance benchmarks for NSA and CISA Kubernetes Hardening Guidance across all of your Kubernetes clusters -- Golang
  • Steampipe Mod - GCP Compliance Mod - Run individual configuration, compliance and security controls or full compliance benchmarks for CIS, Forseti Security and CFT Scorecard for all your GCP projects -- Golang
  • SweetOps - Similar to Gruntwork’s Terragrunt + subscription plan, the Terraspace framework, and BinBash’s Leverage, by CloudPosse -- opinionated
  • TerraCognita - Imports your current cloud infrastructure to Terraform configuration (HCL) or/and to a Terraform State -- various lang
  • Terraform - 1.0 released! - Infrastructure provisioning using existing tools, supports many providers (AWS, Azure, DO, OpenStack, etc) -- Golang Hashicorp
  • Terraformer - Generate terraform files from existing infrastructure (reverse Terraform), supports AWS and GCP, by Google -- Golang
  • Terraform CDK - Use TypeScript or Python to generate Terraform config files -- Golang
  • Terraform Docs - Utility to generate documentation from Terraform modules in various output formats -- Golang
  • Terraform Landscape - Makes terraform plan easier to read -- Ruby
  • Terraform Modules by Cloud Posse - Well written, well maintained, recommended -- collection
  • Terraform pre-commit git Hooks - Format, validate, lint, and all kinds of good stuff -- various lang
  • Terraform Security Tool Comparison - Which Terraform code scanners provide the most comprehensive coverage - click to find out -- collection
  • terraform-exec - Go module for constructing and running Terraform CLI commands -- Golang
  • Terraformer - Generate Terraform files from existing infrastructure (reverse Terraform), supports AWS and GCP, by Google -- Golang
  • Terraforming - Export all AWS resources into Terraform -- Ruby gem
  • Terragoat - Learning and training project that demonstrates how common configuration errors can find their way into production Terraform cloud environments -- various lang
  • Terragrunt - Tools for keeping your Terraform configurations DRY, working with multiple Terraform modules, and managing remote state -- Golang
  • Terragrunt Atlantis Config - Generate Atlantis Config for Terragrunt projects -- Golang
  • Terragrunt Reference Architecture (AWS) - Nice, looking forward to their GCP one -- various lang
  • Terratest - Makes it easier to write automated tests for your infrastructure code, provides a variety of helper functions and patterns for common infrastructure testing tasks -- Golang
  • tflint - Terraform linter for detecting errors that can not be detected by terraform plan -- Golang
  • tfwriter - Auto-generate Terraform code in a non-opinionated way, also great for seeing which parameters a resource provides -- in-browser

Kubernetes

Kubernetes is ancient Egyptian for "he who has won the platform wars", see Learning Resources for tutorials

  • Nomad - Consdier Nomad as a lightweight alternative to Kubernetes, by Hashicorp -- Golang
  • Ambassador - Kubernetes-native API gateway built on Envoy proxy includes gRPC, auth, TLS, and more -- Python
  • Argo - Container-native workflow engine implemented as a Kubernetes CRD (Custom Resource Definition) -- Golang
  • Azure Service Operator - Helps you provision Azure resources and connect your applications to them from within Kubernetes and kubectl -- Golang
  • AWS Controllers for Kubernetes (ACK) - Lets you define and use AWS service resources directly from Kubernetes -- Golang
  • Banzai Cloud - Kubernetes based, open source, multi-cloud with all the good stuff baked in (Prometheus, CICD pipelines, and more) CHECK THIS -- various lang
  • CDK for Kubernetes (CDK8s) - Framework for defining Kubernetes applications and reusable abstractions using familiar programming languages -- various lang
  • ClusterCloner - Reads the Kubernetes clusters in one location (optionally filtering by labels) and clones them into another (or just outputs JSON as a dry run), to/from AWS, GCP, and Azure -- Golang
  • Contour - Kubernetes ingress controller using Lyft's Envoy proxy -- Golang
  • Crossplane - Multicloud control plane, manage 3rd party services as if they were K8s resources -- Golang
  • DevSpace - Build, test and run code directly inside any Kubernetes cluster, no more waiting for rebuild + redeploy, run code instantly -- Golang
  • draft - Streamlined Kubernetes development with sandbox testing & deployment in seconds -- Golang
  • draino - Automatically drains Kubernetes nodes based on labels and node conditions, can be used for auto-remediation -- Golang
  • drone - CI platform built on Docker / containers, can also deploy to Kubernetes -- Golang
  • Eirini - Kubernetes backend for Cloud Foundry -- Golang
  • Envoy Proxy - Sidecar container for distributed applications or microservices, data plane service mesh / edge proxy -- C++
  • Escalator - Batch or job optimized horizontal autoscaler for Kubernetes -- Golang
  • external-auth-server - Allow various authentication schemes in a Kubernetes enviornment -- various lang
  • Fluent Bit Kubernetes Daemonset - The best logger, super light weight -- Golang
  • Flux - Tool that automatically ensures that the state of a Kubernetes cluster matches the config in git via Kubernetes operators -- Golang
  • Flux v2 - Re-written and redesigned -- Golang
  • Flannel - Network fabric for containers via etcd, designed for Kubernetes -- Golang
  • Gangway - Enable authentication flows via OIDC (OpenID Connect Tokens) for a Kubernetes cluster -- Golang
  • Gatekeeper - Gatekeeper is a validating webhook that enforces CRD-based policies executed by Open Policy Agent -- Golang
  • GCP Config Connector - Kubernetes add-on that allows customers to manage GCP resources, such as Cloud Spanner or Cloud Storage, through your k8s cluster's API -- Golang
  • Gitkube - Build and deploy docker images to Kubernetes using git push -- various lang
  • Gloo - Gateway / abstraction layer between upstream services, based off of Envoy -- Golang
  • Goldilocks - Helps you identify a starting point for resource requests and limits -- Golang
  • gravity - Creates snapshots of a Kubernetes cluster that can be restored elsewhere (on-prem, DR situation, etc) -- Golang
  • Helm - tool for managing Kubernetes charts (packages of pre-configured Kubernetes resources) -- Golang
  • Helm Charts - By Bitnami -- collection
  • Heptio Sonobuoy - Kubernetes end to end conformance testing and debugging tool -- Golang
  • Istio - Envoy + auth, policy enforcement, telemetry, traffic flow management etc control plane that runs on top of Mesos and Kubernetes, service mesh control plane -- various lang
  • Istio Operator - An operator that manages Istio deployments on Kubernetes, by Banzai Cloud -- Golang
  • Jenkins Kubernetes Plugin - Use a Kubernetes cluster to dynamically provision a Jenkins agent (using Kubernetes scheduling mechanisms to optimize the loads), run a single build, then tear-down that agent -- Java
  • Jenkins X - Another Kubernetes deployer -- various lang
  • k3s - Lighweight Kubernetes in a 40mb binary, built for the edge or laptop or Pi, by Rancher -- Golang
  • k8s-at-home - Helm charts for applications you run at home, a good foundation for standarized charts -- collection
  • k8spurger - Delete unused resources in your cluster, default mode is dry run -- Python
  • kaniko - Build container images from a Dockerfile, inside a container or Kubernetes cluster -- Golang
  • kaim - Integrate AWS IAM with Kubernetes, associate IAM roles with pods -- Golang
  • Keda - The best autoscaler for k8s -- Golang
  • Keel - Stateless, automated Kubernetes deployment updates -- Golang
  • Kind - Run local Kubernetes clusters using Docker container “nodes", great for local development -- Golang
  • Knative - Run serverless containers on Kubernetes with ease Knative takes care of the details of networking, autoscaling (even to zero), and revision tracking, great for event-driven architectures -- Golang
  • kubepug - Kubernetes PreUpGrade (Checker) -- Golang
  • kube-applier - service that enables continuous deployment of Kubernetes objects by applying declarative configuration files from a Git repository to a Kubernetes cluster -- Golang
  • kube-bench - Compliance checker for Kubernetes CIS benchmarks -- Golang
  • kube-hunter - Hunt for security weaknesses in Kubernetes clusters -- Python
  • kube-prometheus - Use Prometheus to monitor Kubernetes and applications running on Kubernetes -- Golang
  • kube-secrets-init - Kubernetes mutating webhook for secrets-init injection -- Golang
  • kube-state-metrics - It is not focused on the health of the individual Kubernetes components, but rather on the health of the various objects inside, such as deployments, nodes and pods -- Golang
  • kube2iam - Provide IAM credentials to containers running inside a kubernetes cluster based on annotations -- Golang
  • kube2pulumi - k8s yaml in, language of your choice out -- Python
  • Kubecost - Creates cost future and past models so you can see and predict your spend -- Golang
  • kubectl-debug - Debug your pod via a new container with every troubleshooting tools pre-installed -- Golang
  • kubectx - Easily switch between kubectl contexts and namespaces, also includes the kubens tool -- Ruby
  • kubed - Perform periodic cluster snapshots, provide temp storage for deleted objects, automatic event forwarding, deliver notifications via various channels for Kubernetes -- Golang
  • KubeEdge - CNCF project to run Kubernetes at edge -- Golang
  • KubeLinter - Supports k8s and Helm -- Golang
  • kubeplus - Worth checking out for the "connections" option -- Golang
  • Kubernetes - Open source orchestration system for Docker containers, by Google -- Golang
  • kubernetes-cloudflare-sync - Run in your Kubernetes Cluster on GKE and sync DNS records on Cloudflare with your nodes IPs to avoid GCP LB fees -- Golang
  • kubernetes-deploy - tool that helps you ship changes to a Kubernetes namespace and understand the result, by Shopify -- Ruby
  • kubernetes-event-exporter - Exports missed events, there are tons you don't know about -- Golang
  • kubernetes-external-secrets - CRD to pull secrets from AWS Secrets Manager, AWS System Manager, Hashicorp Vault, Azure Key Vault, and Google Secret Manager -- Golang
  • Kubernetes IN Docker (KinD) - Tool for running local Kubernetes clusters using Docker container "nodes" -- Golang
  • Kubernetes Job/CronJob Notifier - Puts a message into Slack -- Golang
  • Kubernetes Network Policy Recipes - Example recipes for Kubernetes Network Policies that you can just copy paste -- collection
  • kubescape - Test if Kubernetes is deployed securely as defined in Kubernetes Hardening Guidance by the NSA and CISA -- Golang
  • kubewatch - Kubernetes event watcher and handler (currently only publishes to Slack channels, integrations wanted!) -- Golang
  • Kube Forwarder - GUI Kubernetes port forwarding manager -- JavaScript
  • kustomize - The preferred templating tool, now built-in to kubectl, official -- Golang
  • Kyverno - Policy engine for Kubernetes -- Golang
  • Lens - An IDE for Kubernetes -- various lang
  • libvirt-k8s-provisioner - Ansible + Terraform, easy clusters -- various lang
  • linkerd2 - The project formerly known as Conduit has been merged into this, a simpler altnernative to Istio if you only need a service mesh -- Golang
  • Lokomotive - Kubernetes distribution with baked in multi-cloud and Terraform support, by Kinvolk -- various lang
  • MetalKube - Bare metal provisioning for Kubernetes, by Red Hat -- Golang
  • MetalLB - Load balancer for bare metal Kubernetes clusters, by Google -- Golang
  • Microk8s - Alternative to minikube, by Canonical -- shell
  • minikube - Kubernetes environments on your laptop -- Golang
  • missing-container-metrics - Exports container exit code, OOM kill status and number of restarts to Prometheus -- Golang
  • mizu - API traffic viewer for Kubernetes enabling you to view all API communication between microservices. Think TCPDump and Wireshark re-invented for Kubernetes -- Golang
  • MKIT (Managed Kubernetes Inspection Tool) - A quick way to assess several common misconfigurations in their Kubernetes environment (AKS, EKS, GKE) -- Dockerfile
  • Octant - Web based representation of a Kubernetes cluster, by VMware -- various lang
  • Open Cluster Management - The upstream for Red Hat Advanced Cluster Management, OpenShift-centric -- various lang
  • Peirates - Kubernetes penetration tool -- Golang
  • Pixie - Instant visibility by giving access to metrics, events, traces and logs without changing code (DaemonSets + eBPF) -- Golang
  • Prometheus Operator - Creates/configures/manages Prometheus clusters atop Kubernetes -- Golang
  • pv-migrate - Tool for migrating PVCs -- Golang
  • Reloader - Reloader can watch changes in ConfigMap and Secret and do rolling upgrades on Pods -- Golang
  • Rook - Self managing, self healing storage orchestrator for Kubernetes via an operator plugin, see also EdgeFS -- Golang
  • shell-operator - Integration layer between Kubernetes cluster events and shell scripts by treating scripts as hooks triggered by events -- Golang
  • skaffold - Easy and repeatable Kubernetes development, test locally then push to a cluster, by Google -- Golang
  • Sloop - Monitors Kubernetes, recording histories of events and resource state changes and providing visualizations to aid in debugging past events, by Salesforce -- Golang
  • Squash - Debug applications from your terminal or IDE while they run in Kubernetes or OpenShift (locally or remotely) -- Golang
  • SuperGloo - Service mesh management and orchtestration -- Golang
  • Telepresense - Local development against a remote Kubernetes or OpenShift cluster -- Python
  • Teleport - Auditing bastion host & middleman, now supports the Kubernetes apifserver protocol -- Golang
  • Typhoon - Minimal and free Kubernetes distro, great for testing and learning on small systems -- various lang
  • Wormhole - CNI plugin for Kubernetes that uses WireGuard for creating a full mesh encrypted network between each host in the Kubernetes cluster. The Kubernetes API is used to coordinate key exchange and configuration -- Golang
  • Vault Secrets Operator - Kubernetes operator for Hashicorp Vault -- Golang
  • vcluster - Virtual Kubernetes clusters - Each vcluster runs inside a namespace of the underlying k8s cluster. It's cheaper than creating separate full-blown clusters and it offers better multi-tenancy and isolation than regular namespaces -- Golang
  • Velero - Manage disaster recovery for your Kubernetes persistent volumes and cluster resources, formerly Heptio Ark -- Golang
  • version-checker - Check image versions running in the cluster vs the newest and then alert via Prometheus when newer ones are available for evaluation -- Golang
  • virtual-kublet - kublet implementation that masquerades as a kubelet for the purposes of connecting Kubernetes to other APIs (Fargate, ACI, IoT, Nomad, Azure, etc) -- Golang
  • Voyager - HAProxy backed secure L7 and L4 ingress controller for Kubernetes -- Golang

Live CD Tools

Security, recovery, bootable USB/CD/DVD creation tools, diagnostics, and more - see also Security Tools and Backups and Provisioning Tools

  • BlackArch Linux - Penetration testing Linux distro, based off of Arch Linux -- various lang
  • CAINE (Computer Aided INvestigative Environment) - Computer forensics on a live CD -- various lang
  • DBAN (Darik's Boot and Nuke) - Spinning disk wiper -- various lang
  • Easy2Boot - Create multiple bootable Linux ISOs on the same USB drive -- Windows
  • GParted Live - Small, bootable ISO that contains gparted - great for resizing a non-LVM root file system -- C
  • Hiren's Boot CD - Re-adding, updated after 6 long years -- closed source
  • Kali Linux - Penetration testing Linux distro -- various lang
  • Network Security Toolkit (NST) - Live CD that includes most tools in insecure.org's top 125 tools list -- various lang
  • NirLauncher - USB live distro that contains all of the NirSoft utilities and more, essential for Windows -- closed source Windows
  • PALADIN - Easy to use Linux-based live CD for forensic analysis -- various lang
  • Rufus - Create bootable USB flash drives -- Windows
  • Security Onion - Linux distro for IDS, NSM, and log management -- various lang
  • Stresslinux - Hardware burn-in and stress testing -- various lang
  • System Rescue CD - System recovery CD that focuses on Linux system recovery -- various lang
  • Tails - Aims at preserving your privacy and anonymity via Tor -- various lang
  • Ultimate Boot CD (UBCD) - Tons of x86 diagnostic and stress test tools on a single CD -- closed source Windows
  • UNetbootin - Create custom, bootable USB Linux CDs -- Windows OS X
  • YUMI - Multiboot USB creator, Linux and -- Windows

Logging

Log management, analysis, analytics, and collection from any source - see also Data Visualization and Dashboards and Metrics and Time Series Data and Monitoring and Alerting

  • Adiscon LogAnalyzer - Slick web interface for syslog messages -- PHP
  • Countly - Mobile and web analytics and marketing platform -- JavaScript
  • Elastalert - Send alerts based on ElasticSearch logs (http 500 increase/spike, any custom string, etc) -- Python
  • ElasticDump - Import / export tools for Elasticsearch -- JavaScript
  • ElasticHQ - Does not yet support ES 5.x -- JavaScript
  • Elasticsearch Exporter - Script to import/export data from ElasticSearch to various other storage systems -- JavaScript
  • Errbit - Self-hosted error catcher, Airbrake API compliant -- Ruby
  • Filebeat - By Elastic, the next generation Logstash Forwarder -- Golang
  • fluentbit - Fast and lightweight log processor, part of the fluentd family -- C
  • Fluentd - Unified logging layer, often used with Kubernetes / OpenShift / containers -- Ruby gem
  • Flume - Distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data -- Java
  • GoAccess - Real-time web log analyzer and interactive viewer that runs in a terminal and/or dashboard -- Windows OS X
  • Graylog2 - Log capture and analysis -- various lang
  • lnav - Advanced log file viewer for the small-scale, terminal/ncurses based -- C++
  • Log Courier - Enhanced fork of Logstash Forwarder -- Ruby
  • Logagent - Lightweight log shipper, like Filebeat and Logstash in one without the JVM footprint -- JavaScript
  • Logplex - Distributed syslog log router, uses Redis, by Heroku -- Erlang
  • logstash - Collect, parse, and store logs. A component of the popular ELK stack -- Ruby
  • Logster - Utility for reading log files and generating metrics to configurable outputs by Etsy -- Python
  • Logwatch - Monitor logs and send an e-mail when event(s) occur -- perl
  • Mamomo - Web analytics platform with a killer UI, formerly PIWIK -- PHP
  • multilog - Reads a sequence of lines from stdin and appends selected lines to any number of logs -- DJB
  • multitail - Monitor multiple log files in a single terminal window -- C
  • netconsole - Kernel module that sends kernel log messages (dmesg, etc) to a remote system without using syslog -- C
  • NXLOG - Universal log collector and forwarder, supports many formats/platforms/sources including Windows -- C Windows
  • OpenSearch - Derived from Elasticsearch 7.10.2, Apache 2.0 license, by Amazon -- Java
  • Open Distro for ElasticSearch - Distro of ElasticSearch with all of the enterprise-grade features added in -- Java
  • Open Web Analytics (OWA) - Track and analyze how people use your websites and applications -- PHP
  • Promtail - Log shipper for Loki users -- Golang
  • Sentry - Application exception logging -- Python
  • Snoopy Logger - Logs commands that are executed and saves the information to syslog -- C
  • Snowplow - Web, mobile and event analytics -- Scala
  • swatch - Simple log watcher -- built-in

Metrics and Time Series Data

Collection, analysis, and storage of metrics, telemetry, and instrumentation data from almost any source - see also Dashboards and Data Visualization and Monitoring and Alerting and Logging

  • collectd - Collects system performance statistics -- C
  • collectd Related Sites - Great tools that integrate with collectd -- collection
  • collectl - sar on steroids -- C
  • Cortex - Multitenant, horizontally scalable Prometheus as a Service -- Golang
  • Diamond - Python daemon that collects system metrics and publishes them to Graphite (or similar), has an API -- Python
  • dim_STAT - Collects almost everything and stores it in a MySQL database, produces reports too -- C
  • FastForward - Flexible system event and metric forwarding agent by Spotify -- Ruby gem
  • Ganglia - Focused on HPC / distributed clusters, uses RRD -- various lang
  • Graphite - Store numeric time-series data and render graphs of the data on demand -- Python
  • Graphite Tools - Tools that work with Graphite -- collection
  • Heka / hekad - Stream processing, can gather logs or performance metrics, by Mozilla, based on Borgmon -- Golang
  • InfluxDB - Distributed time series database with no external dependencies -- Golang
  • jmxtrans - Connector between speaking to a JVM via JMX and whatever stats / TSDB you use -- Java
  • KairosDB - Time series DB written on top of Cassandra -- Java
  • m3 - Distributed TSDB and query dngine, Prometheus sidecar and metrics platform by Uber -- Golang
  • Metricbeat - fetches a set of metrics on a predefined interval from OS & services and ships them to Elasticsearch or Logstash -- Golang
  • Metrics - Metrics and instrumentation at both the JVM and application level -- Java
  • Micrometer - Provides a simple facade (fake interface) over the instrumentation clients for the most popular monitoring systems, allowing you to instrument your JVM-based application code without vendor lock-in -- Java
  • mtail - Extract monitoring data from application logs for collection into a timeseries database, by Google -- Golang
  • OpenTSDB - Store and serve massive amounts of time series data without losing granularity -- Java
  • Prometheus - Metrics collection and storage, can trigger alerts when thresholds are breached, based on Borgmon -- Golang
  • prometheus-am-executor - HTTP server that receives alerts from the Prometheus Alertmanager and executes a given command with alert details set as environment variables -- Golang
  • Sensu Go - Open source monitoring framework, cloud-focused, dynamic, scalable - also does metrics collection -- Ruby
  • SNMPcollector - SNMP collector that saves into InfluxDB for easy visualization -- Golang JavaScript
  • SNMP MIB Archive - Massive archive of SMMP MIBs, please fork and contribute -- collection
  • StatsD - Network daemon that listens for stats/counters/metrics and sends them to backend services (TSDB, Graphite, etc), by Etsy -- JavaScript
  • Telegraf - Agent for collecting, processing, aggregating, and writing metrics -- Golang
  • Thanos - Highly available Prometheus setup with long term storage capabilities -- Golang
  • TimescaleDB - PostgreSQL extension for time series ingestion and queries via SQL -- C
  • Whisper - Store time series info in regular file system files, a modern RRD -- Python

Monitoring and Alerting

Monitor stuff, send alerts, wake you up - see also Metrics and Time Series Data and Logging and Data Visualization and Dashboards

  • Adagios - Web based Nagios configuration interface -- HTML
  • Alerta - Distributed and de-coupled, requires MongoDB -- Python
  • Bosun - Monitoring and alerting system written by Stack Exchange, based on Borgmon -- Golang
  • Cabot - Self-hosted, easily-deployable monitoring and alerts service - like a lightweight PagerDuty -- Python
  • check_mk - New Open Monitoring Distro, extensions / plugins for Nagios -- C
  • Daemon Tools - Service monitoring and management tools -- DJB
  • FastForward (ffwd) - Flexible system event and metric forwarding agent by Spotify -- Ruby gem
  • health - An easy to use, extensible health check library for Go applications -- Golang
  • Icinga - Nagios fork, updated frequently -- various lang
  • Icinga2 - Complete re-write of Icinga by the same folks -- various lang
  • LibreNMS - GPL fork of Observium -- various lang
  • Monit - Includes tools to automatically take action when certain conditions are met (eg: restart a process when it dies) -- C
  • Monitorix - So lightweight that it can be used in mobile devices, aka Mikaku -- perl
  • Naemon - Modular Nagios fork -- various lang
  • Nagios - One of the most widely used OSS monitoring programs -- various lang
  • Nagios Exchange - Centralized repository of Nagios plugins, addons, extensions, etc -- collection
  • NetXMS - Monitoring for all types of devices across the entire data center (hosts + network devices) -- C
  • OpenNMS - Network monitoring, also supports configuration/asset management -- various lang
  • pmacct - IP and network traffic accounting / monitoring -- C
  • PHP Server Monitor - Simple monitoring package that can use built-in public SMS gateways for notifications -- PHP
  • PRTG - Commercial version of Nagios, AD integration, many plugins, excellent price, highly recommended -- C
  • Pynag - Interface with Nagios via Python -- Python pip
  • Sensu Go - Open source monitoring framework, cloud-focused, dynamic, scalable - also does metrics collection -- Ruby
  • Sensu Plugins - Official community site for Sensu plugins -- various lang
  • Shinken - Nagios-compatible monitoring, supports high availability -- Python
  • SigNoz - OSS alternative to New Relic and DataDog -- Golang
  • Statping - THE BEST simple all in one monitoring solution, with mobile app, great for home/small networks -- OMG
  • Uptime Kuma - Self-hosted monitoring tool like Uptime Robot or Statuspage -- various lang
  • Xymon - Fork of Big Brother -- C
  • Zabbix - Stores monitoring data in a DB, has agents for almost every OS, can be a virtual appliance -- various lang
  • Zenoss Core - Supports Nagios plug-in format, based on the Zope application server -- Python

Networking Tools

Hodge-podge of network tools - see also Security Tools and Network Performance Analysis Tools and Orchestration for parallel SSH tools

  • 2ping - Simple bi-directional ping utility, helps determine where packet loss occurs -- Python
  • aria2 - CLI for downloading HTTP/HTTPS, FTP, SFTP, BitTorrent and Metalink -- C Windows OS X
  • ARIN WHOIS in JSON - Authoratative (official) info including CIDR and OriginAS, updated every 8h -- collection
  • arp-scan - Create and send ARP requests -- C
  • Awesome PCAP - Huge list of tools that work with PCAP captures -- collection
  • bbcp - Copies files using multiple TCP streams to greatly increase throughput -- C
  • BGP Looking Glass - A comprehensive list of BGP looking glass servers located in various geographic regions -- collection
  • BIRD Internet Routing Daemon (BIRD) - (Almost) fully functional IP routing daemon for Linux, supports tons of standard routing protocols -- C
  • BPF Tools - Extended BSD Packet Filter (eBPF) and pcap toolkit, by CloudFlare -- Python
  • CERT NetSA Security Suite - Network flow analysis tools -- various lang
  • Cyberduck - GUI FTP, SFTP, WebDAV, S3 & OpenStack Swift browser for Mac and Windows -- itsaduck
  • Data Plane Development Kit (DPDK) - Data plane libraries and framework for fast packet processing -- C
  • dog - DNS CLI like dig but the new hotness -- Rust
  • dsniff - Great for level 2 analysis or service spoofing -- C
  • ElastiFlow - Netflow collection and visualization using the ELK stack -- various lang
  • ExaBGP - The BGP swiss army knife of networking -- Python pip
  • Fast Data Transfer (FDT) - For writing at disk speed over WANs -- Java
  • FBOSS (FaceBook Open Switching System) - FB's software stack for managing and controlling their internal switches -- various lang
  • FreeZTP (Zero Touch Provisioning) - A Zero-Touch Provisioning system built for Cisco IOS -- Python
  • FRRouting - Replacement for / fork of Quagga with more features -- C
  • ftptop - Monitor FTP connections in real time -- built-in
  • Gas Mask - Simple hosts file manager for Mac OS X, switch between host files easily -- Objective C
  • kcptun - Secure and fast tunnel based on KCP -- Golang
  • ipcalc - CLI tool to calculate subnets, netmasks, IP ranges, broadcast addresses, and more -- built-in
  • iptstate - A top-like display of IP Tables state table entries -- built-in
  • GridFTP - Supports parallel streams, optimized for WANs, part of the Globus Toolkit -- C
  • hosts - Consolidates several reputable hosts files and merges them into a unified hosts file with duplicates removed (ads, malware, gambling, porn, etc) -- collection
  • hping3 - Create custom TCP/IP packets, very flexible -- built-in
  • Internet Outages Map - By Thousand Eyes -- in-browser
  • joincap - Alternative to mergecap -- Golang
  • lftp - Supports many protocols (FTPS, HTTPS, SFTP), scheduling, bandwidth throttling, scripting, and more - feature-rich -- C C++
  • lldpd - Daemon that can talk LLDP aka the open version of Cisco Discovery Protocol (CDP), handy for network + host mapping -- C
  • Manito Networks Flow Analyzer - ELK stack netflow analyzer -- Python
  • moloch - Large scale IPv4 full PCAP capturing, indexing and database system -- JavaScript
  • mrsync (multicast remote sync) - Transfers from a master to many remote machines using Unix multicast sockets -- C
  • mTCP - High-performance user-level TCP stack for multicore systems -- C
  • Multipath TCP Checker - Multipath TCP client tester -- in-browser
  • My Looking Glass (myLG) - All-in-one CLI network diagnostic tool -- Golang
  • NAPALM (Network Automation and Programmability Abstraction Layer with Multivendor support) -- Network automation and programmability abstraction layer, for both setting config and config validation -- Python pip
  • ncat - Improved netcat, written by the Nmap team -- C
  • Netconf - Web-based GUI for configuring API-less Cisco devices -- Python
  • netshoot - Docker + Kubernetes network trouble-shooting swiss-army container -- various lang
  • netsniff-ng - High performance, zero-copy networking sniffer -- C
  • NETworkManager - All in one network GUI (config, troubleshooting, etc), neat! -- C Sharp
  • ngrep - Network grep -- C built-in
  • Nornir - Python automation framework without a DSL, alternative to managing network devices with Ansible -- Python
  • nping - Create custom network packets -- C
  • OpenBGPD - Free implementation of BGP v4 -- C
  • OpenBMP - BGP Monitoring Protocol collector with real-time monitoring, looking glass, analytics, etc -- various lang
  • OpenOnload - User-land network stack that requires no modifications to applications to use by intercepting calls -- C
  • OpenWRT - Full Linux distro for consumer-grade routers, allows for tons of non-vendor customization -- various lang
  • PeeringDB - Freely available, user-maintained, database of networks, and the go-to location for interconnection data -- collection
  • PFQ - Framework that allows efficient packets capture/transmission, in-kernel functional processing, and packets steering across sockets/end-points -- C
  • PowerAdmin - Simple Web UI for PowerDNS -- PHP
  • Quagga - Routing software suite, supports most routing protocols -- C
  • SiLK (System for Internet-Level Knowledge) - Tool suite supports the efficient collection, storage, and analysis of network flow data, enabling network security analysts to rapidly query large historical traffic data sets -- C
  • snabb - Worth seeing -- various lang
  • socat - netcat on steroids, supports serial devices -- C
  • sslh - Protocol multiplexer, let multiple daemons listen on a single port -- C
  • tcpflow - TCP demultiplexer, each flow is stored in it's own file -- C++
  • tcpreplay - Capture, edit, and replay network traffic -- C
  • tracepath - Traceroute that doesn't require root -- C
  • vFlow - High-performance, scalable and reliable enterprise netflow collector with Kafka integration, by Verizon -- Golang
  • webterm appliance - Debian-based networking toolbox, runs in a Docker container + Firefox, by the GNS3 team -- neat
  • WireShark - The classic network analyzer -- C
  • WireShark Tools - Superb list of network tools from the WireShark wiki -- collection
  • Zenmap - Official GUI for Nmap -- C Windows OS X

Network Performance Analysis Tools

Load generation, benchmarking, profiling, and latency simulation tools for TCP/IP networks - see also Web and HTTP Performance Analysis Tools and Performance Analysis Tools

  • ARGUS (Audit Record Generation and Utilization System) - Generates network activity reports -- C
  • bmon - Console based network monitor -- C
  • clumsy - Simulate poor network conditions on Windows -- C Windows
  • Comcast - Simulate crappy network connections -- Golang OS X
  • ESnet Fasterdata Knowledge Base - Provides proven, operationally sound methods for troubleshooting and solving performance issues -- collection hpcwisdom
  • EtherApe - Graphical network monitor, pretty output -- various lang
  • Flent - Python wrapper to run mutliple netperf/iperf3/ping in parallel, formerly netperf-wrapper -- Python
  • Flowgrind - Distributed TCP traffic generator -- C
  • iftop - top for network interfaces -- C
  • iperf3 - Supports tuning of various parameters related to timing, protocols, and buffers -- C
  • iptraf-ng - Updated fork of iptraf -- C
  • jnettop - Terminal / ncurses traffic visualizer -- C
  • mtr (my traceroute) - Combines ping and traceroute into a single program -- C
  • Muxy - Muck with your system and application context layers 4-7 -- Golang OS X
  • namebench - Hunts down the fastest DNS servers for your computer to use -- Python Windows OS X
  • netatop - Kernel module for atop to watch and report on network packets -- C
  • netem - Network emulator for testing variable delay, loss, duplication and re-ordering -- C
  • NetHogs - Displays per-process bandwidth usage -- C C++
  • Network Link Conditioner - Simulate bandwidth, latency, and packet loss, by Apple -- closed source OS X
  • nfdump - Captures network flows including sFlow, NetFlow, NetFlow v9, ipfix, etc -- perl
  • nicstat - vmstat for network interfaces -- C
  • nload - Console application that monitors network traffic and bandwidth usage in real time, neat ASCII graphs -- C++
  • ntopng / ntop-ng - New version of the popular ntop tool -- C
  • netperf - Network load generator, by HP -- C
  • Paris Traceroute - Shows proper network topology when load balancers and load-balanced routers are used -- C
  • Ruru - Real-time TCP latency monitoring, utilises Intel DPDK for high speed packet processing with a Node.JS frontend for visualizing the data -- various lang
  • SmokePing - Network latency visualizer, written by the MRTG and RRDtool guy -- perl
  • ss - Socket statistics, a modern netstat -- built-in
  • Stanford Linear Accelerator Center - Network Monitoring Tools - MASSIVE list of network monitoring tools -- collection
  • tc - Built-in Linux kernel traffic control -- built-in
  • TCP Throughput Calculator - See name -- in-browser
  • tcpdive - TCP performance analysis tool, implemented as SystemTap scripts -- C
  • tcplife - Watch the life of TCP connections, uses BCC -- C
  • tcptrack - Console based connection tracker -- built-in
  • trickle - Userspace bandwidth shaper -- built-in
  • vnStat - Console based traffic monitor, supports statistic collecting -- C
  • WinMTR - Windows GUI for MTR -- C++
  • Yconalyzer - Monitor and analyze TCP connections -- C++

Orchestration

OS and VM level orchestration as well as parallel SSH tools - see also Containers and Distributed Systems Tools for container orchestration and Terminal Tools and SSH Clients for SSH clients and SSH Tools for misc SSH tools and Provisioning Tools

  • Ansible - CM and orchestration, also can do provisioning -- Python
  • Batou - Define and perform automated service deployments -- Python pip
  • Capistrano - Use Ruby to run scripts/commands and push software via SSH, uses a Rake DSL -- Ruby gem
  • ClusterSHISH - Cluster SSH for Windows, works with PuTTY and OpenSSH for Windows -- closed source Windows
  • ClusterSSH - Make a change on many servers at the same time -- perl
  • csshX - Cluster SSH for OS X -- C
  • KeyBox - Web-based SSH console that executes commands on multiple shells simultaneously and supports terminal sharing -- Java
  • KeyBox-OpenShift - KeyBox for OpenShift gears -- Java
  • Mass Parallel SSH (mpssh) - Simple parallel SSH -- C
  • Multipass - Super light weight VM manager, easy way to get a fresh Ubuntu machine -- C++
  • OpenLMI (Open Linux Management Infrastructure) - Manage, monitor, and configure servers via API calls (instead of SSH), included in RHEL 7 -- various lang
  • orgalorg - Next generation parallel SSH tool because most other major ones are no longer maintined -- Golang
  • parallel - Execute jobs in parallel using one or more computers -- built-in
  • parallel-ssh (pssh) - Parallel version of OpenSSH tools - comes with prsync, pscp, pnuke, and pslurp too -- Python built-in
  • Parallel Distributed Shell (pdsh) - Kick off many SSH sessions in parallel -- C
  • PyDSH - Python Distributed Shell, parallel SSH -- Python
  • Rundeck - Job scheduler and runbook automation, enable self-service access to existing scripts and tools -- Groovy
  • Salt / Saltstack - Orchestration, server provisioning, and configuration management -- Python
  • Spacewalk - Remote commands/orchestration, patch management, and more - the upstream for Red Hat Satellite 5.x and earlier -- various lang
  • Teleport - Front-end for teams, includes session capture and replay, auditing, 2FA, session sharing, and more -- Golang
  • tmux-cssh - Cluster SSH via tmux -- shell
  • xCAT (Extreme Cloud Administration Toolkit) - Complete all in one management solution (provisioning, orchestration, management, etc) supports almost every UNIX and next generation platform, by IBM -- legit

Package Patch and Repository Tools

Repos, RPM/APT packages, packaging guidelines, patching, and patch management - see also Provisioning Tools

  • apt-dater - Simple ncurses frontend for package management via SSH, also supports yum -- C
  • aptly - Swiss army knife for Debian repository management, has the ability to take snapshots for easy rollback -- C
  • AutoPkg - Packaging and distribution for OS X, great for managing many laptops -- Python OS X
  • CentOS Errata for Spacewalk (CEFS) - Import errata information from CentOS-announce into Spacewalk -- useit
  • CentOS Repositories - Large list of both official and unofficial CentOS software repositories -- collection
  • CentOS Software Collections (SCL) - Use multiple versions of software on a system without disturbing the system default version -- C
  • Copr - Automatic build system providing a package repository as its output, by Fedora -- C
  • cowbuilder - Package builder that uses copy-on-write (COW) to speed up the build process -- C
  • DNF aka Dandified yum - yum v4 packaging system, added to upcoming Fedora/RHEL/CentOS 8 releases -- article
  • ELRepo - Repo that focuses on hardware related packages, supports RHEL and CentOS -- repo
  • Extra Packages for Enterprise Linux (EPEL) - Supports CentOS, RHEL, Scientific Linux, and Oracle Linux -- repo
  • Extra Packaging Guidelines and Policies for EPEL - Packaging guidelines, great even if not creating EPEL stuff -- readit
  • Fedora Packaging Guidelines - Excellent information that can be applied to CentOS & RHEL -- readit
  • fpm (Fucking Package Management) - Build packages for multiple platforms (deb, rpm, etc) with great ease and sanity -- Ruby gem
  • fpm-cookery - Tool for building software packages with FPM -- Ruby gem
  • Habitat - Creates platform-independent build artifacts and provides built-in deployment and management capabilities -- Golang
  • Homebrew (aka brew) - Tons of packages for Mac -- Ruby OS X
  • Koji - Software that builds packages for Fedora, can be used for other stuff too, uses mock -- C
  • Mock - Build packages in a simple chroot so you don't blow up your system -- C
  • mrepo - RPM repository management tool supporting ftp/http/sftp/rsync/rhn/you, formerly Yam -- Python
  • Munki - Managed software installation for OS X, great for managing laptops -- Python OS X
  • OStree - Tool for managing bootable, immutable, versioned filesystem trees (not really a package manager but...) -- thefuture C
  • pkgr - Made deb or RPM packages out of any Ruby, NodeJS, or Go app -- Ruby
  • Pulp - Next generation repository management, a component of Red Hat Satellite 6 -- Python
  • Red Hat Software Collections (SCL) - Use multiple versions of software on a system without disturbing the system default version, use this to get the newest / latest versions of things -- repo
  • reposync - Synchronize yum repositories to a local directory -- built-in
  • rpm-ostree - Hybrid image/package system with atomic upgrades and package layering -- C thefuture
  • RPM Fusion - Provides software that the Fedora Project or Red Hat doesn't ship -- repo
  • schroot - Allow non-root users to create chroot environments, great for package testing -- built-in
  • Spacewalk - Patch management, remote commands, and more - the upstream for Red Hat Satellite -- various lang
  • Tito - Tool for managing RPM based projects using git for their source code repository -- Python
  • yum-presto - yum plugin that provides support for downloading package deltas -- article
  • yum-security - Plugin to only install security updates -- built-in

Performance Analysis Tools

Non-specific and all-in-one performance monitoring tools - see also web, network, storage, and RDBMS and Tracing and Profiling

  • atop - Supports both real-time and historical performance monitoring -- C
  • bashtop - Real time resource monitor -- Python
  • below - Interactive tool to view and record historical system data, by Facebook -- Rust
  • bottom - Inspired by gtop, gotop, and htop -- various lang
  • btop++ - Resource monitor that shows usage and stats for processor, memory, disks, network and processes -- C++
  • Conky - Lightweight system monitor for X windows -- C++
  • Glances - Real-time performance monitoring, written in curses and Python -- Python
  • hazelnut - Python lib to parse /proc/meminfo -- Python pip
  • htop - top replacement, has a few additional features -- C
  • Linux Performance Observability Tools - Awesome graphic that shows you which tool to use, by Brendan Gregg -- yells at drives
  • mem_logger.sh - Monitor a processes' memory usage over time -- shell
  • Munin - Historical performance monitoring to help determine when you server became "slow" -- perl
  • NetData - Real time performance visualization and dashboards -- C
  • nmon - Provides both real-time and historical performance metrics -- C
  • NumaTOP - top for NUMA systems, shows hotspots, call chains, etc -- C
  • PerfKit Benchmarker - Open effort to define a canonical set of benchmarks to measure and compare cloud offerings (disk, network, CPU, etc) -- various lang
  • Phoronix Test Suite - Benchmarking and profiling suite, very feature-rich and versatile -- PHP
  • pidstat - vmstat type output for CPU, disk I/O, page faults, and more on a per-process basis -- built-in
  • pmap - Shell scripts for tracking memory usage using "pmap" -- shell
  • PowerTOP - Real-time power consumption on a per-process & per-thread basis, by Intel -- C++
  • ps_mem - Accurately reports core memory usage for a process -- Python
  • ptop - top/ntop-like task monitor written in Python -- Python pip
  • recap - Collects info from various standard utilities (free, sar, vmstat, etc) at specified intervals, by Rackspace -- various lang
  • saidar - ncurses based program for viewing system statistics -- built-in
  • smem - Reports memory usage based on proportional set size (PSS) instead of the usual resident set size (RSS) -- C
  • sysdig - Linux system exploration and troubleshooting tool with first class support for containers -- C++
  • VMtouch - File system cache diagnostics and control -- C

Provisioning Tools

OS provisioning, image creation, installation, bootstrapping, and lifecycle management - see also Package Patch and Repository Tools and Containers and Distributed Systems Tools and Live CD Tools

  • Box-Cutter - Hashicorp's community repo for Packer & Vagrant templates -- Ruby
  • Clonezilla - Disk image/cloning tool, supports most file system types -- perl shell
  • cloud-init - Configures settings the first time a system spins up (SSH keys, hostname, variables, etc), note NoCloud -- C
  • cookiecutter - Creates projects from cookiecutters (project templates), many cookiecutters to choose from -- Python pip
  • Fog - Computer / OS cloning tool, also has remote client management capabilities -- C++
  • Foreman - Provisioning and life cycle management -- Ruby Windows OS X
  • iPXE - GPL'ed version of PXE, official replacement for gPXE -- C
  • Kickstart - The classic Red Hat tool -- various lang
  • netboot.xyz - Simple iPXE menu and installer -- shell
  • Packer - Automates VM creation for multiple platforms (VMware, AWS, etc) -- Golang
  • Vagrant - Quickly spin up environments for local testing and development -- Ruby
  • Vagrant Plugins - A list of Vagrant plugins on the official Hashicorp wiki -- collection
  • Vagrant Manager - GUI to manage Vagrant boxes -- Windows OS X
  • vagrant-cachier - Cache BLOB downloads to reduce network usage -- Ruby
  • vagrant-host-shell - Simple plugin to run commands on the VM when it boots (think cloud-init) -- Ruby
  • vagrant-hostsupdater - Plugin to add your own /etc/hosts to the VM -- Ruby
  • vagrant-vbguest - Auto-install the latest VirtualBox tools at boot time (if necessary) -- Ruby
  • Salt / Saltstack - Orchestration, server provisioning, and configuration management -- Python

Python Tools and Resources

Python stuff of note

  • argparse - Parser for command-line options, arguments and sub-commands -- Python
  • atexit - Exit handlers -- Python
  • Awesome Python - Very large list of Python resources -- collection
  • bpython - Killer interface for the Python interpreter -- Python
  • ciscoconfparse - Parse, audit, query, build, and modify Cisco IOS-style configurations -- Python pip
  • exscript - Write less code using than either pure paramiko or netmiko -- Python
  • Fabric - Uses paramiko to implement a higher-level API for performing commands over SSH, particularly for deployment sysadmin tasks -- Python pip
  • Faker - Generate fake data easily -- Python pip
  • Fire - Turn any Python module, class, object, function, etc into a CLI -- Python
  • functools - Higher-order functions and operations on callable objects -- Python
  • getpass - Enter a password without echoing what they type to the console -- Python
  • inspect - Inspect live objects -- Python
  • IPython - Interactive Python shell and the kernel for Jupyter -- Python
  • Itertools - Functions creating iterators for efficient looping, iterator building blocks -- Python
  • Jinja2 - Templating language for Python -- Python pip
  • Jupyter - The language-agnostic parts of IPython -- Python
  • Logging - Flexible event logging system for applications and libraries for all modules and more -- Python
  • Mailer - The best e-mail module -- Python
  • more-itertools - More routines for operating on iterables, beyond itertools -- Python pip
  • netaddr - A network address manipulation library for Python -- Python
  • netmiko - Multi-vendor library to simplify Paramiko SSH connections to network devices -- Python
  • netminko_tools - Command line tools built on Netmiko to simplify information gathering -- Python
  • os - Interact with the OS -- Python
  • paramiko - SSH2 protocol library for Python, provides both client and server -- Python pip
  • pdb - Python debugger -- Python
  • pexpect - Expect-like module -- Python
  • pycsco - Python modules to simplify the use of working with Cisco Nexus switches -- Python
  • PyEnv - Simple Python version management that keeps everything within your home directory, virtualenv alternative -- shell
  • PyPI (Python Package Index) - Software repo for Python packages, like Ruby gems or a RPM repo -- collection
  • PyPy - Python alternative with advance features (JIT compiles, sandboxing, etc) -- Python
  • python-prompt-toolkit - Library for building interactive command lines -- Python pip
  • RadSSH - Paramiko-based parallel SSH -- Python pip
  • Requests - The best HTTP library -- Python pip
  • scapy - Interactive packet manipulation for Python -- Python pip
  • selenium - Browser automation -- Python
  • sh - Library that allows you to call any program (shell command) as if it were a function -- Python pip
  • Subprocess - Spawn subprocesse and collect their output -- Python
  • sys - System-specific parameters and functions -- Python
  • tempfile - Generate temporary files and directories -- Python
  • TextFSM - Create a pool of templates to parse text, then use TextFSM to parse useful information from a variety of sources -- Python
  • xlwings - Replace your Excel VBA code with Python -- Python pip

Python Programming Tutorials

Tutorials, exercises, and challenges for learning Python. I have no idea how good/bad these are


RDBMS and SQL Tools

Tools for interacting with and related to the major DBs and SQL

  • Adminer - GUI for database management in a single PHP, formerally phpMyAdmin -- PHP
  • Babelfish for PostgreSQL - Layer that lets PostgreSQL understand queries from applications written for Microsoft SQL Server including T-SQL and TDS, by Amazon -- various lang
  • CockroachDB - Open source version of Google's Spanner storage system -- thefuture Golang
  • DBeaver - OSS multi-platform GUI that supports almost every DB, built from Eclipse -- Eclipse
  • DB Browser for SQLite - GUI database browser for SQLite instances -- C++
  • Flyway - Version control for DB schemas, supports most DBs -- Java
  • gh-ost - Online schema migrations for MySQL, by GitHub -- Golang
  • HeidiSQL - GUI SQL DB browser and editor -- Windows OS X
  • Liquibase - Tracking, managing, and applying database schema changes, SVN for DBs -- Java
  • MaxScale - General purpose DB query proxy, router, and load balancer by MariaDB -- C
  • mycli - CLI for MySQL and derivates with auto-completion and syntax highlighting -- Python
  • mydumper (MySQL Data Dumper) - Much better than mysqldump, works in parallel -- perl
  • MyRocks - RocksDB with a MySQL front-end / interface -- C++
  • MySQL sys schema - Collection of views, functions and procedures to help MySQL administrators get insight into MySQL usage -- SQL
  • MySQL Workbench - The official MySQL GUI for admins, devs, DBAs, and architects -- various lang
  • Oracle TPT Scripts - Tanel Poder's Troubleshooting & Performance Tools for Oracle Databases -- SQL
  • orchestrator - MySQL replication topology management and visualization tool, GUI -- Golang
  • Percona Toolkit for MySQL - Percona's special toolkit -- various lang
  • pgcli - Postgres CLI with autocomplete and syntax highlighting -- Python
  • pgloader - Fast data loader and swiss army knife for PostgreSQL -- Lisp
  • pgweb - Web-based PostgreSQL DB browser -- Golang
  • pg_repack - Remove bloat from tables and indexes without using an exclusive lock -- C
  • Phinx - Database migrations in SQL or PHP -- PHP
  • Postgres-XL - Scale-out version of PostgresSQL -- C
  • Postgres.app - All-in-one version of Postgres for local testing on a laptop -- C
  • PostgREST - Create a REST API for any Postgres DB -- Haskell
  • PostgreSQL GUI Tools - A huge list on the official wiki -- collection
  • Presto - Distributed SQL query engine for big data, by Facebook -- Java
  • Sequel Pro - MySQL management GUI for Mac -- OS X
  • shift - Schema migrations for MySQL, by Square -- Ruby
  • SQLite - Self-contained, serverless, zero-configuration, transactional SQL database engine, great for testing -- C
  • sqlmap - Detect and exploit SQL injection flaws, pen testing tool -- Python
  • SQL Fiddle - Write and test SQL -- in-browser
  • Vitess - Lets you horizontally scale a mysql database at the database level and not the application level -- various lang
  • wal-e - Simple continuous archiving for PostgreSQL -- Python
  • wal-g - Simple continuous archiving for PostgreSQL, successor to wal-e -- Python
  • WWW SQL Designer - Designing RDBMS schemas features saving, exporting to XML, and SQL script creation, free SaaS version here -- JavaScript

RDBMS Performance Analysis Tools

Load generation, benchmarking, telemetry, and profiling for various RDBMS platforms - see also Storage Performance Analysis Tools and Performance Analysis Tools and Tracing and Profiling and Network Performance Analysis Tools and Metrics and Time Series Data

  • Awesome MySQL Performance - Collection -- collection
  • HammerDB - Load testing and benchmark tool, supports most DBs -- Tcl
  • innotop - "top" for MySQL -- NOTE: beware of exposing your password -- perl
  • MySQLtuner-perl - Analyzes a MySQL installation and provides suggestions to increase performance -- perl
  • MySQL Performance Analyzer - Open sourced by Yahoo -- Java
  • Open PostgreSQL Monitoring (OPM) - Includes a web console, nice GUI -- JavaScript
  • Percona Toolkit for MySQL - Percona's internal tools -- various lang
  • pg_activity - htop for PostgreSQL -- Python
  • pg_view - PostgreSQL monitoring, supports ncurses, console, and JSON output -- Python pip
  • pgBadger - PostgreSQL log analyzer -- perl
  • PgHero - A performance dashboard for Postgres - health checks, suggested indexes, and more -- JavaScript
  • PGObserver - Killer PostgreSQL monitor, includes dashboard -- Python
  • Silly Little Oracle Benchmark 2 (SLOB2) - Stresses and benchmarks Oracle DBs, works at the RDBMS layer -- various lang
  • Swingbench - Oracle load generator -- closed source
  • SysBench - Evaluates OS parameters that are important for DBs, does not require a DB to be installed -- C
  • Tsung - Distributed stress tester, also supports stress testing DBs -- Erlang

Regular Expressions

regex checkers, creators, evaluators, debuggers, and tutorials - see also Shell Scripting and Tools and Software Development Tools

  • Debuggex - Online regex debugger -- in-browser
  • ExtendsClass - Online visual regex tester -- in-browser
  • perlretut - perl-focused but very useful for all regex -- article
  • Refiddle - Online, supports JavaScript, Ruby, and .NET only -- in-browser
  • regex101 - Online regex tester and debugger, supports multiple languages -- in-browser
  • RegexOne - Learn regular expressions with simple, interactive examples -- tutorial
  • regexper - Regex visualizer using railroad diagrams, great for debugging -- in-browser
  • RegExr - Another online regex tool that includes cheat sheets, examples, and community-contributed expressions -- collection
  • RegularExpressions.info - THE BEST regex site -- collection
  • Regular Expressions - A Gentle User Guide and Tutorial - The easy way -- tutorial
  • RexV.2 - Online Regex evaluator, supports multiple languages -- in-browser
  • Rubular - Online Ruby-only regex evaluator -- in-browser
  • txt2re - Regex generator, recommended -- in-browser
  • txt2regex - Converts human sentences to regex, written in bash -- shell

Secrets Management

Password, private key, and API key storage and management tools - see also Security Tools and SSL Tools and Two Factor Authentication and VPNs and Tunnels

  • Bitwarden - The new hotness in password management -- various lang
  • blackbox - Safely store secrets in Git, by Stack Exchange -- shell
  • Diceware - Neat IRL passphrase generator -- doitirl
  • encpass - Lightweight solution for using encrypted passwords in shell scripts -- shell
  • GPG (GNU Privacy Guard) - The original! Store em in text files and encrypt -- C pro
  • GRC Password Generator - Generates 63 bit passwords -- in-browser
  • Kee - Auto-fill website logins using KeePass as a source, like LastPass -- C# Firefox
  • Keybase - Upload your public key and find other people's public key via their social media user name(s) -- in-browser
  • KeePass - Lightweight, easy to use, GUI password manager, runs on most platforms -- C# Windows
  • Keepass2Android - Compatible with KeePass 2.x -- Android
  • KeePassDroid - Open source version of KeePass 1.x for Android -- Android
  • KeePassXC - Supports YubiKey and TOTP, forked becasue KeePassX is no longer being updated -- C++
  • KeeWeb - Cross-platform password manager compatible with KeePass -- JavaScript
  • Keywhiz - A system for distributing and managing secrets (API keys, certificates, etc), by Square -- Java
  • kpcli - Command line interface / CLI for KeePass -- perl
  • lastpass-cli - CLI for LastPass -- C
  • MacPass - A free, open-source, KeePass-compatible password manager for macOS -- Objective-C
  • msktutil - Keytab client for a Microsoft Active Directory environment -- built-in
  • One-Time Secret - Create links that self-destruct after a single viewing, great for sharing passwords -- Ruby
  • pass (passwordstore) - Uses GPG at it's core, supports tracking password changes in git -- shell
  • passff - Addon for interacting with the a pass aka passwordstore repository -- Firefox
  • Password Pusher - Links to passwords expire after a certain number of views and/or time has passed, RoR app -- Ruby
  • Password Safe - Designed by Bruce Schneier himself -- bam
  • pwd.sh - Script to manage passwords in an encrypted file using GPG -- shell
  • pwgen - Password generated, included in most Linux distros -- built-in
  • Secrets OPerationS (SOPS) - Secrets management, by Mozilla -- Python
  • TeamPass - Collaborative password management using the LAMP stack -- PHP
  • Vault - Tool for storing secrets (API keys, passwords, certs, etc) by Hashicorp -- Golang

Security Tools

Misc security tools - see also Live CD Tools and Networking Tools and Secrets Management and SSL Tools and Two Factor Authentication and VPNs and Tunnels and Logging

  • Advanced IP Scanner - For Windows, recommended -- closed source
  • afl-fuzz (American Fuzzy Lop) - One of the best fuzzers -- C
  • AIDE (Advanced Intrusion Detection Environment) - File integrity checker, alternative to Tripwire -- various lang
  • Amass - Automatically obtains subdomain names in a variety of ways and uses that info to build maps of the target network, by OWASP -- Golang
  • Angry IP Scanner - GUI network scanner, supports plugins, -- Java Windows OS X
  • Armitage - GUI for Metasploit -- Java
  • asecurecloud - A free library of 400+ customizable AWS security configurations and best practices (CF, Terraform, and AWS CLI) -- collection
  • ATA Secure Erase - Send a signal to an ATA drive to perform a hardware-based erase, the ONLY way to wipe a SSD properly -- article
  • Atomic Red Team - Library of tests mapped to the MITRE ATT&CK® framework. Security teams can use Atomic Red Team to quickly, portably, and reproducibly test their environments -- Ruby
  • Attack Surface Analyzer 2.0 - See exactly what changed post-OS install or post-software install, by Microsoft -- Windows
  • Attack Surface Framework - ASF will discover assets/subdomains, enumerate their ports and services, track deltas and serve as a continuous and flexible attacking and alerting framework -- various lang
  • auditd-attack - A Linux auditd rule set mapped to MITRE's Attack Framework -- collection
  • authconv - Web app authorization coverage scanning -- JavaScript
  • Autopsy - Great for grabbing artifacts for DIFR -- closed source
  • Awesome Honeypots - Curated collection of honepots and honeypot resources -- collection
  • AzGovViz - Azure Governance Visualizer PowerShell script that captures Azure Governance related information -- PowerShell
  • Azucar - Multi-threaded plugin-based tool to help you assess the security of your Azure Cloud environment -- PowerShell
  • bettercap - Swiss Army knife for 802.11, BlueTooth, and Ethernet networks reconnaissance and attacks -- Golang
  • Brida - Bridge between Burp Suite & Frida, lets you use and manipulate applications own methods while tampering the traffic exchanged between the applications and their back-end services/servers -- various lang
  • bro - Framework for network analysis and security monitoring -- C++
  • CALDERA - Automated adversary emulation system that performs post-compromise adversarial behavior within Windows Enterprise networks -- Python
  • Checkov - Scans cloud infrastructure provisioned using Terraform, Cloudformation or Kubernetes and detects security and compliance misconfigurations, similar to a sub-component of Twistlock, Policy as Code -- Golang
  • cherrytree - Hierarchical note taking application, excellent for red team / pentest notes -- Python
  • chkrootkit - Rootkit checker, best used from a live CD -- C
  • CIS Linux Benchmarks - Linux OS hardening guides, superb! -- collection
  • CloudSploit - Scans for risks, multi-cloud -- various lang
  • Cloud Security Suite (cs-suite) - One stop tool for auditing the security posture of AWS/GCP/Azure infrastructure -- Python pip
  • Common Vulnerability Scoring System Calculator - Version three -- collection
  • ConfigServer Security and Firewall (CSF) - Stateful Packet Inspection (SPI) firewall, Login/Intrusion Detection and Security application for Linux servers -- various lang
  • Conftest - Write tests against structured configuration data (Kuberetes, Terraform, Serverless, etc) -- Golang
  • CSP Evaluator - Check if a Content Security Policy (CSP) serves as a strong mitigation against cross-site scripting attacks, by Google -- in-browser
  • cuckoo - Feed in malware URL, it fires up VM, and provides a report about the actions the malware took -- in-browser
  • CyberChef - A web app for encryption, encoding, compression and data analysis -- in-browser JavaScript
  • Cyber Security Transformation Chef - CyberChef as a Burp Suite extension -- Java
  • dcfldd - Enhanced dd with security and forensics features -- C
  • Dradis CE (Community Edition) - Reporting framework for generating one-click reports (vuln scanning, pentest, etc) -- Ruby
  • Dshell - Network forensic analysis framework, written by the US Army -- Python
  • EncFS - Encrypted file system in user space via FUSE -- C++
  • fail2ban - Watches log files to ban IPs based on rules (too many failed logins, exploit attempts, brute force attacks, etc) -- Python
  • ffuf - Fast web fuzzer written in Go -- Golang
  • Forefront Identity Manager - State-based multi-platform identity management, by Microsoft -- closed source
  • Forensics Acquisition of Websites (FAW) - Download and save social media sites, paid version has more features -- closed source Windows
  • Forensics Wiki - Digital forensics wiki, tons of tools and information -- wiki
  • Free Cybersecurity Services and Tools - A collection by CISA -- collection
  • FTimes - System baselining and evidence collection tool -- C
  • fwknop (FireWall KNock OPerator) - Single Packet Authorization (SPA), authoriation packet from you opens firewall rules so only you can get in -- various lang
  • GGCR - GRC, by Google -- Python
  • Google Safe Browsing Site Status - Use Google for site/URL malware analysis -- in-browser
  • Grant - OAuth proxy that supports >200 providers -- various lang
  • GRR Rapid Response - Live forensics for incident response via a Python agent, dump memory, isolate host, snoop syscalls etc -- Python
  • hashcat - World's fastest CPU password cracker / password recovery -- C
  • HIPAA Security Risk Assessment Tool - Helps healthcare providers conduct a security risk assessment, results of the assessment are displayed in a report which can be used to determine risks in policies, processes and systems and methods to mitigate weaknesses -- Windows
  • Hybrid Analysis - Dree malware analysis service that detects and analyzes unknown threats using a unique Hybrid Analysis -- in-browser
  • icebreaker - Automates internal network attacks against Active Directory to deliver you plaintext credentials -- various lang
  • Information Security Cheat Sheets and Checklists - An assortment of IS checklists and cheat sheets, -- collection
  • IPv6 Attack Toolkit - Title -- collection
  • Jenkins Pentesting - Title, please contribute -- various lang
  • Just the Basics (JTB) Investigator - Simple menu & CLI to automate repetitive everyday tasks -- Python
  • geoiplookup - Uses the GeoIP DB and library to determine which physical country an IP or host originates in, includes PAM library -- built-in
  • Hindsight - Internet history forensics for Google Chrome/Chromium -- Python
  • Jailkit - chroot toolkit -- C
  • Joe's Sandbox (Cloud Basic) - Deep automated malware analysis -- in-browser
  • Kali NetHunter Linux Root Toolkit (LRT) - Collection of bash scripts that setup and install Kali Linux NetHunter from a Linux/OSX environment onto a NetHunter supported device -- shell
  • LDAP Password Hunter - Wraps features of getTGT.py (Impacket) and ldapsearch in order to look up for password stored in LDAP database -- shell
  • LinEnum - Enumerate a local Linux environment -- shell
  • Lynis - Auditing and hardening tool (CIS, PCI-DSS, etc), supports most Unix-like operating systems -- shell
  • macOS Security Compliance Project - By NIST -- guide
  • Malcom - Easily deployable network analysis tool suite for full packet capture artifacts (PCAP files) and Zeek logs -- various lang
  • Massscan - Scans in parallel and async for the fastest scans around -- C
  • Metasploit - Classic exploit framework -- various lang
  • MicroBurst - A PowerShell toolkit for attacking Azure -- Powershell
  • Microsoft Security Tools by Zolderio - Tools to secure O365 and Azure, by Zolder -- various lang
  • MITRE ATT&CK - Knowledge base of adversary tactics and techniques based on real-world observations, used for threat modeling -- collection
  • MobSF (Mobile Security Framework) - Automated, all-in-one mobile application (Android/iOS/Windows) pen-testing, malware analysis and security assessment framework capable of performing static and dynamic analysis -- various lang
  • National Checklist Program - U.S. government repository of publicly available security checklists (or benchmarks) that provide detailed low level guidance on setting the security configuration of operating systems and applications -- collection
  • National Cybersecurity Assessments and Technical Services (NCATS) - Free vuln scans, phishing campaigns, architecture reviews, pen tests, and more by the CISA -- service
  • National Software Reference Library (NSRL) - Large collection of diskprints from various software and malware -- collection
  • National Vulnerability Database - Gotta catch em all -- collection
  • NetworkMiner - Network Forensic Analysis Tool (NFAT) for parsing PCAP files, paid version is worth it -- open and closed source
  • Nikto2 - Web & app server vulnerability scanner -- perl
  • nmap - Classic port scanner -- various lang
  • nmap_vulners - Emumerate and list vulnerabilities during an nmap scan similar to Nessus -- collection
  • nmap Scripting Engine (NSE) - Tons of scripts for nmap -- various lang
  • oauth2_proxy - A reverse proxy that provides authentication with Google, Github or other provider -- Golang
  • oclHashcat - World's fastest CPU + GPU password cracker / recovery software -- C
  • OpenSCAP - NIST Certified SCAP 1.2 toolkit -- C
  • OpenVAS - Vulnerability scanner, forked from the now closed-source Nessus scanner -- C
  • Open Policy Agent - Unifies policy enforcement in the cloud, the new standard -- Golang
  • Open Web Application Security Project (OWASP) - Focus on web application security -- organization
  • Oracle Database Attacking Tool (ODAT) - Open source penetration testing tool that tests the security of Oracle databases remotely -- Python
  • OS X Auditor - Forensics tool for Mac -- OS X JavaScript
  • OSSEC - Host based intrusion detection system (HIDS), supports most Unix-like OSes -- C Windows OS X
  • OWASP Cheat Sheet Series - Contains all OWASP cheat sheets, new v2 -- collection
  • p0f - Utilizes an array of sophisticated, purely passive traffic fingerprinting mechanisms to identify the players behind any incidental TCP/IP communications (often as little as a single normal SYN) without interfering in any way -- C
  • PALADIN - Easy to use Linux-based live CD for forensic analysis -- various lang
  • Penetration Testers Framework - Script to pull down all the latest greatest tools -- Python
  • pfsense - The best firewall software, supports appliances and live CDs -- C
  • PowerZure - PowerShell framework to assess Azure security -- PowerShell
  • preeny - Some helpful preload libraries for pwning stuff -- C
  • Prey - Open source anti-theft software for almost all platforms -- shell Java mobile
  • ProcDOT - All in one visual malware analysis and visualization, by CERN -- closed source various lang
  • Purple Team ATT&CK Automation - Metasploit automation of MITRE ATT&CK TTPs -- Ruby
  • Qubes OS - A reasonably secure operating system -- various lang
  • Red October - Go server for two-man rule style file encryption and decryption -- way cool
  • Red Teaming Toolkit - Collection of open source and commercial tools that aid in red team operations -- collection
  • Regshot - Snapshot and compare the Windows Registry for before/after analysis -- C
  • Rootkit Hunter - Compares hashes of important files with known good hashes that are stored in online databases -- perl shell
  • SalSA (Salvaging Static Analysis) - Windows PE file parsing in-browser, can be locally hosted, by the DoD -- Python
  • Samhain - HIDS, file integrity checker, rootkit detection, log file monitoring, and more -- C
  • SANS Tools - OSS tools and toolkits from SANS instructors -- collection
  • Scout Suite - Multi-cloud security-auditing tool, which enables security posture assessment of cloud environments. Using the APIs exposed by cloud providers, Scout Suite gathers configuration data for manual inspection and highlights risk areas -- various lang
  • scrub - Supports many disk-wiping standards including military / government grade wipes -- built-in
  • scrypt - More secure against hardware brute-force attacks than alternatives such as PBKDF2 or bcrypt, key stretching -- C
  • searchsploit - CLI to search Exploit DB -- shell
  • SecLists - Collection of multiple types of lists used during security assessments collected in one place -- collection
  • Security Onion - Linux distro for IDS, NSM, and log management -- various lang
  • Security Stack Mappings - Maps Azure and AWS product security controls to MITRE ATT&CK -- collection
  • Security Technical Implementation Guide (STIG) - Guides to securing almost every application, by the US military -- collection
  • SELKS - Debian based based IDS/IPS with ELK stack, installable or live CD -- various lang
  • Shodan - The security search engine -- oh yeah
  • SIFT Workstation - Collection of free and open-source incident response and forensic tools designed to perform detailed digital forensic examinations in a variety of settings, by SANS -- various lang
  • simplewall - Simple app to configure Windows Filtering Platform (WFP), VERY powerful -- Windows
  • Snort - The classic network intrusion prevention system (NIPS) -- C
  • SPARTA - GUI to simplify the scanning and enumeration phases -- Python
  • Spiderfoot - Automate the process of gathering intelligence about a given target -- Python
  • SubBrute - Subdomain enumeration tool for penetration testers -- Python
  • sudosh2 - Records all keystrokes and output and can play back the session as just like a VCR -- C
  • SuperTokens - OSS alternative to Auth0, Firebase Auth and AWS Cognito -- Java
  • Suricata - IDS + IPS + NSM + PCAP processing all in one -- various lang
  • SWORD Dropbox -- $15 OpenWRT + Pi based DIY disposable pen-test tool -- various lang
  • symon-config - Sysmon configuration file template with default high-quality event tracing -- XML!!!! Windows
  • SysmonForLinux - Uses eBPF, can be used for SIEM or troubleshooting / debugging -- C
  • tcpbin - A simple TCP dumping server/host for pentesting -- Python
  • terraform-compliance - BDD for Terraform, security and compliance-focused -- Python pip
  • tfsec - Static code analysis for Terraform -- Golang
  • theZoo - These are real and they are smarter than you, do not fuck around -- collection
  • tink - Smiple, small, secure crypto library by Google -- C++
  • Tomb - zsh wrapper script for cryptsetup + gpg + LUKS volumes -- shell
  • Tripwire - File integrity checker and monitor, replacement for the now closed-source Tripwire -- C
  • URLquery - Free service to scan a URL for web-based malware -- in-browser
  • URL Canary - Create canary URLs so you know if someone is inspecting the source code of your applications -- in browser
  • urlscan.io - Similar to URLquery -- in-browser
  • usbkill - Kill switch that takes action when a USB device is connected -- Python
  • VeraCrypt - OSS successor to and fork of TrueCrypt, supports Linux, Windows, and Mac -- C C++
  • VirusTotal - Allows you to upload a file and have it scanned by tons of virus scanners -- in-browser
  • Volatility - Open source memory forensics -- Python
  • VulnHub - ISOs and more for hands-on security practice -- collection
  • w3af - Web application attack and audit framework, OSS vun scanner -- Python
  • Wazuh - Host-based open source security platform -- various lang
  • Xplico - Useful for looking at pcaps of web sessions -- C
  • Zed Attack Proxy (ZAP) - Pen testing too that focues on web applications -- Java
  • ZMap - Scanner designed for large address spaces -- in-browser
  • zxcvbn - Password strength estimator, written by Dropbox, operates in a browser window -- CoffeeScript in-browser
  • zzuf - Transparent application input fuzzer that works by intercepting file and network operations and changing random bits in the program's input -- C

Shell Scripting and Tools

Shell scripting tutorials, collections of note, Linux shells for Windows, and misc neat tools - see also Terminal Tools and SSH Clients and Orchestration for parallel SSH tools and Regular Expressions

  • Advanced Bash-Scripting Guide - By the Linux Documentation Project (LDP) -- tutorial
  • awk Tutorial - Easy to understand awk tutorial -- tutorial
  • Awesome dotfiles - All kinds of . files -- collection
  • autojump - Small database of directories that you visited in the past, used to quickly navigate complex directory structures -- Python
  • autoenv - Autoruns a .env file in a directory when you cd into the directory -- Python
  • Awesome Shell - Massive collection of shell tools -- collection
  • Babun - Pre-configured Cygwin with many more features and a better design -- various lang
  • Bash-it - bash version of the oh-my-zsh shell environment -- various lang
  • BashGuide - Targeted at beginners -- wiki
  • Bash Hackers Wiki - Human-readable bash documentation so you don't have to dig through the man page -- wiki
  • Bash Pitfalls - Common errors that bash programmers make -- wiki
  • cheat - create and view interactive cheatsheets on the command-line -- Python
  • comm - Display lines that two files have in common (eg: the opposite of diff) -- built-in
  • CommandlineFu - Killer code snippets -- collection
  • CRUSH (Custom Reporting Utilities for SHell) - Killer toolset for working on delimited data, by Google -- C
  • Cygwin - GNU shell and tools for Windows -- C Windows
  • dotfiles by Paul Miller - Beautiful and flexible Mac terminal configuration files and utilities, ZSH-based -- shell OS X hawt
  • dotfiles - Unofficial guide to dotfiles on GitHub -- collection
  • Environment Modules - Dynamic modification of your shell environment using modules -- Tcl Windows
  • fzf - Command line fuzzy finder, supports tmux/bash/zsh -- Golang
  • icdiff - diff tool that highlights the differences -- Python OS X
  • moreutils - "Collection of the unix tools that nobody thought to write long ago when unix was young" -- neat
  • notify - Send a notification from your Linux system to an Android app on your phone, good for long running shell commands -- JavaScript Android
  • pigz - Parallel gzip for multi-processor/core systems -- C
  • pv - Shows the progress of data as it flows through a pipe -- built-in
  • ShellCheck - Checks shell scripts for common mistakes, essentially a linter / static analysis -- in-browser
  • tldr - Simplified and community-driven man pages, cuts out a lot of cruft -- collection
  • Unix Toolbox - A collection of Unix/Linux/BSD commands and tasks for advanced users -- collection

Software Development Tools

Build systems, stubs/mocks, CI/CD, cheat sheets, and other - see also Git Tools and Collaboration Tools and Editors and Shell Scripting and Tools and Web and HTTP Tools and Tracing and Profiling and Web and HTTP Performance Analysis Tools

  • ack2 - grep-like tool designed to search source code -- perl
  • afl-unicorn - Fuzz any piece of binary that can be emulated by Unicorn Engine -- C
  • ag (aka The Silver Searcher) - Source code searching tool, a better grep -- C
  • Bazel - Google's build system -- Java
  • Binary Optimization and Layout Tool (BOLT) - A linux command-line utility used for optimizing performance of binaries, by Facebook -- various lang
  • Buildbot - CI framework -- Python
  • Conventional Commits - Specification that provides an easy set of rules for creating an explicit commit history; which makes it easier to write automated tools on top of -- article
  • Create React App - Create React apps with no build configuration, webpack and Babel are handled behind the scenes, by Facebook -- various lang
  • cwrap - Wrappers for creating test scenarios and faking behavior/ stubbing, mostly network focused, by the Samba guys -- C
  • dev-setup - Automated setup scripts for laptop tools like Sublime Text, AWS, Spark, Android dev, and more -- collection
  • drone - CI platform built on Docker / containers, can also deploy to Kubernetes -- Golang
  • dropwizard - Simple library for building production-ready RESTful web services -- various lang
  • Fossil - Simple all-in-one SCM -- various lang
  • fswatch - Cross-platform for watching files and taking action when they change -- C++
  • gdb TUI - Curses / menu-based interface for GDB, much easier than REPL mode -- C
  • GoCI - Go continuous delivery platform by ThoughtWorks -- Java
  • Guard - Flexible framework to take action on file system change event -- Ruby gem
  • Gulp - Built system / toolkit that helps you automate time-consuming tasks in your development workflow -- JavaScript
  • Ionic Framework - Mobile UI kit that allows you to write multi-platform mobile apps in JS, Angular, Vue, or React - no Swift or Android knowledge needed -- various lang
  • Jenkins - The most popular CI orchestration tool, supports a billion plugins -- various lang
  • jenkins-job-dsl (Jenkins Job DSL Plugin) - Groovy-based DSL for writing Jenkins jobs -- Groovy
  • Jenkins job-config-history Plugin - Tracks changes to system and job configurations -- Java
  • Jenkins Job Builder - Takes simple descriptions of Jenkins jobs in YAML or JSON format and uses them to configure Jenkins -- Python pip
  • Jenkins Log Recorder - Helps you group relevant logs together while filtering out the noise -- Java
  • Jenkins Pipeline Mutlibranch Plugin - Automatically creates a new Jenkins job whenever a new branch is pushed to a source code repository -- Java
  • Jenkins ThinBackup - Jenkins plugin that backups configurations (not workspaces or archives) -- Java
  • JSON Server - Full fake REST API for quickly prototyping and mocking in 30 seconds -- JavaScript
  • Meld - Diff tool, recommended -- Python
  • Mockoon - Create mock APIs in seconds, runs locally -- JavaScript Electron
  • MockServer - Web server to remotely or locally mock HTTP/HTTPS and similar -- Java
  • mountebank - Stub downstream resources for testing, supports HTTP HTTPS SMTP TCP -- JavaScript Windows OS X
  • Ninja - Small build system with a focus on speed -- Python
  • Pact - HTTP contract tests and without contract testing, the only way to ensure that applications will work correctly together is by using expensive and brittle integration tests -- various lang
  • PatchELF - Simple utility for modifying existing ELF executables and libraries -- C
  • PEview - Easily and quickly view the structure and content of Windows EXE DDL LIB Portable Executable (PE) files -- closed source
  • PRoot - chroot, mount --bind, and binfmt_misc without privilege/setup -- C
  • Proxygen - Modern C++ HTTP library, by Facebook -- C++
  • REST-assured - Java DSL for testing of REST services -- Java
  • RocksDB - Library that provides an embeddable, persistent key-value store for fast storage - by Facebook -- C++
  • SonarQube - Platform and dashboard for managing code quality -- Ruby Java
  • Sonatype Nexus - Software / binary artifact storage -- Java
  • SourceGraph - Perhaps the best code search and navigation engine -- Golang
  • SymbolHound - Search engine that doesn't ignore special characters, great for programming questions -- try it
  • Visual Studio Code Remote - Containers - Lets you use a Docker container as a full-featured development environment. It allows you to open any folder inside (or mounted into) a container and take advantage of Visual Studio Code's full feature set -- neat
  • watchman - Watch files and take action when they change (eg: kick off the CI system), by Facebook -- C
  • WireMock - Flexible stubbing and mocking services -- Java

SSH Tools

Misc SSH stuff - see also Terminal Tools and SSH Clients for clients and Orchestration for parallel SSH tools and Shell Scripting and Tools

  • autossh - Automatically restart SSH sessions that stop passing traffic -- C
  • Corkscrew - Tunnel SSH through HTTP proxies -- C
  • Dropbear - Small / minimal SSH client and server, often used in IoT and embedded devices -- C
  • Keychain - Manage SSH and GPG keys, acts as a frontend to ssh-agent, only enter passphrase once per reboot -- shell
  • Match - Creates a conditional block, great for controlling actions on a per-user and/or per-host basis in sshd_config -- built-in
  • Mosh (Mobile Shell) - Remote shell that supports roaming (client IP address changes) and intermittent connectivity, by MIT -- C++
  • Shuttle - A simple SSH shortcut menu for macOS -- Objective-C
  • ssh-chat Instead of a shell you get a chat prompt -- Golang
  • ssh-ldap-helper - Store public keys in LDAP -- built-in
  • SSHFS - Mount remote file systems using a SSH tunnel -- built-in
  • sshmuxd - SSH jumphost style proxy -- Golang
  • sshttp - Port multiplexer that hides a SSH daesmon behind HTTP, HTTPS, or SMTP on a single port -- C
  • SSH Guard - Think fail2ban for SSH -- C
  • SSH Power Tool (sshpt) - Execute commands and upload files to many servers simultaneously via SSH without using pre-shared keys -- Python pip
  • storm - CLI and GUI tool to manage your SSH connections (add, delete, list, search) -- OS X

SSL Tools

SSL, TLS, CAs, and similar tools for interacting with HTTPS and SSL certificates - see also Web and HTTP Tools and Security Tools

  • BadSSL.com - Test various clients (browsers, etc) against bad SSL configs -- in-browser
  • BoringSSL - Google's fork of OpenSSL, does not guarantee API and ABI compatibility -- C
  • cfssl - PKI/TLS swiss army knife, has CLI and a HTTP API server for signing, verifying, and bundling TLS certificates -- Golang
  • cipherscan - Find out which SSL ciphersuites are supported by a target -- Python
  • Dogtag Certificate System - PKI component of FreeIPA, by Fedora -- C
  • Fizz - C++14 implementation of the TLS-1.3 standard, by Facebook -- C++
  • HSTS Preload Submission - Submit your site to be preloaded in major browsers -- security
  • Let's Encrypt - Free SSL certs from a real CA -- in-browser
  • Mozilla Server Side TLS - Mozilla's extensive server side TLS configuration guide -- wiki
  • Mozilla SSL Configuration Generator - Generate SSL configs for Apache, Nginx, ELB, HAproxy and more -- in-browser
  • nogotofail - Spot and fix weak TLS/SSL connections and sensitive cleartext traffic, by Google -- Python
  • Qualys SSL Server Test - Evaluates and provides recommendations for the SSL settings of any web site -- in-browser
  • s2n - Amazon's implementation of the TLS/SSL protocols in C99 (simple, small, fast, secure) -- C
  • sslconfig - CloudFlare's Internet facing SSL cipher configuration, patches for Nginx and OpenSSL -- C
  • ssldump - The de-facto repo -- C
  • SSLsplit - Transparent and scalable SSL/TLS interception -- C
  • sslyze - Fast and full-featured SSL scanner, written in Python -- Python OS X
  • stunnel - Create simple TLS tunnels for existing services (eg: telnet, nc, etc) -- C

Storage Tools

The storage junk drawer - see also Backups and Cloud File Sync and Sharing and Live CD Tools

  • BeeGFS - Parallel cluster file system, worth examining -- C++
  • CrystalDiskInfo - S.M.A.R.T. GUI tool for Windows -- closed source Windows
  • FlashCache - General purpose, write-back block cache -- C
  • fs-cache - Modern NFS client-side caching -- built-in
  • Linux-IO Target (LIO) - Create and share iSCSI, Fibre Channel, FC over Ethernet, and other storage targets from Linux VMs -- built-in
  • lsblk - List block devices -- built-in
  • ncdu - ncurses version of "du" -- built-in
  • Parted Magic - Resize, grow, shrink, clone, recovery, wiping, benchmarking, and more. Supports Linux and Windows file systems -- Windows
  • snapper - CLI tool to manage Btrfs snapshots, snapshot timelines, and more -- various lang
  • System Storage Manager (SSM) - Generic CLI for managing all types of storage (DM, LVM, multipath), added in RHEL 7 -- built-in

Storage Performance Analysis Tools

Grind your disks into dust - see also Performance Analysis Tools and Tracing and Profiling

  • Bonnie++ - The classic, still updated -- built-in
  • Connectathon Test Suite - NFS stress testing and benchmarking tools -- various lang
  • fio - Supports 19 different I/O engines (sync, mmap, libaio, posixaio, etc), very powerful -- C
  • Fnotifystat - Dumps the file system activity in a given period of time -- C
  • ioping - Monitor I/O latency in real time -- C
  • iorate - Originally written by EMC, now open source -- C
  • iotop - top for I/O requests, displays information on a per-process basis -- Python
  • IOzone - Supports NFS, still being updated! -- C
  • Threaded I/O Tester (tiobench) - Threaded I/O tester, tiotest and tiobench -- C

Terminal Tools and SSH Clients

Fancy SSH clients, terminal sharing, and similar tools - see also SSH Tools and Orchestration for parallel SSH tools and Shell Scripting and Tools

  • Bitvise SSH Client - Feature-rich SSH & SFTP client for Windows, free for individual use -- closed source Windows
  • Byobu - An enhanced version of the "screen" utility -- shell OS X
  • ChromaTerm-- - Pipe stdin to this program which highlights based on user defined regexs in .conf file -- C
  • ConnectBot - Open source SSH client for Android -- Java Android
  • Guacamole - Clientless (in-browser) remote desktop gateway, supports VNC and RDP -- various lang
  • i2cssh - csshX like ssh tool for iTerm2 -- Ruby
  • iTerm2 - Killer terminal replacement for Mac -- OS X -- Objective-C
  • KiTTY - PuTTY fork with additional features -- Windows
  • MobaXterm - Tabbed SSH, VNC, and RDP client for Windows, free for personal use -- closed source Windows
  • mRemoteNG - Open source, tabbed, multi-protocol, remote connections manager -- Windows
  • MTPuTTY (Multi-Tabbed PuTTY) - Multiple PuTTY sessions in a single window -- Windows
  • NoVNC - Client-less VNC in a web browser, uses HTML5 and WebSockets -- in-browser
  • PuTTY - The classic SSH client -- Windows
  • PuTTYtray - PuTTY in the systray + additional features -- C
  • reptyr - Reparent a running program to a new terminal -- C
  • screen - Detatch and re-attach to shell sessions while they continue to run in the background -- C
  • SuperPutty - Allows the PuTTY SSH client to be opened in tabs -- Windows
  • tmate - Instant terminal sharing, a tmux fork -- OS X
  • tmux - Alternative to GNU screen, also used for terminal sharing -- C
  • tmux Resurrect - Persists tmux environment across system restarts -- shell
  • Warp - Securely share your terminal, like handing a co-worker your keyboard -- Golang
  • WinSCP - The well-known SCP client -- C Windows
  • x2go - One of the best remote-desktop-over-SSH clients -- Windows OS X
  • Xshell - SSH client for Windows, free for personal use -- closed source Windows

Tracing and Profiling

OS kernel and process-level tracing and profiling tools - see also Performance Analysis Tools and Network Performance Analysis Tools and Web and HTTP Performance Analysis Tools and Metrics and Time Series Data and RDBMS Performance Analysis Tools

  • bcc - Next generation, Linux 4.x kernel tracing tool suite, uses eBPF (Extended Berkeley Packet Filters) -- C
  • eBPF - DTrace + SystemTap, requires 4.x kernel, mostly uses bcc -- various lang
  • kdump - Linux kernel dump facility (where to save it, what to save, etc) -- C
  • Linux Trace Toolkit - Next Generation (LLTng) - Linux kernel tracer and profiler, lower overhead than System Tap -- C
  • lttng-analyses - Official collection of LLTNG scripts and snippets -- collection
  • Mastif Visualizer - Visualizer for the Valgrind's Mastiff utility -- C++
  • OpenSnoop - Continually monitor for file opens -- shell
  • OProfile - System-wide statistical profiling tool -- C
  • perf - User-land performance analysis tool, a sampling profiler -- C
  • perf-tools - Uses perf and ftrace, includes iosnoop -- Brendan Gregg
  • Record and Replay (rr) - Record the failure once, then debug the recording deterministically, supports C and C++, by Mozilla -- C C++
  • strace - System call tracer for user space processes -- built-in
  • SystemTap (stap) - Linux kernel tracing and performance analysis tool -- C C++
  • SysmonForLinux - Uses eBPF, can be used for SIEM or troubleshooting / debugging -- C
  • Valgrind - Tool suite that includes cache profilers, heap profiles, thread race condition checkers, and more - a CPU-level emulator -- C

Two Factor Authentication

Universal 2nd Factor (U2F) is the successor to two-factor authentication / TFA / 2FA and multi-factor authentication / MFA. Avoid using SMS if possible - see also VPNs and Tunnels and Secrets Management

  • Authelia - 2FA and SSO for your apps via Docker on Kubernetes, supports Yubikey, Google Authenticator, and e-mail based password reset -- Golang
  • CentOS 7 2FA VPN - VPN with CentOS 7 + FreeRADIUS + FreeIPA + Google Authenticator -- tutorial
  • FreeOTP - Open source fork of Google Authenticator, by Red Hat -- C mobile
  • Google Authenticator - Official project, supports Android, iOS, and has a PAM module for SSH 2FA -- C
  • Nitrokey - Open source thumb drive for authentication -- various lang
  • One Time Password - PAM module allowing single-use passwords to login to a system -- C
  • pam-u2f - PAM module for auth via U2F compatible devices, by YubiKey -- C
  • privacyIDEA - All in one solution for two-factor authentication across all your organization's devices such as OTP tokens, SMS, VPNs, SSH keys, Windows, keyfob, etc -- various lang
  • Titan Security Key - Physical key for FIDO 2FA, unphishable (really) -- physical
  • W3C's Web Authentication - Standards driven, supports various tokens and OTP generators -- specification
  • yubico-pam - PAM module for use with YubiKey devices -- C
  • yubikey-full-disk-encryption - Encrypt storage on a LUKS partition using a Yubikey -- Shell
  • YubiKey GPG & SSH Guide - Guide to using YubiKey as a SmartCard for GPG and SSH -- tutorial

Virtualization and SDN

OS, network, and storage virtualization, emulation, and simulation including SDN, load balancers, and firewalls - see also Containers for container-only network meshes and VMware Tools

  • BlueStacks - Android emulator that you can run on Windows, run apps from the Play store, etc -- various lang
  • Boxedwine - WINE that uses Emscripten (wasm and asm.js) to run in a browser -- various lang
  • BusyBox - Bootable Linux with tiny versions of many common UNIX utilities in a single small executable -- C
  • Calico - L3 fabric that runs a vRouter on each node, supports containers -- Python
  • DOSbox - Open source DOS emulator, great for running old games or utilities -- C C++
  • Firecracker - Micro-VM for serverless computing, by Amazon -- Rust
  • FreeNAS - BSD-based NAS, supports ZFS -- C
  • GitHub Load Balancer Director - Layer 4 load balancer -- C
  • GNS3 - Cisco and other network simulator that runs in VirtualBox or Qemu/KVM -- Python
  • HAproxy - Open source software load balancer -- C
  • haproxyctl - Wrapper to talk to the HAProxy socket, as well as regular init (start stop restart) shit -- Ruby gem
  • Katran - A high performance layer 4 load balancer library, by Facebook -- C++
  • KVM (Kernel Virtual Machine) - The one, the only -- C
  • kvm-tools - CLI tools for managing qemu-kvm domains -- C
  • KVM Management Tools - Great list of KVM management tools on the KVM wiki, updated frequently -- collection
  • libvirt - Open source API, daemon and management tool, used with many virtualization solutions -- C
  • Linux-IO Target (LIO) - Create and share iSCSI, Fibre Channel, FC over Ethernet, and other storage targets from Linux VMs -- built-in
  • LVS (Linux Virtual Server) - Linux-based load balancer, also includes the IPVS kernel module -- C
  • Mininet - Easily setup networks for testing on your laptop with VirutalBox -- various lang
  • Minio - An open source object storage server compatible with Amazon S3 APIs -- Golang
  • ns-3 - Network simulator, mostly focuses on wireless IP such as Wi-Fi, WiMAX, or LTE and routing protocols such as OLSR and AODV -- C
  • Open vSwitch (OVS) - Production quality software switch -- C
  • OpenFiler - Linux-based NAS, supports most protocols and storage types -- C
  • Oracle VM VirtualBox - Easy, simple virtualization -- C C++ Windows OS X
  • oVirt - Virtualization management platform, the upstream for Red Hat Enterprise Virtualization (RHEV) -- Java
  • Packet Tracer - Cisco network simulator, by Cisco -- closed source Windows OS-X
  • pfsense - The best firewall software, supports appliances and live CDs -- C
  • phpVirtualBox - Web front-end for VirtualBox -- PHP
  • QEMU (Quick EMUlator) - Also supports hardware emulation (SPARC, RISC, etc) -- C
  • Seesaw - Load balancer based on Linux Virtual Server (LVS), by Google -- Golang
  • Shadow - Network simulator that runs real applications like Tor and distributed systems of thousands of nodes on a single machine -- C
  • Squid - Reverse proxy, caching server, web traffic filter, and more -- C++
  • Traefik - Modern HTTP reverse proxy and load balancer, supports many backends -- Golang
  • Unicorn Engine - CPU emulator framework (ARM, AArch64, M68K, Mips, Sparc, X86) via QEMU -- various lang
  • virt-manager - KVM / Xen / LXC GUI -- Python
  • WANem (Wide Area Network Emulator) - WAN emulator -- C
  • WINE - Compatibility layer for running Windows apps on POSIX-compliant OSes -- C OS X
  • Xen - It's a hypervisor! -- C
  • XenServer - Successor to Xen Cloud Platform (XCP) -- C
  • xhyve - Lightweight OS X virtualization based on bhyve -- C OS X
  • ZeroTier - Cloud / provider-agnostic private network backplanes, network virtualization -- C++ all platforms

VMware Tools

Mostly unofficial VMware tools and scripts - see also Virtualization and SDN

  • Compliance Checker for vSphere - Provides detailed compliance checks against vSphere hardening guidelines, official tool -- closed source
  • FastSCP - Super fast SCP client for Windows for moving stuff around in a VMware environment (ISOs, VMDKs between data stores, etc) -- closed source
  • Flings - Tons of very useful apps written by VMware engineers, worth exploring A++++ -- closed source? various lang
  • Onyx - Do something in vSphere and it will create a PowerCLI script to do that thing that you just did -- closed source PowerShell Windows
  • open-vm-tools - OSS alternative to VMware Guest Tools -- C
  • PowerCLI - VMware's CLI -- closed source
  • pyrvtools - Extract useful information from an RVTools ESX inventory file -- Python pip
  • RVtools - Killer GUI for viewing information about your VMware environment -- closed source .NET Windows
  • RVtools Export - Wrapper script to save daily snapshots of your environment using RVtools -- PowerShell
  • Sexigraph - Graphite-based visualization appliance for Sexilog -- various lang
  • Sexilog - ELK stack virtual appliance designed for vSphere / VMware ESXi logs -- various lang
  • vCheck - Get an overview of a new environment or check the health of an existing one -- PowerShell
  • vDisk Informer - Check vDisks alignment and see if they have wasted space -- closed source Windows
  • vGhetto Script Repository - Various scripts from virtuallyGhetto -- various lang
  • VMware Community PowerPack - A variety of scripts from Virtu-al.net and now other blogs such as ict-freak.nl and ntpro.nl -- lost-to-the-internet PowerShell
  • VMware on Github - VMware's Github repos -- various lang
  • VMware Sample Exchange - Code samples & scripts by both VMware and the community -- various lang
  • vSphere Health Check Report - Reports a massive amount of information, run it on a schedule -- closed source perl
  • vSphere Mobile Watchlist - Monitor VMs on your phone -- closed source mobile

VPNs and Tunnels

VPN client & servers, tunneling proxies, DNS tunneling, simple host-to-host tunnels, and related tools - see also SSL Tools and SSH Tools and Virtualization and SDN

  • Algo VPN - IKEv2 Ubuntu-based strongSwan VPN server deployed to any cloud via Ansible -- Python
  • AutoVPN - Spin up and autoconfigure OpenVPN instances in AWS -- Python
  • BrowserLeaks - See if info is leaking from your browser (your real IP when you're behind a VPN, etc) -- in-browser
  • CentOS 7 2FA VPN - VPN with CentOS 7 + FreeRADIUS + SSSD + Google Authenticator -- stepbystep
  • Corkscrew - Tunnel SSH through HTTP proxies -- C
  • DNS Leak Test - See if DNS queries are leaking outside of your VPN / secured network -- in-browser
  • dnscrypt-proxy - DNS proxy, with support for modern encrypted DNS protocols such as DNSCrypt v2 and DNS-over-HTTP/2 -- Golang
  • dnscrypt-wrapper - Add dnscrypt support to any resolver, server-side dnscrypt proxy -- C
  • fwknop (FireWall KNock OPerator) - Single Packet Authorization (SPA), authoriation packet from you opens firewall rules so only you can get in -- various lang
  • IP Leak - Test to see if your browser is leaking information -- in-browser
  • kcptun - Secure and fast tunnel based on KCP, can increase throughput -- Golang
  • localtunnel - Share a HTTP/web service on your local development machine without messing with DNS and firewall settings -- JavaScript Windows OS X
  • MACsec aka 802.1AE - Use me for confidentiality and integrity at layer 2, useful for WAN links -- standard
  • Magic Wormhole - Safely and simply send arbitrary-sized files and directories (or short pieces of text) from one computer to another -- Python
  • n2n - L2 over L3 VPN that uses a peer-to-peer architecture -- C
  • nipe - A script to make Tor Network your default gateway -- perl BOOYA
  • OpenConnect - Supports Cisco's AnyConnect SSL VPN -- C
  • OpenVPN - The one and only -- C
  • Pritunl - Distributed enterprise VPN server built using the OpenVPN protocol, supports Google sign-in -- Python
  • Project V - A set of network tools that helps you to build your own computer network. It secures your network connections and thus protects your privacy, a newer Shadowsocks -- Golang
  • Shadowsocks - A secure socks5 proxy, designed to protect your Internet traffic -- various lang most platforms
  • SoftEther - Perhaps the best VPN software out there -- C
  • spiped - Create encrypted pipes between socket addresses using pre-shared keys (PSKs), similar to ssh -L -- C
  • sshttp - Port multiplexer that hides a SSH daemon behind HTTP, HTTPS, or SMTP on a single port -- C
  • sshuttle - Transparent proxy server / VPN, doesn't need admin, forwards over SSH, supports DNS tunneling -- Python OS X
  • SSH Through or Over Proxy - How to create a HTTP(s) tunnel for your SSH traffic -- article
  • Streisand - Set up a server running a wide variety of privacy software, so easy that grandma can use it, Ansible-based -- Python
  • strongSwan - IPsec-based -- C
  • stunnel - Create simple TLS tunnels for existing services (eg: telnet, nc, etc) -- C
  • tinc - Simple, multi-platform VPN -- C
  • WireGuard - Performant in-kernel VPN server with DJB Crypto and very modern primitives -- C
  • Ubuntu IKEv2 VPN Setup - Simple script that sets up a IKEv2 VPN with strongSwan on Ubuntu -- shell

Web and HTTP Tools

RPC is dead, long live RPC - see also SSL Tools and Networking Tools

  • API Blueprint - Supports bindings/plugins for many APIs -- various lang
  • Caddy - Web server with automatic TLS and more -- Golang
  • Fiddler - HTTP/HTTPS debugging proxy for any browser, system or platform -- closed source
  • gRPC - High performance RPC via HTTP/2, by Google -- C
  • gRPC-Gateway - gRPC to JSON proxy generator following the gRPC HTTP spec -- Golang
  • gRCPCurl - curl for gRPC servers -- Golang
  • h2i - Go's interactive HTTP/2 console debugger, send raw frames, etc -- Golang
  • Hamms - Simulate/create connection failures, malformed response data, slow servers, fat headers, and more! -- Python
  • htaccess Snippits - Huge collection of common and useful .htaccess snippets, please contribute -- collection
  • http-traceroute - Shows the entire route including cookies, redirects, and response codes -- Ruby
  • httpdiff - Perform the same reuqest against two HTTP servers and diff the results -- Golang
  • httpie - curl replacement with many new features -- Python
  • httptoolkit - Open-source HTTP(S) debugging proxy, analyzer & client -- Typescript
  • htty - A console application for interacting with web servers -- Ruby gem
  • Huginn - Build agents that perform automated tasks/workflows for you online or locally, like IFTTT -- Ruby
  • Insomnia REST Client - An alternative to POSTman, additional features -- Electron Windows OS X Linux
  • Ionic Framework - Mobile UI kit that allows you to write multi-platform mobile apps in JS, Angular, Vue, or React - no Swift or Android knowledge needed -- various lang
  • JSON Server - Full fake REST API for quickly prototyping and mocking in 30 seconds -- JavaScript
  • jq - Command line JSON processor and manipulator -- C
  • localtunnel - Share a HTTP/web service on your local development machine without messing with DNS and firewall settings -- JavaScript Windows OS X
  • mitmproxy - Intercept, modify, replay and save HTTP/S traffic - even edit flows on the fly -- Python pip
  • mountebank - Stub downstream resources for testing, supports HTTP HTTPS SMTP TCP -- JavaScript Windows OS X
  • Newman - CLI companion for Postman -- JavaScript
  • nghttp - CLI HTTP/2 client, similar to curl/wget and more -- C
  • nginxconfig - Web-based nginx config generator -- in-browser
  • ngrok - Reverse proxy that creates a secure tunnel from a public endpoint to a locally running web service and captures & analyzes all traffic over the tunnel for later inspection and replay -- rocks Golang
  • OpenResty - Turn nginx into a non-blocking API server -- C
  • PageKite - Makes HTTP servers or SSH publicly available on any server -- Python
  • POSTman - Create and share API and HTTP requests, great for testing and sharing -- Chrome
  • Puppeteer - Provides a high-level API to control headless Chrome over the DevTools Protocol -- JavaScript
  • Pushpin - Proxy server that adds WebSockets to existing request-response APIs -- C++
  • react-admin - Add an React admin GUI to any RESTful API -- JavaScript
  • Redirect Detective - See the complete path a redirected URL goes through -- in-browser
  • Repose - REST proxy, solutions to API tasks such as auth, rate limiting, API validation, HTTP logging, and much more -- Java
  • Resty - CLI REST client you can use in shell/bash/zsh pipes -- shell
  • SecurityHeaders.io - Scan your web site's HTTP headers -- in-browser
  • Swagger - Popular API development library, now the OpenAPI standard -- various lang
  • tengine - A distribution of Nginx with some advanced features, by Alibaba -- C
  • tortilla - Easily wrap web APIs -- Python pip
  • Tyk - API gateway -- Golang
  • urlscan.io - Displays tons of stats and info about any given URL -- in-browser
  • Varnish - Caching HTTP accelerator -- C
  • vcr - Record and play back HTTP sessions -- Ruby gem
  • webhook - Super simple webhook server -- Golang
  • wuzz - Interactive cli tool for HTTP inspection (menu-based) -- Golang

Web and HTTP Performance Analysis Tools

Load generation, debugging, benchmarking, and profiling for SaaS, webapps, and HTTP(S) - see also Performance Analysis Tools and Network Performance Analysis Tools

  • Awesome Web Performance Optimization - Collection of web performance optimization (WPO) tools, articles, and more -- collection
  • Betwixt - Web debugging proxy with a Chrome DevTools look -- JavaScript
  • Brotli - Modern alternative to gzip, better packing, performs well with HTTP/2, HTTPS only -- zoooooom
  • django-debug-toolbar - Panels that display profiling information about the current request/response -- Python pip
  • Chrome DevTools - Many built-in tools for performance analysis -- mostly OSS
  • Gatling - HTTP, JMS, and WebSocket load generator -- Scala
  • Google Web Tracer - Helps you identify and fix performance problems in your web applications, by Google -- Chrome
  • Firefox Developer Tools - A full list of built-in Firefox developer tools including performance tools -- various lang
  • h2load - HTTP/2 and SPDY load generation tool, part of the nghttp2 suite -- C
  • HAR Analyzer - HTTP Archive analyzier for troubleshooting from a browser perspective, by Google -- in-browser
  • httping - Simple program that "pings" a URL and shows response time -- C
  • Jaeger - OpenTelemetry compatible distributed tracing system, works well with Istio and Envoy, by Uber -- Golang
  • JMeter - Designed to load test functional behavior and measure performance, written in Java -- Java
  • Locust - Load generation tool written in Python that allows you to define user behavior -- Python
  • ngxtop - Real time top for nginx -- Python
  • OpenTelemetry - Vendor-neutral distributed tracing, a merger of OpenTracing and OpenCensus -- various lang
  • OpenZipkin - Distributed tracing systems for SaaS and webapps, by Twitter, based on Google's Dapper -- Scala
  • Packetbeat - Distributed packet monitoring system that can be used for application performance management -- Golang
  • PageSpeed Insights - Analyzes the content of a web page, then generates suggestions to make that page faster, by Google -- in-browser
  • PageSpeed Module - Open-source server modules that optimize your site automatically (nginx and Apache), by Google -- various lang
  • peep - Heap inspector for live memcached instances -- Ruby
  • redis-faina - Query analyzer that parses Redis' MONITOR command for counter/timing stats about query patterns, by Facebook -- Python
  • Tempo - Cost-efficient, requiring only object storage to operate, and is deeply integrated with Grafana, Prometheus, and Loki and OpenTelemetry -- various lang
  • Tsung - Distributed stress tester, also supports stress testing DBs -- Erlang
  • Web Page Test - Free website speed test from multiple locations around the globe using real browsers (IE and Chrome) and at real consumer connection speeds -- in-browser
  • wrk - Multi-threaded CLI-based HTTP load generation tool -- C
  • wrk2 - Fork of wrk that fixes the "coordinated omissions problem" -- C
  • UpTrends Uptime Checker - Check a URL's response time from ~30 different sites around the globe -- in-browser
  • Varnish Dashboard - Realtime dashboard for Varnish cache servers -- JavaScript

Misc Tools of Note

Everything else

  • Adblock Radio - An adblocker for live radio streams and podcasts -- various lang
  • Alma Linux - A replacement for CentOS which was discontinued in Dec 2020 by IBM in favor of CentOS Stream -- operating system
  • Animated Knots - For business or pleasure -- collection
  • Anti-Ablock Killer - Keep your ad-blocker active when pages take anti-blocking measures, requires Greasemonkey or similar -- JavaScript
  • Annie - Fast and simple video download library and CLI tool written in Go -- Golang
  • AppLock - Lock individual apps on Android -- Java
  • Architecture Decision Record - Document that captures an important architectural decision made along with its context and consequences -- template
  • Archive.org aka The Internet Archive aka The Wayback Machine - Yo donate, this is beyond important -- the biggest collection
  • AutoHotKey - Automate input (desktop and web forms, data entry, keybinds, etc) -- C++ Windows
  • Awesome AutoHotKey - A collection of AutoHotKey scripts and libraries -- various lang
  • AutoIT - Scriptable GUI input for Windows (think AutoHotKey) but with a BASIC-like syntax -- closed source
  • Backstage - Unifies all your infrastructure tooling, services, and documentation with a single, consistent UI so developers can easily provision and view their resources -- various lang
  • Bypass Paywalls - Shhh -- Firefox Chrome
  • Bypass Paywalls - Better maintained but Chrome only -- Chrome
  • c4builder - Lightweight nodejs cli tool for building, maintaining and sharing a software architecture project using only text, combines PlantUML C4Model, Markdown, and more -- various lang
  • CheapCharts - Historic price trends for digital movies and TV on iTunes, Vudu, Amazon Prime, and similar -- collection
  • Clone Wars - Open-source clones of popular sites like Airbnb, Amazon, Instagram, Netflix, Tiktok, Spotify, Trello, Whatsapp, Youtube, etc -- various lang
  • conserver - Virtual console server with many features (multi-user, console log history, etc) -- C
  • Container Proxy - Set up proxies and then assign proxy to a container. Each request comming from any tab in the container will use this proxy -- Firefox
  • Container Tabs Sidebar - Show tabs in a sidebar grouped by privacy containers -- Firefox
  • Cookie Autodelete - Browser add-on that's essentially a cookie whitelist / grey list -- Firefox Chrome
  • CRIU (Checkpoint Restore in Userspace) - Freeze a process, save it to disk, then resume it later -- insane
  • crumbs - Simple mind maps with asterisks - great for brain dumping with little transcription overhead -- Golang
  • Dashkiosk - Manage dashboards on multiple screens, simple and effective -- JavaScript
  • DevOps Conferences - Add and remove via PR, please contribute -- collection
  • Discount for Student Dev - Programming and DevOps related discounts for .edu addresses, updated frequently, please contribute -- collection
  • Downtify - Open source Spotify downloader -- various lang
  • Etcher - SD card writing software, easy -- various lang
  • Exploit Database - Collection of current and past exploits -- collection
  • f.lux - Changes your screen from blue light to yellow light when the sun sets to tell your brain it's night time -- closed source OS X Windows
  • fast-data-dev - Docker image for Kafka developement that includes a boatload of related tools -- various lang
  • FigmaToCode - Outputs Figma objects to Tailwind, Flutter, or SwiftUI code -- JavaScript
  • Firefox Multi-Account Containers - Cookies are separated by container, allowing you to use the web with multiple identities or accounts simultaneously -- Firefox
  • Fossor - Automate on-call investigation steps, by LinkedIn -- Python pip
  • free-for.dev - Massive list of services with free tiers -- collection
  • Geocities-izer - Make any HTML web page look like a Geocities page -- sweeeeeeet
  • GlossaryTech - Browser addon to learn tech terms with easy to understand defintions -- JavaScript
  • googler - Google search, news, and site search from the terminal, slick -- Python
  • Google Advanced Operators for Web Search - Search modifiers for more accurate results, by Google -- collection
  • GoodRX - Coupons for prescriptions, vaccines, and more, can be used pre-deductible, USA-centric -- ez savings
  • Google Search Verbatim Mode - Searches for exactly what you want, no "intelligence" added -- article
  • gosu - Simple Go-based setuid+setgid+setgroups+exec for stepping down privs, use me to avoid weird su and sudo TTY bugs -- Golang
  • Highlight This - Multi-highlight, regex, all data local, tons of features -- Chrome
  • Homomorphic Encryption - Allows data to be encrypted and out-sourced to 3rd parties for processing, all while encrypted (or remove HIPAA barriers) -- article
  • Hoopla - Digital movies, music and ebooks for USA public library members -- explore yours today
  • HTML5 Up - HTML5 website templates all free and under a CC license -- collection
  • HTTPS Everywhere - Uses client-side tricks to force misconfigured sites to use HTTPS all the time, by the EFF (donate) -- Firefox Chrome
  • Huginn - Build agents that perform automated tasks/workflows for you online or locally, like IFTTT -- Ruby
  • ICANN Domain Lookup - Use this when searching for a domain to buy because they cannot use your searches to jack up the price -- in-browser
  • inxi - "a full featured system information script" (hardware info, etc) -- shell
  • IPTV - Collection of 8000+ publicly available IPTV channels from all over the world -- screw off Bundesliga -- collection
  • Jellyfin - OSS alternative to Plex -- various lang
  • JustWatch - Search for movies and TV across dozens of streaming services, it shows you which services it is available on, and you can click to watch, app and in-browser -- collection
  • Kanopy - Streaming movies for USA public library members -- explore yours today
  • Kodi - Open source home theater, run me on a Pi, think open source Fire Stick / Roku / Apple TV / TV app store / etc -- various lang
  • L3AF - Complete lifecycle management of eBPF programs in the kernel, by Wal-Mart -- various lang
  • Libby - App for ebooks and audio books for USA public library members, from OverDrive -- explore yours today Android iOS Windows 10
  • Library Extension - Browser extension that shows you which Amazon books are available free at your local libraries, please donate -- closed source
  • Mail Tester - Test the spammyness of your e-mails -- in-browser
  • maybe - Allows a dry run of almost any Linux binary, see the files it will modify, calls made, etc -- Python pip
  • Maza - Like Pi-hole but local and using your operating system -- shell
  • Mjolnir - Automation for OS X via Lua, think AutoHotKey for Mac -- C OS X
  • MultiRBL - SMTP blackhole lookup tools -- in-browser
  • MX Toolbox - SMTP blacklist lookup, header analysis, and more -- in-browser
  • Network UPS Tools (NUT) - Manage power devices from over 100 manufacturers using a single web interface -- C Windows
  • NoRoot Firewall - Android app that logs connections to show overly chatty background apps and more -- closed source
  • NoScript - JavaScript, XSS, tracker, Flash blockers and more - your condom for the Internet -- Firefox
  • OISD Domain Block List - The only Pi-Hole and domain blocklist you'll need -- collection
  • Omni Calculator - 1500+ free calculators -- collection
  • oomd - Userspace OOM killer, highly configurable and much improved over the kernel's built in OOM killer -- C++
  • Open19 - Open data center hardware for standard size racks -- neat
  • OpenBMC - Framework to build a complete Linux image for a Board Management Controller (BMC), by Facebook -- C
  • OpenBoard - Open source keyboard for Android that respects your privacy -- Android
  • OpenHaystack - Framework for tracking personal Bluetooth devices via Apple's massive Find My network, build your own Airtags -- various lang
  • OverDrive - Ebooks and audio books for USA public library members -- explore yours today
  • PeerTube - P2P version of YouTube, don't let your videos get taken down, now supports live streaming -- free as in freedom
  • Pi-Hole - Ad and tracking blackhole that covers your entire network with a single device -- hardware
  • PineTime - An open source smartwatch -- hardware
  • Popcorn Time - Multi-platform, free software BitTorrent client that includes an integrated media player -- various lang
  • priceMedic - Compare procedures and services at hospitals near you, currently in beta -- collection
  • Privacy Badger - The best in-browser privacy and anti-tracking tool, by the EFF (donate) -- Chrome Firefox
  • PrivacyTools.io - All in one collection, web site source is on GitHub -- collection
  • Puffer - Re-transmits free over-the-air broadcast television signals received by an antenna located on the campus of Stanford University, USA geo-locked, by Stanford Platform Lab and many others -- in-browser Android
  • Redfish - An industry standard protocol providing a RESTful interface for the management of servers, storage, networking, and converged infrastructure -- article
  • repl.it - In-browser REPLs for a ton of languages -- in-browser
  • RISC-V - The open source CPU that can run at >5Ghz while using 1/100th the power of a Xeon E7 -- hardware
  • Rocky Linux - A replacement for CentOS which was discontinued in Dec 2020 by IBM in favor of CentOS Stream -- operating system
  • RSS-Bridge - Generate RSS and Atom feeds for websites that don't have one -- PHP
  • runwhen - Utilities for running commands at particular times, cron on steroids -- C
  • Say What - Using speech-to-text to fully check out during conference calls -- Python
  • scrcpy - Remote display and control of Android devices connected on USB or TCP/IP -- C
  • Secure Messaging Apps - True comparison of the technical features and merits of various apps, which are TRULY secure? -- collection
  • SessionBox - Use websites with multiple accounts at the same time made easy -- Chrome
  • Spectacle - Control desktop windows via keyboard shortcuts -- Objective C
  • SponsorBlock - Skip sponsorships, subscription begging and more on YouTube videos. Report sponsors on videos you watch to save others' time -- Chrome
  • StackStorm - IFTTT for Ops, auto-remediation -- Python
  • Stack on a Budget - Collection of services with great free tiers for developers on a budget, excellent for learning -- collection
  • Steps Recorder (aka Problem Steps Recorder) - Windows 7+, user records themselves reproducing a problem, they send video to you, you playback -- closed source
  • SymbolHound - Search engine that doesn't ignore special characters, great for programming questions -- try it
  • Temporary Containers for Firefox - Open tabs, websites, and links in automatically managed disposable containers which isolate the data websites store (cookies, storage, and more) from each other, enhancing your privacy and security while you browse -- addon
  • Twilight - F.lux for Android -- closed source
  • Two Factor Auth (2FA) - Web sites that do and don't support 2FA, organized by category, submit PRs for changes -- collection
  • uBlock Origin - Ad blocker, only use Origin, do not use another uBlock -- Firefox Chrome
  • unPaywall - When you view a paywalled scholarly article, Unpaywall automatically checks its open database of 28 million legal, open-access articles and tells you if you can get it elsewhere for free -- JavaScript
  • WhosHere - Monitor an area for WiFi and Bluetooth probe requests to see when people (devices) come and go, with web UI and IFTTT webhook integraiton for Slack/SMS pushes -- PHP
  • yt-dlp - A youtube-dl fork with additional features and fixes -- various lang

Learning Resources

Free learning resources and collections of note (DevOps/SRE, cloud, information security, Kubernetes, Docker/containers, Python, Golang)

my-links's People

Contributors

cyrilbois avatar da5209e7f827 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.