aws / homebrew-aws

Homebrew is a package manager for macOS which provides easy installation and update management of additional software. This Tap (repository) contains the Formulae that are used in the macOS AMI that AWS offers.

EC2 macOS Homebrew Tap

AWS offers EC2-optimized macOS AMIs for developer use. The EC2 Mac AMIs provided by AWS include this Tap by default, which provides a simple way to get these tools and updates to them.

What is Homebrew?

Homebrew is a package manager for macOS, which provides easy installation and update management of additional software.

What is a Tap?

A third-party (in relation to Homebrew) repository, which provides installable packages on macOS.

See more at https://docs.brew.sh/Taps.

What packages are available in this Tap?

This Tap (repository) contains the formulae that are used in the macOS AMI(s) that are offered by AWS for launching EC2 Mac instances.

This includes:

Name	Description	Type	Package Name
Amazon ENA	EC2 Mac ENA Network Driver for macOS	Cask	amazon-ena-ethernet
Amazon ENA DriverKit	EC2 Mac ENA Network Driver for macOS Monterey and later (arm64 only)	Cask	amazon-ena-ethernet-dext
Amazon EFS	Amazon Elastic File System	Keg	amazon-efs-utils
Amazon SSM Agent	Amazon SSM Agent	Cask	amazon-ssm-agent
EC2 macOS Init	Instance initialization and configuration, including performance optimization	Cask	ec2-macos-init
EC2 System Monitor for macOS	For collecting system monitoring CloudWatch metrics on mac1.metal instances	Cask	ec2-macos-system-monitoring
EC2 macOS Utils	Provides common command-line tool for customizing EC2 Mac instances	Cask	ec2-macos-utils
EC2 Instance Connect	EC2 Instance Connect	Cask	ec2-instance-connect

How do I install the packages from this Tap?

This Tap follows standard Homebrew commands, for more information, please refer to the Homebrew Documentation. Updating homebrew itself is done via the following command:

brew update

The above command will update homebrew itself and this comamnd should always be run prior to attempts to install or update any packages with brew.

Installing the Tap

The macOS AMIs provided by AWS already come with Homebrew installed and this Tap "pretapped". To manually install the Tap, use the the brew tap command:

brew tap aws/homebrew-aws

Removing the Tap

If this Tap needs to be removed for any reason, Homebrew includes a specific command for this. (Note: This will only remove the Tap and it will preserve anything that was previously installed)

brew untap aws/homebrew-aws

Updating Packages

To update a specific package (Keg or Cask) that has been installed from this tap. Kegs use the default brew upgrade <package> command, while Casks have their own sub-command: brew upgrade --cask <package>.

For example:

Type	Update Command
Full System	brew upgrade
Keg	brew upgrade amazon-efs-utils
Cask	brew upgrade --cask amazon-ena-ethernet-dext

Installing Packages

There are two primary ways to install software from the Tap. Kegs use the default brew install <package> command, while Casks have their own sub-command: brew install --cask <package>.

For example:

Type	Install Command
Keg	brew install amazon-efs-utils
Cask	brew install --cask amazon-ena-ethernet-dext

Removing Packages

Removing software is similar to installing software. Kegs and Casks now use the same brew remove <package> command.

For example:

Type	Uninstall Command
Keg	brew remove amazon-efs-utils
Cask	brew remove amazon-ssm-agent

Documentation

To get more information about brew you can run brew help or man brew on a mac1.metal instance or check Homebrew's documentation for Homebrew's complete documentation.

License

This project is licensed under the Apache License, version 2.0.

homebrew-aws's Issues

Package Request: Unified CloudWatch Agent

This is a request for the Unified CloudWatch Agent

• https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/download-cloudwatch-agent-commandline.html

• https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/install-CloudWatch-Agent-commandline-fleet.html

x86_64

curl -sLO 'https://s3.amazonaws.com/amazoncloudwatch-agent/darwin/amd64/latest/amazon-cloudwatch-agent.pkg'
sudo installer -verbose -pkg amazon-cloudwatch-agent.pkg -target /

aarch64

curl -sL 'https://s3.amazonaws.com/amazoncloudwatch-agent/darwin/arm64/latest/amazon-cloudwatch-agent.pkg' -O
sudo installer -verbose -pkg amazon-cloudwatch-agent.pkg -target /

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard

cat /opt/aws/amazon-cloudwatch-agent/bin/config.json

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json

# To check the status
/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a status

# To view the logs
tail -f /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log

Amazon Managed Roles:

CloudWatchAgentServerRole

OR

CloudWatchFullAccess

Creating a custom policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "cloudwatch:PutMetricData",
                "ec2:DescribeVolumes",
                "ec2:DescribeTags",
                "logs:PutLogEvents",
                "logs:DescribeLogStreams",
                "logs:DescribeLogGroups",
                "logs:CreateLogStream",
                "logs:CreateLogGroup"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ssm:GetParameter"
            ],
            "Resource": "arn:aws:ssm:*:*:parameter/AmazonCloudWatch-*"
        }
    ]
}

Example configuration:

{
 "agent": {
  "metrics_collection_interval": 60,
  "run_as_user": "root"
 },
 "metrics": {
  "aggregation_dimensions": [
   [
    "InstanceId"
   ]
  ],
  "append_dimensions": {
   "AutoScalingGroupName": "${aws:AutoScalingGroupName}",
   "ImageId": "${aws:ImageId}",
   "InstanceId": "${aws:InstanceId}",
   "InstanceType": "${aws:InstanceType}"
  },
  "metrics_collected": {
   "collectd": {
    "metrics_aggregation_interval": 60
   },
   "cpu": {
    "measurement": [
     "cpu_usage_idle",
     "cpu_usage_iowait",
     "cpu_usage_user",
     "cpu_usage_system"
    ],
    "metrics_collection_interval": 60,
    "resources": [
     "*"
    ],
    "totalcpu": false
   },
   "disk": {
    "measurement": [
     "used_percent",
     "inodes_free"
    ],
    "metrics_collection_interval": 60,
    "resources": [
     "*"
    ]
   },
   "diskio": {
    "measurement": [
     "io_time",
     "write_bytes",
     "read_bytes",
     "writes",
     "reads"
    ],
    "metrics_collection_interval": 60,
    "resources": [
     "*"
    ]
   },
   "mem": {
    "measurement": [
     "mem_used_percent"
    ],
    "metrics_collection_interval": 60
   },
   "netstat": {
    "measurement": [
     "tcp_established",
     "tcp_time_wait"
    ],
    "metrics_collection_interval": 60
   },
   "statsd": {
    "metrics_aggregation_interval": 60,
    "metrics_collection_interval": 10,
    "service_address": ":8125"
   },
   "swap": {
    "measurement": [
     "swap_used_percent"
    ],
    "metrics_collection_interval": 60
   }
  }
 }
}

The amazon-efs-utils formula is incompatible imcompatible

Expected Behavior

An execution of brew install amazon-efs-utils successfully install amazon-efs-utils.

Current Behavior

The installation of amazon-efs-utils is failing at docutils-0.15.2 build after upgrading Homebrew's [email protected] from version 3.9.9 to 3.9.10.

ec2-user@ip-172-31-52-246 ~ % /usr/local/bin/python3 -V
Python 3.9.10
ec2-user@ip-172-31-52-246 ~ % brew install amazon-efs-utils
(....)
==> /usr/local/Cellar/amazon-efs-utils/1.30.2/libexec/bin/pip install -v --no-deps --no-binary :all: --ignore-installed /private/tmp/amazon-efs-utils--docutils-20220120-59577-1y9ckfz/docutils-0.15.2
Last 15 lines from /Users/ec2-user/Library/Logs/Homebrew/amazon-efs-utils/07.pip:
:all:
--ignore-installed
/private/tmp/amazon-efs-utils--docutils-20220120-59577-1y9ckfz/docutils-0.15.2

Using pip 21.3.1 from /usr/local/Cellar/amazon-efs-utils/1.30.2/libexec/lib/python3.9/site-packages/pip (python 3.9)
Processing /private/tmp/amazon-efs-utils--docutils-20220120-59577-1y9ckfz/docutils-0.15.2
  Preparing metadata (setup.py): started
  Running command python setup.py egg_info
  Error: The "distutils" standard module, which is required for the
  installation of Docutils, could not be found.  You may need to
  install a package called "python-devel" (or similar) on your
  system using your package manager.
  Preparing metadata (setup.py): finished with status 'error'
WARNING: Discarding file:///private/tmp/amazon-efs-utils--docutils-20220120-59577-1y9ckfz/docutils-0.15.2. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

If reporting this issue please do so to (not Homebrew/brew or Homebrew/core):
  aws/aws

Steps to Reproduce

Run the following commands on an EC2 Mac instance with the Big Sur 11.6.2 AMI.

brew upgrade [email protected]
brew install amazon-efs-utils

Possible Solution

The newer versions of docutils don't have the problematic class imports in setup.py. Upgrading docutils would be one option.

Enforcing to use stdlibs distutils could also work.

Some troubleshooting info

It seems because of the upgrade of setuptools at the homebrew-core repository commit for [email protected]. The setup.py in the above docutils-0.15.2 include some class imports from distutils, and the classes are included in stdlib's distutils but setuptools' local distutils. The difference was introduced at a commit of setuptools which caused a difference of /usr/local/lib/python3.9/site-packages/distutils-precedence.pth from

import os; var = 'SETUPTOOLS_USE_DISTUTILS'; enabled = os.environ.get(var, 'stdlib') == 'local'; enabled and __import__('_distutils_hack').add_shim();

to

import os; var = 'SETUPTOOLS_USE_DISTUTILS'; enabled = os.environ.get(var, 'local') == 'local'; enabled and __import__('_distutils_hack').add_shim();

If I use stdlib, I can successfully import the class.

ec2-user@ip-172-31-63-81 ~ % python3
Python 3.9.10 (main, Jan 20 2022, 08:53:44)
[Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils.command.build_py import build_py_2to3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'build_py_2to3' from 'distutils.command.build_py' (/usr/local/lib/python3.9/site-packages/setuptools/_distutils/command/build_py.py)
>>>
ec2-user@ip-172-31-63-81 ~ % SETUPTOOLS_USE_DISTUTILS=stdlib python3
Python 3.9.10 (main, Jan 20 2022, 08:53:44)
[Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils.command.build_py import build_py_2to3
>>>

v1.35.1 not available, blocking usage of amazon-efs-utils on Sonoma

aws/efs-utils#191

What could we do to make progress on publishing 1.35.1 and not be blocked by

ec2-user@ip-172-31-23-112 ~ % sudo mount -t efs fs-016ae2b9856e2a054 efs
We do not support EFS on MacOS Kernel version 23.3.0
mount: /Users/ec2-user/efs failed with 1

The amazon-efs-utils formulae post install steps is the wrong launchd Job

The amazon-efs-utils formulae provides additional steps on post install. Currently the user is supposed to copy the amazon-efs-mount-watchdog.plist to the /Library/LaunchAgents directory.

issue 1

This should be updated to refer to /Library/LaunchDaemons as LaunchAgents are intended for per-user agents that run on behalf of a user when they they login to the system. LaunchDaemons will run on system startup and without a user logged in to the system.

issue 2

The launchctl load .. and launchctl unload .. commands are considered deprecated and should be updated.

Current:

To enable watchdog for TLS mounts:
  sudo cp #{libexec}/amazon-efs-mount-watchdog.plist /Library/LaunchAgents
  sudo launchctl load /Library/LaunchAgents/amazon-efs-mount-watchdog.plist

To disable watchdog for TLS mounts:
  sudo launchctl unload /Library/LaunchAgents/amazon-efs-mount-watchdog.plist

Updated

To enable watchdog for TLS mounts:
  sudo cp #{libexec}/amazon-efs-mount-watchdog.plist /Library/LaunchDaemons
  sudo launchctl bootstrap system /Library/LaunchDaemons/amazon-efs-mount-watchdog.plist

To disable watchdog for TLS mounts:
  sudo launchctl bootout system /Library/LaunchDaemons/amazon-efs-mount-watchdog.plist

References:

• https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/Introduction.html
• https://developer.apple.com/library/archive/technotes/tn2083/_index.html
• https://wiki.freepascal.org/macOS_daemons_and_agents