geerlingguy / ansible-role-glusterfs

Ansible Role - GlusterFS

Ansible Role: GlusterFS

Installs and configures GlusterFS on Linux.

Requirements

For GlusterFS to connect between servers, TCP ports 24007, 24008, and 24009/49152+ (that port, plus an additional incremented port for each additional server in the cluster; the latter if GlusterFS is version 3.4+), and TCP/UDP port 111 must be open. You can open these using whatever firewall you wish (this can easily be configured using the geerlingguy.firewall role).

This role performs basic installation and setup of Gluster, but it does not configure or mount bricks (volumes), since that step is easier to do in a series of plays in your own playbook. Ansible 1.9+ includes the gluster_volume module to ease the management of Gluster volumes.

Role Variables

Available variables are listed below, along with default values (see defaults/main.yml):

glusterfs_default_release: ""

You can specify a default_release for apt on Debian/Ubuntu by overriding this variable. This is helpful if you need a different package or version for the main GlusterFS packages (e.g. GlusterFS 3.5.x instead of 3.2.x with the wheezy-backports default release on Debian Wheezy).

glusterfs_ppa_use: true
glusterfs_ppa_version: "LATEST"

For Ubuntu, specify whether to use the official Gluster PPA, and which version of the PPA to use. See Gluster's Getting Started Guide for more info.

glusterfs_gpg_key_version: "7"
glusterfs_deb_version: "LATEST"

For Debian, specify the version of the GPG key and apt package repository to use. See Gluster's Getting Started Guide for more info.

Dependencies

None.

Example Playbook

- hosts: server
  roles:
    - geerlingguy.glusterfs

For a real-world use example, read through Simple GlusterFS Setup with Ansible, a blog post by this role's author, which is included in Chapter 8 of Ansible for DevOps.

License

MIT / BSD

Author Information

This role was created in 2015 by Jeff Geerling, author of Ansible for DevOps.

ansible-role-glusterfs's Issues

Glusterfs doesn't work for debian 8

I checkout project from master branch and I ran molecule test for geerlingguy/docker-debian8-ansible:latest. I don't know why but got an error. My ansible version is ansible 2.7.1

TASK [ansible-role-glusterfs : Ensure debian repo is present.] ***************** An exception occurred during task execution. To see the full traceback, use -vvv. The error was: , E:Some index files failed to download. They have been ignored, or old ones used instead. fatal: [instance]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"/root/.ansible/tmp/ansible-tmp-1551168034.8251243-176084680185581/AnsiballZ_apt_repository.py\", line 113, in <module>\n _ansiballz_main()\n File \"/root/.ansible/tmp/ansible-tmp-1551168034.8251243-176084680185581/AnsiballZ_apt_repository.py\", line 105, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/root/.ansible/tmp/ansible-tmp-1551168034.8251243-176084680185581/AnsiballZ_apt_repository.py\", line 48, in invoke_module\n imp.load_module('__main__', mod, module, MOD_DESC)\n File \"/tmp/ansible_apt_repository_payload_IJ5oNY/__main__.py\", line 550, in <module>\n File \"/tmp/ansible_apt_repository_payload_IJ5oNY/__main__.py\", line 542, in main\n File \"/usr/lib/python2.7/dist-packages/apt/cache.py\", line 443, in update\n raise FetchFailedException(e)\napt.cache.FetchFailedException: W:Failed to fetch https://download.gluster.org/pub/gluster/glusterfs/LATEST/Debian/jessie/amd64/apt/dists/jessie/main/binary-amd64/Packages HttpError404\n, E:Some index files failed to download. They have been ignored, or old ones used instead.\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

In my opinion the gluster 5 version doesn't support debian jessie https://download.gluster.org/pub/gluster/glusterfs/LATEST/Debian/

GlusterFS role Debian installs fail

See builds: https://travis-ci.org/geerlingguy/ansible-role-glusterfs/builds/272741717

Basically, we need to follow the directions here:

• https://gluster.readthedocs.io/en/latest/Install-Guide/Install/
• Or more up to date: https://download.gluster.org/pub/gluster/glusterfs/LATEST/Debian/

The PPA doesn't install on Debian :P

glusterfs EPEL repo urls does not work

For example http://download.gluster.org/pub/gluster/glusterfs/LATEST/EPEL.repo/pub.key does not work. There are http://download.gluster.org/pub/gluster/glusterfs/LATEST/EPEL.repo/rsa.pub file availible.

File http://download.gluster.org/pub/gluster/glusterfs/LATEST/EPEL.repo/glusterfs-epel.repo is updated.

Use 'LATEST' for PPA on Debian/Ubuntu

Ref to #14 - basically the LATEST release should be a little better for ongoing maintenance. Will have to increment major version on this role though.

Also update the link to the installation docs: https://docs.gluster.org/en/v3/Install-Guide/Install/

Failed install on Raspberry pi 3/4 32bit

as requested in #33 i have created a new ticket for this issue.

when using this role to install to a raspberry pi 3/3 on the "Ensure Debian Repo is present" step you get the following error.

{"changed": false, "msg": "apt cache update failed"}

running in debug mode produces the following.
The full traceback is: File "/tmp/ansible_apt_repository_payload_nycaw82d/ansible_apt_repository_payload.zip/ansible/modules/packaging/os/apt_repository.py", line 548, in main File "/usr/lib/python3/dist-packages/apt/cache.py", line 589, in update raise FetchFailedException(e) fatal: [192.168.0.200]: FAILED! => { "changed": false, "invocation": { "module_args": { "codename": null, "filename": "gluster", "install_python_apt": true, "mode": null, "repo": "deb [trusted=yes] https://download.gluster.org/pub/gluster/glusterfs/LATEST/Debian/buster/armhf/apt buster main", "state": "present", "update_cache": true, "validate_certs": true } }, "msg": "apt cache update failed" }

performing an apt update on the rpi manually provides no error and multiple other roles also update the cache. after doing some investigation i can see that in this step it adds the glusterFS repository and searches for the following.

https://download.gluster.org/pub/gluster/glusterfs/LATEST/Debian/buster/armhf/

but armhf does not exists and only builds for arm64 and amd 64.

this could be a side issue as ansible is reporting an apt update error but this is the only thing i could see wrong.

Hope this helps.

Installation of GlusterFS fails

Hi,
The installation of GlusterFS fails.

I created a role that I want to roll out to some nodes in my home lab.

The role is defined as follows:

---
# tasks file for single_data_volume
- include_role:
    name: gluster-node
    tasks_from: single_data_volume
  vars:
    gluster_brick_dir: '/data1'
    gluster_mount_dir: '/gluster_volume1'
    gluster_brick_name: 'brick1'

- include_role:
    name: gluster-node-partition
    tasks_from: default_partitioning

- name: Create default partitioning
  include_role:
    name: gluster-node-partition

- name: Create gluster volume for partition 1
  include_role:
    name: gluster-node
  vars:
    gluster_brick_dir: /data1/glusterbrick1
    gluster_mount_dir: /gluster_volume1
    gluster_brick_name: 'brick1'

- name: Create gluster volume for partition 2
  include_role:
    name: gluster-node
  vars:
    gluster_brick_dir: /data1/glusterbrick2
    gluster_mount_dir: /gluster_volume2
    gluster_brick_name: 'brick2'

- name: Create gluster volume for partition 3
  include_role:
    name: gluster-node
  vars:
    gluster_brick_dir: /data1/glusterbrick3
    gluster_mount_dir: /gluster_volume3
    gluster_brick_name: 'brick3'

It makes use of another role I defined which contains the code from you tutorial (https://www.jeffgeerling.com/blog/simple-glusterfs-setup-ansible)

---
# tasks file for single_data_volume
- include_role:
    name: gluster-node-partition
    tasks_from: default_partitioning

- include_role:
    name: geerlingguy.glusterfs

- name: Install glusterfs
  include_role:
    name: geerlingguy.glusterfs

- name: Ensure Gluster brick and mount directories exist.
  file: "path={{ item }} state=directory mode=0775"
  with_items:
    - "{{ gluster_brick_dir }}"
    - "{{ gluster_mount_dir }}"
  become: true

- name: Configure Gluster volume.
  gluster_volume:
    state: present
    name: "{{ gluster_brick_name }}"
    brick: "{{ gluster_brick_dir }}"
    replicas: 2
    cluster: "{{ groups.storageservers | join(',') }}"
    host: "{{ inventory_hostname }}"
    force: yes
  run_once: true

- name: Ensure Gluster volume is mounted.
  mount:
    name: "{{ gluster_mount_dir }}"
    src: "{{ inventory_hostname }}:/{{ gluster_brick_name }}"
    fstype: glusterfs
    opts: "defaults,_netdev"
    state: mounted
  become: true

I do the rollout using the following command:

ansible-playbook -i inventories/playground/hosts.yaml site.yml -K

This is the error message I get:

fatal: [node2]: FAILED! => {"changed": false, "msg": "'/usr/bin/apt-mark manual dirmngr' failed: E: Could not create temporary file for /var/lib/apt/extended_states - mkstemp (13: Permission denied)\nE: Failed to write temporary StateFile /var/lib/apt/extended_states\n", "rc": 100, "stderr": "E: Could not create temporary file for /var/lib/apt/extended_states - mkstemp (13: Permission denied)\nE: Failed to write temporary StateFile /var/lib/apt/extended_states\n", "stderr_lines": ["E: Could not create temporary file for /var/lib/apt/extended_states - mkstemp (13: Permission denied)", "E: Failed to write temporary StateFile /var/lib/apt/extended_states"], "stdout": "dirmngr set to manually installed.\n", "stdout_lines": ["dirmngr set to manually installed."]}
fatal: [node1]: FAILED! => {"changed": false, "msg": "'/usr/bin/apt-mark manual dirmngr' failed: E: Could not create temporary file for /var/lib/apt/extended_states - mkstemp (13: Permission denied)\nE: Failed to write temporary StateFile /var/lib/apt/extended_states\n", "rc": 100, "stderr": "E: Could not create temporary file for /var/lib/apt/extended_states - mkstemp (13: Permission denied)\nE: Failed to write temporary StateFile /var/lib/apt/extended_states\n", "stderr_lines": ["E: Could not create temporary file for /var/lib/apt/extended_states - mkstemp (13: Permission denied)", "E: Failed to write temporary StateFile /var/lib/apt/extended_states"], "stdout": "dirmngr set to manually installed.\n", "stdout_lines": ["dirmngr set to manually installed."]}

Both nodes run latest Ubuntu 18.04 server edition.

What do I need to do to make it work?

TASK [geerlingguy.glusterfs : Import GlusterFS GPG key.] failed

fatal: [host1.local]: FAILED! => {"changed": false, "failed": true, "msg": "failed to fetch key at http://download.gluster.org/pub/gluster/glusterfs/LATEST/rsa.pub , error was: HTTP Error 404: Not Found"}
        to retry, use: --limit @1_prepare_gluster.retry

Maybe the repo isn't there anymore?

IPC connect call failed - in task: Ensure PPA for GlusterFS is present. (Ubuntu 22)

Hello Jeff, i've been using your roles a lot and this one sometimes give me an error in the task Ensure PPA for GlusterFS is present
This only happens sometimes and may be a problem with my hoster(hetzner) or other things on my Ubuntu box.

vars:

glusterfs_default_release: ""
glusterfs_ppa_use: true
glusterfs_ppa_version: "10"

error

TASK [geerlingguy.glusterfs : Ensure PPA for GlusterFS is present.] ****************************************************************************
fatal: [data-01-5c156b41-1c43-5588-8b28-d382b21ba464]: FAILED! => changed=false 
  cmd: /usr/bin/apt-key adv --recv-keys --no-tty --keyserver hkp://keyserver.ubuntu.com:80 F7C73FCC930AC9F83B387A5613E01B7B3FE869A9
  msg: |-
    Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
    gpg: key 13E01B7B3FE869A9: public key "Launchpad PPA for Gluster" imported
    gpg: can't connect to the agent: IPC connect call failed
    gpg: Total number processed: 1
    gpg:               imported: 1
  rc: 2
  stderr: |-
    Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
    gpg: key 13E01B7B3FE869A9: public key "Launchpad PPA for Gluster" imported
    gpg: can't connect to the agent: IPC connect call failed
    gpg: Total number processed: 1
    gpg:               imported: 1
  stderr_lines: <omitted>
  stdout: |-
    Executing: /tmp/apt-key-gpghome.fRR92Wrx6D/gpg.1.sh --recv-keys --no-tty --keyserver hkp://keyserver.ubuntu.com:80 F7C73FCC930AC9F83B387A5613E01B7B3FE869A9
  stdout_lines: <omitted>

This may be a problem with me running the playbook too many times in development but i don't know just wanted to post maybe you have a idea for a workaround to ensure the task won't fail.

Replication: Unfortunately i am not able to replicate this.

edit: without ppa i have no issues.

Thanks for all your work!

wrong apt repo address for raspberry pi

TASK [geerlingguy.glusterfs : Ensure debian repo is present.] ******************************************************************************************************************************************************************************************************************
fatal: [pi3modelb]: FAILED! => {"changed": false, "msg": "Failed to update apt cache: E:The repository 'https://download.gluster.org/pub/gluster/glusterfs/LATEST/Debian/bullseye/armhf/apt bullseye Release' does not have a Release file."}

i believe the right address is https://download.gluster.org/pub/gluster/glusterfs/LATEST/Debian/bullseye/arm64/apt/ but its currently empty.

is there any way to ignore this error and move on?

Packages actualization

Maybe it would be good to switch off GlusterFS pakages upgrading (for example for Ubuntu "apt-mark hold glusterfs*") for security reasons.

Error adding glusterfs3.5

I get this error when adding your role to a server - it looks like an issue with a missing Release file in the 3.5 ppa. Changing glusterfs_ppa_version from 3.5 to 3.8 gets around the issue

TASK [geerlingguy.glusterfs : Add PPA for GlusterFS.] ************************** fatal: [site02]: FAILED! => {"changed": false, "failed": true, "module_stderr": "", "module_stdout": "Traceback (most recent call last):\r\n File \"/tmp/ansible_ezhESi/ansible_module_apt_repository.py\", line 522, in <module>\r\n main()\r\n File \"/tmp/ansible_ezhESi/ansible_module_apt_repository.py\", line 512, in main\r\n cache.update()\r\n File \"/usr/lib/python2.7/dist-packages/apt/cache.py\", line 454, in update\r\n raise FetchFailedException(e)\r\napt.cache.FetchFailedException: W:The repository 'http://ppa.launchpad.net/gluster/glusterfs-3.5/ubuntu xenial Release' does not have a Release file., W:Data from such a repository can't be authenticated and is therefore potentially dangerous to use., W:See apt-secure(8) manpage for repository creation and user configuration details., E:Failed to fetch http://ppa.launchpad.net/gluster/glusterfs-3.5/ubuntu/dists/xenial/main/binary-amd64/Packages 404 Not Found, E:Some index files failed to download. They have been ignored, or old ones used instead.\r\n", "msg": "MODULE FAILURE", "parsed": false}

Task: Ensure GlusterFS is installed fails

It looks like the repo has moved from http://download.gluster.org/pub/gluster/glusterfs/LATEST/EPEL/ to http://buildlogs.centos.org/centos/7/storage/x86_64/gluster-3.8/

volume create: gluster: failed: Host worker0 is not in 'Peer in Cluster' state

I am getting this error when I run my playbook for the first time.
After that, the error disapear but I would like to run my playbook flawlessly from the first go.

I created a repo with a very easy step by step guide to explain and reproduce my error:
https://github.com/rolele/glusterfs-ansible-test

I am using vagrant so it is very easy to reproduce.

Add ability to configure Debian/Ubuntu default_release

Add a glusterfs_default_release like the nginx_default_release var in my Nginx role.

See how this role would work with Ansible's new gluster_volume module

Using Ansible's gluster_volume module (available starting in 1.9.x) might simplify this role's ambition a bit; I was originally looking into ways to use this role to communicate with multiple servers and coordinating bricks and service starts... but it looks like this role might be best suited to simply installing all the requisite software and configuration.

Then playbooks would be responsible for all the actual brick setup using gluster_volume.

Automated Ubuntu test builds failing

E.g.

TASK [role_under_test : Add PPA for GlusterFS.] ********************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "failed to fetch PPA information, error was: HTTP Error 404: Not Found"}

See: https://travis-ci.org/geerlingguy/ansible-role-glusterfs/jobs/401136438

GlusterFS in clients only mode.

I am using GlusterFS in a small setup to share files between Servers.

My Setup:
2 GlusterFS server for redudency.
4 GlusterFS clients which mounts the server brick.

While this Ansible Role is perfect for setting up GlusterFS server, This playbook also installs glusterfs-server on the client machines.

Proposal: We can have a variable such as glusterfs_server_use Which will not install glusterfs-server on the client machines.

glusterfs_server_use: true

I can make the above changes and send a PR in some time.

Use official Gluster PPA for Ubuntu

See: http://www.gluster.org/community/documentation/index.php/Getting_started_install

Wrong service name

I have Ubuntu 16.04.4 LTS. Role cannot run the system service named "glusterfs-server" because it doesn't exist. Exists the "glusterd" service.

New CentOS release 7.6 breaks repo install

With the new CentOS release the installation of the repository package now installs centos-release-gluster-legacy from the extras, which by default which doesn't give access to any gluster packages.

Packages available:

centos-release-gluster-legacy.noarch : Disable unmaintained Gluster repositories from the CentOS Storage SIG
centos-release-gluster40.x86_64 : Gluster 4.0 (Short Term Stable) packages from the CentOS Storage SIG repository
centos-release-gluster41.noarch : Gluster 4.1 (Long Term Stable) packages from the CentOS Storage SIG repository
centos-release-gluster5.noarch : Gluster 5 packages from the CentOS Storage SIG repository

Looking at the setup-RedHat.yml file it specified centos-release-gluster{{ glusterfs_default_release }} so I guess specifying 40, 41 or 5 should work but in the default file this variable suggests usage is only for Ubuntu.

I fixed this in my env by changing the variable above in the defaults to 41 (LTS repo for Gluster)

Not sure what the best approach would be to resolve this, other than ensure a relevant repo version is specified. Shame CentOS/RHEL don't have a package without version in the title which works.

Use TravisCI/Docker for multi-platform tests

This will help ensure RHEL users don't get a broken role after certain changes are made that aren't currently tested via Travis (only Ubuntu 12.04 is covered right now).

glusterfs installation fails due to missing python3-pyxattr package

Summary
The installation of glusterfs fails on a fresh CentOS 8 Stream installation with the following error:

TASK [geerlingguy.glusterfs : Ensure Gluster packages are installed.]
*******************************************************************************
fatal: [centos8stream]: FAILED! => {"changed": false, "failures": [], "msg": "Depsolve Error occured: \n Problem: 
cannot install the best candidate for the job\n  - nothing provides python3-pyxattr needed by 
glusterfs-server-9.1-1.el8.aarch64", "rc": 1, "results": []}

The package python3-pyxattr is provided by the PowerTools repo, which is not activated per default. The repository can be activated with dnf config-manager --enable PowerTools

[fgierlinger@centos8stream ~]$ sudo dnf repolist
repo id                                                  repo name
AppStream                                                CentOS-8 - AppStream
BaseOS                                                   CentOS-8 - Base
centos-gluster9                                          CentOS-8 - Gluster 9
extras                                                   CentOS-8 - Extras
[fgierlinger@centos8stream ~]$ sudo dnf config-manager --enable PowerTools
[fgierlinger@centos8stream ~]$ sudo dnf repolist
repo id                                                  repo name
AppStream                                                CentOS-8 - AppStream
BaseOS                                                   CentOS-8 - Base
PowerTools                                               CentOS-8 - PowerTools
centos-gluster9                                          CentOS-8 - Gluster 9
extras                                                   CentOS-8 - Extras

The parameter enablerepo for yum might solve this problem, but I have not tested it.

Steps to reproduce
Execute the following playbook on a CentOS 8 Stream host.

- hosts: all
  gather_facts: yes
  roles:
    - role: geerlingguy.glusterfs
      vars:
        glusterfs_default_release: 9

Expected behaviour
Glusterfs installation succeeds when executing the role and specifying glusterfs_default_release: 9

Actual behaviour
The installation of glusterfs fails, because the package python3-pyxattr can not be installed, which is located in the PowerTools repository. The PowerTools repository is not enabled in a CentOS 8 Stream minimal installation

Host information
OS: CentOS 8 Stream aarch64
Python: 3.8.2
Ansible: 2.10.7
geerlingguy.glusterfs: 3.1.2

missing release-file in gluster repo for Debian Bullseye

Hi Jeff,

at first thank you for your role.
Actually it fails for for Debian Bullseye, because of a missing Release file at the configured path.

E: The repository 'https://download.gluster.org/pub/gluster/glusterfs/LATEST/Debian/bullseye/amd64/apt bullseye Release' does not have a Release file. N: Updating from such a repository can't be done securely, and is therefore disabled by default. N: See apt-secure(8) manpage for repository creation and user configuration details.

There is a simple solution for it.
There's a path with a Release file.

I will commit a patch soon.

Rocky Linux 8 installation fails with 'failed to download metadata for repo "centos-gluster9"'

In CI, RL8 fails with:

TASK [geerlingguy.glusterfs : Ensure Gluster packages are installed.] **********
[117](https://github.com/geerlingguy/ansible-role-glusterfs/runs/5356276882?check_suite_focus=true#step:5:117)
  fatal: [instance]: FAILED! => {"changed": false, "msg": "Failed to download metadata for repo 'centos-gluster9': Cannot prepare internal mirrorlist: No URLs in mirrorlist", "rc": 1, "results": []}

For now I'm disabling those tests.

Add link to jeffgeerling.com blog from README

Hi,

I was having problems getting glusterfs working until I found your blog article about this role:
http://www.jeffgeerling.com/blog/simple-glusterfs-setup-ansible

It would probably help others to have a link to that article from the README.

Thanks for the great roles.

Install fails for Raspberry Pi 3B

Hi,

I am trying to use the role and it fails for Rpi B3+:

TASK [geerlingguy.glusterfs : Ensure debian repo is present.] *********************************************************************************************************************
Sunday 26 January 2020 12:14:12 -0800 (0:00:01.080) 0:03:11.539 ********
fatal: [pi1]: FAILED! => {"changed": false, "msg": "apt cache update failed"}

Thanks.

Fix role build failures in certain OSes and OS versions

Current status:

• CentOS 6
• CentOS 7
• Ubuntu 12.04
• Ubuntu 14.04

See: https://travis-ci.org/geerlingguy/ansible-role-glusterfs

Apt repository specification contains trusted=yes

I do not think trusted=yes should be there. After all you are installing the pubkey in

ansible-role-glusterfs/tasks/setup-Debian.yml

Line 12 in b831a71

url: 'https://download.gluster.org/pub/gluster/glusterfs/{{ glusterfs_gpg_key_version }}/rsa.pub'

The official docs do not recommend [trusted=yes] either: https://docs.gluster.org/en/latest/Install-Guide/Install/