santandermetgroup / ansible-thredds-cluster Goto Github PK

If collections with no catalog service are linked in the gateway's catalog, this will return HTTP 404 when accessed.

This would allow catalog tree hierarchies to define multiple <datasetRoot>. Actually they are restricted to one. This requires to add the appropriate entries in the corresponding gateways.

collections:
  - &collection1
    name: collection1
    path: [path1,path2,path3]
    catalogs: data/collection1
    services: [catalog, fileServer, dodsC]

Role: tomcat-memcached

In two default variables of the role tds_\major_version needs to be defined previously by installing with either yum or source roles.

• memcached_session_manager_tc_jar: memcached-session-manager-tc {{ tomcat_major_version }} -{{ memcached_session_manager_version }}.jar

• memcached_session_manager_tc_url: https://repo1.maven.org/maven2/de/javakaffee/msm/memcached-session-manager-tc {{ tomcat_major_version }}/ {{ memcached_session_manager_version }}/{{ memcached_session_manag

Instead of the current process of manually connect to hosts and execute the update_proxy.py script, try to find a way to add workers using a playbook from the control machine, discovering facts from the hosts (saved in .json files) and adding them into the proxy using delegate_to.

Currently we use YAML anchors to reference tds_instances and collections in the replicas dict of the deployment model. This requires collections and tds_instances dicts to be defined in the same yaml file where replicas are defined. This is unwanted behavior since there are scenarios where all the hosts share the same collections but the replicas dict differ and this force us to copy the collections dict in both yaml files.

Example:

tomcat_base: '/usr/local'

collections:
  - &collection1
    name: collection1
    path: collection1
    catalogs:
      src: catalogs/collection1
    services: [dodsC, fileServer, catalog]

  - &collection2
    name: collection2
    path: collection2
    catalogs:
      src: catalogs/collection2
    services: [dodsC, fileServer, catalog]

tds:
  - &tomcat
    name: tomcat
    tds_content_root: /esg/content

replicas:
  - gateway: esgf
    host: "{{ ansible_eth1.ipv4.address }}"
    route: "{{ ansible_nodename }}"
    port: 8223
    tds_instance: *tomcat
    collection: *collection1

tds_version: 5.0.0

collections:
  - &collection1
    name: collection1
    path: collection1
    catalogs:
      src: catalogs/collection1
    services: [dodsC, fileServer, catalog]

  - &collection2
    name: collection2
    path: collection2
    catalogs:
      src: catalogs/collection2
    services: [dodsC, fileServer, catalog]

tds:
  - &backend_instance
    name: instance1
    connectors:
      - port: 8223
        protocol: AJP/1.3

replicas:
  - gateway: esgf
    host: "{{ ansible_eth1.ipv4.address }}"
    port: 8223
    route: "{{ ansible_nodename }}"
    tds_instance: *backend_instance
    collection: *collection1

  - gateway: esgf
    host: "{{ ansible_eth1.ipv4.address }}"
    port: 8223
    route: "{{ ansible_nodename }}"
    tds_instance: *backend_instance
    collection: *collection2

Ideal behavior would be to refer to the tds_instance and collection using their name. However this requires, to all the tasks that loop over the replicas, to discover the appropriate dict using the name key and its value, e.g.

- name: Create directories for collections in instances 
  file:
    dest: "{{ instance.tds_content_root | default(tomcat_base ~ '/' ~ instance.name ~ '/content') }}/thredds/{{ collection.path }}"
    state: directory
  with_items: "{{ replicas }}"
  vars:
    collection: "{{ collections | selectattr('name', 'equalto', 'collection1') | first }}"
    instance: "{{ tds | selectattr('name', 'equalto', 'instance1') | first }}"

tomcat_manager default value in role tomcat is not propagated to role tds, causing the role to fail if the variable is not explicitly defined

• Roles: httpd
• OS: CentOS/6
• httpd version: 2.2

When trying to start httpd v2.2 with the template provided by the role httpd this error message pops out:

httpd: Syntax error on line 9 of /etc/httpd/conf/httpd.conf: Cannot load /etc/httpd/modules/mod_access_compat.so into server: /etc/httpd/modules/mod_access_compat.so: cannot open shared object file: No such file or directory

After commenting out line 9 (i.e. stop loading the module mod_access_compat.so) the following error pops out:

httpd: Syntax error on line 11 of /etc/httpd/conf/httpd.conf: Cannot load /etc/httpd/modules/mod_authn_core.so into server: /etc/httpd/modules/mod_authn_core.so: cannot open shared object file: No such file or directory

After commenting out both lines, httpd is able to start and seems to work fine (not sure about this)

mod_access_compat.so Documentation

mod_acces_compat.so
Compatibility: | Available in Apache HTTP Server 2.3 as a compatibility module with previous versions of Apache httpd 2.x. The directives provided by this module have been deprecated by the new authz refactoring. Please see mod_authz_host

mod_authn_core.so Documentation

mod_authn_core.so
Compatibility: | Available in Apache 2.3 and later

collections:
  - &collection1
    name: collection1
    path: collection1 # Id of the collection and logical path in THREDDS
    catalogs: /data/collection1
    services: [catalog, fileServer, dodsC]
    datasets:
      - content: |
           <datasetScan name="NCEP Data" path="ncep" location="/data/ldm/pub/native/grid/NCEP/">
             <metadata inherited="true">
               <serviceName>fileserver</serviceName>
               <authority>meteo.unican.es</authority>
               <dataType>Grid</dataType>
             </metadata>
           </datasetScan> 

The content of every dataset will be rendered into the main catalog of the collection, before the </catalog> tag.

  - name: Deploy tds
    hosts: tds4gws
    vars_files: 
      - tds4gws-secret.yml
    vars:
      tomcat_instances:
        - name: tds4gws_1800
          base_port: 1800
          ajp: 
            port: 18009
            proxyPort: 8008
            proxyName: 192.168.33.12
          gws_instance: 
            - name: "{{ gws.0.name }}"

should be:

  - name: Deploy tds
    hosts: tds4gws
    vars_files: 
      - tds4gws-secret.yml
    vars:
      tomcat_instances:
        - name: tds4gws_1800
          base_port: 1800
          shutdown: ...
          connectors:
            - type: ajp # mandatory
              port: 18009
              proxyPort: 8008
              proxyName: 192.168.33.12
            - type: http
              port: ...
              proxyPort: ...
              proxyName: hostname/ip or ip only? -> hostname, use it in httpd <VirtualHost>
          gws_instance: 
            - name: "{{ gws.0.name }}"

The following configuration shows an example of a deployment of two hosts that should be implemented in two different plays since the ip_address couples the play to the host being deployed by Ansible. It should be replaced by the ip used by Ansible to allow identical hosts to be deployed in the same play.

- name: Deploy worker1
  hosts: worker1
  vars_files: 
    - workers-secret.yml
  vars:
    tomcat_install_from_source: true
    tomcat_system: false
    proxyPort: 8008
    tomcat_instances:
      - name: worker1_1800 -> dynamic ({{ whatever }})
        base_port: 1800
        ip_address: 192.168.50.12 -> dynamic ({{ whatever }})
        gws_instance: 
          - name: "{{ gws.0.name }}"
    gws:
      - name: gws1
        path: gws1/test
        location: "/group_workspaces/jasmin/gws1"

  roles:
    - role: tds 

- name: Deploy worker2
  hosts: worker2
  vars_files:
    - workers-secret.yml
  vars:
    tomcat_install_from_source: true
    tomcat_system: false
    proxyPort: 8008
    tomcat_instances:
      - name: worker1_1800
        base_port: 1800
        ip_address: 192.168.50.13
        gws_instance: 
          - name: "{{ gws.0.name }}"
    gws:
      - name: gws1
        path: gws1/test
        location: "/group_workspaces/jasmin/gws1"
  roles:
    - role: tds 

The 'lib' role, inside the roles folder, is actually used from other roles to perform specific tasks, instead of being a role per se, creating a dependency from roles that make use of it. Should it be refactored and its functionality included in the corresponding role?