aws-samples / aws-refarch-moodle

Moodle Reference Architecture with supporting Cloud Formation templates

Hosting Moodle™ on AWS

Version 2.0.2

Overview

This repository provides set of CloudFormation nested templates that deploy a highly available, elastic, and scalable Moodle™ 4.4 environment on AWS. Moodle™ offers a learning platform that provides educators, administrators and learners a single robust, secure and integrated system for personalized learning environment.

These nested templates can be used to deploy Moodle™ on AWS using Amazon Virtual Private Cloud (Amazon VPC), Amazon Elastic Compute Cloud (Amazon EC2), Auto Scaling, Elastic Load Balancing (Application Load Balancer), Amazon Aurora, Amazon ElastiCache, Amazon Elastic File System (Amazon EFS), Amazon CloudFront, Amazon Route 53, Amazon Certificate Manager (Amazon ACM) with AWS CloudFormation in YAML format.

This architecture is expansive enough to meet the needs of large institutions / organizations. Smaller organizations can choose to run a subset of the template to meet their needs. These templates can also be run individually and may be modified.

This template currently uses Moodle™ 4.4 stable version downloaded directly from download.moodle.org. Details for downloading are available in the templates/03-pipelinehelper.yaml template file.

Deployment guide

Read the reference architecture and the steps below to understand the deployment scope and options. While following the steps and guidelines to deploy Moodle™, pay careful attention to the parameters and their descriptions.

Pre-requisites

1. Select an AWS Region (for example: us-east-1) for your deployment.
2. Give a meaningful Stack Name that does not have any special characters including hyphen(-) Eg: MoodleDevDeploy or MoodleProd
3. If you plan to use HTTPS, you must create or import your certificate into Amazon Certificate Manager (ACM) and provide its ARN when deploying the CloudFormation stack.
4. Alternatively, if you plan to use an SSL Certificate with Amazon CloudFront, you must create or import your certificate into Amazon Certificate Manager in the us-east-1 region before launching Moodle™ and provide it's ARN when deploying the CloudFormation stack.

Steps

1. Deploy the 00-main.yaml stack. You can also click the Launch Stack button below to launch the stack in your logged-in AWS Account.

2. After the stack deployment completes, you will see a DNS Name entry under the Outputs tab under the main CloudFormation template. This DNS Name value will be your Moodle™ app URL. You can configure aliases or CNAMEs to point to this DNS Name if you want to customize this.

3. Navigate to the Moodle™ application URL to complete the installation.

   1. NOTE: You may encounter a 504 Gateway Timeout or CloudFront error on the final step of the Moodle™ application installation wizard (after configuring the administrator password). You can safely ignore this error and refresh the page to complete the installation.

   2. You may also see "Installation must be finished from the original IP address, sorry." If this is the case, update your database and set the lastip field of the mdl_user table to the internal IP address of your Application Load Balancer which can be found under the Network Interfaces section of the EC2 section of the AWS Console. To update the value in the database, run these commands on the EC2 web server:

       psql -h <hostname> -U<Username> 
       update mdl_user set lastip='<ip address>';
      

4. Once the Moodle™ installation wizard completes successfully, you need to update the value of the SSM Parameter IsMoodleSetupCompleted to 'Yes'.

   1. In your main CloudFormation template, check Outputs tab to see parameter IsMoodleSetupCompleted. Click the link under Value to get details of the parameter.
   2. Edit the parameter and change the value to Yes.
   3. Go back to Outputs tab to see link for MoodleCodePipeline. Click on the link to open Code Pipeline. Click on the Release Change button. This will re-run the deployment pipeline and update the Moodle™ configurations post-installation, in order to adjust the auto-scaling configuration and the session cache configuration.

5. This template can optionally deploy Amazon ElastiCache as the Moodle™ Session and/or Application cache(s). When this feature is activated, you still need to configure the Application Cache within Moodle™ after deployment (see how-to guide). The cache endpoint is listed under the CloudFormation Output tab as ApplicationCacheServerEndpoint.

*NOTE: To connect to your EC2 web servers, select an EC2 Instance and click on the Connect button in the AWS Console. Open the Session Manager tab and click on the Connect button. Note that this feature uses the AWS SSM Agent that is installed on the instances, allowing you to connect to EC2 Instances without opening the SSH port to Internet traffic. An alternative approach to connect to your instances would be to enable the bastion host through the CloudFormation stack parameters.

Launch the CloudFormation Template

You can launch this CloudFormation template in different AWS Regions. Below are links to help you get started quickly, but note that you can always change the region yourself once you are in the AWS Console.

AWS Region Code	Name	Launch
us-east-1	US East (N. Virginia)
us-east-2	US East (Ohio)
us-west-2	US West (Oregon)
eu-west-1	EU (Ireland)
eu-central-1	EU (Frankfurt)
ap-southeast-1	AP (Singapore)
ap-southeast-2	AP (Sydney)
ap-south-1	India (Mumbai)
ca-central-1	Canada (Central)

Architecture

The following sections describe the architecture and its components. This architecture uses a similar approach to the one used in the WordPress Reference Architecture.

AWS Certificate Manager

AWS Certificate Manager lets you easily provision, manage, and deploy Secure Sockets Layer/Transport Layer Security (SSL/TLS) certificates for use with AWS services. You should use SSL/TLS to protect data in transit, including sessions and passwords. If you plan to use SSL/TLS, you must create or import a certificate using AWS Certificate Manager before you deploy the template. In addition, if wish to use CloudFront and host Moodle™ in a region other than us-east-1, you must create or import the certificate in both us-east-1 and the region you are hosting Moodle™ in. CloudFront requires certificates in the us-east-1 region.

Application Load Balancer

The Application Load Balancer distributes incoming application traffic across multiple EC2 instances in multiple Availability Zones. You achieve high availability by clustering multiple Moodle™ servers behind this load balancer. You can review Moodle™'s overview of Server Clustering before proceeding.

Amazon Autoscaling

Amazon EC2 Auto Scaling helps ensure that the appropriate number of Amazon EC2 instances are available to handle the load of the application. The template configures autoscaling based on CPU utilization. An additional instance is added when the average CPU utilization exceeds 75% for three minutes and removed when the average CPU utilization is less than 25% for three minutes. Based on the instance type, cache configuration, and other factors, you may find that other metrics are better predictors of load. You can change the metrics to better meet your operational needs.

Note: that the installation wizard causes spikes in CPU that could cause the cluster to scale unexpectedly. To avoid an issue with this during installation, initial deployment starts with minimum and maximum autoscaling values of 1. Once you complete the Moodle™ installation wizard, update the SSM parameter IsMoodleSetupCompleted and run the Moodle™ pipeline, the minimum and maximum autoscaling values will be updated according to your parameters.

Amazon Elastic File System (EFS)

Amazon Elastic File System (Amazon EFS) provides simple, scalable file storage in the AWS Cloud. Using EFS makes Moodle™ operations and management (shared files, updates, patches, etc.) easier. However, Moodle™ performance may suffer when the application code itself is run from mounted volumes like EFS. Moodle™ recommends dirroot to be on local or high-performance storage. This template follows that recommendation, and uses a combination of Elastic Block Storage (EBS) and EFS for storage. Each web server in the Moodle™ Cluster employs the following directory structure:

$CFG->dirroot = '/var/www/moodle/html'        #Stored on root EBS volume
$CFG->localcachedir = '/var/www/moodle/local' #Stored on root EBS volume 

$CFG->dataroot = '/var/www/moodle/data'       #Stored on shared EFS filesystem
$CFG->cachedir = '/var/www/moodle/cache'      #Stored on shared EFS filesystem
$CFG->tempdir = '/var/www/moodle/temp'        #Stored on shared EFS filesystem

With elastic throughput type, Amazon EFS automatically scales throughput performance up or down to meet the needs of your workload activity. You don't need to specify or provision the throughput capacity to meet your application needs.

*Moodle™ recommends the dirroot be set as read only for the apache process in a clustered environment [Reference]. You should not install plugins to a server cluster from the admin page. Moodle™ recommends manually installing plugins on each server during planned maintenance. To follow the infrastructure-as-code methodology, installation/upgrade of plugins can be managed using AWS CodePipeline scripts. See the .pipeline folder inside your AWS CodeCommit Moodle™ repository.

AWS CodePipeline

This CloudFormation templates use AWS Services to create a CI/CD pipeline to help manage your Moodle™ environment. AWS CodeCommit will host a git repository for your Moodle™ environment. It initially pulls the source from download.moodle.org.. It also adds files required to automate the deployment pipeline. You can explore these files under the .pipeline folder. This template also creates an AWS CodePipeline configuration that build artifacts to deploy on EC2 with autoscaling groups using AWS CodeBuild and AWS CodeDeploy. It can optionally support a BLUE_GREEN deployment.

You can customize the overall pipeline for your Moodle™ setup.

AWS Systems Manager - Parameter Store

This template also uses the Parameter Store to host Moodle™ environment configurations parameters like the database endpoint, the database credentials, the application and session cache endpoints, etc. This allows easy management of these configuration parameters. You can change these parameters and refresh your deployment to quickly implement them.

Caching

---

Caching can have a dramatic impact on Moodle™'s performance. This template configures various forms of caching including OPcache, CloudFront and ElastiCache.

OPcache

PHP OPcache speeds up PHP execution by caching precompiled scripts in memory. This template configures OPcache as described here.

Amazon ElastiCache

Amazon ElastiCache for Memcached is a Memcached-compatible in-memory key-value store service that can be used as a cache or a data store. Moodle™ recommends that you don't use the same memcached server for both sessions and MUC Refer. Events triggering MUC caches to be purged leads to MUC purging the memcached server]. This template configures two ElastiCache clusters, one for session caching and one for application caching.

This template also allows you to create Amazon ElastiCache for Redis as Redis compatible in-memory key-value store service that can be used as a cache or a data store.

Session Caching

Moodle™ recommends that you store user sessions in one shared memcached server. The template configures session caching as described here.

Note: This template doesn't configure the Session Cache during initial deployment. It waits for you to finish the initial Moodle™ installation wizard and update the Parameter IsMoodleSetupCompleted value to Yes in the SSM Parameter Store. Once the installation is completed, you need to run the Moodle™ pipeline to enable session caching and finalize the other remaining configuration.

Application Caching

The template deploys an ElastiCache cluster for application caching, but the application caching must be configured after the Moodle installation is completed. You can configure memcached or Redis by filling in the auto-discovery endpoint to the list of Servers under both Store Configuration and Enable Clustered Servers (see image below). You can find the ApplicationCacheServerEndpoint address in the Outputs of the CloudFormation stack. Finally, scroll to the bottom of the caching administration page in Moodle™ and set ElastiCache as the default store for application caching.

Amazon CloudFront

Amazon CloudFront is a global content delivery network (CDN) service that securely delivers data, videos, applications, and APIs to your viewers with low latency and high transfer speeds. It also helps in caching content closer to user's geography and reduces loads on the web servers.

Amazon Route 53

Amazon Route 53 is a highly available and scalable cloud Domain Name System (DNS) service. The template will optionally configure a Route53 alias that points to either the Application Load Balancer or CloudFront. If you are using another DNS system, you should create a CNAME record in your DNS system to reference either the Application Load Balancer or CloudFront (if deployed). If you don't have access to DNS you can leave Domain Name blank and the template will configure Moodle™ to use the auto-generated Application Load Balancer domain name.

---

License

This library is licensed under the Apache 2.0 License.

Portions copyright.

• Moodle™ is licensed under the General Public License (GPLv3 or later) from the Free Software Foundation.
• OPcache is licensed under PHP License, version 3.01.

Please see LICENSE for applicable license terms and NOTICE for applicable notices.

The word Moodle and associated Moodle logos are trademarks or registered trademarks of Moodle Pty Ltd or its related affiliates.

aws-refarch-moodle's Issues

Rotate logs automatically to S3

We have been seeing Apache logs fill our disk, is there any possibility of adding a rotate log automatically to S3? For example every week
Thanks!

Aurora Postgresql parameter group no longer valid

the parameter group created by the stack is no longer valid.

Stack failing to launch due to DB Cluster creation

Thank you for building this reference architecture!

I tried launching it several times with default values. It keeps failing at creating the DB cluster with the following error.

DBClusterParameterGroup not found: default.aurora-postgresql9.6 (Service: AmazonRDS; Status Code: 404; Error Code: DBClusterParameterGroup NotFound; Request ID: 53b6cfe2-fc05-4caa-9d51-03b307218a0b)

Tried to launch it in US-East 1 and AP.

Change in version aurora-postgresql

Change in version aurora-postgresql 03-rds.yaml

DatabaseInstanceType:
AllowedValues:
- db.r4.large ##version aurora-postgresql13 no accept r4
- db.r4.xlarge
- db.r4.2xlarge
- db.r4.4xlarge
- db.r4.8xlarge
- db.r4.16xlarge
- db.r5.large ## version aurora-postgresql12 accept r5
- db.r5.xlarge
- db.r5.2xlarge
- db.r5.4xlarge
- db.r5.8xlarge
- db.r5.16xlarge

DBClusterParameterGroupName: default.aurora-postgresql13 ##Change version aurora-postgresql to from 12 to 13

Allowed instances update for Cape-Town af-south-1 region

Good day

Several of the instance types used in the reference architecture aren't available in the Cape-Town af-south-1 region. An example is the db.r4.large instance for the RDS. Could this perhaps be updated to enable deployment to the Cape-Town region?

Kind regards
Jaco-Louis

Database instance size options

Hi,
I noticed database options start at db.r4.large. Is there any reason why smaller options were not included, or could I edit the template to add more options?

Updating Moodle version

How can I update a new Moodle version without loss data?. Only I must to update the stack modifing the yaml file?

Stack stuck on REVIEW_IN_PROGRESS

The template creation process seems to be stuck on REVIEW_IN_PROGRESS

Do I need to tweak something?

Pricing

Hi there i would just like to find out what the estimated costs would be to run this stack. Im afraid of running the stack without knowing the costs. i dont want to get a nasty bill from aws

deploy Moodle in exisiting VPC

which lines required to be updated in 01-newvpc.yaml to deploy Moodle in existing VPC

Error in after_install.sh pipeline

can't read /var/www/moodle/html/cache/stores/memcached/lib.php

i wanna use moodle 3.8 with this architecture

i changed 04-web.yaml file:

• php version 7.0 => 7.2
• php packages php70-intl => php72-intl
• postgrey 93 => 10
• https://download.moodle.org/stable34/moodle-3.4.tgz => https://download.moodle.org/stable38/moodle-3.8.tgz
  but it isnt work can you help

Constant 504 Error

504 Error is always displayed for the processes like backup and restore.

Plugin installation and Moodle version upgrade

This template you all put together is amazing. I have a couple of doubts, to install plugins and upgrade Moodle version is there any example that we may follow on how to modify any of the .pipeline files?

Existing Moodle setup migrate to this architecture

Hi, We have an existing Moodle setup which is customized as per our requirements. We do have a GitHub repository.

How can we migrate that to this architecture?

Please help.

Update to Amazon AMI Linux 2 and create separate RDS for mysql

Hi,

First of all thank you for creating this cf template. Can you please update moodle to latest moodle version 3.6.3, update Amazon AMI Linux versions to Amazon AMI Linux version 2 and also provide mysql compatible version of RDS.

Thanks
Rama

Config.php requires "use Aws\\Credentials\\CredentialProvider"

Hi,
I have an infrastructure with moodle installed as described in aws-refarcg-moodle.

Since August 4th I have a 500 HTTP server error and I found the following error.

ERROR

#######################################################################
Fatal errorPHP Fatal error: Uncaught Aws\Exception\CredentialsException: Error retrieving credentials from the instance profile metadata service. (cURL error 6: Could not resolve host: https (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)) in phar:///var/www/moodle/html/lib/aws.phar/Aws/Credentials/InstanceProfileProvider.php:272
Stack trace:
#0 phar:///var/www/moodle/html/lib/aws.phar/Aws/Credentials/InstanceProfileProvider.php(159): Aws\Credentials\InstanceProfileProvider->handleRetryableException(Object(GuzzleHttp\Exception\ConnectException), Array, 'Error retrievin...')
#1 [internal function]: Aws\Credentials\InstanceProfileProvider->Aws\Credentials\{closure}()
#2 phar:///var/www/moodle/html/lib/aws.phar/GuzzleHttp/Promise/Coroutine.php(160): Generator->throw(Object(GuzzleHttp\Exception\ConnectException))
#3 phar:///var/www/moodle/html/lib/aws.phar/GuzzleHttp/Promise/Promise.php(204): GuzzleHttp\Promise\Coroutine->_handleFailure(Object(GuzzleHttp\Exception\ConnectException))
#4 phar:///var/www/moodle/html/lib/aws.phar/Guzzl in phar:///var/www/moodle/html/lib/aws.phar/Aws/Credentials/InstanceProfileProvider.php on line 272

#######################################################################

To resolve it I have to change the initial part of config.php file from:

              require 'lib/aws.phar';
              use Aws\\SecretsManager\\SecretsManagerClient;
              use Aws\\Exception\\AwsException;
              use Aws\\Iam\\IamClient;

              $client = new SecretsManagerClient([
                  'version' => '2017-10-17',
                  'region' => ''
              ]);

to:

              require 'lib/aws.phar';
              use Aws\\SecretsManager\\SecretsManagerClient;
              use Aws\\Exception\\AwsException;
              use Aws\\Iam\\IamClient;
              use Aws\\Credentials\\CredentialProvider;

              $provider = CredentialProvider::defaultProvider();

              $client = new SecretsManagerClient([
                  'version' => '2017-10-17',
                  'region' => '',
                  'credentials' => $provider
              ]);

Basically seems that a breaking change has been introduced in aws.phar.

Please can you help me to understand how to prevent that from happening?
Is it available an url point for a specific v3 revision ?
What I did is correct ?
You too have the same anomaly as me?

Thanks

Can't continue Moodle Installation

After successfully deploying the stack. I can't access the webserver to continue the installation.

AWS Memcached or Redis Type Moodle error

Hi, i have installed the template in the AWS Milan region and if i choose Memcached for Session and Application. After to put IsMoodleSetupCompleted to "Yes" and restart the Pipeline i have a Moodle error because the memcached module is not found.

In the install_cacheclient.sh i found that seems to be an error, the command tar -xf tar -xf '/tmp/latest-64bit-X86 seems has not the openssl3 suffix so it is impossible to copy the amazon-elasticache-cluster-client.so in /usr/lib64/php/modules/ .

Moreover I have to modify the /etc/php.d/50-memcached.ini and the /etc/php.d/20-redis.ini file to change the extension=/usr/lib64/php/modules/amazon-elasticache-cluster-client.so in extension=amazon-elasticache-cluster-client .

After this change I have another issue infact on my AWS AMI the openssl version is 1.0.2k but seems i need openssl 3.0

I tried to change amazon-elasticache-cluster-client.so for my openssl version as descibed in https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/Appendix.PHPAutoDiscoverySetup.Installing.html but than i have issues with other components.

Moreover the extension=igbinary.so seems to be not present.
Am I doing something wrong ?

Thank you

Best Regards

Andrea Malatesta

failed to create: [NatEIP0, NatEIP1]

When creating the stack, I get the following error:
Embedded stack arn:aws:cloudformation:us-east-2:004749571273:stack/MoodleLink2-vpc-4WE3TZH1DORN/3ff97630-c822-11ed-bc0e-02905d19378a was not successfully created: The following resource(s) failed to create: [NatEIP0, NatEIP1].

Limitation DB Connections

Hi, there is some DB connection limit on Moodle?

We have installed from this stack, and everything is working fine, but sometimes when we have a lot of users, the Database cluster fails.

Database is Cluster Aurora PostgreSQL db.r4.2xlarge, CPU and Memory usage is not high and we receive
Error: Database connection failed.
It is possible that the database is overloaded or otherwise not running properly.

Any idea where to see? Logs in DB only says could not receive data from client: Connection reset by peer, but no more info

template for ap-south-1

Hi John

We need cloud formation templates for ap-south-1 region. There is a line in config.php that limits the region:

'2017-10-17', 'region' => 'us-east-1', ]); ..... Thanks a lot for your time. Best Prasad

Older Moodle version in template

actually Moddle has 4.2+ version but the template has Moddle 4.1 version. What must I change in files to install the latest Moddle version?

03-efsfilesystem.yaml not updated in S3 (still using python2.7 as runtime)

Line 533 of 03-efsfilesystem.yaml species Python 3.9 as runtime. This is correct. Nevertheless, the S3 file referenced as the template url for the EFS File System still uses python2.7.

This causes the stack to fail with the following message:

Resource handler returned message: "The runtime parameter of python2.7 is no longer supported for creating or updating AWS Lambda functions. We recommend you use the new runtime (python3.9) while creating or updating functions. (Service: Lambda, Status Code: 400, Request ID: 79d0f135-d0c4-4fb0-a771-17da637c3774, Extended Request ID: null)" (RequestToken: 909c90cf-5c0a-849e-94ed-e7bae428f3b5, HandlerErrorCode: InvalidRequest)

moodle url name change

The default moodle alb dns name gets populated in the url (e.g.http://moodl-publi-vm8x7xxxxxxxx-878763876.ap-south-1.elb.amazonaws.com/). Wanted to check on the url replacement option - is it better to do it with {wwwroot}/admin/tool/replace/index.php and changing $CFG_root in config.php? or would you recommend any other option considering the same dns name population at other places as well? Please advise. Thanks in anticipation.

Error: Database connection failed

Hi,

I have successfully created the stack but when I go to the Moodle setup site, this error comes up:

I checked the config.php file but saw nothing wrong.

How can this error gets fixed?

Best regards,
Yusuf

Cron not running?

Hi I deployed this stack a couple days ago however I don't understand how it handles the Cron jobs for Moodle.

Under site admin > reports I am seeing the following critical error:
"The admin/cli/cron.php script has never been run and should run every 1 min"

Exception - Class "Redis" not found

The following error appears after installing and enabling the cache

Issues trying to lunch the stack

Sorry for this question but how can i lunch this stack. I try it but i get the same error all time.

rds | CREATE_FAILED | Embedded stack arn:aws:cloudformation:us-east-1:########:stack/moodlecftest2-rds-#############-###-###-############## was not successfully created: The following resource(s) failed to create: [DatabaseCluster].
-- | -- | -- | --

How are we handling the Email Service ?

I was looking at the option how email is being configured in this architecture

Release Change

hi vrioux
when the deployment is completed , release change give me error & i got this email
The launch template associated with this Auto Scaling group cannot be found. Each time a scale out operation is performed, it will be prevented until you make a change that fixes the issue.
thank you in advance

Only 150-200 users can login out of 2000 users

Removed.

Invalid config.php (error 500)

The file generated at /var/www/moodle/html/config.php does not parse as PHP, because the templates/04-web.yaml file of this repo uses backtick quotes.

When /var/www/moodle/html/config.php is generated, the backticks are substituted by question marks, leading to a parsing error and an error 500 when accessing the website.

This bug was introduced in #1.

AWS EFS is very slow for administration in moodle

Hi ,

AWS EFS for the administration side in moodle are very slow ..... do you have a workaround for this ?

Thanks a lot