Student Management on the AWS Cloud

The Department already had a significant investment in on-premise hardware and software licences, so any cloud infrastructure had to operate in a hybrid environment. For the solution to work several key on-premise repositories and resources had to be accessible from the Cloud and vice-versa. The Modis team used a relatively new feature of Route 53 called Route 53 Endpoint Resolvers to integrate the on-premise private DNS namespace, with the Virtual Private Cloud environment. From the point of view of servers in the Cloud, DNS resolution was seamless.

Deploying the software artefacts handed over by the vendor proved to be a complex, multi-step process involving sequenced database updates, provisioning multiple Microsoft IIS servers containing multiple applications, and procedural elements such as clearing the Elasticache cache at specific times.

To make the deployment to AWS quick and reliable with these constraints, automation was critical. All AWS resources were created by CloudFormation templates, broken down by function or lifecycle, and parameterised so they could be orchestrated and executed in many ways as required. Detailed configuration of the Microsoft Windows application servers or complex behaviour was handled by PowerShell scripts, modularised to share common behaviour, and parametrised for maximum configurability.

Given the changing nature of the deployable artefacts during development, regular lessons-learned sessions were held so that feedback could be provided to the third-party vendor on ways to make the deployment process even faster and more efficient.

All servers were placed into auto-scaling groups (ASG) behind application load balancers (ALB) to enable scaling in the face of demand. This allowed the EC2 fleet to grow and shrink as required and also ensured that the Department was minimising what it was paying for. Certain peak periods in the morning and afternoon were well know, so a feature of ASG was used to schedule the scale-up to ensure resources were pre-warmed to meet this demand.

AWS Systems Manager was used extensively throughout the project. All variable configuration items — including secrets and confidential information, such as usernames, passwords, and other tokens — were abstracted and stored in an encrypted format in a Parameter Store hierarchy. This ensured no sensitive information was ever committed to the source code repository. Systems Manager Maintenance Window and Automation was also used to ensure Windows and Linux security updates were automatically applied across the EC2 instance fleet.

Given the highly confidential nature of the data being processed and stored, end-to-end encryption of all data in-flight and at rest was mandatory. AWS Certificate Manager was used to provision and manage all CloudFront and ALB certificates with the latest cipher suites. Self-signed certificates were then used for the hop between the ALB and EC2 instances themselves. All S3 buckets had versioning, encryption, and public access-denial enabled by default and built into the associated CloudFormation templates.

Continuing the defence-in-depth strategy, AWS Web Application Firewall was used to implement geo-restriction to known access points, along with a series of Open Web Application Security Project (OWASP) proactive controls such as cross-site scripting and SQL injection.

To maintain the security posture of the AWS infrastructure itself, CloudWatch metrics, alarms and events were used to monitor key resources such as the root account, security groups, S3, and IAM permissions. Any out-of-band changes triggered a real-time message to an SNS topic, which forwarded a email to the DevOps team to investigate.

Managed security services such as Trusted Advisor and Guard Duty were activated as a matter of course. Again, CloudWatch events were used to flag any deviations from the known steady-state.

To support the eventual transition to production and support, all application server logs and key operating system logs for the EC2 fleet were pushed to CloudWatch Logs along with VPC flow logs and CloudTrail logs. An Elasticsearch cluster then ingested this log stream. Technical and customer support teams could then use Kibana to search and visualise the data for support purposes.