Good feedback is specific, timely and actionable, therefore the Guardian’s Digital team decided to distill and centralise the information coming from Amazon Web Services. Whilst we have now started using AWS StackSets for other purposes, one of our first applications was aggregating the security reports from Trusted Advisor.
What is Trusted Advisor and why is it awesome?
Trusted Advisor is an optimisation tool for AWS, with an entire section dedicated to security. It is a good place to get started and we use it to check all our S3 bucket configurations.
Trusted Advisor will also flag open or overly permissive security groups. Security groups are the virtual firewalls of AWS and you can use them to control the traffic to your instances.
a system should only be able to access the information and resources that are necessary for its legitimate purpose
If you think you have locked down your Security Groups and Trusted Advisor is findings false positives, be aware that if a security group has more than one rule for a specific port, then it is the most permissive rule that will be applied. Trusted Advisor helped us uncover and remove some duplicated rules. You may also find it necessary to delete overly permissive rules after creating rules that are more restrictive.
Trusted Advisor can help developers adhere to the principle of least-privilege. This means a system should only be able to access the information and resources that are necessary for its legitimate purpose.
Why did we need to aggregate reports?
Most of our infrastructure is held in AWS, divided across many accounts that are owned and managed by different development teams. Reviewing the Trusted Advisor reports for each account by logging into the console would be a long and repetitive task for anyone to be attempting on a regular basis.
Centrally monitoring accounts simplifies the task of checking, and can also widen access to the reports and findings, which hopefully empowers more developers to raise questions or take corrective actions.
Getting started with AWS StackSets
AWS StackSets are not a security tool directly, but they are a tool that can help you on your quest towards improving AWS security, or at least your AWS monitoring. In the context of AWS, a stack is a collection of resources that can be managed as a single unit. StackSets are a CloudFormation tool that extends the functionality of stacks by allowing the creation, update, or deletion of stacks across multiple accounts.
The initial setup of a StackSet is quite manual, however if you think you will use more than one StackSet it is definitely worth doing.
You will need to select one of your accounts to become the StackSet Administrator and give it a stackset-admin role. The other accounts will require a stackset-target role. The stackset-admin can also be a stackset-target for itself, which is fun.
Be aware that whichever account you have made the stackset-admin now holds incredible power over every stackset-target, so choose it wisely and consider how to minimise the risks.
Creating a trusted relationship between accounts may weaken or break some of the security controls that are currently in place, so consider where in the sandboxing lifecycle you are and what trade-offs you are willing to make.
A dedicated account that is only used for StackSet administration is one mitigation we have in place. Developers can request access to the account as needed and relinquish permissions when they are done.
Working with StackSets is also a good opportunity to pair on deployment with someone. Talking someone through each step as you do it is a great opportunity to share knowledge, along with letting them double-check your work.
StackSets in practice
We used StackSets to create a role in each of our accounts that our centralised tool (Security HQ) can assume. The role has the required permissions for the services we want to request information from with API calls.
If we want to expand the permissions so we can integrate more services with Security HQ we can use the StackSet to update all of the roles at once with a single CloudFormation change.
Challenges
StackSets can open you up to a new form of CloudFormation drift, as nothing prevents a stack that is part of a StackSet from being modified or deleted from within the account. However, is possible to perform drift detection on a StackSet so you can at least identify which stacks have been modified.
Since we were creating stacks in accounts that may have other teams managing them, we found it helpful to add an ‘Owner’ tag to resources created by the StackSet. This gives teams some guidance on who to ask about strange roles and resources suddenly appearing in their accounts.
Conclusion
If you are using one or more cloud platforms, it is worth investigating ways to centrally audit and monitor your environment. StackSets and Trusted Advisor, along with some internal tooling for aggregating and enriching reports, has helped standardise security checks across our AWS accounts and make them accessible in a single location, rather than requiring someone to check each account.
Finally, if we are giving developers ownership of building and deploying our infrastructure we should be giving them ownership of securing it. We should aim for a culture where we share security expertise and responsibility the same ways we share operations expertise and responsibility.