The myth of attackers breaking through layers of firewalls or decoding encryption with their smartphones makes for great movies, but poor real world examples. In the majority of cases, attackers go for easy targets: web frameworks with security vulnerabilities, out of date systems, administration pages open to the Internet with guessable passwords or security credentials mistakenly leaked in open source code are all popular candidates. The goal of Test Driven Security is to take care of the baseline: apply elementary sets of controls on applications and infrastructures, and test them continuously, directly inside the DevOps deployment pipeline.
A baseline of security controls defines the minimal requirements applications should match before being deployed to production. The controls are simple and specific, such as:
- All websites must implement a Content Security Policy
- Form submission must require CSRF tokens, unless explicitely whitelisted
- SSH Root login must require sudo on all systems
- The rules in firewalls and security groups must be tested at every deployment
- HTTP traffic is prohibited, HTTPS endpoints must use Mozilla's modern guidelines
- Outdated and vulnerable dependencies must be upgraded
The list of security best practices is established by the security team with the help of developers and operators to make sure everyone agrees on their value. A list of baseline requirements can be assembled quickly by collecting those best practices and adding some common sense. The controls themselves are simple and do not require particular expertise, the difficulty comes from testing and implementing them everywhere and all the time.
This is where Test Driven Security comes in. TDS is a similar approach to Test Driven Development (TDD) which recommends developers to write tests that represent the desired behavior first, then write the code that implements the tests. TDS proposes to write security tests first, thus representing the expected state, and then implement the controls that pass the tests.
The TDS approach brings several benefits:
1. Writing tests forces security engineer to clarify and document expectations. Engineers can build products with the full knowledge of the required controls rather than catching up post-implementation.
2. Controls must be small and very specific units which are easy to tests. Vague requirements such as “encrypt network communication” are avoided, instead we will prefer the explicit “enforce HTTPS with ciphers X, Y and Z or all traffic”, which clearly states what is expected.
3. Re-usability of the tests across products is high, as most products and services share the same base infrastructure. Once a set of baseline tests is written, the security team can focus on more complex tasks.
4. Security regressions are caught in real-time, prior to deployment, rather than periodically during manual reviews.
Let’s take an example. A good practice for web applications is to enforce the use of HTTPS on all traffic, which can be done using HTTP Strict Transport Security (HSTS). A website can set a HSTS header returned with HTTP responses to tell web browsers to always HTTPS when connecting to the site, never HTTP. It’s a simple control, trivial to configure at the web server or application level, that we can very easily test for by looking at the headers returned by a website.
The code sample below shows how a simple Bash script can check the value of HSTS on a local application (https://localhost:8080/). This test can easily be part of the integration pipeline, perhaps run automatically as part of a pull request. The test should be defined ahead of implementing HSTS itself. It will fail when the HSTS header is unset or when its value is lower than 90 days. As soon as the website returns an acceptable value, the test will succeed.
#!/usr/bin/env bash
hsts=$(curl -si https://localhost:8080 |
grep 'Strict-Transport-Security:' |
awk '{print $2}' |
cut -d '=' -f 2 | sed 's/\r//')
if [ "$hsts" != "" ] && [ "$hsts" -gt 7776000 ]; then
echo "HSTS test passes. value is $((hsts/86400))days"
exit 0
else
echo "Strict transport security is lower than the expected 90days value"
exit 1
fi
Tests in the TDS approach will fail initially. This is expected to verify their correctness once they pass after the feature is implemented. Security teams should help developers and operators implement controls in their software and infrastructure at first, taking each test one by one and providing guidance on implementation, and eventually transfer ownership of the tests to the DevOps teams. When a test passes, the teams are confident the control is implemented correctly and the test should never fail ever again.
An important part of TDS is to treat security as a feature of the product. This is achieved by implementing controls directly into the code or the systems of the product. Security teams that do not collaborate with developers, and implement security outside of the applications and infrastructure, instigate a culture of distrust that eventually puts organizations at risk. You should shy away from this approach. Not only does it create tensions between teams, it also provides poor security as controls are not aware of the exact behavior of the application and miss things. A security strategy that isn’t owned by the engineering teams will not survive very long, and will slowly degrade over time. It is critical for the security team to define, implement and test, but it is equally critical to delegate ownership of key components to the right people.
TDS adopts the DevOps principles of automating the pipeline and working closely with dev and ops teams. It forces security folks to build and test security controls within the environments adopted by developers and operators, instead of building their own separate security infrastructure.
In the presentation, I will show how we implement TDS using a variety of open source tools:
- OWASP Zaproxy takes care of the baseline scan of web applications. We have worked closely with the core team of ZAP to implement Test Driven Security. We run baseline scan in two ways: 1) In the CI pipeline, like Travis-ci or CircleCI, to give feedback on pull requests and 2) in Jenkins as part of the deployment pipeline of each service, to test staging endpoint every time they are deployed.
- Static analysis tools, like jshint or gas, are used to inspect the source code of applications in CI.
- Dependency management tools, like NSP (Node Security Project) or goreport, are used to check for issues in third party of the applications. We also use closed source platforms like requires.io or greenkeeper.
- Infrastructure auditing is done via tools like Pineapple, which is used to verify the state of security groups in AWS.
Over the past 18 months, we have implemented TDS over a hundred of applications, websites and services and successfully improved our security posture. In this presentation, I will describe the techniques we developed to integrate TDS deep into the DevOps pipeline. The tooling we created will be presented to the audience, with links to resources they can use freely. I will also discuss the human aspect of integrating security processes into development and operations workflows, and how to succeed without disrupting slowing down organization’s SDLC.
