Congratulations, you failed!

For those of you that know me, or have seen one of my conference talks, you know that one of my favorite phrases is: “if it hurts, do it more often”.

Reading that, you might think I’m some sort of masochist. And it’s probably a phrase you won’t hear your doctor say to you. He’ll probably tell you to stop doing whatever it is you were doing that ended up hurting you.

Full disclosure: I didn’t coin that phrase.

What does it mean though? What’s hurting, what’s so painful? It’s the cumulative effect of frustrating experiences, of performing activities that didn’t work, of doing things that failed. The result? To quote Martin Fowler: “The natural human response, therefore, is to put off doing it for as long as possible.”

On failure

First, some thoughts about failure.

For a lot of people, failure has a negative connotation. For a lot of people, failure is associated with pain. I think I understand where that’s coming from. Our culture directly links failure with negative emotions and negative consequences. One only has to look up “failure” in a dictionary.

failure
Pronunciation /ˈfālyər/ /ˈfeɪljər/ 
NOUN

1. Lack of success.
2. The omission of expected or required action.
3. The action or state of not functioning.

Oxford English Dictionary

Personally, I don’t consider “failure” to be a negative thing. Like Arianna Huffington, I don’t see failure as being the opposite of success. As a matter of fact, I think failure is an opportunity!

An opportunity to learn, grow and improve. Failure can be that wake-up call you need, that helps you strip away the cobwebs, find out what’s really going on and use that knowledge to your betterment. Use failure as motivation, use failure to help you find (and go beyond) your limits.

Paraphrasing Taleb, failures are not something to get hung-up about, the best results come from trial and error. And that means trying a lot. Failure then is not something to waste by quitting, stepping away. That’s the easy way out! “Never give in. Never give in. Never, never, never, never.” (Winston Churchill)

It’s time to embrace failure!

Failure and IT

How does failure relate to IT? And how does failure relate to modern software delivery methodologies such as Cloud Native, DevOps and Continuous Delivery? They have a closer relation than you might think.

Hardware fails. Power supplies break, disks die, motherboards quit. This is particularly relevant for public cloud providers, as they are usually built on commodity hardware.

Hardware is not the only thing that can fail though. In any virtualized environment, you will be confronted with network latency and bandwidth problems, noisy neighbours, virtualization issues, storage problems, etc.

“Everything fails all of the time in the cloud”

Werner Vogels (CTO, Amazon)

Failure happens, failure is inevitable. It’s how you respond that matters. So we need to be aware of, prepare for and embrace the concept of failure in those environments.

That’s also why chaos engineering tools like Netflix’s Chaos Monkey (that actively and intentionally introduces infrastructure failures, amongst others), were created. For the sole purpose of exposing weaknesses, errors or design flaws in environments and enabling organizations to design for failure and improve how their systems responds to failures. And it’s not limited by mere tools!

Additionally, patterns such as Circuit Breaker were developed to allow for, and contain failures in service-to-service communication. Health checks were introduced to quickly detect failing instances of applications and replace them with new instances.

Failure and Continuous Delivery

Back to “if it hurts, do it more often”.

This phrase was made famous by Jez Humble, one of the godfathers of Continuous Delivery. Continuous Delivery helps in dealing with failures by detecting issues early in the software delivery process, significantly reducing the cost of rework. This is achieved by splitting big problems into smaller problems. Instead of big steps, we take small steps.

“Continuous Delivery is the ability to get changes of all types—including new features, configuration changes, bug fixes and experiments—into production, or into the hands of users, safely and quickly in a sustainable way.”

At the core of Continuous Delivery is automation. Automating the repetitive, tedious and repeatable tasks, freeing up human decision making capacity where it matters and adds the most value. We automate building, deployment, various types of testing, configuration, provisioning, etc.

All these steps are executed in pipelines that deliver fast feedback. Failures are detected quickly and notified to the engineers so they can take appropriate action and recover. When a pipeline fails (because of issues with a deployment, a failing test, etc.), we treat it as a “stop-the-world” event where a team drops what they’re doing to fix the pipeline.

Now, when it comes to pursuing Continuous Delivery, it’s important to gather some metrics. Metrics that can help track your progress along the way and give insight in (for example) delivery stability and speed. Steve Smith discussed some of those metrics in his book Measuring Continuous Delivery.

In summary

So, it looks like failure is all around us, especially in IT.

We can choose to put off doing things to avoid the pain, or we can embrace failure. For failure is an essential part of learning. And if you’re not learning, what’s the point of living?

Is this easy? No. But then, most interesting things aren’t.

Congratulations, you failed!

PS: The above might sound like I’ve got it all under control. My own failures – especially when I’m not reaching the high bar I set for myself – still frustrate me from time to time though. Nobody’s perfect.