Thoughts on HBO Max
Recently, in case you weren't aware, HBO Max sent out an integration testing email to their real mailing list. The email got sent to quite a few people and it started trending on social media, especially if you follow a lot of software development stuff. For the most part I've enjoyed reading about the various ways in which people have brought down production. I shared one of the times it happened to me and I could share several more stories about bringing down our live application. However, after reading a thread by @justkelly_ok it made me shift how I looked at what was happening on social media. Then, this morning, I saw something I thought was so cringe-worthy I had to stop what I was doing and write this post.
Focus on HBO Max
The focus this entire time has been on the intern. While I want other developers to share stories about how they've also brought down production and words of encouragement we should also spend more time critically analysing what happened from an organizational standpoint. While HBO not simply firing the intern and helping them work through the problem is commendable it is also a pretty low bar if we're being honest with ourselves. If you look at what happened from a systems standpoint there's some serious concerns going on here, most of them @justkelly_ok mentioned in her thread.
Lack of Security 101
I'm not an infosec guru but I do know the basics of securing software and business operations. One of the first, and most important, is the principle of least privilege. This is something that I have discussed with my CTO on numerous occassions and we're always auditing our access controls to make sure people have access to what they need and only what they need. That an organization the size of HBO would not have sufficient controls around this is astounding to me. There is simply no excuse for an intern-level employee to have access to the production mailing list, let alone the servers to actually do anything with that list. If their security around this is lax there is reason to believe that other important security features are not implemented correctly.
It is never 1 person's fault
Unless you're literally building the app, deploying it, and running the business operations by yourself it is likely never the sole fault of any one individual. More than 1 person should be involved in virtually all projects. Whether it be interactions in the design process, pair programming during implementation, or code review before deployment there are numerous steps along the way for other people to provide feedback and catch potential problems. Sometimes those problems still get through but it isn't solely because of one individual. Software is a team sport and you should be working as a team. If this intern was able to get all the way to running integration tests with no feedback from somebody else that's on the organization, not the intern.
I would leave HBO Max
Although the company didn't fire me and helped me work through my mess-up that's the minimum I would expect! That shows you're not just total shitheads. It doesn't necessarily show that you're a good company. Whether they intended to mean harm by it or not the truth of the matter is that HBO Max threw that intern under the social media bus when they effectively blamed it on them. Sure, we saw a lot of #hugops but I bet there was a lot of nasty, vile shit from the trolls and douchebags that inhabit the Internet. Not only did I mess up on an epic scale but my company made sure the whole social media world was talking about me? Nah, I'm good with that kind of anxiety…as soon as my internship was up I would be out of there.
Thank you, @justkelly_ok for making me look at this situation with a different perspective. I would have likely just smiled at the stories of encouragement and went along my way. However, now I'm looking at HBO Max with a little more scrutiny.