Nightmare on Dev Street: 5 Terrifying Situations to Make Your Spine Tingle

Oct 24, 2022 2:38:15 PM | Software Development Nightmare on Dev Street: 5 Terrifying Situations to Make Your Spine Tingle

In this blog, we'll discuss 5 nightmare situations developers tend to deal with, as well as some free solutions you can use to prevent these nightmares from coming to life.

Errors are already a nightmare, but they're not the only thing keeping developers up at night (Thanks, PagerDuty). As a developer, some of your worst fears may include servers going down, losing all your data, or latent errors, to name a few. 

In this blog, we'll discuss 5 nightmare situations developers tend to deal with, as well as some free solutions you can use to prevent these nightmares from coming to life. 

A Data Center or Zone Goes Down 

In theory, you should design your systems to handle any failure (that's why there are multiple zones and regions). The keyword here is "theory." When you're setting up your systems, there's always a chance you missed something in your planning. 

A great example of this horror story is when Amazon brought down one too many servers, shutting down the entire Virginia region! In 2017, the Amazon Simple Storage Service attempted to debug an issue slowing down their billing system. A member of the team executed a command meant to remove a small number of servers and unintentionally removed more servers than expected. Yup, even giants like Amazon can make mistakes. That's why you need to know precisely when a zone goes down or an outage occurs.

Free Tool Suggestion: The Outages Project

The Outages Project tracks outages via "git-scraping." You can also start monitoring outages by answering a simple series of questions.

Losing All Your Data 

Have you ever heard about a Junior Developer deleting their production environment? Yeah, it happens. See this Reddit post. To sum up: New Junior developer accidentally deletes the company's production database. Here's the scariest part: the devs couldn't restore the database using their backups. 

Yeah, this whole situation is a nightmare. But, seriously, who gives a brand new Junior Developer the keys to the production database and doesn't provide ANY supervision? 

That's why you need to back up your data, then back up that data, and so forth. More importantly, the backup process should be completely automated.

Free Tool Suggestion: Mongodb Backup

The Mongodb-Backup and Mongodb-restore allow you to automate the process of automating your backups to Mongodb (if that's what you're using). If you're looking for something a bit more robust or you're not using Mongodb, here's a great list of 7 database backup tools. The important thing is to back up your data and, you know, make sure it's usable.

Latent Errors

No matter how much you test your code in staging, there's always a chance that a sneaky error is hiding in wait, ready to pounce as soon as you activate specific paths in production. These are called Latent Errors, and they're evil little buggers. Essentially, these errors already exist in production but have yet to surface. 

Here's a frightening example of a latent error from this Stackoverflow answer. A Boeing 777 aircraft suffered from a latent error due to their system using data from a failed accelerometer, which activated the stall warning while in autopilot mode. Thankfully, the pilots could manually fly the plane back from where they took off, and no one was hurt. While this is an extreme case of the impact of a latent error, it does an excellent job of how these errors can pop up out of nowhere. 

So, what can you do about an invisible error?

Free Tool Suggestion: Error Monitoring

In the case of errors, whether they're active or latent, you need a reliable error monitoring tool. If you're not too familiar with error monitoring, it's a tool that alerts you when an error occurs within your project. That means real-time error alerts, perfect for catching latent errors before they impact too many users. One tool we're fond of is Airbrake Error and Performance Monitoring (obviously since we're Airbrake, lol), which has a free forever developer tier.

Memory Leaks

Memory Leaks can also be rather ghoulish. Pierre DeBois, in his article, "Understanding Memory Leaks in Programming," describes a memory leak as "a type of resources mismanagement in programming…They occur when programming objects are stored in computer memory, be it a laptop or smartphone, but then the allocated memory is not released as designed with the programming." The result is a degradation in performance. This leak, if left unchecked, will begin to consume available memory. 

Fortunately, you most likely already own a free solution to detect memory leaks! 

Free Tool Suggestion: Chrome Developer Tools

Chrome Developer Tools allows you to take heap snapshot images and Allocation Instrumentation on Timeline recordings, which provide metrics such as jsHeapSizeLimit, usedJSHeapSize, and totalJSHeapSize. For more information, check out the article we linked above! DeBois goes through the entire process step by step.

Slow Response Time

I don't need a specific example here because we've all experienced this! You're on a website or application, and it takes FOREVER for the page to load. Slow load times are rather common in programming. With every new commit or a new feature, there's a chance of performance degradation. For a full explanation of why this occurs, check out Johny's Software Lab's blog post, "Why do programs get slower with time.

Every SaaS app is at risk of slow response times. The question is, how do you know that your app is slowing down? 

Free Tool Suggestion: Lighthouse and Airbrake 

Google Chrome Lighthouse should be a staple in monitoring slow response times within your application. Lighthouse analyzes the Performance, Accessibility, SEO, and other essential metrics related to the performance of web apps and web pages.

If you want a free tool that shows you slow paths within your application, once again, we have to recommend Airbrake, which can pinpoint routes with slow response times. 

And those are just a few of the nightmare issues developers face. But, we're hoping that by providing some free tools, you can alleviate some of the pain points associated with servers going down, data loss, latent errors, memory leaks, and slow response times. 

Written By: Alexandra Lindenmuth