Stopping DDoS in its Tracks

I often get asked, "What’s the best equipment one can buy to protect my company from a Distributed Denial of Service (DDoS) attack?" While this is a valid question it is making a very large number of assumptions – namely, that you have a very large volume of traffic and a very large budget. While, performing mitigation themselves is valid for some companies, this is rarely the case and, in this blog post, we are going to uncover the considerations you must make.

The asset

Before we start talking about how to defend, we need to understand what we are defending, and namely how much an hour of downtime would cost if the asset is not available. This can be from a fiscal or purely reputational point of view. In some cases, like online retailers, there is the added issue of “customer stickiness”. Studies suggest that consumers will stick to the last supplier they used for a particular item, so an hour outage due to a DDoS may have a lasting effect beyond the hour. On the other hand, for something like a local manicure shop, it may not be critical losing their online presence as this would only cause them to receive more phone calls to make those bookings over the phone and the cost will be the lost time to make those bookings.

Either way, once we have the cost of an hour, a minute, or a day of an outage; we can proceed to the next step which is the risk of such an event occurring. Knowing those two numbers we can establish the impact of a DDoS event, which in turn will tell us what level of resources we can invest in to proactively prepare for such an event.

While the cost and risk are something very individual to a company and outside of the scope of this article, it is important to talk about one of the variables in this equation.

The attack(er)

A component of the risk is the skill set of the attacker. While some sectors are more likely to be attacked by sophisticated attackers, there is a new player on the block that levels the field – enter the “Booters.”

The so-called Booter services emerged about 5 years ago around the massive adoption of real-time online gaming. In those games, it is of significant advantage if one can delay their opponent during a raid or campaign. This can easily be achieved with a short DDoS on the opponent, and this is how the Booter services were born. They would offer a fairly cheap way to execute a DDoS attack in the form of a monthly subscription to the service varying between $50-150 US. Even as of 2019, some of those services were able to execute attacks reaching 40Gbps.

The optimal strategy

Now that we know what we are protecting, what the risk is, and the ROI of that effort, we need to pick a strategy. In summary, there are three options – onsite (or DIY, do-it-yourself), outsource using a service provider, and hybrid.

Do-it-yourself (Onsite)

Onsite is the first thing that comes to mind for most people when they think about DDoS mitigation. In this case, the company would procure and install DDoS mitigation equipment in the same location where their origin servers are, usually in a data center close to their business and sometimes on their premises. Note the term “origin server”, it designates the server originating the content.

One serious downside of this method is that the majority of the DDoS attacks are volumetric and before they even reach this mitigation equipment, they must be transported to it. As we saw earlier, a Booter service can deliver 40 Gbps of traffic, so one must be able to transport it to the mitigation equipment before it can mitigate it. This means the procurement of at least 40 Gbps of connectivity to the Internet. Most websites would be running less than 2-3 Gbps of traffic and maybe over-provisioned to 10 Gbps in case of a traffic spike. Being able to withstand 40 Gbps, means significant investment for 40 Gbps transport all the time. But what if the attack is 41 Gbps?!? How much bandwidth should be provisioned? And do you have the budget to pay for this excess capacity for the rare occasion you may be attacked?

Now, how about personnel? In my experience, most network teams are not well suited to deal with DDoS and many of the assumptions they have do not work. In an attempt to offset this expense, some hardware vendors offer remote management services. Usually, this is a model that does not work well and adds an excessive amount of additional cost coming from the cost of the support contract and the vendor push for equipment upgrades. So, if you think you should outsource equipment management, then it’s probably better to have the vendor manage their own equipment and thus their own capital investment, not yours.

This is why some companies would completely outsource DDoS mitigation to a service provider who uses the economy of scale and aggregate a large amount of connectivity and computes resources. Furthermore, they have a very distributed infrastructure and their deployments are strategically placed in Internet Exchanges (IXs).

DDoS Mitigation Service Provider (Outsourced)

There are two models of operation – on-demand and always on. In the former case, traffic flows to the origin server, and when an attack is detected the service provider starts announcing the origin server IP space out of their points-of-presence (POPs), which attracts all traffic there. Then it is “scrubbed” and the “clean” traffic is sent back to the origin servers, usually over GRE tunnel. See animation.

One downside with this method is that some traffic may be lost in the windows of detection, decision, and route propagation. Another problem arises if the attacker is sophisticated and has mapped the infrastructure before executing the attack. In this case, they can simply attack the IP address of the network interface between the Internet service provider and the origin server, thus preventing the clean traffic from reaching the origin servers. Just as an illustration, Google recently experienced a similar attack of the size of 800 Gbps attacking 100Gbps peering link.

This is why it is better to use the always-on model, where the defense infrastructure is not exposed. Note that there are added benefits. As already noted, their POPs are in Internet Exchanges, so they are very close to the end-users. This decreases the round-trip time and drastically decreases the time for the 3-way handshake. Many of them also support connection pooling, which means the POP servers have a long-lasting connection with the origin server so they do not need to establish a new connection, and the window is already very large so entire objects can be sent at once without waiting on an ACK packet.

Some of the DDoS Mitigation providers are also a Content Delivery Network (CDN), or to be precise CDNs usually offer DDoS mitigation. A CDN adds additional benefit as it caches the static objects and they do not need to be retrieved from the origin server but can directly be served. And yet another benefit is that some CDNs even allow the execution of custom code in their POPs which can further accelerate the application.

A potential downside with the distributed edge architecture may be regulated industries that cannot share their TLS keys with the service provider so they can terminate their traffic. Even in those cases, it is possible to find a middle ground. For example, Cloudflare offers the Keyless SSL feature, which allows for the TLS temporal key negotiation to happen on the origin server but the actual decryption of traffic encrypted with that key may happen on the edge node.

Hybrid

Now that we have covered the two extremes let’s look at the hybrid approach. In this approach, the idea is to mitigate small attacks on-premise and when the attack size passes certain threshold mitigation is moved to the cloud. To a very large extent, it is similar to On-demand DDoS mitigation, but it requires expensive equipment onsite to do the initial mitigation.

While this method may appear to combine the best of both worlds it indeed combines the worst. As described in the On-demand method, if the attacker runs a traceroute before the attack starts, they will be able to map the clean path where GRE tunnel packets will flow after the server IP space is advertised by the service provider and attack it during the attack. Another problem with this type of mitigation is that it is very limiting as not all hardware vendors support all types of cloud mitigation.

The Hybrid method can still be useful if a company is hesitant to move all of its mitigation capacity to the cloud and wants to try it first while leveraging already existing deployments. However, it is important to note that an advanced attacker would very likely exploit the lack of protection of the clean path.

Summary

While there are three groups of DDoS mitigation strategies, one of them appears to be optimal for most companies. Due to the unbalance of how easy an attacker can send large amounts of attack traffic and how expensive it is to keep dedicated equipment for that rare occasion, it is becoming apparent that it is better to outsource this capability.

Of course, there will be exceptions and this is why a company needs to carefully analyze their risk, the value of online presence, and ROI on equipment and personnel.