17 November 2022

The mistake cascade increases the drop in resistance to DDoS risks

Mistakes in the organization of protection against the risks associated with DDoS attacks almost always lead to a reduction in the resilience of their Internet resources to these risks, and it is impossible to compensate for them solely by connecting anti-DDoS services, even when they are most advanced. The situation is often aggravated by the fact that the combination of several flaws increases their overall negative impact.

In this article, we will analyze some of these flaws we encountered while building protection against DDoS risks at a rather large client managing several hundred websites.

Mistake # 1: DDoS protection covers only part of the Internet resources

The client connected only a portion of their resources to our cloud DDoS protection services, leaving about a hundred websites unprotected. More specifically, the protection was only connected at the L3/L4 layers according to the OSI model, while the HTTP application layer (L7) remained unprotected at that time. Obviously, the client believed that no one would ever find these websites and would not be interested in them. Perhaps this would also be the case if they were normal users.

However, the attackers had no rest, they found these websites and took them down with a DDoS attack at the application layer. 

This example proves once again that partial protection against DDoS attacks proves to be "leaky" and thus ineffective in practice. To reliably protect Internet resources from DDoS risks, it is necessary to secure them across the board and take a whole range of interrelated measures to reduce these risks (we have outlined the details of our approach in this article).

And since the attack was very intense, the anti-DDoS services were able to quickly identify the main sources of illegitimate traffic and suppress most of the attack at the packet level. However, the volume of the remaining traffic was enough to make the client's unprotected sites inaccessible at the L7 layer.

To fully solve the problem, it was necessary to include filtering services at the application level. We managed to do this relatively quickly: after receiving a complete list of the incompletely protected sites from the customer, we placed them under the protection of our L7-layer anti-DDoS services.

Mistake # 2: The customer's network was announced not only to us, but also to two other ISPs

Our customer decided to "hedge their bets": they registered their network not only through StormWall, but also through two ISPs, and considered their channels as backups. In order to route traffic primarily through the StormWall network, he attempted to degrade his path through two other providers by using the announcement with BGP prepend mechanism.

However, this technique did not work: since announcements with BGP prepend parameters are filtered by many providers, the traffic went not only through StormWall, but also through other providers. And when the DDoS attack started, three quarters of the illegitimate traffic was directed to the client's Internet resources, bypassing StormWall protection.

It took more than an hour to provide an exclusive announcement of the network (in other words, so that the traffic only went through StormWall filters), while the attack continued.

There are three ways to ensure the protection of Internet applications and websites that work over HTTPS:

• with the disclosure of the SSL private key
• with the receipt of the certificate and the “Let's Encrypt” private key by the anti-DDoS service provider, without decrypting SSL
• by providing the anti-DDoS provider with the content of the application server's system logs

In any case, the anti-DDoS provider must receive data about how data is exchanged at the L7 level - without this information it is impossible to build reliable protection of websites and Internet applications.

In our particular case, we applied protection with the disclosure of the SSL private key, as the resources were informative in nature and there were no strict requirements for the protection of the transmitted data.

Mistake #3: The ACL mechanism was applied on GRE tunnels

After all the traffic was redirected to the StormWall filters, it turned out that some of the network packets had been lost somewhere. The reason could be, for example, excessive load on the routers. However, our customer claimed that his routers were very powerful and, according to him, were not overloaded.

After diagnosing the situation, our specialists found that the loss of network packets occurred in the GRE tunnel. At the same time, the bandwidth of the channel was quite sufficient and the traffic volume in the tunnel GRE was not too high.

Upon further analysis, it turned out that the ACL (Access Control List) mechanism on the customer's routers connecting through the GRE tunnels was applied to the tunnel interfaces, which resulted in the packets being processed programmatically rather than hardware-wise, which significantly affected the performance of these routers.

After the customer's specialists disabled the ACL mechanism on routers connected to GRE tunnels, there was no more packet loss.

* * *

Although the listed errors are not serious, their combination caused a significant portion of our client's Internet resources to be unavailable. Normal access to these resources was restored only after about four hours.

And here are the three most important recommendations as a summary for this case:

• For DDoS protection to be effective, it must be comprehensive, covering all Internet resources and all layers that can be used to launch DDoS attacks against those resources.
• To minimize the volume of unauthorized traffic in DDoS attacks, all traffic must pass through the filters of anti-DDoS providers.
• To ensure high resilience of Internet resources against DDoS attacks, the operation of routers must be optimized to achieve their maximum performance. In particular, it is necessary to ensure that the ACL mechanism in the tunnel GRE does not work.