16 June 2022

How to tell legitimate visitors from bots: non-obvious cases

As you know, it is extremely difficult to correct mistakes made at the design phase of a new product. This is completely true for various aspects of protecting applications, electronic and digital services from DDoS attacks: These aspects must be worked out in advance, establishing their resistance to DDoS attacks at the early stages of software product development.

In particular, it is very important to clearly define what signs can be used to identify legitimate requests so that the anti-DDoS provider can accurately identify the beginning of an attack on the application and defend against illegitimate requests. Application developers and their customers should have an idea of how a DDoS defender thinks and acts, and ensure up front that it can detect and disable bots. This is especially important for applications that implement HTTP services that do not use a browser, and for UDP services.

In very general terms, the DDoS protection system can be described as follows. First, a detailed description of how the application works (its profile) is created: from which websites the requests are coming, what headers and methods are used, with what intensity, etc. Based on this description, a model of normal interaction is created, with which all requests made to this application by different visitors and customers are compared.

If there are no signs that can help distinguish legitimate requests from illegitimate requests generated by DDoS bots, it becomes very difficult to detect the activities of these bots. Here is an example. One day, we detected a suspicious wave of requests to a remote banking service application of one of the financial organizations. Normally there was a flow of 500 requests per second, but then suddenly there was a sharp increase - the flow grew to about 10,000 requests per second. Interestingly, 200 requests per second came from each source address, and all of them received a 401 Unauthorized response ("access denied"). Suspecting the beginning of the attack, we decided to block such requests, but soon the bank contacted us and expressed their confusion at the fact that legitimate visitors were blocked. In order to understand the situation, we contacted the developers of the application, and they explained to us that nothing unusual had happened, the application had just started downloading data, so this wave of requests should be considered normal behavior. We were surprised: if this behavior is considered normal, what does a DDoS attack on this application look like? And how to distinguish the normal activity from the one caused by intruders?

Most often, such incidents occur in companies that support their own APIs or mobile apps. Their developers and customers need to think in advance about how a DDoS protection provider will be able to distinguish legitimate visitors from bots for HTTP services designed for non-browser interaction and for UDP services. If these aspects are not worked out, the only signs of suspicious activity will be the presence of the IP addresses of the sources of the requests in some databases of unwanted IP addresses and a suspiciously large increase in the flow of requests. And if the bot's activity does not exceed the activity of normal visitors, no defender will be able to block it.

This is especially true for applications that use UDP messages. Since it is more difficult to filter UDP traffic than TCP traffic, a DDoS defender must have a thorough understanding of the protocols of interaction between your UDP applications to accurately determine whether the next sender is legitimate. Sometimes legitimacy can be judged by the pre-approval process (if it is possible). In some cases, there are clear rules for a UDP application's interaction with legitimate clients, which the anti-DDoS provider's filtering system can use to determine that this client is legitimate. And sometimes it is possible to assess legitimacy using a TCP service.

In any case, you and your developers should clearly understand and describe how you will distinguish legitimate from illegitimate requests at the very beginning of building an application. This knowledge must be clearly recorded and then passed on to the DDoS defender.

Or talk to your anti-DDoS provider and find out how your application works so they can build their profile detailing what their normal activities are. Anything that deviates significantly from this can be considered the action of bots and cut off, freeing up the application and the computing environment in which it operates for interaction with legitimate visitors.