Why Data Matters in AI-Powered Bot Detection

How Can We Distinguish Bots That Behave Like Humans?

Automated bot traffic now accounts for more than half of all web traffic, according to the 2026 Bad Bot Report. For businesses, blocking malicious bots and macros has become essential to maintaining fairness and preventing unnecessary resource consumption.

But the real challenge is not simply that there are more bots. It is that bots are increasingly behaving like humans.

In the past, automated programs were relatively easy to identify. Their machine-like behavior was often obvious—for example, sending hundreds of requests from the same IP address within a short period or repeatedly accessing the same URL at fixed intervals.

Today’s macros and malicious bots are much more sophisticated. They can rotate IP addresses and User-Agents, imitate real browser environments, and adjust request intervals to make their activity resemble that of legitimate users.

As a result, it is becoming increasingly difficult to identify bots based on a single condition such as “too many requests” or “requests arriving too quickly.” Bot detection must ultimately answer a more fundamental question:

“Is this behavior something a real user would reasonably do?”

This is where AI and machine learning become increasingly important.

AI Detection and Data Management

Instead of relying on people to manually identify rules across countless access requests, AI can analyze a wide range of traffic characteristics and behavioral data to detect patterns that differ from normal user behavior.

However, there is an important prerequisite.

For AI to distinguish normal from abnormal behavior, it first needs a reliable basis for understanding what normal user behavior actually looks like.

Traffic data collected from real services may contain not only legitimate user requests, but also repetitive requests generated by macros, abnormal API calls, and automated purchase attempts. If all of this data is treated as normal service usage without proper classification, the model’s understanding of what constitutes “normal” behavior can become distorted.

Contaminated training data can reduce detection accuracy. What matters is not simply how much data is available.

Which data should be considered legitimate user data?
Which behaviors should be classified as automated or abnormal?
And how should those decisions be reflected in the training data?

The quality of AI-based bot detection can begin to diverge at this very stage of data selection and classification.

AI Detection and Service-Specific Context

The definition of normal behavior can vary significantly from one service to another.

On a typical web service, repeated requests within a short period may appear abnormal. But when ticket sales open or course registration begins, legitimate users may also refresh pages and submit requests repeatedly within a very short period.

Conversely, sophisticated macros may intentionally slow down their requests or imitate a normal sequence of page navigation.

This means that simple rules such as “fast requests are bots” and “slow requests are humans” are not enough to make accurate decisions.

To define the boundary between normal and abnormal behavior more precisely, bot detection must take both the characteristics of the service and the situation in which the traffic occurs into account.

What Makes AI-Powered Bot Detection Competitive?

Detection methods may vary, but conceptually, an AI-powered bot detection system can be simplified into the following process:

• Collect data

• Select valid and relevant data

• Classify and label human, bot, and suspicious traffic

• Analyze behavior and access patterns

• Learn normal patterns and establish baselines

• Detect anomalies

• Feed detection results back into data and policies

As bots continue to imitate human behavior and new forms of automation emerge, bot detection must do two things at the same time: identify the characteristics of known bots and discover abnormal behaviors that have not yet been seen before.

In other words, the competitiveness of AI-powered bot detection comes from the combination of AI, data, service context, and detection experience.

It is also important to recognize that not every bot should be blocked.

Some forms of automated traffic—such as search engine crawlers and certain AI crawlers or agents—may be useful or necessary for service operations and information distribution.

Bot management therefore needs to evolve toward understanding the characteristics and purposes of different types of traffic—including real users, search engines, AI crawlers, suspicious automation, and malicious bots—and responding to each of them differently.

This is why detection accuracy matters.

AI Bot Detection Ultimately Requires an Understanding of Traffic

STCLab’s BotManager does not determine whether traffic is generated by a bot based on a single condition. Instead, it analyzes multiple signals together, including access environments and behavioral patterns.

BotManager evaluates traffic from multiple perspectives using various detection policies, including IP-, country-, and ASN-based firewall rules, custom filters, behavioral patterns, and access environment analysis. It also analyzes request patterns and risk levels based on identifiers such as login IDs, session IDs, and IP addresses.

More recently, BotManager has introduced AI-powered risk analysis and policy recommendations, as well as policy-specific threshold recommendations based on historical traffic.

However, the competitiveness of AI-powered bot detection cannot be explained by AI models or individual features alone.

Across environments such as ticketing, e-commerce, finance, education, and reservation services, user behavior and traffic characteristics differ significantly. Understanding which requests are legitimate and which behaviors are abnormal ultimately requires traffic data accumulated from real-world services and the experience to analyze it.

Since 2010, STCLab has developed and delivered NetFUNNEL, its virtual waiting room solution for controlling large-scale access traffic, to customers in global markets. Through NetFUNNEL, which has been deployed across the critical service environments of more than 600 customers, STCLab has accumulated extensive experience in analyzing and controlling high-volume traffic.

Going forward, this experience will serve as an important foundation for advancing AI-powered bot detection—helping distinguish legitimate users from automated requests with greater precision and establish normal behavior baselines tailored to the characteristics of each service.

What Completes AI-Powered Bot Detection?

AI can make bot detection more sophisticated. But the level of a detection technology cannot be judged simply by asking whether it uses AI.

• What data is being analyzed?

• What is classified as legitimate user data?

• How accurately can the system distinguish the boundary between normal and abnormal behavior for each service?

As more bots learn to behave like humans, competitive bot detection will not come simply from adding more rules or applying more complex AI models.

The ability to understand legitimate user behavior and classify data according to service-specific criteria will become a core competitive advantage in AI-powered bot detection.