epoll vs. Busy Polling in Low-Latency Trading Systems | Trading Engineering — Deep Dive #5

epoll vs. busy polling in low latency systems

epoll vs. Busy Polling in Low-Latency Trading Systems

Trading Engineering — Deep Dive #5

When developing network applications on Linux, epoll() is one of the most commonly used mechanisms for handling I/O events.

It is highly efficient for general-purpose servers.

However, low-latency trading systems sometimes require a different approach.

When market data or order messages must be processed with extremely low latency, the way the CPU waits for data can itself become part of the latency problem.

This article compares epoll and Busy Polling / Busy Looping, and explains why some low-latency systems deliberately use CPU resources to minimize waiting time.

---

1. How a Typical Network Application Works

In a conventional server application, there is no reason to continuously consume CPU resources while waiting for network data.

A typical flow looks like this:

Application
     ↓
   epoll()
     │
     │  Waiting...
     │
     ↓
Network Event
     │
     ↓
read()
     │
     ↓
Process Data

When there is no data, the application waits inside epoll_wait().

This allows the CPU to perform other work.

In other words:

When there is no data, the CPU does not need to keep running the application.

This is a very efficient design for web servers, API servers, and conventional TCP applications.

---

2. What Is epoll?

Linux epoll is an I/O event notification mechanism designed to efficiently monitor multiple file descriptors.

For example, a single process may manage many sockets:

Socket 1 ─┐
Socket 2 ─┤
Socket 3 ─┤
Socket 4 ─┤──→ epoll_wait()
Socket 5 ─┤
Socket 6 ─┘

When data becomes available on one or more sockets, epoll_wait() returns the corresponding events.

A simplified example looks like this:

struct epoll_event events[64];

int n = epoll_wait(
    epfd,
    events,
    64,
    -1
);

for (int i = 0; i < n; i++) {
    int fd = events[i].data.fd;

    if (events[i].events & EPOLLIN) {
        read(fd, buffer, sizeof(buffer));
    }
}

For general-purpose network applications, this is an extremely useful model.

---

3. The Core Idea of epoll: Wait for Events

One of the key characteristics of epoll_wait() is that the application can wait until an event occurs.

Conceptually:

CPU
 │
 ↓
epoll_wait()
 │
 │
 │  Sleep / Wait
 │
 ↓
Packet Arrival
 │
 ↓
Wake-up
 │
 ↓
Process Packet

While no data is available, the CPU does not need to continuously execute the application.

This reduces CPU consumption and allows the system to use those resources elsewhere.

---

4. What Is Busy Polling?

Busy Polling takes a fundamentally different approach.

Instead of waiting for an event, the application repeatedly checks whether data is available.

For example:

while (running) {

    if (data_available()) {
        process_data();
    }

}

Even when there is no data, the loop continues executing.

In other words:

The CPU keeps running and continuously checks whether data has arrived.

This is also commonly referred to as a Busy Loop or Polling Loop.

---

5. epoll vs. Busy Polling

At a high level:

epollBusy Polling

When no data is available	Wait	Keep checking
CPU usage	Low	High
Wake-up process	Required	Not required
Implementation	Common	Specialized
Latency	Generally higher	Can be very low
CPU resources	Efficient	Dedicated
Power consumption	Lower	Higher

The important point is:

Busy Polling is not automatically faster in every environment.

The result depends on the workload, hardware, operating system configuration, and application architecture.

---

6. Why Can Busy Polling Be Faster?

One important factor is the waiting and wake-up path.

A simplified event-driven flow may look like:

Packet Arrival
      ↓
Kernel
      ↓
Event Notification
      ↓
Wake-up
      ↓
Scheduler
      ↓
Application
      ↓
read()
      ↓
Process

With Busy Polling:

Application
     ↓
Polling
     ↓
Data Arrived?
   ↙      ↘
 No       Yes
 │         │
 └─ Loop   ↓
         Process

Because the application is already running, it does not depend on the same sleep/wakeup mechanism.

In certain environments, this can reduce latency and, particularly, tail latency.

---

7. But Busy Polling Has a Cost

There is an important trade-off.

Busy Polling continuously consumes CPU resources.

For example:

Core 0
  └── Network Polling Thread
       └── High CPU Utilization

Even when no data arrives, the CPU continues executing the polling loop.

This can result in:

• Higher CPU utilization
• Higher power consumption
• Less CPU capacity for other workloads
• Increased heat
• Potential CPU resource contention

In other words:

Busy Polling is a strategy for spending CPU resources to reduce waiting time.

---

8. Trading Systems Change the Equation

For a general-purpose server, conserving CPU resources is often important.

For a latency-sensitive trading system, however, minimizing latency may be the primary objective.

Consider a simplified trading flow:

Market Data
     ↓
Packet Arrival
     ↓
Decode
     ↓
Strategy
     ↓
Risk Check
     ↓
Order
     ↓
FEP

If this entire path needs to be processed within a very short time, the mechanism used to wait for incoming packets becomes important.

In some high-rate market data environments, data may arrive so frequently that the application spends very little time actually idle.

In such an environment, continuously polling for packets can be a reasonable architectural choice.

---

9. Is Busy Polling Simply "Wasting CPU"?

Not necessarily.

For general-purpose applications, it is easy to think:

CPU 100%
   ↓
Bad

But a low-latency system may intentionally dedicate an entire CPU core to network processing:

Core 0 → Network Polling
Core 1 → Market Data
Core 2 → Strategy
Core 3 → Order

In this configuration, high CPU utilization on Core 0 is not necessarily a problem.

The important question is:

What is the CPU actually doing?

If the CPU is continuously polling a high-rate network interface in order to minimize latency, the utilization is a deliberate part of the architecture.

---

10. epoll and Polling Are Not Always Mutually Exclusive

It is useful to avoid thinking of epoll and Busy Polling as completely opposing technologies.

Linux networking provides various mechanisms for polling and event notification, and some systems combine different techniques depending on their latency requirements.

For extremely latency-sensitive systems, engineers may consider approaches such as:

• Socket Busy Polling
• NIC polling
• Kernel bypass
• User-space networking
• DPDK
• Onload-style technologies

Conceptually, there are several levels of optimization:

Traditional I/O
      ↓
epoll
      ↓
Polling Optimization
      ↓
Busy Polling
      ↓
Kernel Bypass
      ↓
User-Space Networking

Not every system needs to go to the bottom of this stack.

---

11. Busy Polling and CPU Affinity

Busy Polling is closely related to CPU Affinity, which we discussed in Deep Dive #3.

For example:

Core 0
 └── Network Polling Thread

Core 1
 └── Market Data Thread

Core 2
 └── Strategy Thread

Core 3
 └── Order Thread

Assigning dedicated responsibilities to CPU cores can reduce thread migration and help maintain cache locality.

However, if the polling thread shares a CPU core with unrelated work:

Core 0
 ├── Polling
 ├── Logging
 ├── Monitoring
 └── Other Tasks

the polling thread must compete for CPU resources.

Therefore, Busy Polling should be considered together with CPU topology and thread placement.

---

12. NUMA Matters Too

In Deep Dive #4, we discussed NUMA and Memory Locality.

Now the concepts begin to connect:

CPU Affinity
      ↓
Busy Polling
      ↓
NUMA Locality

For example, if a NIC is attached to NUMA Node 0:

NUMA Node 0

NIC
 ↓
Polling Thread
 ↓
Market Data
 ↓
Strategy

Keeping the processing path within the same NUMA domain may help maintain locality.

On the other hand:

NIC
 ↓
NUMA Node 0
 ↓
Interconnect
 ↓
NUMA Node 1
 ↓
Strategy

may introduce additional cross-NUMA data movement.

This connects the concepts from the previous articles:

CPU → Cache → Memory → NUMA → NIC

as one continuous data path.

---

13. TCP and UDP Have Different Characteristics

Busy Polling should also be considered in the context of the transport protocol.

For example, market data is often distributed using UDP multicast:

Exchange
   ↓
Multicast
   ↓
NIC
   ↓
UDP
   ↓
Market Data

Order transmission or session-based communication may use TCP:

Application
    ↓
TCP
    ↓
NIC
    ↓
Broker / Exchange

TCP and UDP have different characteristics, so the appropriate polling strategy depends on the entire system design.

For UDP-based market data, packet loss, sequence numbers, gap detection, and recovery mechanisms must also be considered.

Therefore:

UDP does not automatically mean Busy Polling.

---

14. Do Not Measure Latency Alone

Suppose Busy Polling reduces latency.

Does that automatically mean the system has improved?

Not necessarily.

Other metrics should be evaluated as well.

Latency

• Average
• P50
• P95
• P99
• P99.9

CPU

• CPU utilization
• CPU cycles
• IPC
• Context switches
• CPU migration

Network

• Packet rate
• Packet loss
• Receive queue
• NIC utilization

Application

• Throughput
• Queue depth
• Processing time
• End-to-end latency

For trading systems, tail latency can be more meaningful than the average.

---

15. When Should You Use epoll?

epoll remains an excellent choice for many systems.

It is particularly suitable when:

• Many sockets must be managed
• CPU efficiency matters
• Latency requirements are moderate
• The system is a general-purpose server
• Many connections are handled concurrently
• Scalability is important

For example:

Client 1 ─┐
Client 2 ─┤
Client 3 ─┤
Client 4 ─┤
Client 5 ─┤
           ↓
        epoll
           ↓
       Event Loop

This architecture is efficient and widely applicable.

---

16. When Should You Consider Busy Polling?

Busy Polling may be worth considering when:

• Extremely low latency is critical
• Packet rates are very high
• Dedicated CPU cores are available
• CPU resources can be traded for latency
• CPU/NIC topology can be controlled
• The workload resembles HFT or high-rate market data processing

But the final decision should always be based on measurement.

If Busy Polling significantly increases CPU usage while producing little or no latency improvement, it may not be a useful optimization.

---

17. The Most Important Concept: Trade-off

The choice can be viewed as a trade-off:

                 CPU Efficiency
                       ↑
                       │
                 epoll ●
                       │
                       │
                       │
                       │        ● Busy Polling
                       │
                       └────────────────────→
                              Low Latency

General-purpose systems may prioritize CPU efficiency.

Certain HFT systems may prioritize extremely low and predictable latency.

There is no single approach that is optimal for every system.

---

18. FontesFintech Engineering Approach

When designing network processing for a low-latency system, the important question is not simply which API to use.

The important question is:

Where is the latency actually coming from?

A simplified end-to-end path might look like:

NIC
 ↓
Kernel / User Space
 ↓
Polling / Event Notification
 ↓
Packet Processing
 ↓
Market Data
 ↓
Strategy
 ↓
Risk
 ↓
Order
 ↓
FEP

We need to understand where latency is introduced along this path.

For example:

How frequently does data arrive?

How long does the CPU actually spend waiting?

Is event wake-up a meaningful part of the latency?

Can a dedicated CPU core be allocated?

What is the NIC-to-CPU NUMA topology?

Does the latency improvement from Busy Polling justify the additional CPU cost?

These questions should be answered through measurement before changing the architecture.

---

Conclusion

epoll() and Busy Polling are not simply "slow" and "fast" alternatives.

They represent two different approaches to using CPU resources while waiting for data.

epoll allows the CPU to wait efficiently when there is no work.

Busy Polling continuously consumes CPU resources in order to minimize waiting time and can provide very low and predictable latency in certain environments.

Therefore, in low-latency trading systems:

Saving CPU is not always the same as optimizing performance.

Sometimes dedicating an entire CPU core to polling can be a reasonable engineering trade-off.

But that decision should always be validated through measurement and benchmarking.

Key Takeaway

In low-latency systems, how you wait for data can be as important as how you process it.

Wait Less. Poll Smarter. Measure Everything.

---

Next: Trading Engineering — Deep Dive #6

Interrupts, Polling and Network Latency

In the next article, we will go one level deeper:

What actually happens between the moment a network packet arrives at the NIC and the moment application code begins processing it?

We will follow the path:

NIC Interrupt → IRQ → Kernel → NAPI → Socket → Application

and examine why the choice between Interrupts and Polling matters in low-latency systems.