Linux io_uring architecture & high-performance I/O
Engineered by Linux block maintainer Jens Axboe, io_uring is the next-generation asynchronous I/O interface for Linux. It completely replaces legacy libaio and POSIX AIO by solving the fundamental performance bottleneck of high-speed storage: the system call overhead and context switching required to submit and reap millions of I/O operations per second.
Ring buffer architecture: SQ & CQ
At the core of io_uring are two lockless circular ring buffers allocated by the kernel and mapped directly into user-space memory:
| Ring Buffer | Structure Name | Role & Data Flow |
|---|---|---|
| Submission Queue (SQ) | io_uring_sqe (64 bytes) | Application writes I/O descriptors (opcode, fd, buffer pointer, length, offset, user_data) directly into the ring. Tail is updated in user space. |
| Completion Queue (CQ) | io_uring_cqe (16 bytes) | Kernel posts completed I/O events (result status/bytes transferred, flags, matching user_data tag) into the ring. Application reads head without locking. |
Asynchronous I/O interface comparison
| Feature | Synchronous (pread/pwrite) | Linux AIO (libaio) | io_uring |
|---|---|---|---|
| Syscalls Per Batch | 1 syscall per operation | 2 syscalls (submit + reap) | 0 to 1 syscall (0 with SQPOLL) |
| File Support | All files / Sockets | O_DIRECT files only (blocks on buffered) | All files, Sockets, Pipes, Direct & Buffered |
| Polled I/O Support | No | No | Yes (IOPOLL kernel polling) |
| Fixed Memory Buffers | No (Page pin per I/O) | No (Page pin per I/O) | Yes (Zero page pinning overhead) |
Advanced acceleration modes
1. Submission Queue Polling (SQPOLL)
When initialized with IORING_SETUP_SQPOLL, a dedicated kernel thread polls the submission queue. The application enqueues requests and updates the queue tail using memory atomic instructions: zero system calls are made to submit I/O.
2. Polled Hardware I/O (IOPOLL / High Priority)
When paired with enterprise NVMe controllers via IORING_SETUP_IOPOLL, completion interrupts are disabled. The CPU actively spins on the hardware completion register, eliminating interrupt latency and achieving very low tail latencies on supported hardware.
Benchmarking io_uring with fio
Modern storage benchmarks use fio to measure the true hardware limits of NVMe devices via ioengine=io_uring:
# 1. High-IOPS Random Read Benchmark (io_uring engine with fixed buffers and direct I/O)
fio --name=nvme_iouring_benchmark \
--filename=/dev/nvme0n1 \
--ioengine=io_uring \
--fixedbuffers=1 \
--registerfiles=1 \
--direct=1 \
--rw=randread \
--bs=4k \
--iodepth=64 \
--numjobs=8 \
--runtime=30 \
--time_based \
--group_reporting
# 2. Ultra-Low Latency Polled Benchmark (IOPOLL)
fio --name=nvme_iopoll_test \
--filename=/dev/nvme0n1 \
--ioengine=io_uring \
--hipri=1 \
--direct=1 \
--rw=randread \
--bs=4k \
--iodepth=1 \
--numjobs=1 \
--runtime=15 \
--group_reportingFrequently asked questions
What makes io_uring fundamentally faster than POSIX AIO (libaio)?
Legacy Linux libaio only supported asynchronous operations with O_DIRECT on block devices and required two syscalls (io_submit() and io_getevents()) per batch, introducing severe context-switch overhead. io_uring utilizes two circular ring buffers (Submission Queue and Completion Queue) shared in memory between user-space and kernel via mmap(), enabling true zero-syscall asynchronous submissions across all file types, network sockets, and devices.
What is SQPOLL (Submission Queue Polling)?
With IORING_SETUP_SQPOLL enabled, the kernel spawns a dedicated kernel thread that continuously polls the Submission Queue ring. When user-space writes an I/O request into the ring, the kernel executes it immediately without the application ever issuing an enter() system call, achieving true zero-overhead I/O.
What are fixed buffers and registered files in io_uring?
Normally, each I/O requires the kernel to pin and unpin user memory pages and look up file descriptor table structures. With io_uring_register(), memory buffers and file descriptors are registered once at startup, allowing subsequent operations to bypass page reference counting and file table lookups completely.