Throughput is a performance metric that measures the amount of work a system can process within a given period of time. In software engineering, it typically refers to the number of requests, transactions, or operations a system can handle per second, minute, or hour. Throughput is a critical indicator of system capacity and is closely monitored in web applications, APIs, databases, messaging systems, and data processing pipelines.
Throughput is often discussed alongside latency, but the two measure different things. Latency measures how long a single operation takes to complete, while throughput measures how many operations are completed in aggregate. A system can have low latency but also low throughput if it can only handle one request at a time, or high throughput with moderate latency if it processes many requests concurrently. Optimizing both metrics requires balancing concurrency, resource allocation, and architectural design.
Several factors influence application throughput:
- Hardware resources: CPU, memory, disk I/O, and network bandwidth
- Software architecture: Synchronous vs. asynchronous processing, connection pooling, and caching
- Database performance: Query optimization, indexing, and read replicas
- Concurrency model: Thread pools, event loops, and message queues
Measuring and improving throughput matters for applications that must serve large numbers of users or process high volumes of data. Throughput optimization is a key part of performance engineering, ensuring that software can handle production workloads efficiently and scale to meet growing demand.