Why System Calls are Slow

Java is an abstraction around the underlying operating system. Normally, you wouldn't have to worry about what calls the JVM makes to the OS on your behalf. But these system calls can be expensive as the girls and buys down at LMAX know. They've put together a framework that tries to minimize them.

As with any other process, Linux allows you to see what underlying operating system calls the JVM is making with the strace command. When threads vie for synchronized blocks, for instance, you'll see  output like:


futex(0xb7651344, FUTEX_WAIT_PRIVATE, 1, {0, 999854465}) = 0

where the futex system call is asking for the operating system to arbitrate the contention.

When a call is made the following steps are executed (from the excellent The Linux Programming Interface):

1. Although your C code may look like you're calling a system call as if it were any other function, it actually "invokes a wrapper function in the C library".
2. "Argument are passed to the wrapper via the stack [as with any C function call], but the kernel expects them in specific registers. The wrapper function copies the arguments to these registers".
3. "Since all system calls enter the kernel in the same way, the kernel needs some method of identifying the system call. To permit this, the wrapper function copies the system call into a specific register (%eax)"
4. The "machine instruction (int 0x80), which causes the processor to switch from user mode to kernel mode and execute code pointed to be location 0x80" is run.
5. In response to this, "the kernel invokes its system_call() routine". This is covered in the next section.
6. "If the return value of the system call service routine indicated an error, the wrapper function sets the global variable errno ... using this value. The wrapper function then returns to the caller, providing an integer return value indication the success or failure of the system call"

Step #5 mentions the "system_call() routine". This does the following:

1. "Saves the register values onto the kernel stack".
2. "Checks the validity of the system call number."
3. "Invokes the appropriate system call routine, which is found by using the system call number to index a table of all system call services (the kernel variable sys_call_table). If the system call service routine has any arguments, it first checks their validity". The list of kernel calls and their memory addresses can be found at /proc/kallsyms. [1]
4. Restores registers from the stack and places the return value on the stack.
5. Drops back to user mode and returns to the wrapper function.

If this looks like a lot of work, it's because it is. This is why system calls are largely avoided in the Disruptor where even the smallest delay is to be avoided.

[1] The Ksplice Blog.