Interception library

The libioil library can be used to improve the performance of read and write operations to DFuse from applications which use libc. The library bypasses the kernel by intercepting POSIX read and write calls from the application and servicing them directly in user-space.

Using libioil

To use the interception library, set LD_PRELOAD to point to the shared library in the DAOS install directory:

LD_PRELOAD=/usr/lib64/libioil.so

For example:

$ dd if=/dev/zero of=./foo bs=1G count=20
20+0 records in
20+0 records out
21474836480 bytes (21 GB, 20 GiB) copied, 14.1946 s, 1.5 GB/s

$ LD_PRELOAD=/usr/lib64/libioil.so dd if=/dev/zero of=./bar bs=1G count=20
20+0 records in
20+0 records out
21474836480 bytes (21 GB, 20 GiB) copied, 5.0483 s, 4.3 GB/s

It is also possible to link applications with the interception library at compile time with the -lioil flag.

Monitor activity

To enable activity reports for the interception library, set the D_IL_REPORT environment variable. This returns a short summary of activity in the shared library destructor, typically as a program exits.

The value assigned to D_IL_REPORT defines the number of read and write calls to log to stderr. To log all calls, set the value to -1. To print the summary at program exit only, set the value to 0.

For example:

$ D_IL_REPORT=1 LD_PRELOAD=/usr/lib64/libioil.so dd if=/dev/zero of=./bar bs=1G count=20
[libioil] Intercepting write of size 1073741824
20+0 records in
20+0 records out
21474836480 bytes (21 GB, 20 GiB) copied, 5.17297 s, 4.2 GB/s

$ D_IL_REPORT=3 LD_PRELOAD=/usr/lib64/libioil.so dd if=/dev/zero of=./bar bs=1G count=5
[libioil] Intercepting write of size 1073741824
[libioil] Intercepting write of size 1073741824
[libioil] Intercepting write of size 1073741824
5+0 records in
5+0 records out
5368709120 bytes (5.4 GB, 5.0 GiB) copied, 1.27362 s, 4.2 GB/s

$ D_IL_REPORT=-1 LD_PRELOAD=/usr/lib64/libioil.so dd if=/dev/zero of=./bar bs=1G count=5
[libioil] Intercepting write of size 1073741824
[libioil] Intercepting write of size 1073741824
[libioil] Intercepting write of size 1073741824
[libioil] Intercepting write of size 1073741824
[libioil] Intercepting write of size 1073741824
5+0 records in
5+0 records out
5368709120 bytes (5.4 GB, 5.0 GiB) copied, 1.29935 s, 4.1 GB/s

Some programs, including most GNU utilities from the coreutils package, have a destructor function to close stderr on exit. This means that, while the interception library works for many basic commands such as cp and cat, it's not possible to see the summary generated by the interception library.