ROTANG/Cray Inc. iota(1)
NAME
iota - Input Output Test Apparatus
SYNOPSIS
iota [-h] -options ...
DESCRIPTION
iota is a parallel-serial-parallel nested I/O load generator
which issues input or output work in serial and in parallel
at the filesystem file and file handle/descriptor levels.
iota experiments are defined via command line arguments.
Extensive performance data is logged for printout at execu-
tion termination. iota can optionally generate and test
data patterns written to and read from files, and supports
multiple I/O APIs (fe: UnixIO, STDIO, POSIX AIO). iota is
implemented using Pthreads to achieve parallelism at its
inner and outermost loops, controlling execution flow using
mutex locks rather than repeatedly creating new threads.
iota implements a triply nested loop (from outter to inner):
1. Over files (-nfiles) -- Open one or more files on the
filesystem to be operated on in parallel. Each file has its
own data buffer and test pattern, and has a unique filesys-
tem filename.
2. Over Operations (-nops) -- Perform the seek (if
requested, see -noseek), read/write the entire buffer
(-bufsz), and self-test (if requested, see -test) cycle one
or more times sequentially.
3. Over stripes (-nstripes) -- Break each
seek/read/write/self-test into one or more concurrent sub-
tasks responsible for a contiguous portion of the file
buffer. Stripes are the buffer size (-bufsz) divided by the
number of stripes, with the remainder being allocated to the
last (highest numbered) stripe. By default, stripes use
their own file handle/descriptor, but can share one across
the entire file (-share).
OPTIONS
-h Complete list of command arguments iota is parsing
for, listing default values and brief descriptions
in a tabular format.
-op
r, the default, performs a read experiment; w a
write experiment
Version 2.1 Last change: Fri Jul 14 10:43:35 PDT 2000 1
ROTANG/Cray Inc. iota(1)
-nfiles integer
Number of filesystem files (unique filenames) to
open concurrently. A unique suffix (.filen.99999)
is automatically added to the base filename of each
file.
-bufsz integer
Size of buffer, in bytes, to serve as a single oper-
ation. Each file process allocates a buffer associ-
ated with only that file.
-nstripes integer
Number of stripes to break an operation buffer
(-bufsz) into. Stripe functions are performed con-
currently.
-nops integer
Number of sequential operations of a buffer size
(-bufsz) will perform for each file (-nfiles).
-test
none, the default, performs no error checking of the
data written or read. boundaries checks only one
wordlength's worth of bytes (aligned or unaligned)
at the boundaries between two stripes. This integer
key is stored in reverse byte order of the native
representation. buffer checks all aligned words in
the buffer, as well as the boundaries checking.
-scope
sys requests that the created threads are scheduled
by the kernal in contention with all other processes
on the system. proc requests that created threads
are scheduled by the Pthread runtime library at the
process level. OS support for the scheduling scopes
varies by vendor.
-io_api
Base filename used to name filesystem files.
-lock_atomic
Serialize access to a filesystem file (-nfiles),
regardless of the number of file handles/descriptors
(-nstripes) open to that file, for the scope of a
single atomic seek through read/write operation.
The lock is not actually performed if seek is not
used (-noseek).
EVENT LOG REPORT
iota logs raw timing event data in a cache which is printed
in tabular format to the standard output upon exit (normal
termination or abnormal end) of the experiment. The event
log is preceded by a header which indicates the timestamp of
when the program was invoked and the runtime flags used to
specify the experiment. Each log event covers a scope
delimited by starting a timer before the function(s) begin,
and stopping it after the function(s) is (are) done. The
scopes of the individual event types are described below.
If a functionality is not used (for example, the -noseek
option is specified), no log events are generated for that
functionality.
The columns, from left to right, are:
Event name
Described below
File number
Identifier of the file, from 0 through Nfiles - 1
Stripe number
Identifier of the stripe, per file, from 0 through
Nstripes - 1
Operation number
Version 2.1 Last change: Fri Jul 14 10:43:35 PDT 2000 3
ROTANG/Cray Inc. iota(1)
Identifier of the operation number, per stripe, per
file, from 0 through Nops - 1
Time
Total time, in milliseconds, to perform the event, whose
scopes are described below.
Absolute time
Absolute time, in milliseconds, since the first logged
event.
MB/s
MegaBytes per second transfer rate, where MegaBytes is
calculated as 2**20 8bit bytes.
Log event names, scopes, and descriptions:
open includes the time required to open a file descriptor or
handle.
seek is reported only if a seek is being performed before a
read/write operation (specifically, if -noseek is not being
used). The seek log event indicates the amount of time
required to perform the seek prior to the read/write opera-
tion.
op reports the time to perform the read or write of an
atomic stripe unit of the operation.
atomic_op indicates the amount of time required to seek to
an offset in the file and perform the atomic stripe unit of
work. -atomic_op is reported only if a seek is being per-
formed before a read/write operation (specifically, if
-noseek is not being used).
stripes is the total time to perform all the operations on a
given stripe. The exact scope of this timer starts before
the first seek/read/write of a stripe until the last
read/write returns.
all_stripes is similar to the stripes scoped event, except
it covers all the stripes for a file.
close indicates the time required to perform the file close
operation per stripe, or for file 0 when file han-
dle/descriptor sharing (-share) is used.
file log events indicate the total time to open all the
stripes, perform all the requested seeks, reads or writes,
and all the close functions for all the stripes, for a sin-
gle filesystem file (-nfiles).
Version 2.1 Last change: Fri Jul 14 10:43:35 PDT 2000 4
ROTANG/Cray Inc. iota(1)
total is always the last reported event, and is similar to
the file log event except it covers the scope of all files
in the experiment.
Following all the log events is a statistical summary of the
experiment on a per-file basis. Columns represent the file
identifier (0 through Nfiles - 1), the minimum, arithmetic
mean (average), and maximum MegaByte per second transfer
rate as measured at the op scope.
NOTES
Total file handle/descriptor usage is nfiles * nstripes, and
it is easy to exceed the system limits, which are typically
about 60 file handles per process.
The total log event does not report a timestamp because the
total start time is observed before the first log event
(file open), which causes the timestamp relative to the
first event to be incorrect by the amount of time between
when the total timer is started and the first event is
logged.
SEE ALSO
POSIX, Pthreads, STDIO, UnixIO, limit.
BUGS
When striped buffers are not evenly divisible, the remainder
should be distributed equally among the lower numbered
stripes, not allocated to the last stripe.
There is no synchronization between stripes. That is, it is
possible for one stripe to finish its sequence of operations
before any others even start. There should be an optional
operation synchronization (-op_sync) so no stripe is ever
more than one operation ahead of any other stripe.
The actual scheduling priority is never checked against the
requested one, so it is not known whether or not the
requested scheduling mode is being observed.
The test patterns for self testing could be be more robust.
The statistical summary should be user-selectable for op,
atomic_op, stripes, or all_stripes events.
The default reported for -noseek by -h is actually the
inverse of what is happening since the internal variable is
"use_seek", not "noseek".
Version 2.1 Last change: Fri Jul 14 10:43:35 PDT 2000 5
ROTANG/Cray Inc. iota(1)
The term "stripe" actually refers to what happens to IO
requests in the OS, and shouldn't be used to describe the
operation decomposition that iota does.
iota has a tendancy to expose bugs in vendor runtime
libraries and operating systems when imposing high stress
levels. If you think you've found a bug in iota (read: not
a vendor runtime or OS bugs), please let me
(mogill@cray.com) know immediately! I promise I'll fix it
(eventually).
FUTURE WORK
Interface to use POSIX AIO.
Resolution of everything on the BUGS list.
DISTRIBUTING
Some of this software is derived from ROTANG software by
permission of Jace A Mogill. All other portions of iota are
owned by Cray Inc.
AUTHOR
Jace A Mogill, Cray Inc. (mogill@cray.com), maintains
numeric libraries, two commercial finite element simulation
programs, a slew of climate, ocean, and weather (COW) codes,
a MUD, two I/O benchmarking tools, and a variety of other
stuff on workstations, Multi-Threaded Architectures, dis-
tributed memory RISC machines, and distributed memory vector
machines.
Jace A Mogill, ROTANG (mogill@rotang.com), maintains an
interactive color slice displayer (HMV), a pipelined movie
player (Hum), a pipelined version of gunzip (punzip, work in
progress), and a bunch of other stuff in his copious spare
time.
Version 2.1 Last change: Fri Jul 14 10:43:35 PDT 2000 6
ROTANG -- ZOOM! NOT BOOM!
Find out more about who ROTANG is
and how it is powered by M4.
ROTANG produces the world's finest optional software.
IOTA questions welcome at iota4 youknowwhat rotang DELETE com
This browsing experience is Copyright 2003, ROTANG.