layout status flags for PNFS_LAYOUT_RECALLED and PNFS_LAYOUT_CHANGED are removed, and replaced by a list of recalled ranges. recalls during io are added to the list and processed on pnfs_layout_io_finished(). recalls outside of io are processed immediately
new function layout_recall_range() loops through all existing layout segments, and removes ranges that intersect with the range recalled. deals with 4 cases per segment:
-only the beginning of the segment is recalled
-only the end of the segment is recalled
-the entire segment is recalled
-only a middle part of the segment is recalled
new function pnfs_layout_recall_status() is called before each unit of io, allowing io threads to bail out early if a recall is detected. takes a layout segment as an argument, and only returns an error if that segment intersects a recalled range
new function pnfs_layout_recall_fenced() is called when map_ds_error() in pnfs_io.c detects fencing. also takes a layout segment as an argument, and appends a recall matching the range of the segment
pnfs_layout_state_prepare() now checks the given range against the list of recalled ranges
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
pnfs_layout_state now stores a list instead of a single pnfs_file_layout entry. when new segments are acquired through LAYOUTGET, they are inserted into the list in order of increasing offset
functions related to pnfs_layout_state_prepare() now operate on the list to find missing layout ranges and segments missing devices
pattern_init() in pnfs_io.c now allocates and initializes io threads for each layout segment in the range
new function pattern_join() will call WaitForMultipleObjects() in a loop, to support io patterns with more than 64 threads. if pattern_fork() is called with a thread count of 1, the thread function is called directly instead of spawning a new thread
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
struct pnfs_io_pattern, pnfs_io_thread, and pnfs_io_unit are not referenced outside of pnfs_io.c, so they don't need to be in pnfs.h
also grouped the inline helper functions together at the bottom of pnfs.h
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
added stripe_next_unit() in pnfs_io.c, removed pnfs_file_device_io_unit() from pnfs_device.c
moved get_sparse_fh()/get_dense_fh() to pnfs_io.c, now only called once on pattern_init()
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
when starting io, both pnfs_read() and pnfs_write() need a guarantee that their range is covered by layout segments. because we have to drop the lock for LAYOUTGET and GETDEVICEINFO, earlier layout segments may be recalled during this process. to avoid this, new function pnfs_layout_state_prepare() gets called repeatedly until it can verify under a single lock that 1) the entire desired range is covered with layouts and 2) each of these layouts has an associated device. whenever pnfs_layout_state_prepare() has to drop its lock for LAYOUTGET or GETDEVICEINFO, it returns PNFS_PENDING
on PNFS_SUCCESS, the caller knows that all segments in the range are valid and can dispatch io to those segments without worrying about recalls, because it still holds the pnfs_layout_state lock
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
the thread-by-server model breaks down when using dense layouts, because multiple stripes could be mapped to a single data server, and the per-data-server thread would have to send a COMMIT for each stripe
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
sorry, earlier Casey, but the patch 'threading by io unit instead of stripe' from 6/20/2010 was nuts! with PNFS_THREAD_BY_SERVER disabled, we definitely -don't- want to create a separate thread for each io unit (each READ/WRITE request to a ds). we just want the one per stripe, as the intended alternative to PNFS_THREAD_BY_SERVER
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
added 2011 year to the copyright line
added authors info to the license
added UofM license to libtirpc files that we modified
(but i probably missed some)
exclusive locks are no longer held over LAYOUTGET, LAYOUTRETURN, or GETDEVICEINFO rpcs. this prevents a deadlock when CB_LAYOUTRECALL needs an exclusive lock while another operation is on the wire
introduced a 'pending' condition variable to protect access to state->layout while the layout's lock is not held
updated file_layout_recall() to compare the stateid sequence numbers to determine if the server has processed an outstanding LAYOUTGET or LAYOUTRETURN, where we're required to reply with NFS4ERR_DELAY
LAYOUTGET, LAYOUTRETURN, and GETDEVICEINFO can now be sent with try_recovery=TRUE because they no longer hold an exclusive lock. this makes it possible for recover_client_state() to recall all of the client's layouts without deadlocking
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
nfs41_lock_stateid_arg() is now called only once in handle_read()/handle_write(), and pnfs_read()/pnfs_write() no longer depend on nfs41_open_state
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
on a successful LAYOUTGET, file_layout_fetch() calls layout_update() to copy the first layout segment returned and update the layout stateid
on a successful LAYOUTRETURN, file_layout_return() frees the layout segment and updates/clears the stateid
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
moved state data (stateid, flags, locks, and reference counts) out of struct pnfs_layout, which should represent a layout segment returned by LAYOUTGET
struct pnfs_layout_state now holds this state, along with a pointer to a single pnfs_file_layout
struct pnfs_file_layout_list is now a list of pnfs_layout_states, and was renamed to pnfs_layout_list
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
added pnfs_layout.open_count to count open references, and only return the layout when pnfs_open_state_close() takes the open_count to 0
use InterlockedIncrement/Decrement to avoid an exclusive lock on the layout
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
20.3. CB_LAYOUTRECALL
"LAYOUTRECALL4_FSID and LAYOUTRECALL4_ALL specify that all the storage device ID to storage device address mappings in the affected file system(s) are also recalled."
pnfs_file_layout_recall() now takes a nfs41_client instead of just the pnfs_file_layout_list, because both the layout list and device list are accessible from nfs41_client. for bulk recalls, calls new function pnfs_file_device_list_invalidate(). each device with layout_count=0 is removed and freed, and devices in use are flagged as REVOKED and freed when layout_count->0
layout_recall_return() now takes a pnfs_file_layout instead of pnfs_layout for access to pnfs_file_layout.device. pnfs_layout_io_start() and pnfs_layout_io_finish() do the same, because pnfs_layout_io_finish() calls layout_recall_return(). layout_recall_return() calls pnfs_file_device_put() to release its reference on the device
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
pnfs_device.status remembers whether a given device has been GRANTED/REVOKED
pnfs_device.layout_count tracks the number of layouts using the device, incremented by pnfs_file_device_get() and decremented by pnfs_file_device_put(). when pnfs_file_device_put() takes layout_count to 0, remove and free the device only if it's flagged as REVOKED
because pnfs_file_device_get() modifies pnfs_device.layout_count, we can no longer use a shared lock; changed pnfs_file_device.lock from SRWLOCK to CRITICAL_SECTION, and moved to pnfs_device.lock to document the fact that it's used for pnfs_device.status and pnfs_device.layout_count
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
operations that require a stateid now take stateid_arg for recovery information. these operations include close, setattr, lock/unlock, layoutget, and read/write (including pnfs)
nfs41_open_stateid_arg() locks nfs41_open_state and copies its stateid into a stateid_arg
nfs41_lock_stateid_arg() locks nfs41_open_state.last_lock and copies its stateid into a stateid_arg; if there is no lock state, it falls back to nfs41_open_stateid_arg()
pnfs_read/write() now take nfs41_open_state so they can generate stateid_args
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
fixed a few cases of warning 4242: possible loss of data
wincrypt.h appears to come with windows.h in later versions of the ddk, but nfs41_client.c fails to compile in WDK 6001 without #include <wincrypt.h>
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
