when open_update_cache() calls open_delegation_return() to return its delegation, it doesn't update its local variable 'delegation_type' to reflect the new delegation->type. this causes the next call to nfs41_name_cache_insert() to continue failing
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
the namecache/delegation feedback code in nfs41_ops.c:open_update_cache() is interfering with delegation recovery. if its call to nfs41_name_cache_insert() fails with ERROR_TOO_MANY_OPEN_FILES, the delegation is returned instead of being recovered. this shouldn't happen during recovery, because we're replacing a delegation rather than adding a new one
nfs41_open() now remembers whether we already had a delegation. if we did, open_update_cache() will pass OPEN_DELEGATE_NONE to nfs41_name_cache_insert() to prevent it from returning ERROR_TOO_MANY_OPEN_FILES. if we didn't already have a delegation, the nfs41_delegreturn() needs to be called with the same try_recovery flag from nfs41_open()
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
nfs41_name_cache_remove() needs to update the 'numlinks' attribute for other links, even if the file being removed is not found in the cache. to search for its attr cache entry, nfs41_name_cache_remove() now requires a fileid argument. nfs41_remove() only gets a pointer to the parent's filehandle, so it also needs the target fileid argument
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
we were previously only verifying that the server didn't reboot between WRITEs. COMMIT returns a verifier that needs to be checked as well
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
open_update_cache() wasn't retrying insert after the open_delegation_return() error case
nfs41_name_cache_delegreturn() needs a check for if (attributes->delegated) to avoid an extra deref when the call to nfs41_name_cache_insert() failed
added a TODO comment to nfs41_client_delegation_return_lru() for improvements to the naive algorithm
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
to prevent delegations from claiming all available slots in the attribute cache, nfs41_name_cache_insert() now returns ERROR_TOO_MANY_OPEN_FILES when the number of delegated entries reaches 50% of the cache capacity. see comment 'delegations and cache feedback' in name_cache.c for details
when nfs41_open() sees this error from nfs41_name_cache_insert(), it calls new function nfs41_client_delegation_return_lru() to return the least-recently-used delegation and, if successful, loops back to nfs41_name_cache_insert(). nfs41_client_delegation_return_lru() returns NFS4ERR_BADHANDLE if all delegations are currently in use (associated with an existing open), in which case nfs41_open() returns the newly-granted delegation
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
on opens that grant a delegation, return the delegation immediately if attribute cache insertion fails
nfs41_name_cache_insert() now returns success in the 'out_err_deleg' case, where name cache insertion failed but attr cache insertion was successful
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
added 2011 year to the copyright line
added authors info to the license
added UofM license to libtirpc files that we modified
(but i probably missed some)
for delegations without an associated open, try the optional WANT_DELEGATION operation with CLAIM_PREVIOUS before falling back to OPEN/CLAIM_PREVIOUS. the advantage of WANT_DELEGATION is that it doesn't generate an open stateid, so we don't need the corresponding CLOSE as in recover_delegation_open(). if WANT_DELEGATION fails with NFS4ERR_NOTSUPP, don't try it again during that round of state recovery
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
10.2.1. Delegation Recovery (re: client restart/lease expiration)
A server MAY support claim types of CLAIM_DELEGATE_PREV and CLAIM_DELEG_PREV_FH, and if it does, it MUST NOT remove delegations upon a CREATE_SESSION that confirm a client ID created by EXCHANGE_ID. Instead, the server MUST, for a period of time no less than that of the value of the lease_time attribute, maintain the client's delegations to allow time for the client to send CLAIM_DELEGATE_PREV and/or CLAIM_DELEG_PREV_FH requests. The server that supports CLAIM_DELEGATE_PREV and/or CLAIM_DELEG_PREV_FH MUST support the DELEGPURGE operation.
if there's a delegation to reclaim, recover_open_no_grace() now tries CLAIM_DELEGATE_PREV (supported by emc server, but not linux) before falling back to CLAIM_NULL
nfs41_client_delegation_recovery() sends DELEGPURGE to indicate that we're finished reclaiming delegations
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
delegation stateid arguments to DELEGRETURN and OPEN are now stateid_arg, for use with recover_stateid_delegation(). added a nfs41_delegation_state pointer to stateid_arg, for when a delegation stateid is used in the absence of nfs41_open_state (DELEGRETURN, SETATTR)
recovery during a call to nfs41_delegation_to_open() requires special attention; recover_stateid_delegation() has to handle the case where recover_open() already reclaimed the open stateid. it does this by returning BAD_STATEID instead of retrying the OPEN (which would generate yet another open stateid)
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
recover_client_state() flags all delegations as revoked before starting open state recovery. if recover_open() finds that its delegation is revoked, it attempts to recover it using CLAIM_PREVIOUS. if its delegation has already been reclaimed by another open, it can skip reclaiming the open stateid (provided it has no byte-range locks to reclaim). after all opens have been reclaimed, any delegations still marked 'revoked' are passed to recover_delegation(). recover_delegation() also uses CLAIM_PREVIOUS (or CLAIM_NULL outside of the grace period) to reclaim the delegation, but has to throw away the open stateid with CLOSE
added a try_recovery argument to nfs41_delegreturn() and nfs41_delegation_granted(), so it can be called by recover_open() if OPEN grants an unexpected open
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
nfs41_open() in open.c is no longer used for recovery, so made static and renamed to do_open(). renamed nfs41_rpc_open() back to nfs41_open()
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
previously we'd query owner and group file attributes on all getattrs and
storing that in nfs41_file_info structure which was caused a problem for
readdirs.
when a delegation is granted by OPEN, its delegation type is passed to nfs41_name_cache_insert(). as long as the delegation is held, its name_cache_entry is kept out of the cache.exp_entries list to prevent it from expiring. an extra reference is held on the attr_cache_entry as well, so it sticks around even if the name_cache_entry is removed (a parent expires, for example). new function nfs41_name_cache_delegreturn() adds the name_cache_entry back to the list, and releases the extra attr_cache_entry reference
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
combined nfs41_open() and nfs41_open_reclaim() into nfs41_rpc_open() by factoring out the open_claim4 argument. new function nfs41_open() in open.c deals with the nfs41_open_state, adding it to the client's list, and handles any delegations granted
added xdr for OPEN CLAIM types CLAIM_DELEGATE_CUR, CLAIM_DELEG_CUR_FH, CLAIM_DELEGATE_PREV, CLAIM_DELEG_PREV_FH (the _FH types are new to 4.1, and not supported by linux server)
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
in create_session we were sending auth_none as available security types
for the callback channel. Adding auth_sys to the list. No enforcement of
these creds happens.
instead of updating the attribute cache with the values given to SETATTR, add a GETATTR to the compound; this will capture changes to time_modify and change that the client could otherwise miss, and get the server's value of timestamps sent with SET_TO_SERVER_TIME4
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
handling receiving WRONGSEC error in compound_encode_decode function by
sending either SECINFO or SECINFO_NONAME op to find out available
security flavors from the server. then try to establish new security
context given the ordered list returned by the server.
Not handling if parent directory doesn't permit a security flavor of
its child directory. Example "/" exported with only auth_sys and
"/sec" exported with only "gss".
if we are doing CREATE_NEW file creation, then based on whether or not
we have a persistent session, we'll send either GUARDED4 create for
persistent session and EXCLUSIVE4_1 create otherwise.
nfs41_cb_session stores the last cb_compound reply (whether or not cachethis was set) to handle retry attempts, along with the cb_compound arguments for improved NFS4ERR_SEQ_FALSE_RETRY detection
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
exclusive locks are no longer held over LAYOUTGET, LAYOUTRETURN, or GETDEVICEINFO rpcs. this prevents a deadlock when CB_LAYOUTRECALL needs an exclusive lock while another operation is on the wire
introduced a 'pending' condition variable to protect access to state->layout while the layout's lock is not held
updated file_layout_recall() to compare the stateid sequence numbers to determine if the server has processed an outstanding LAYOUTGET or LAYOUTRETURN, where we're required to reply with NFS4ERR_DELAY
LAYOUTGET, LAYOUTRETURN, and GETDEVICEINFO can now be sent with try_recovery=TRUE because they no longer hold an exclusive lock. this makes it possible for recover_client_state() to recall all of the client's layouts without deadlocking
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
LAYOUTGET xdr now supports decoding of multiple layout segments, which are returned in a list with pnfs_layoutget_res_ok
LAYOUTRETURN no longer operates on an existing pnfs_file_layout. it now takes a copy of the layout stateid, and returns the new stateid with pnfs_layoutreturn_res
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
send CREATE_SESSION with compound_encode_send_decode() instead of nfs41_send_compound() for its NFS4ERR_DELAY and NFS4ERR_STALE_CLIENTID handling
added 'try_recovery' argument to nfs41_create_session(), which is passed on to compound_encode_send_decode(). nfs41_session_renew() uses try_recovery=FALSE, because it handles the NFS4ERR_STALE_CLIENTID error on its own. nfs41_session_create() uses try_recovery=TRUE to make use of the NFS4ERR_STALE_CLIENTID error handling. modified the NFS4ERR_STALE_CLIENTID block to call nfs41_client_renew() and retry the operation (i.e. CREATE_SESSION), instead of falling through to session recovery
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
12.7.4. Recovery from Metadata Server Restart
"The client MUST stop using layouts and delete the device ID to device address mappings it previously received from the metadata server."
during client state recovery, call pnfs_file_layout_recall() to revoke all layouts and devices held by the client
LAYOUTGET, LAYOUTRETURN, and GETDEVICEINFO are all sent under their respective locks, and pnfs_file_layout_recall() requires a lock on each layout and device it operates on, so this would cause a deadlock if one of those operations triggered the recovery. to avoid this, LAYOUTGET, LAYOUTRETURN, and GETDEVICEINFO are all sent with try_recovery=FALSE. this behavior is preferable for recovery, because errors in the pnfs path cause us to fall back to the metadata server
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
if we see NFS4ERR_NO_GRACE from recovery operations, it means we lost our state due to a lease expiration rather than a server reboot. in this case, it's possible that conflicting locks were granted to other clients, so we have to try normal OPEN/LOCK operations to recover our state. because they're sent during recovery, nfs41_open() and nfs41_lock() take a new 'bool_t try_recovery' argument so we can avoid recursion
if these operations fail due to conflicting locks, we have no choice but to return errors to the application. using a stateid that was revoked due to lease expiration results in NFS4ERR_EXPIRED, and we map this error to ERROR_FILE_INVALID: The volume for a file has been externally altered so that the opened file is no longer valid.
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
nfs41_open_state maintains a list of outstanding byte-range locks by calling open_lock_add() and open_lock_remove() in lock.c
during client state recovery, after reclaiming each OPEN stateid, send LOCK requests with reclaim=TRUE for each lock it owns, and update the open's lock stateid with the result
added 'bool_t reclaim' argument to nfs41_lock(); when set, compound_encode_send_decode() is called with try_recovery=FALSE to avoid recursive recovery
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
avoid the recursive case where state recovery operations (OPEN for reclaim and RECLAIM_COMPLETE) return BADSESSION, which kicks off another round of recovery
added a 'bool_t try_recovery' argument to compound_encode_send_decode() in place of its unused 'bufsize_in' and 'bufsize_out'. when try_recovery=FALSE, return BADSESSION/STALE_CLIENTID errors instead of attempting recovery. nfs41_open_reclaim(), nfs41_reclaim_complete(), and nfs41_destroy_session() now pass try_recovery=FALSE
during state recovery, we can now check the return values of nfs41_open_reclaim() and nfs41_reclaim_complete() for BADSESSION, and use a goto to restart session recovery
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
operations that require a stateid now take stateid_arg for recovery information. these operations include close, setattr, lock/unlock, layoutget, and read/write (including pnfs)
nfs41_open_stateid_arg() locks nfs41_open_state and copies its stateid into a stateid_arg
nfs41_lock_stateid_arg() locks nfs41_open_state.last_lock and copies its stateid into a stateid_arg; if there is no lock state, it falls back to nfs41_open_stateid_arg()
pnfs_read/write() now take nfs41_open_state so they can generate stateid_args
Signed-off-by: Casey Bodley <cbodley@citi.umich.edu>
