Jan-

Thanks for the reply!

   Impressive, my caches tend to be up to 500Mb or so. I think you may
   possibly be the first one to ever use a 10Gb client cache.

10Gb is small, if you use the proper unit of measurement: it's $4 worth of
storage. I have a little over a terabyte of disk on my personal office
computer, so dedicating 10Gb to my cache is trivial.

My home directory is about 20Gb -- that is, about $7 of storage -- so 10Gb is
only half of what it takes to hold my homedir.

My intended use mode for coda was to have a filesys shared between my laptop
and my office machine. I want my entire volume resident on both systems; I
want complete access to the volume when I am disconnected on my laptop. So the
mode of use is to arrange for my coda cache to be larger than the entire data
set, and run with *everything* hoarded. I am trying this out to see if it will
work.

   > - hoard it all with
   >     hoard add /coda/lambda.csail.mit.edu d+

   Do you use a single volume on the server, or are there multiple volumes?

Single volume.

   I am not sure why it would want to revalidate the attributes of that CVS
   directory. Something like that would indicate that we either don't have
   cached access rights for the user who is trying to access it, or that
   the object is considered 'demoted' (i.e. we got a callback or returned
   from disconnection).

   Since there are no callbacks (incoming packets are dropped) I assume it
   is trying to check if the current user has access.

   >   and the find tree walk hangs. After a minute or two, codacon says
   >     unreachable lambda.csail.mit.edu ( 11:58:10 )
   >   and the find walk resumes with the following output:
   > 
   >     find: ./research/mrlc/mrlc/spim/CVS: Permission denied
   ...
   >     find: ./research/mrlc/mrlc/confpaper: Permission denied

   Interesting, when we are disconnected the permission check is extremely
   permissive. Even if the cached rights would otherwise be considered
   stale we still allow access, and if they are missing we fall back on the
   System:AnyUser rights.

   I guess your volume is protected by an ACL, so that anonymous users
   cannot read those files. 

Yes, which is important. I wouldn't want to lose access to my protected files
just because I disconnected. So to test that out, I removed all rights from
System:AnyUser on the top-level dir.

   And we don't actually remove rights, but only
   mark them as stale (removal of cached rights only happens when you
   explicitly drop tokens with cunlog).

   So this would indicate that we never actually cached rights for those
   object, which is strange.

Yes, that does seem strange. 

   Both the earlier connected treewalk didn't actually walk the complete tree
   and hoard didn't cache rights. The hoard not caching rights may very well
   be possible, I may only try to make sure the file attributes and data are
   'fresh'.

   I know GNU find has some optimizations that may make it skip
   subdirectories when the directory linkcount is off. You can disable
   those by using 'find -noleaf'. Maybe the workaround we try to use isn't
   working.

It seems very weird that the hoard wouldn't cache rights.

I have also had these timeout / no-permission errors when doing a big
"cp -pr" copy into the coda filesys.

   cfs listacl and setacl only work while we are connected. This is because
   the client doesn't really know anything about users and groups so it is
   not useful to locally cache ACL data. 

Ahh. Would be nice if there was something about this in the cfs man page;
I would never have guessed that on my own.

   What the client caches is what set of rights the user that fetched the
   object had. 

I would also never have guessed this, either!

   > - Then I reconnect with
   >     cfs reconnect
   >   Now I can see the problem directories with no trouble. I redo the find
   >   tree-walk a third time and it completes with no problems.
   >   + codacon shows *no* server->client file motion. 

   Where there one or more 'validateattr' or 'validatevols' calls after
   reconnection?

I don't understand what you are asking here.

I also am having problems with hoard authentication. The man page says
the hoard user must be either
  - root
  - the console user
  - the "primary" user from /etc/venus.conf
but I have a hard time making this work. I'm logged into my computer, with
X running on the console (vt7, which is standard). But hoard loses:

    % hoard add /coda/lambda.csail.mit.edu/user/shivers/text/3min.txt 
    canonicalize: chdir(shivers) failed (Permission denied)

Then I su and try again:
    # hoard add /coda/lambda.csail.mit.edu/user/shivers/text/3min.txt 
    canonicalize: chdir(shivers) failed (Permission denied)

Then I note that I have no tokens while su'd as root, so I clog and try again:

    # ctokens
    Tokens held by the Cache Manager for root:
	@lambda.csail.mit.edu
	    Not Authenticated
    # clog shivers_at_lambda.csail.mit.edu
    username: shivers_at_lambda.csail.mit.edu
    Password: 
    # hoard add /coda/lambda.csail.mit.edu/user/shivers/text/3min.txt 
    pioctl:Add(7f000000_at_lambda.csail.mit.edu, ./user/shivers/text/3min.txt, 10, 0, 0): Permission denied
    # 

By the way, I made /usr/bin/hoard suid-root as suggested by the man page
before trying any of this; no good. I also switched to the console's
virtual tty vt1 and logged in there before trying any of this. No good.

Then, I switched away from X and back to virtual tty 1 and tried the hoard. No
good. Then, I logged out from vt1, logged back in as root, and was *finally*
able to get my hoard to complete successfully.

What's the deal with hoard?

Two other problems with hoard:
 - the "hoard walk" command is not documented anywhere in the man page.
   I just stumbled over it in a scenario described in the admin & user's
   manual.

 - The "hoard clear", "hoard list" and "hoard modify" commands appear in
   the man page, but it never defines what they do.
    -Olin