So many questions,

On Tue, Nov 16, 2004 at 03:28:32PM -0600, Jerry Amundson wrote:
> I have several GB of maildirs in a compressed tar file. When I try to 
> untar (to venus on the SCM), it gets a ways before giving me "File too 
> large" errors. But the SCM has plenty of space.

Either there is a single file > 100MB, or a directory has become > 256KB
(although that second case still crashes the client far too often).

> [root_at_uhura home]# cfs lv /coda/pbs.com

I guess you only have a single volume in which you are trying to untar
the tarfile. In that case there will be other limits that will affect
you. There probably is some arbitrary ceiling wrt the total number of
files in a single volume and I know there is a (configurable) limit for
the number of directories in a volume. The known directories per volume
limit is related to the size of the resolution log, even if there is
nothing to resolve, every directory has a ResolveNULL placeholder entry.
And since the administrative limit is somewhere around 4K or 8K
resolution log entries this means we run out once we create the same
number of directories. This shouldn't affect volumes with only a single
replica. To increase the administrative limits we'd need to run for
every replica of the replicated volume, something like,
    'volutil setlogparms <replica-id> reson 4 logsize 16384'

> Can I bypass venus, or manipulate cache size somehow?

You cannot bypass venus, but the venus cache size can be increased
(/etc/coda/venus.conf:cacheblocks=500000). You will need to reinit the
client before it picks up the new value though.

> Also, how to I make sure the data is replicated to my second codasrv? 
> Ie. venus on the second codasrv could really just be looking at the 
> first.

Ehh, that should be automatic as long as the server is reachable and the
volume actually was created as a doubly replicated volume on both
servers. A simple check on which servers a volume is replicated,
    'cfs whereis /coda/pbs.com'

On Tue, Nov 16, 2004 at 11:52:04PM -0600, Jerry Amundson wrote:
> On Tue November 16 2004 8:57 pm, Stephen J. Turnbull wrote:
> > this limit is given in bytes (256K, IIRC), not dentries, I suppose
> > that a somewhat imprecise "file too large" error might be issued when
> > you run out of space for more entries in a directory.
> 
> Could be, I didn't think that would rear it's ugly head so soon, though.
> 
> [root_at_uhura vicepa]# find 0 | wc -l
> 2717

If the filenames are shorter than about 32 characters I would expect the
limit to hit around 4000 entries in a single directory (approximately 32
for the name + probably another 32 for other data per entry ). My
maildir filenames are already 25 characters long but only seem to use
only the hostname and not the fully qualified domain name. If it were to
use the fqdn most files exceed 32 bytes and end up getting padded to the
next multiple of 32, limiting the total size to 256K/96 = 2730.

All of these numbers are approximate, there is some additional padding
going on and I think there is some reserved space for an allocation
bitmap and such. The 'rvmsizer' tool contains the full formula and will
display a warning whenever it encounters a directory that won't fit in
the existing Coda directory structure.

Considering how close your measured 2717 and my estimated 2730 entries
are I guess that your filenames contain the fully qualified domain name
and are typically longer than 32 bytes.

> >     Jerry> Also, how to I make sure the data is replicated to my
> >     Jerry> second codasrv? Ie. venus on the second codasrv could
> >     Jerry> really just be looking at the first.
> >
> > Shut down the first codasrv, and take a look via either venus, is the
> > obvious brute-force approach.  I don't think looking in the server's

That is definitely a brute-force solution. First thing would probably be
to check if the volume is actually replicated by using 'cfs whereis'.

> Looks like createvol_rep has to specify both server - the second server 
> cannot be added to an existing volume...?
> http://www.coda.cs.cmu.edu/maillists/codalist/codalist-2004/6180.html

Correct, growing and shrinking existing replicated volumes is definitely
something I want, but right now it requires a series of magic volutil
incantations and various editing/regenerating of VRList/VRDB and VLDB
files (and some required crossing of fingers). Definitely not
recommended if you're just getting started with Coda.

On Wed, Nov 17, 2004 at 09:05:53AM -0600, Jerry Amundson wrote:
> [root_at_uhura mail]# cfs lv /coda/pbs.com
>   Status of volume 0x7f000000 (2130706432) named "/"
>   Volume type is ReadWrite
>   Connection State is WriteDisconnected
>   Minimum quota is 0, maximum quota is unlimited
>   Current blocks used are 214647
>   The partition has 6392816 blocks available out of 11294220
>   *** There are pending conflicts in this volume ***
>   There are 13528 CML entries pending for reintegration
> 
> [root_at_monamie mail]# cfs lv /coda/pbs.com
>   Status of volume 0x7f000000 (2130706432) named "/"
>   Volume type is ReadWrite
>   Connection State is WriteDisconnected
>   Minimum quota is 0, maximum quota is unlimited
>   Current blocks used are 214647
>   The partition has 6392816 blocks available out of 11294220
>   *** There are pending conflicts in this volume ***
>   There are 9421 CML entries pending for reintegration

Ehh, are these from different startups, or right after each other? It
looks like the client is rapidly reintegrating as over 4000 operations
got pushed to the server between the two calls. But if there really is a
conflict it should have frozen the volume.

> Q1: How do I get back to "Connected"? "cfs wr" didn't seem to change 
> anything.

Really only 2 ways out, either find and repair the conflict or throw
away all local (but uncommitted) changes.

To dump the local CML the 'friendly way', cfs purgeml /path/to/volume
Alternatively to completely wipe any bad state from the Coda client,
kill the client, unmount /coda, and restart venus with a -init flag or
create a file named INIT in the venus.cache directory.

> Q2: How do I repair the conflict? beginrepair prompts "Pathname of 
> object in conflict?", which I do not know.

Typically it is the first entry in /usr/coda/spool/<uid>/<volname>.cml
(/var/lib/coda/spool/?) which is the first modification log entry that
failed to reintegrate. Alternatively, one that I tend to use a lot as it
can be used to find server-server conflicts as well,

    find /coda/path/to/volume -lname '@*'

> Q3: Do I need to use "cfs fl" in some controlled fashion to copy the 
> maildir's to this box? And throttle my tar extract, somehow?

Ehh, no. cfs fl simply dumps things from the cache. If your client is
(write-)disconnected it will automatically start throttling when the CML
fills up. Ofcourse disconnected operation does tend to be more sensitive
to conflicts, although most conflicts occur on the boundary where we
switch between connected and disconnected because a connected operation
typically times out and is retried during reintegration. Some operations
have no problem with this, but others complain that we're trying to
modify an already modified file, or create an already existing object,
etc.

You could use 'cfs strong ; cfs cs ; cfs wr', cfs strong tells venus to
not switch to write-disconnected operation as a result of sluggish
server behaviour. cfs cs then makes sure that all servers are reachable.
And finally 'cfs wr' tells the client to try to move all volumes to
'write-reconnected' or fully connected state.

Jan