The British Caving Association

David Gibson wrote:Well, obviously it depends on what youre doing. For your 'specialised' uses it may not work, but its fine for lots of other things where you are just
fetching data - e.g. scans of publications, photos, e.g. http://bcra.org.uk/pub/candks/catalogue.html

But all these pieces of data, deserve and will get the special treatment

try

https://views.scraperwiki.com/run/cave_ ... 20-%202005

(prototype, so a bit buggy) yes it is currently running on different machines, so it is possible, but it just adds an administrative burden, and complicates it for the geek and none geek user alike.

David Gibson wrote: Hmmm. What happens to the data is a "management" issue, I think; not concerned with where it is stored.

Totally agree, however, single point makes that management easier. In the end it is a compromise, the BCA is cloud then there is other cloud storage, my arguement is make things easy (and at a reasonable cost) for people and it is more like to be used.
Archive properly on the BCA seems to fit today's technology, compromise along the way may need to happen.....

David Cooke wrote:The figures are different to what I was told also the maths doesn't seem to add up. Can we be more precise?

I do not have direct access to the china svn server that the data is coming. so sorry no. Best figure I have is that it is 5.6Gb on disk with 2500 odd change sets, and this is only one of the servers, there is more on another one (but I believe a magnitude less).

cave-registry has about 8Gb free and at about change set 1800 I got and out of memory error. cpanel was showing almost full, cannot remember exact figures.

I am told the Matienzo dataset is 5Gb, that is not a transfer, so it should start at being that small

To try to get an idea CheddarCatchment is 132Mb on server with 194 changes. on my machine it is 115.5Mb. So managed correctly not much of an overhead.

Update: As of 23rd Sept. I've organised it so the total space is now 20GB.