Request for more space for Cave Registry

[David Cooke]

I have added a few more things to the forum, is it appropriate to ask on the forum, for more room (see previous email) on the cave-registry site. Getting good hint that we might get Peak stuff sometime, also Footleg has started to work on the Swildons data, so we are cutting it a little fine with only 150Mb space. Don't want people getting error messages due to lack of space, it has to be a totally reliable service

I've added a further 300MB.

I don't supposes there is any easy way to get SVN to use cloud storage a la http://www.memset.com/cloud/storage? It would be cheaper there. I'm not too concerned about price at the moment, but thinking of the future.

[Andrew]

Thanks

Don't think it can be done easily, as it is database, not at real file structure, I'll put it on my list of things to look at

[David Gibson]

I don't supposes there is any easy way to get SVN to use cloud storage a la http://www.memset.com/cloud/storage? It would be cheaper there. I'm not too concerned about price at the moment, but thinking of the future.

Im happy to use that Cloud storage for a lot of BCRA's data. Ive just taken delivery of 2.2 GB of scans of old issues of C&KS and it would take the pressure off the web server if this data could go elsewhere. My main question, Cookie, is "is this a service BCA will be providing?" - in which case, "how soon will it be available?", or is it a case, now youve pointed it out to me, of my arranging my own cloud storage at Memset? The reason Im asking is that, otherwise, Im going to need 2GB more web space.

[Andrew]

I would like to transfer another couple of svn databases, for this more room is needed, please can we increase the storage. 400Mb would do but 1Gb would be better

thanks

[David Cooke]

You now have 2.2GB

[Andrew]

Ta

[Andrew]

Hi

Could we up the space available to 10 Gb, it looks like we are about to get another 5 Gb of surveying data in the registry (I do not want to say from where on an open forum until it actually happens, so as not to cause embarresment/pressure if it does not work out, but would like it ready, as they have asked for a username and pw so should happen)

Andrew

[David Cooke]

Done.

Starts with "M"?

[Andrew]

Thank you for such quick action

Mmmmm, you do know everything

Andrew

[Andrew]

Hi

Could we have some more please, and lots, just tried to port the hmg data that I am told that has been agreed, well ran out of space very quickly. I would suggest another 20Gb as the hmg is at 8 and we will have 5Gb+ of Matienzo data coming in.

thanks

Andrew

PS cpanel is not responding, asks for username and pw but just keeps asking, no errors, just times out in the end, happened when we ran out of space, deleted hmg by command line for now, so hope it will come back, will post separately if it does not

[David Gibson]

I would suggest another 20Gb as the hmg is at 8 and we will have 5Gb+ of Matienzo data coming in.

Nothing to do with me, of course, but Im just remembering how reluctant Cookie was to give me additional space for BCRA's archive a few months ago In fact, he was quite right, there was no need for all the info I was archiving to go on a "web site" at all, and it was suggested that I set up some cloud storage at MemSet... which I did. Seems to work well... it has its advantages and disadvantages, but Im now in the process of thinking about (note the double qualifier - "process of" => wont happen soon; "thinking of" => wont happen soon either) moving all the active Cave & Karst Science PDFs off the web site and into cloud storage. Access to these, at http://bcra.org.uk/cks, is via what is probably called, by proper programmers, a content delivery system, and the salient point is that I can put the files "anywhere" - even on an FTP site - and the user himself wont notice any (much) difference.

So I just wondered whether, given the large amount of data storage space you need, Andrew, you'd be better off with Cloud Storage? Ultimately, though, if Cookie gives you the space, you dont have a problem

[Andrew]

Fortunately svn does not work with cloud storage.

Why do I say fortunately. Well, it is crazy, storage should be cheap as chips (pick your variety) The BCA is a part of the cloud, why would I want to put my data on a different server. Cloud storage does have its uses, but not if you own, or rent a server, use that and then your data can become so much more. The real time analysis of data that can go on now is so powerful, take a look at what CUCC are doing with there log books and extracting of data from them, it is in its infancy, but it will get better. The survey data in Therion could be automatically run when an update is received and the surveys that are already available updated in real time. The list is endless, it could all be done with the data on another (cloud) server, but the barriers and the speed degradation would be terrible, we will be doing this with photos shortly. Photos should be stored at full resolution. Currently ubss internet provider is unlimited, as do most others, 50 quid a year, all in.

Using a cloud server for the other data is similar to sticking it in a loft and waiting for it to be lost.

The BCA and BCRA should have the foresight to have archiving as a priority, and online and available. Both organisations should be encouraging data storage on there servers, with the ability to enhance the information but having excellent access, maybe a discussion to be had at HE.

Sorry, should try not to rant.....

Andrew

[David Gibson]

The list is endless, it could all be done with the data on another (cloud) server, but the barriers and the speed degradation would be terrible

Well, obviously it depends on what youre doing. For your 'specialised' uses it may not work, but its fine for lots of other things where you are just fetching data - e.g. scans of publications, photos, e.g. http://bcra.org.uk/pub/candks/catalogue.html

Using a cloud server for the other data is similar to sticking it in a loft and waiting for it to be lost.

Hmmm. What happens to the data is a "management" issue, I think; not concerned with where it is stored. For example, I keep a mirror of my cloud site on a local 1TB network drive, which is synchronised daily to what's in the cloud storage. People who dump data on any sort of server and forget about it are just asking for trouble?

The BCA and BCRA should have the foresight to have archiving as a priority, and online and available. Both organisations should be encouraging data storage on there servers, with the ability to enhance the information but having excellent access, maybe a discussion to be had at HE.

I agree.

When I originally asked for "more space" and Cookie suggested that I used Memset's cloud storage his philosophy was, IIRC, that BCA neednt be involved because Memset's cloud storage was free, so there were no accounting issues. (Memset is BCA's service provider for web space). However, I found that the free service was not sufficient and so Im now using paid-for cloud storage. This is a nuisance because every month I get a bill for a few pounds (download access is something like 10p per MB IIRC) which I need to remember to claim from BCRA (or BCA?). So, now that Im having to pay, I think it makes more sense for BCA to manage the payments. So,how about it, Cookie? I request that BCA manages a cloud storage system for its members.

[David Cooke]

Hi

Could we have some more please, and lots, just tried to port the hmg data that I am told that has been agreed, well ran out of space very quickly. I would suggest another 20Gb as the hmg is at 8 and we will have 5Gb+ of Matienzo data coming in.

Andrew, I'll probably have to purchase more disk space so it'll have to wait until I'm back from Czech.

The figures are different to what I was told also the maths doesn't seem to add up. Can we be more precise?

[David Cooke]

Hi Dave,

if I could push it out to cloud storage I would but I don't think the archive software (SVN) will play well with it.