External High Capacity Photo Storage

[John Luxton]

I am reaching the point where my 2TB hard drive on my main PC is nearly full.

Space started shrinking rapidly once I started shooting in DNG at the end of 2016 and using Lightroom to create jpgs.

As well as the DNG/RAW images I also have the outputted jpgs stored on the hard drive, along with most of my other jpg images since 2000!

I seriously need to rethink my storage strategy within the next few days before the 2022 shooting season gets underway.

In addition as the collection has grown backing up takes longer - I also keep multiple back ups on external drives.

My next step is to move my collection off the PC onto an an external drive. (I do not wish to use a cloud already get 1GB free as part of my Microsoft sub but any more has to be paid for.)

Basically I see my options are:

RAID drive which is an external comprising two disks which can be configured to mirror each other so that there is always a copy should one fail.
NAS Storage - multiple hard drives held in an enclosure
External SSD drives - no moving parts longer life - but higher prices per GB of storage.

If anyone has used SSD just how much faster are they compared to a standard USB external hard drive?

Just wondering what others facing the same dilemma have done?

John

 

[AM9]

I've been using NAS for long-term storage of media files (there isn't much option when most of them are video files). Things that I've learnt over the years:

storing anything locally on a regularly used PC is always at risk of a PC hardware failure. As a device that is regularly switched on and off daily, the thermal stresses within key components (e.g. HDD, power supply, motherboard and/or components plugged into it), a failure can do considerable damage to the installation.​

​

if the PC is used for e-mail & general on-line browsing, any malware attack may cause damage to the data held locally, - possibly being subject to a ransom demand to retrieve it as well.​

​

if there is more than a single PC in the house, access to the media will probably only be from the keyboard of the machine that has them stored on.​

There was a time where a NAS was problematic in some simple home installations as large amounts of data held on them could be slow to access when relying on slow router wi-fi or even ethernet links. That has changed now with virtually every router having gigabit ethernet and wi-fi N or ac local connections.
A NAS has several advantages over local storage:

modern commercial NASs are small, consume minimal energy and have high reliability compared to workstations. They also have many self-management options including automatic backup, media format conversion, and abilities to act a media servers where any device capable of displaying/playing a media file can do so. In your example of camera images, most viewing devices are by dafault compatible with jpegs, so although you are wisely storing raw files for editing and printing, they are generally only compatible with workstations that have the appropriate drivers/modules installed to handle them.​

​

Storing single copies of data also has it's risks so a viable backup strategy should also be considered. Most NAS models have external USB ports where SSD/HDD drives can be left plugged in whilst their internal firmware/software manages scheduled back-up automatically.
If you have a router with a USB port on it you can sample some of the experience of using a NAS by connecting any USB pocket drive (or even a large USB pen drive) to it. If you then map that drive on your PC, you will be able to send and retrieve data from it in the same was as a NAS allows. Placing data on the drive will allow you to access your images form any device connected to the router.

 

[John Luxton]

I've been using NAS for long-term storage of media files (there isn't much option when most of them are video files). Things that I've learnt over the years:

storing anything locally on a regularly used PC is always at risk of a PC hardware failure. As a device that is regularly switched on and off daily, the thermal stresses within key components (e.g. HDD, power supply, motherboard and/or components plugged into it), a failure can do considerable damage to the installation.​

​

if the PC is used for e-mail & general on-line browsing, any malware attack may cause damage to the data held locally, - possibly being subject to a ransom demand to retrieve it as well.​

​

if there is more than a single PC in the house, access to the media will probably only be from the keyboard of the machine that has them stored on.​

There was a time where a NAS was problematic in some simple home installations as large amounts of data held on them could be slow to access when relying on slow router wi-fi or even ethernet links. That has changed now with virtually every router having gigabit ethernet and wi-fi N or ac local connections.
A NAS has several advantages over local storage:

modern commercial NASs are small, consume minimal energy and have high reliability compared to workstations. They also have many self-management options including automatic backup, media format conversion, and abilities to act a media servers where any device capable of displaying/playing a media file can do so. In your example of camera images, most viewing devices are by dafault compatible with jpegs, so although you are wisely storing raw files for editing and printing, they are generally only compatible with workstations that have the appropriate drivers/modules installed to handle them.​

​

Storing single copies of data also has it's risks so a viable backup strategy should also be considered. Most NAS models have external USB ports where SSD/HDD drives can be left plugged in whilst their internal firmware/software manages scheduled back-up automatically.
If you have a router with a USB port on it you can sample some of the experience of using a NAS by connecting any USB pocket drive (or even a large USB pen drive) to it. If you then map that drive on your PC, you will be able to send and retrieve data from it in the same was as a NAS allows. Placing data on the drive will allow you to access your images form any device connected to the router.

Thanks for feed back - I am aware of the risks of PC failure or being compromised by malware. That is why my existing collection is backed up regularly to external drives - usually three copies - one of which I take away with me on trips just in case any think happens when I am out!!

Although I have a second PC in the lounge this is an old machine I seldom use and is only there for use in emergency and certainly doesn't have the power memory to edit large DNG files.

I don't really need to access the drives from anything other than my main work station -where I do all my photo editing, cataloging, web site editing etc. Thus if I went for the NAS option I would probably want to hard wire it to the PC with an Ethernet connection, though I suppose an alternative could be to hide a the NAS in an under stair cupboard which has a power supply and is conveniently located half way between router and PC.

 

[Ken H]

I have had for some years a 500gb disk drive connected to my router via ethernet which I back up to.
I since bought a 1Tb data stick that I have plugged in the USB port on the router. So i have 2 drives independent of my laptop, accessible from any pc on the network.
My needs are more modest than the OP's. But I imagine they can be multiplied up to his requirement.
Dunno if you can still get drives that have ethernet capability. But you could use a USB hub to have more than one drive connected to the router
I use the router to hold these devices. The 1tb stick is very small so easy to lose. Stuck in the router permanently stops that risk.
My lappy has a 128Gb sd card in a slot so I have capacity on my pc.

all these things are mapped in windows explorer so are visible and usable as drive letters in windows and DOS

 

[pdq]

You'll have to pay for a new NAS or similar so why not pay for Cloud storage?

 

[John Luxton]

You'll have to pay for a new NAS or similar so why not pay for Cloud storage?

I prefer something tangible and own for my storage. Not reliant on a third party for which a subscription is payable.

Twice in my computing life I had problems with web site providers who "disappeared" and with it the web site that was stored on their servers.

Perhaps now you can see why I am hesitant to use any form of cloud for my principal storage?

 

[Ken H]

You'll have to pay for a new NAS or similar so why not pay for Cloud storage?

I always wonder how secure cloud is. what if someone goes bust?

I scan old timetables. I upload them to wayback machine ( upload.archive.net ) in the hope they are preserved after I am gone. They are on Timetable World also.

 

[John Luxton]

I always wonder how secure cloud is. what if someone goes bust?

My thought exactly!

Or what happens if you have a problem with the subscription payment?

If you own the drive it can't disappear other than be stolen by an intruder - which I suppose to some extent is a possibility, but probably a much lower one.

 

[AM9]

... I don't really need to access the drives from anything other than my main work station -where I do all my photo editing, cataloging, web site editing etc. Thus if I went for the NAS option I would probably want to hard wire it to the PC with an Ethernet connection, though I suppose an alternative could be to hide a the NAS in an under stair cupboard which has a power supply and is conveniently located half way between router and PC.

Unless your PC has more than one ethernet port, you can't wire a NAS directly to it without losing your internet connection. In addition, two devices on a network with no control capability would make ip address allocation difficult. Depending on your layout the NAS could live next to the router which would mean that a short, (<1m) cable could be used for a direct connection, - much mpre reliable than a wi-fi connection.

 

[John Luxton]

Unless your PC has more than one ethernet port, you can't wire a NAS directly to it without losing your internet connection. In addition, two devices on a network with no control caqpability would make ip address allocation difficult. Depending on your layout the NAS could live next to the router which would mean that a short, (<1m) cable could be used for a direct connection, - much mpre reliable than a wi-fi connection.

BT upgraded my phone system to full fibre some weeks ago which means my router no longer lives on my computer desk my pc now communicates with the router via Wifi rather than Ethernet connection.

The Ethernet port on the PC is currently used by my laser printer which means it can be used via the wifi from elsewhere in the house. However, I could switch the printer to USB to free up the PC Ethernet socket to attach a NAS.

 

[dgl]

I would convert the printer to wireless using a suitable access point rather than via the PC, it's much better that wa and then I would also either connect the NAS to the same access point (assuming it's fast enough) or have it connected directly to the router.
Power line could also be another option, again you would want one of the higher speed systems.

Naturally if it's only the main machine that the NAS will be used with then the speed of the AP is less of an issue as you could just use a gigabit switch to connect the PC, printer and NAS together.

I've had my NAS for about 10 years and it's been faultless, do get one with mirrored drives as it protects you against HDD failure.

 

[AM9]

The Ethernet port on the PC is currently used by my laser printer which means it can be used via the wifi from elsewhere in the house. However, I could switch the printer to USB to free up the PC Ethernet socket to attach a NAS.

The problem there is that if you do want to print from any other device, the PC must first be turned on and logged into.

 

[Ken H]

If speed is an issue, connect PC to router with Ethernet. Connect NAS to router either USB or ethernet. Printer is wifi or usb from pc.
I cant ethernet from PC cos no ethernet port Last laptop I used to use Wifi except then doing heavy file transfer stuff, when I would use ethernet.
For me the slowest thing is uploading to google Drive. Seems to take ages. Wish I could FTP to it, but I dont know how to.

 

[John Luxton]

The problem there is that if you do want to print from any other device, the PC must first be turned on and logged into.

The PC is on for most of the time when I am at home anyway - so not a major issue for me.

 

[172007]

I researched my own cloud back up drive options earlier this year. I could not find what I was looking for. None of the drives I found could be backed up them selves with another. I.e. a could drive at my home that is then backed up by a clip drive at my parents. This solves the Docklands bombing problem where quite a few backups where lost as they were stored in the same building as the servers etc.

 

[Ken H]

I researched my own cloud back up drive options earlier this year. I could not find what I was looking for. None of the drives I found could be backed up them selves with another. I.e. a could drive at my home that is then backed up by a clip drive at my parents. This solves the Docklands bombing problem where quite a few backups where lost as they were stored in the same building as the servers etc.

You really need an off site back up. Whether its a spare disk you keep in your office drawer or in your car glove compartment - it doesnt matter. Consider losing all your stuff if there is a fire at home. For most it doesnt matter but if you consider your stuff to be valuable then you need to do something.

 

[AM9]

I would convert the printer to wireless using a suitable access point rather than via the PC, it's much better that wa and then I would also either connect the NAS to the same access point (assuming it's fast enough) or have it connected directly to the router.
Power line could also be another option, again you would want one of the higher speed systems.

Naturally if it's only the main machine that the NAS will be used with then the speed of the AP is less of an issue as you could just use a gigabit switch to connect the PC, printer and NAS together.

I've had my NAS for about 10 years and it's been faultless, do get one with mirrored drives as it protects you against HDD failure.

My issue with using wi-fi for any internal data moving is that depending on the competition from neighbours for wi-fi channels, - particularly on 2.4GHz, the total data load must be shared on a single connection whereas an ethernet switch can handle as many full speed channels as there are pairs of devices to converse.

 

[AM9]

I researched my own cloud back up drive options earlier this year. I could not find what I was looking for. None of the drives I found could be backed up them selves with another. I.e. a could drive at my home that is then backed up by a clip drive at my parents. This solves the Docklands bombing problem where quite a few backups where lost as they were stored in the same building as the servers etc.

That would probably be do-able via a link if both drives were in a DMZ (a DeMilitarised Zone whic is accessible from both within and outside thew firewall function of the router). Maybe you know some nerdy gamers who are the main domestic users of DMZs. Another possibility might be a mini VPN at each end, but this may be getting a bit complex for your parents.

 

[Furrball]

Your router should have multiple LAN ports, you can put the NAS next to the router

Possible though that the router may only be 10/100 ports depending its age in which case a 5 port Gigabit switch an be obtained for under 20 quid and placed next to your computer

Or get a USB 3 raid device. USB 3 will actually be faster for you than network

 

[John Luxton]

Your router should have multiple LAN ports, you can put the NAS next to the router

Possible though that the router may only be 10/100 ports depending its age in which case a 5 port Gigabit switch an be obtained for under 20 quid and placed next to your computer

Or get a USB 3 raid device. USB 3 will actually be faster for you than network

Yes the BT router does have multiple land ports.

Didn't realise a USB3 raid would be any faster.

That might be the better solution as I only work on photos via one PC.

Just wondering how SSD via USB compares to USB Raid in term of back up speed.

SSD does appear to becoming cheaper and is more compact.

 

[pdq]

Don't worry about the the number of ethernet ports. A Netgear or TP Link switch costs peanuts and will distribute the network resource across extra devices. From my router I have a desktop PC, printer, NAS, TV, hifi, DVD player and pvr all connected, with the help of a powerline connection and two switches. Phones, another couple of laptops, Chromecast, Chromecast audio and the heating hub are connected with wifi. There is no issue with connectivity across those devices.

 

[Furrball]

Yes the BT router does have multiple land ports.

Didn't realise a USB3 raid would be any faster.

That might be the better solution as I only work on photos via one PC.

Just wondering how SSD via USB compares to USB Raid in term of back up speed.

SSD does appear to becoming cheaper and is more compact.

The cable between your computer and the storage device will likely be the limiting factor rather than the speed of the drives in the external storage

Save the money on the external drive and go with good quality mechanical drives and treat yourself to a SSD or two for in the computer itself

 

[John Luxton]

After further research I finally decided on the G-RAID 12TB Thunderbolt drive which as well as being USB compatible also has Thunderbolt connectivity which will be useful for when I upgrade my PC which I will probably do in 2023 as I tend to upgrade every 6-7 years.

Appears to be be well made in a metal enclosure and fairly bullet proof with a 5 year warranty, the internal drives are removable as well.

Configured for RAID1 mirroring will deliver 6TB storage and units can be daisy chained.

Anyway have just ordered it for delivery around the weekend.

 

[Islineclear3_1]

As someone with limited computer knowledge, I too need to start thinking about reviewing my backup storage options. I concur with John; cloud storage isn't for me and I prefer physical tangible units that I can retrieve quickly if necessary.

I generally back up to 2 x external HDD drives and USB pen drives. I have about 20 external drives , only had 1 failure in 10 years but the digital files are all RAW files and will be a retirement project to convert them all to usable JPEGs. I keep most of the drives away from the house and still using a 20-year old PC....

This system has worked for me since I "went digital" in 2006 but I agree, there must be a better, more efficient and secure way

 

[vedereds]

A RAID disk might be a good option too. I became a little more calm about my data, but this is not enough. True, it seems to me that the simplest and best solution would be to buy a flash drive. In this case, your main task is not to forget where you kept it. In the case of everything related to storing files on a computer, you should be prepared for the fact that at any time you will have to call the hard drive data recovery service. I have encountered this problem many times, so I make at least two copies of my files while working with them, and then two more copies of the finished format to other flash media.

 

[dakta]

I've just started playing with a gopro I bought and have learnt if you do want to take quality footage at todays resolutions you can go through gigabytes like butter.

I thought a 1tb secondary mechanical disk (its fast enough) would be enough however that's not looking likely to be the case.

I run my PC (in terms off operating system and programs) off a 1tb ssd with a 1tb mech disk for the files - so what i'll likely do in the future is buy 2 x 4tb mechanical disks as price doesn't seem too daft and mirror them. Footage will likely be rubbish but you don't want to lose anything, anything that's actually worth a keeper can get shoved onto the cloud - though i only have 1tb of that. So that's my own plan unless I can think of something better.

I say 'only' tongue firmly in cheek as I remember getting my first 1gb drive and it doesn't feel that long ago.