Why you should back up your data

A place to discuss anything.
Post Reply
User avatar
ODPaul82
Posts: 683
Joined: Sun May 08, 2011 6:32 am
Location: Digswell Herts

So, I've never been one to shy away from when I balls up, and as a software developer for a day job I should know better.

Over the weekend I visited my mums and decided to take my external hard drive with me which contains everything trading wise.
All my models, all my daily notes, pretty much the vast majority of everything new that I'm thinking off.

As something as innocuous as "can you move your stuff off the table, I want to change the table cloth", led to me moving hard disk and laptop to the ground.

When I went to plug the hard disk back into my main trading rig at home, I went to access my football model in Excel.... "File not found". Checked the drive, a whole chunk of files missing. My main football model (FATI - Football Analysis Trading Intelligence [Also my stomach is a fatty and has its own gravity but that's a whole new topic]), all of my daily notes across May, a tool I developed to simplify matching individual entry/exit bets from betfair extract, all gone. My play-pen for football data to mess about with strategies. Gone. A ratings system for horses/dogs, disappearo.... Analysis I'd been doing on Australian greyhound and horses biases based on recent performance and trap/grading/course, disappeared in thin air.

Stuff I had been working on for 2+ years, just gone.

Having utiilised the microsoft "please dont panic but we'll try our best" free download to recover files and something more commercial I managed to recover FATI and a greyhound version I'd been developing of ratings I can figure out where I was with the other sports.

The whole TLDR is..... no matter how tech literal you are..... Make A F******* Backup
User avatar
ShaunWhite
Posts: 9731
Joined: Sat Sep 03, 2016 3:42 am

It's not too late to try EaseUS Data Recovery. It's the dogs bollocks. It's saved my bacon several times and recovers stuff you forgot you even had.

And while you're checking it out, their backup software is really good too. Just schedule an incremental backup and you're all good.
User avatar
Derek27
Posts: 23477
Joined: Wed Aug 30, 2017 11:44 am
Location: UK

I've used IDrive for years, for free. Great backup software, you can do incrementals, but when they scrapped free accounts they had the audacity to offer me a 90% discount for the first year. Made me feel a drug pusher was offering me free samples of heroin to get me hooked.

But here's an example of a free alternative. Just have Microsoft OneDrive running and install 7-Zip. Create a cmd file, add/change the directories you want backed up, user folder/7zip path names, remove the ATTRIB commands and "-Full" from the file name for an incremental backup and it will bung a 7-Zip file into your OneDrive folder. :)

Code: Select all

ECHO off

:: Reset all archive attributes for a full backup, comment out for an incremental backup
ATTRIB +a "C:\Users\Derek\AppData\Roaming\Bet Angel\Bet Angel Professional\*.*" /s
ATTRIB +a  D:\Data\*.* /s

:: Point to temporary folder
SET TempStore=D:\Backup_Temp

:: Point to OneDrive folder and subfolder(s)
SET BackupFolder=C:\Users\Derek\OneDrive\File Backup

:: Set password for 7-Zip file
SET PW=password

:: Create date/time file name
SET Filename=%Date:~8,2%
SET Filename=%Filename%%Date:~3,2%
SET Filename=%Filename%%Date:~0,2%
SET Filename=%Filename%-%Time:~0,2%
SET Filename=%Filename%%Time:~3,2%-Full.7z :: Remove "-Full" for incremental backup
SET Filename=%Filename: =0%

:: Copy files with archive attribute set
XCOPY "C:\Users\Derek\AppData\Roaming\Bet Angel\Bet Angel Professional\*.*" %TempStore%\BA\*.* /m
XCOPY "C:\Users\Derek\AppData\Roaming\Bet Angel\Bet Angel Professional\Automation\*.*" %TempStore%\BA\Automation\*.* /m
XCOPY "C:\Users\Derek\AppData\Roaming\Bet Angel\Bet Angel Professional\Shortcuts\*.*" %TempStore%\BA\Shortcuts\*.* /m
XCOPY D:\Data\*.* %TempStore%\Data\*.* /m /s

CD /d D:\

:: Create 7-Zip file
"C:\Program Files\7-Zip\7Z" A -t7z -p%PW% "%BackupFolder%\%Filename%" .\Backup_Temp\*

:: Delete temporary folder
RD /q /s %TempStore%

PAUSE
WisdomOfCrowds
Posts: 48
Joined: Tue Nov 09, 2021 2:55 pm

Gee, I'm really sorry to hear about the mayhem the lost data has caused.

I lost a lot of data about three years ago when a power cut occurred as I was booting up. Thankfully, I'd backed up most of the data a week before. I religiously made sure I backed everything up every night after that for a about year.

It's easy to become complacent - especially when nothing has gone wrong for a while - but thanks for the reminder. We can never be too careful.

I use a detachable hard-drive. I suppose the chances of that failing at the same time as my main computers is low but even that thought crosses my mind!

With the help and advice shown above, I hope you manage to retrieve everything. Good luck!

Mike
User avatar
paspuggie48
Posts: 611
Joined: Thu Jun 20, 2013 9:22 am
Location: South-West

Been there too ODPaul.

Back in 2010 I was living in Luxembourg and I developed macros to scrape the horse races from the Sporting Life website (not knowing at the time BA Pro can do it). I spent months collating all back data and then each and every day I religiously downloaded every days races and collected a mass of data over nearly 5 years.

When I came back to England I continued these tasks each day, always saying to myself "I must make a backup". I did on memsticks, but probably once every 6 months thinking "it will be just fine".

Then a few years later I had a small syndicate with my brother and some friends and we adopted some strats and started making some steady profit. All of a sudden my laptop died of death. No probs I thought, I'll get my backup...alas, it was 6 months old. The amount of data & VBA tools I had collected and developed in just that 6 months alone meant it was basically impossible to re-write and collect again and devise the strats as there were so many criteria and formulas.

It was then I decided to invest in BA Pro and the godsend was one can download the races each day as a CSV and with my new found PQ skills I could get all race details pretty damned quick. So, my new automated journey began.

Alas, I never learnt my lesson!!!

Last November my laptop died of death again and did I make a backup? Not really, I had some old snippets that I could work with to regenerate but basically it set me back a bit.

So, after getting another laptop I spent all of December re-developing my PQs and BAF files in BA in readiness for Jan 1st.

It was at this time I realised I really must invest in a back up system. Well, at work we use O365 SharePoint & OneDrive, so I dug into my pocket and subscribed for the annual Microsoft O365 package (around £70/yr I think). Now I use OneDrive for all my files and everything is auto-saved by OneDrive and is backed up online.

I doubt I will lose anything again but lesson learned for sure :D :) ;)
User avatar
ShaunWhite
Posts: 9731
Joined: Sat Sep 03, 2016 3:42 am

Daily incremental backups, weekly full backup alternating between at least 2 drives. Always use a physical drive and keep the weekly backup in the shed or somewhere else a house fire won't reach. I wouldn't use a cloud service, my data is too important to leave to someone else. A 1tb hdd is only about 30 quid so there's no excuses these days really. I've got an hdd caddy in the front of my machine and just swap them in/out. Backups aren't optional they're part of the job.
sniffer66
Posts: 1666
Joined: Thu May 02, 2019 8:37 am

ShaunWhite wrote:
Fri Jun 02, 2023 11:04 am
Daily incremental backups, weekly full backup alternating between at least 2 drives. Always use a physical drive and keep the weekly backup in the shed or somewhere else a house fire won't reach. I wouldn't use a cloud service, my data is too important to leave to someone else. A 1tb hdd is only about 30 quid so there's no excuses these days really. I've got an hdd caddy in the front of my machine and just swap them in/out. Backups aren't optional they're part of the job.
Agreed, but I do think there's value in doubling up by using a cloud service as a 3rd option. I do a daily\weekly backup to my RAID'ed NAS, and I also have the Google Drive agent backing up all files from certain folders on the fly. It doesn't hurt to have the security of having storage offsite
User avatar
ShaunWhite
Posts: 9731
Joined: Sat Sep 03, 2016 3:42 am

Fair enough, I can't argue with that sniffer. But maybe use it for incremental backups. I wouldn't fancy paying for, or downloading terabytes. :shock:
User avatar
ShaunWhite
Posts: 9731
Joined: Sat Sep 03, 2016 3:42 am

One more thing..... Test your recovery procedures periodically or its all a waste of time and money. Data is easy, but a dead C: drive will be fun. At work, once a year we used to relocate to a facility full of blank servers to simulate a disaster at HQ (esp after 9/11). The aim was to get the dealers back up to full ops inside 24hrs and back office in 48.
User avatar
ODPaul82
Posts: 683
Joined: Sun May 08, 2011 6:32 am
Location: Digswell Herts

Thanks everyone for words of advice and condolance with it, it's a true arse ache
I had been able to recover an older older version of FATI, I think I'm most annoyed about losing daily notes
User avatar
Derek27
Posts: 23477
Joined: Wed Aug 30, 2017 11:44 am
Location: UK

ShaunWhite wrote:
Fri Jun 02, 2023 11:04 am
Daily incremental backups, weekly full backup alternating between at least 2 drives. Always use a physical drive and keep the weekly backup in the shed or somewhere else a house fire won't reach. I wouldn't use a cloud service, my data is too important to leave to someone else. A 1tb hdd is only about 30 quid so there's no excuses these days really. I've got an hdd caddy in the front of my machine and just swap them in/out. Backups aren't optional they're part of the job.
For years I felt the same way about cloud backups. But you can use high-strength encryption, security should prevent technicians from accessing the data unless they need to and when you've got thousands of people backing up pictures of their cat or holiday snaps who in their right mind's gonna wast an hour looking to see what people are storing?

Ever since cameras have been added to phones the dreaded holiday photos have become mobile with people photographing anything and everything. I even met somebody who takes photos of signs with poor spelling/punctuation!
User avatar
napshnap
Posts: 1189
Joined: Thu Jan 12, 2017 6:21 am

Damn you all, I now feel unrest about my laptop!)
weemac
Posts: 1216
Joined: Mon Sep 16, 2013 8:16 pm

Years ago, while coming downstairs to put the morning kettle on, I decided it was ridiculous I had no back-up of my PC, so I went straight to my desk to do just that. Switched on the PC, only to be met with the blue screen of death.

Just unbelievable.
Post Reply

Return to “General discussion”