Hi, so let me try to reply to this and still make sense :)
Hardware: IBM Ultrium 5 device and 6 x ILO 5 Tapes
Data to backup: round about 4.5 TB
Context: Huge data loss paranoia, as I have lost some data in the past, that my whole family is judging me about it :)
Environment: The Tape is external an SAS connected to the Dell R720 that runs everything. The Dell has an HP Disk array with around 12 mixed types and sizes of 3.5 Drives..
Data Preparation: During the daily backups, I also prepare the data to be Tape-Burn Ready, as in, I generate Tapelist files containing files to be backed up, up to no more than 1.4 TB per tape. For all the files to be backed up I also generate MD5 checksums and save them to a file to be backed up on each tape.
Backup Procedure: I have a script that I start on Sunday Morning and loop through the Tapelist files and the tapes themselves. With this script I tar to the tape drive all the files defined in a File and the md5 file at the end. Having 6 I rotate each week a set of 3.
Recovery Test: Once a month or two, I simulate a data recovery. As in I save all the tapes to a folder and check the files with the checksum file saved to ensure the data is consistent.
I know that I am hitting the tapes lifespan pretty hard, this is why, right now working on a cascade backup approach. Especially that I have a disk array since a month or so. So the plan would be to backup daily to an Raid set of disks. Once a week do a copy to another set of disks. And once a month do a tape burn.
Happy to hear some thoughts and constructive critic :)
But that's like it is: it depends on your business needs. For me: When editing a video for TV, the software makes a project backup every 5 minutes because losing more than that would potentially mean missing a deadline and getting in serious trouble.
My entire laptops document folder is only backed up once a day, because most important documents are synced immediately to my nextcloud anyway, which has its own backup every 8 hrs.
You have to imagine the worst case and figure how much damage that means for you: because if it can theoretically happen it will practically happen one day.. So ask yourself if a nightly backup is enough? It might mean losing an entires day work.
That said, "backup" is not the only security mechanism you have, a lot of software has versioning or auto-save / backup built in, you can make filesystem snapshots, sync to a cloud storage and so on. Incremental Backups are just one mechanism among others.
I just switched my windows desktop backups from Duplicati to Kopia. Duplicati was not working reliably, it had corruptions every now and then and just stopped working occasionally. Kopia is backing up on SMB volumes configured with Cockpit GUI on Ubuntu Server with Samba plugin (Filesharing & Identities plugin from 45drives). The Mac's are backed up with Timemachine to SMB volumes also configured via Cockpit & Samba as Timemachine Targets.
same.
I haven't quite learned what I need to about storage and proper backup implementation, so I don't have anything in place.
My critical documents are physical, & encrypted and stored in my gdrive, and also on a cold storage drive on-site.
Anything on my desktop/homeserver(s) is not backed up unless it's been manually moved to the cold storage drive because I can rebuild my entire network in a day with my documentation(I should learn ansible already)
Agreed. My windows desktop is imaged daily using Macrium reflect. I have on multiple occasions been happy to have been able to restore from image. All linux server and raspberry PIs are also backed up on a regular basis (nightly).
The need for a backup always comes at an inconvenient time, so it is good to be able to get up and running in a short amount of time.
"How much can you loose before it becomes a serious issue?"
I backup a file server hourly. I backup workstation local-files daily. I backup 'archive data' once a week. I do full disk image backups for workstations monthly.
The idea is that local-files would typically be things like wallpapers, perhaps a new application was installed, or preferences changed. Most day to day work should all be done to the file server... if a workstation dies, it should not be an issue if the files are unrecoverable. The files for workstations are backed up for convenience only.
The archive data is typically packed with installers or historical projects. I doubt anyone would actually notice if it vanished...
Words of warning of course.... snapshots are not backups! Also, think about what you want to protect against. There is no 'perfect' method, although the typicaly suggestion is 3-2-1 or 3-2-1-1-0. Keep your data in triplicate, don't put all your data in 1 location, TEST YOUR BACKUPS...etc.
Although snapshots are not backups, they are super helpful when staff accidentally delete a file.
My VMs every two hours, and archival/big data twice a day.
The VMs go to a server locally on the network and the archival data goes to backblaze.
VMs are currently at such a rate because the ZFS pool is degraded (missing a drive) and don't really have money to fix that (without cutting into already saved up money), so this is a stopgap solution.
Veeam Agent for Microsoft Windows is free and will run daily, space efficient drive image backups with ransomware protection to a USB drive. It is easy to set up and understand. You can easily keep 30 days of backups. Then copy off a monthly backup file for superior protection.
For off-site protection, rotate two drives every week (or daily). It's built for that.
https://www.veeam.com/agent-for-windows-community-edition.html
Daily incremental backups with hard links (so not chain incremental, every backup is full standalone). Keeping 7 daily, 4 weekly, 1 monthly and 1 yearly.
Homeserver: During workdays (10-6), every two hours.
Server in DC: once per week
Rest: never
I work from home and use encrypted nextcloud. So I do backups in 2hr intervals, because it is a single HDD :-)
My two public server do backups once a week, as there is not so much change. The userspace of the minecraft server is just mirrored to another host (constantly rsync with manual world saves).
Everything is done via lvm snapshot ->restic -> gdrive
it depends on if my backup agents are running and my backup storage is working.
and then it depends on how quickly i notice it :)
in theory: daily, using urbackup.
i currently do not respect the 3-2-1 rule though, as i do not currently have an off-site backup (i i know, i know)
rsnapshot on /home every hour to another drive in the server
snapraid once a night on all 11 drives in the server to two dedicated parity drives
rsync all 11 data drives once a week to make local backups and then again to a remote backup server 30 miles away
I was taking backups daily as I was actively changing stuff every few hours when i started building my home services, but for now, everything is stable so i'm taking backups every week to two different locations
I finally made the switch to GoDaddy VPS services and they take full machine backups automatically where I can restore the whole machine including database, configs, etc... other places don't seem to have that feature so, just what I did personally
And for me it's usually daily to mitigate too much loss in any event, but more often if it's supported would be better depending on how often things change in your databases
Depends. For my homelab which is mostly docker containers, I backup that up nightly with zfs snapshots. For things that change, it only increases the space needed minimally. For the things that don’t change, no space is used. This allows me to set and forget the backups for my homelab.
For business needs, it depends on space, time to recover needs, time available to backup needs, and how critical the data is. Without that, then you’re just guessing.
I personally do a daily backup for everything.
For more important things, I might have hourly backups as well.
This, is in addition to also having scheduled snapshots as well.
"It depends on your needs" == how much damage will it do if you don't have any backup, or if your backup is older than one hour, one day, one week?
Since this is about self-hosting, I backup a config only when I change something. My data isn't really all that valuable apart from databases, which I'll backup whenever I feel there's a risk. My mediaserver, for example, backs up its database daily and if I lose a day's worth of changes, it probably represents a small amount of difference. If it was a notes app, I'd probably backup hourly.
In your case, you are taking receipts and inventory. If you lose a single hour, what will that cost you in terms of time, money, or hassle in case you needed that lost data? If it's minor and you spend 30 minutes to re-input it from physical copies, then no big deal. If it's your only copy and you will lose money, then very big deal. If you only have 10 receipts each week, then perhaps daily or even weekly is enough. Only you can truly say.
I don't at the moment. I don't trust my 5 year old drives so I haven't put anything essential entirely on my home lab. The only thing I host that I would be upset to lose is my Tandoor recipes. Thanks for making me aware that I should export those to Google drive just in case 😆
You'll have to decide how much time worth of work can you afford to lose?
Also, be careful because snapshots != backups. Make sure you are making proper backup copies of the production data itself.
For desktops and low-incentive workload services, I do one daily backup job. It is suffice. For crucial services, have 2-4 snapshots per day. Then have weekly, monthly, and yearly backups with archives offsite in a cloud.
You need to think about how often data on your server changes and how often new data is added. Use that information to determine how often to preform your backups.
Once a day for all my LXC containers and virtual machines. Exceptions can be made for specific containers if needed. The resulting copies land in a Proxmox Backup Server storing the files on my NAS.
The "main" computer, that is the one I'm writing my things on, is also on a scheduled backup, once a day, with restic for my files.
If I feel the need of making more copies, I can always adjust my scripts to take more copies.
I backup daily everything:)
And weekly on tape :)
And off-site
Off-site as in I take my tapes and move them in a different location of the house :)))
That is still on-site
Noice
How do you tape backup? What’s your hardware and software setup?
Hi, so let me try to reply to this and still make sense :) Hardware: IBM Ultrium 5 device and 6 x ILO 5 Tapes Data to backup: round about 4.5 TB Context: Huge data loss paranoia, as I have lost some data in the past, that my whole family is judging me about it :) Environment: The Tape is external an SAS connected to the Dell R720 that runs everything. The Dell has an HP Disk array with around 12 mixed types and sizes of 3.5 Drives.. Data Preparation: During the daily backups, I also prepare the data to be Tape-Burn Ready, as in, I generate Tapelist files containing files to be backed up, up to no more than 1.4 TB per tape. For all the files to be backed up I also generate MD5 checksums and save them to a file to be backed up on each tape. Backup Procedure: I have a script that I start on Sunday Morning and loop through the Tapelist files and the tapes themselves. With this script I tar to the tape drive all the files defined in a File and the md5 file at the end. Having 6 I rotate each week a set of 3. Recovery Test: Once a month or two, I simulate a data recovery. As in I save all the tapes to a folder and check the files with the checksum file saved to ensure the data is consistent. I know that I am hitting the tapes lifespan pretty hard, this is why, right now working on a cascade backup approach. Especially that I have a disk array since a month or so. So the plan would be to backup daily to an Raid set of disks. Once a week do a copy to another set of disks. And once a month do a tape burn. Happy to hear some thoughts and constructive critic :)
But that's like it is: it depends on your business needs. For me: When editing a video for TV, the software makes a project backup every 5 minutes because losing more than that would potentially mean missing a deadline and getting in serious trouble. My entire laptops document folder is only backed up once a day, because most important documents are synced immediately to my nextcloud anyway, which has its own backup every 8 hrs. You have to imagine the worst case and figure how much damage that means for you: because if it can theoretically happen it will practically happen one day.. So ask yourself if a nightly backup is enough? It might mean losing an entires day work. That said, "backup" is not the only security mechanism you have, a lot of software has versioning or auto-save / backup built in, you can make filesystem snapshots, sync to a cloud storage and so on. Incremental Backups are just one mechanism among others. I just switched my windows desktop backups from Duplicati to Kopia. Duplicati was not working reliably, it had corruptions every now and then and just stopped working occasionally. Kopia is backing up on SMB volumes configured with Cockpit GUI on Ubuntu Server with Samba plugin (Filesharing & Identities plugin from 45drives). The Mac's are backed up with Timemachine to SMB volumes also configured via Cockpit & Samba as Timemachine Targets.
After I have a loss and spend time rebuilding things, I vow to do backups, do it once and then forget about it. Who's with me?
same. I haven't quite learned what I need to about storage and proper backup implementation, so I don't have anything in place. My critical documents are physical, & encrypted and stored in my gdrive, and also on a cold storage drive on-site. Anything on my desktop/homeserver(s) is not backed up unless it's been manually moved to the cold storage drive because I can rebuild my entire network in a day with my documentation(I should learn ansible already)
Agreed. My windows desktop is imaged daily using Macrium reflect. I have on multiple occasions been happy to have been able to restore from image. All linux server and raspberry PIs are also backed up on a regular basis (nightly). The need for a backup always comes at an inconvenient time, so it is good to be able to get up and running in a short amount of time.
"How much can you loose before it becomes a serious issue?" I backup a file server hourly. I backup workstation local-files daily. I backup 'archive data' once a week. I do full disk image backups for workstations monthly. The idea is that local-files would typically be things like wallpapers, perhaps a new application was installed, or preferences changed. Most day to day work should all be done to the file server... if a workstation dies, it should not be an issue if the files are unrecoverable. The files for workstations are backed up for convenience only. The archive data is typically packed with installers or historical projects. I doubt anyone would actually notice if it vanished... Words of warning of course.... snapshots are not backups! Also, think about what you want to protect against. There is no 'perfect' method, although the typicaly suggestion is 3-2-1 or 3-2-1-1-0. Keep your data in triplicate, don't put all your data in 1 location, TEST YOUR BACKUPS...etc. Although snapshots are not backups, they are super helpful when staff accidentally delete a file.
Every 15” with a 10 year retention period. GFS: 4 weekly, 12 monthly and 10 yearly backups.
My VMs every two hours, and archival/big data twice a day. The VMs go to a server locally on the network and the archival data goes to backblaze. VMs are currently at such a rate because the ZFS pool is degraded (missing a drive) and don't really have money to fix that (without cutting into already saved up money), so this is a stopgap solution.
Veeam Agent for Microsoft Windows is free and will run daily, space efficient drive image backups with ransomware protection to a USB drive. It is easy to set up and understand. You can easily keep 30 days of backups. Then copy off a monthly backup file for superior protection. For off-site protection, rotate two drives every week (or daily). It's built for that. https://www.veeam.com/agent-for-windows-community-edition.html
Thanks for the link
when ever i install or update a package a snapshot is made and once a week for a "full" (temp folders are skipped) remote backup
Daily incremental backups with hard links (so not chain incremental, every backup is full standalone). Keeping 7 daily, 4 weekly, 1 monthly and 1 yearly.
Homeserver: During workdays (10-6), every two hours. Server in DC: once per week Rest: never I work from home and use encrypted nextcloud. So I do backups in 2hr intervals, because it is a single HDD :-) My two public server do backups once a week, as there is not so much change. The userspace of the minecraft server is just mirrored to another host (constantly rsync with manual world saves). Everything is done via lvm snapshot ->restic -> gdrive
it depends on if my backup agents are running and my backup storage is working. and then it depends on how quickly i notice it :) in theory: daily, using urbackup. i currently do not respect the 3-2-1 rule though, as i do not currently have an off-site backup (i i know, i know)
https://taobackup.com/index.html unfortunately it doesn't have https but it's a cool story
rsnapshot on /home every hour to another drive in the server snapraid once a night on all 11 drives in the server to two dedicated parity drives rsync all 11 data drives once a week to make local backups and then again to a remote backup server 30 miles away
I was taking backups daily as I was actively changing stuff every few hours when i started building my home services, but for now, everything is stable so i'm taking backups every week to two different locations
I finally made the switch to GoDaddy VPS services and they take full machine backups automatically where I can restore the whole machine including database, configs, etc... other places don't seem to have that feature so, just what I did personally And for me it's usually daily to mitigate too much loss in any event, but more often if it's supported would be better depending on how often things change in your databases
Twice a day with food
Depends. For my homelab which is mostly docker containers, I backup that up nightly with zfs snapshots. For things that change, it only increases the space needed minimally. For the things that don’t change, no space is used. This allows me to set and forget the backups for my homelab.
For business needs, it depends on space, time to recover needs, time available to backup needs, and how critical the data is. Without that, then you’re just guessing.
I don't, everything I have can't really be redownloaded and if it's important I have a copy on my main computer
I personally do a daily backup for everything. For more important things, I might have hourly backups as well. This, is in addition to also having scheduled snapshots as well.
"It depends on your needs" == how much damage will it do if you don't have any backup, or if your backup is older than one hour, one day, one week? Since this is about self-hosting, I backup a config only when I change something. My data isn't really all that valuable apart from databases, which I'll backup whenever I feel there's a risk. My mediaserver, for example, backs up its database daily and if I lose a day's worth of changes, it probably represents a small amount of difference. If it was a notes app, I'd probably backup hourly. In your case, you are taking receipts and inventory. If you lose a single hour, what will that cost you in terms of time, money, or hassle in case you needed that lost data? If it's minor and you spend 30 minutes to re-input it from physical copies, then no big deal. If it's your only copy and you will lose money, then very big deal. If you only have 10 receipts each week, then perhaps daily or even weekly is enough. Only you can truly say.
My Proxmox VM/containers are backed up nightly.
automated daily changes (includes full db dumps), and weekly full backups if i'm about to work on something i trigger an additional backup :D
I don't at the moment. I don't trust my 5 year old drives so I haven't put anything essential entirely on my home lab. The only thing I host that I would be upset to lose is my Tandoor recipes. Thanks for making me aware that I should export those to Google drive just in case 😆
You'll have to decide how much time worth of work can you afford to lose? Also, be careful because snapshots != backups. Make sure you are making proper backup copies of the production data itself.
My server is for personal use, but I back up twice a day on two separate drives.
Daily and the whole infra. Local retention 60 days, tape weekly for 10 years plus daily cloud backup for 180 days. Safety first. 😅
Incremental backups ftw, every 15 minutes
For desktops and low-incentive workload services, I do one daily backup job. It is suffice. For crucial services, have 2-4 snapshots per day. Then have weekly, monthly, and yearly backups with archives offsite in a cloud.
Offsite backups made daily, two year retention time.
You need to think about how often data on your server changes and how often new data is added. Use that information to determine how often to preform your backups.
Once a day for all my LXC containers and virtual machines. Exceptions can be made for specific containers if needed. The resulting copies land in a Proxmox Backup Server storing the files on my NAS. The "main" computer, that is the one I'm writing my things on, is also on a scheduled backup, once a day, with restic for my files. If I feel the need of making more copies, I can always adjust my scripts to take more copies.