You might not even like rsync. Yeah it’s old. Yeah it’s slow. But if you’re working with Linux you’re going to need to know it.
In this video I walk through my favorite everyday flags for rsync.
Support the channel:
https://patreon.com/VeronicaExplains
https://ko-fi.com/VeronicaExplains
https://thestopbits.bandcamp.com/
Here’s a companion blog post, where I cover a bit more detail: https://vkc.sh/everyday-rsync
Also, @BreadOnPenguins made an awesome rsync video and you should check it out: https://www.youtube.com/watch?v=eifQI5uD6VQ
Lastly, I left out all of the ssh setup stuff because I made a video about that and the blog post goes into a smidge more detail. If you want to see a video covering the basics of using SSH, I made one a few years ago and it’s still pretty good: https://www.youtube.com/watch?v=3FKsdbjzBcc
Chapters:
1:18 Invoking rsync
4:05 The --delete flag for rsync
5:30 Compression flag: -z
6:02 Using tmux and rsync together
6:30 but Veronica… why not use (insert shiny object here)
If you’re trying to back up Windows OS drives for some reason, robocopy works quite similarly to rsync.
I would generally argue that rsync is not a backup solution. But it is one of the best transfer/archiving solutions.
Yes, it is INCREDIBLY powerful and is often 90% of what people actually want/need. But to be an actual backup solution you still need infrastructure around that. Bare minimum is a crontab. But if you are actually backing something up (not just copying it to a local directory) then you need some logging/retry logic on top of that.
At which point you are building your own borg, as it were. Which, to be clear, is a great thing to do. But… backups are incredibly important and it is very much important to understand what a backup actually needs to be.
I would generally argue that rsync is not a backup solution.
Yeah, if you want to use rsync specifically for backups, you’re probably better-off using something like
rdiff-backup, which makes use of rsync to generate backups and store them efficiently, and drive it from something likebackupninja, which will run the task periodically and notify you if it fails.rsync: one-way synchronizationunison: bidirectional synchronizationgit: synchronization of text files with good interactive merging.rdiff-backup:rsync-based backups. I used to use this and moved torestic, as thebackupninjatarget forrdiff-backuphas kind of fallen into disrepair.That doesn’t mean “don’t use
rsync”. I mean,rsync’s a fine tool. It’s just…not really a backup program on its own.Having a synced copy elsewhere is not an adequate backup and snapshots are pretty important. I recently had RAM go bad and my most recent backups had corrupt data, but having previous snapshots saved the day.
Don’t understand the downvotes. This is the type of lesson people have learned from losing data and no sense in learning it the hard way yourself.
How would you pin down something like this? If it happened to me, I expect I just wouldn’t understand what’s going on.
Why videos? I feel like an old man yelling at clouds every time something that sounds interesting is presented in a fucking video. Videos are so damn awful. They take time, I need audio and I can’t copy&paste. Why have they become the default for things that should’ve been a blog post?
Ad money.
They linked blog post with the video: https://vkc.sh/everyday-rsync/
Ive personally used rsync for backups for about…15 years or so? Its worked out great. An awesome video going over all the basics and what you can do with it.
And I generally enjoy Veronica’s presentation. Knowledgable and simple.
It works fine if all you need is transfer, my issue with it it’s just not efficient. If you want a “time travel” feature, your only option is to duplicate data. Differential backups, compression, and encryption for off-site ones is where other tools shine.
I have it add a backup suffix based on the date. It moves changed and deleted files to another directory adding the date to the filename.
It can also do hard-link copied so that you can have multiple full directory trees to avoid all that duplication.
No file deltas or compression, but it does mean that you can access the backups directly.
Thanks! I was not aware of these options, along with what other poster mentioned about
--link-dest. These do turn rsync into a backup program, which is something the root article should explain!(Both are limited in some aspects to other backup software, but they might still be a simpler but effective solution. And sometimes simple is best!)
Surely restic or borg would be better for backups?
Rsync can send files and not delete stuff, but there’s no versioning or retention settings.
Duplicity also works really well for backups and you can encrypt them with your own openpgp key.
Tangentially, I don’t see people talk about rclone a lot, which is like rsync for cloud storage.
It’s awesome for moving things from one provider to another, for example.
@calliope It’s also great for local or remote backups over ssh, smb, etc.
It has been remarkably useful! I keep trying to tell people about it but apparently I am just their main use case or something.
I would have loved it when I was using Samba to share files on my local network decades ago. It’s like a Swiss Army knife!
Rsnapshot. It uses rsync, but provides snapshot management and multiple backup versioning.
Yes, but a few hours writing my own scripts will save me from several minutes of reading its documentation…
It took me like 10 min to setup rsnapshot (installing, and writing systemd unit /timer files) on my servers.
I’m sure I could script something similar in under 10 (hours).
Man ilove being autostic
Yeah it’s slow
What’s slow about async? If you have a reasonably fast CPU and are merely syncing differences, it’s pretty quick.
It’s single thread, one file at a time.
It’s slow?!?
