General :: Backup Only The Files Which Have Changed?
May 18, 2011
if there an application which does the following:
1. I have a laptop with an internal 200GB HD.
2. I run the application & it creates a list of all files (size & time-stamp) without actually storing them. Let's call this the "snapshot list".
3. I update some of the files on the laptop.
4. Now I run the application & it only copies the files which have changed on the laptop, that have different size/time-stamp from the snapshot list, onto some external media, such as a memory card. Of course, the files should be copied onto their proper location in the directory tree & not just pile up in one place.
Why is this useful? although the laptop has a 200GB HD I typically only update a small number of files, whose total size is maybe 10MB or so. If I could only backup those which have changed, I could do this with a tiny SD card instead of lugging around an external usb HD.
I'm just setting up a partition on a seperate HDD in my system. I plan to use the partition to backup the important files on my main HDD (to guard against HD crash).
The question I have is about where would be the typical location to auto mount this partition? Which would it be normal to go for:
I have a directory (let's say A) with a few hundred files and subdirectories and a copy of this directory (let's say B). All the files and subdirectories in B have a newer date than A. However only a few of them are really changed files. Is there a way to find those files and directories? Basically they have different sizes so this could be used to compare them with the files from A.
I'm going to make a nightly backup copy from one server to another, using rsync. If I have a sufficiently large file, say 4+ GB or so, I'm not interested in copying the whole file if only a small change has been made. Can rsync detect small changes on block level and backup only those if needed?
I use a PHP IDE that has no built-in ability to upload a project to a site. So, I'm looking for common easy to use tool for Linux that would able to upload modified documents to the server instead of uploading of the whole site. I also accept shell scripts that would be able to do this.
I have bought an external usb hard drive on which I back up my three computers every once in a while.Space will quickly be used up.I can't find that little bit of research that I need yesterday.Here is what I would like to find:An application that eliminates doubles in identical files and renames files that have changed by appending the last saved date yyyymmdd to the file name.Does such an application already exist?
I have a situation where a directory has about 1.5 million files in it. On an hourly basis, I want to be able to find any files that have changed in the last hour, compress them, encrypt them and then copy them to both a local backup machine and an off site backup.
Is there any kind of utility or kernel module that creates some type of log of modified files? I know I can use find, but the search for -mtime in this directory takes quite a while and will not suffice for an hourly backup.
the permissions for my home directory were accidentally changed from 'access files' to 'create and delete files', and I changed them back, but ever since then I am not able to change any preferences/settings at all. power management, themes, panels, emerald, anything. my user account is supposed to be the administrator, and all the user privliges are checked. how to get control of my computer back?
I'm using gnome. I created some text files. After I change something in myfile.txt there's myfile.txt~ automatically created as a backup. Ok... now, on KDE I see these ~ files and I can delete them if I want but they're hidden in gnome... Gnome seems to treat them as special hidden files (just like .mydir directories) How do I make ~ files visible in gnome so I could edit or delete them?
I am using MySQL as the database system for my application on a Linux system. Every week I update the system and take backups (mysqldump) of the databases changed (2 databases). I then .tar.gz them and ftp the resulting file to a remote server, after which I remove the original backups and tar.gz files from the Linux server. Being a complete novice when it comes to Unix systems, I would like to know if it is possible to write a script which would do all this automatically, i.e. perform the following steps.
1) Backup database A to A.sql (mysqldump) 2) Backup database B to B.sql (mysqldump) 3) tar -cvzf dest.tar.gz A.sql B.sql 4) ftp dest.tar.gz to ftp@remoteserver.com 5) Delete A.sql, B.sql, dest.tar from local server
I then installed a new version of Ubuntu 10.04 from disk and copied the files in /home from the cd to the hard. I am able to open, view etc. all the files in most directories except those in /home/documents. There are text files created by gedit, OOWP and several PDF files. I cannot open or view these files, depending: gedit and pdf files gets a Err.Msg. "Don't recognize file type" (it is clearly marked PDF) . The OO files look like rows of 'high bits' and a dialogue box opens giving me the options to change Char. Set, Font, Language, Paragraph break.
I have around 100 users. I want to take backup of files which are on desktop for every user. My user directory path is -: /home/dr/<user_name>/Desktop
1) Script has to run on a particular time everyday 2) Script has to take backup of all files present in "Desktop" directory 3) Make a tar with name "yyyy-mm-dd-desk-files" 4) Make directory outside "Desktop" with name "Desktop-Backup", if already exist then don't make this folder. 5) The tar have to moved in this folder. 6) Remove the files from "Desktop" directory. (i.e. Desktop should be empty) 7) Mail the status that "Backup Successful"
I keep a backup of a bunch of files on a flash drive, so that whenever I change distributions I can just restore all my Android stuff (saves on re-downloading everything). One of these is the Android SDK.
In my ~/.bashrc I add the paths to some executables in the SDK, only if the directory exists, and only if the path is not already in $PATH. For the Android NDK this works fine, but for the SDK I get this:
Code: snfo@snfo:~$ adb devices bash: /home/snfo/Android/sdk/platform-tools/adb: No such file or directory snfo@snfo:~$ ls -F /home/snfo/Android/sdk/platform-tools/adb /home/snfo/Android/sdk/platform-tools/adb*
Everything else is fine though, just that one path is causing trouble.
Now, I've saw something similar to this before whenever you move an executable from one place to another. If you don't re-source your bash config it will continue to keep looking wherever it used to be located. But I've never moved these files.
I am backing up parts of my computer with DD, and i was wondering if there was a quick way to split the files created into 4.4GB sized files that will fit onto a DVD. Anyone have any idea of how to do this?
Today I tried to compress some folders containing backup files from last year. I right-clicked on the folders and selected compress as tar.gz. I let it work, and found that hours later, the folders were still compressing. How long is it supposed to take, anyway? I was trying to compress the two sets of backups simultaneously; together they're around 1.5 GB. They have many subdirectories.
I'm trying to create backup/archive my Ubuntu 10.04 system files (so I can restore it in case my system get corrupted). More specifically, I'm trying to zip the important files in my root directory not including my home directory (which includes my documents which I backup separately/more frequently) to an external hard drive attached via USB (called 'My Book').
Since File Roller didn't give me quite the level of control I was looking for, I created a script that I could execute to backup and archive regularly. Here's a snippet: cd /media/"My Book"/"Linux Backups" NOW=$(date +"%b-%d-%y") LOGFILE=Backup_Root_FileSystem-$NOW.log sudo zip -r -T -v Backup_Root_FileSystem-$NOW / -x /media/'My Book'* /media* /proc* /sys* /mnt* /dev* /cdrom* /home* /'lost+found'* | tee -a $LOGFILE
rdiff-backup keeps recreating a symlink: /backup/ext -> /mnt/vg1/vol1, which is kind of missing the point, because then it doesn't actually back up the files in /mnt/vg1/vol1/installs, etc.
I have two NASes. I work off of one, and the other is used as a backup. As I have it set up now, it's slow. Running a backup takes a week. Even for 7 TB, with 1,979,407 files, this seems a bit outlandish,particularly as both systems are RAID-5 and the network is all gigabit. I've been digging about in the rsync man pages, and I really don't understand what differentiates the various topologies.Right now, all the processing is being done on the backup NAS, which has the main volume from the main NAS mounted locally over SMB. I suspect that the SMB overhead is killing me, particularly when dealing with lots of files.
I think what I need is to set up rsync on the main nas as a daemon, and then run a local rsync client to connect to it, which would hopefully allow me to completely avoid the whole SMB-in-the-middle affair, but aside from mentioning that it's there, I can find very little information on why one would want to use the daemon mode for rsync.
Here's my current rsync command line: rsync -r -progress --delete /cifs/Thecus/ /mnt/Storage/input? Is there a better way/tool to do this? Edit:Ok, to address the additional questions: The "Main" NAS is a Thecus N7700. I have additional modules installed that give me SSH, and it has rsync, but it's not in the $PATH, and I havn't figured out how to edit the local $PATH in a way that persists between reboots. The "Backup" NAS is a DIY affair, built around a 1.6Ghz Via Mobo with a Adaptec Hardware RAID card. It's running CentOS 5 with a full desktop environment. It's the hardware I'm running rsync from. (Gigabit is through a additional PCI card).
Further Edit: Ok, got rsync over SSH working (thanks, lajuette!).I had to do a bit of tweaking on my command line, I'm running rsync with the args:rsync -rum --inplace --progress --delete --rsync-path=/opt/bin/rsync sys@10.1.1.10:/raid/data/Storage /mnt/Storage (Note: I'm specifically not using -a, because I want to change the ownership to the local account, to not freak-out SELinux)
I have installed luckybackup software on my ubuntu 10.10 notebook edition. But I don't know how to use it to backup files to an external hard drive. The Hard Drive is a 1 TB Seagate. I don't think that the Destination Drop down menu in luckybackup even shows the External HD.
During my backups I'm finding that rsync is copying all files, instead of just what's changed.
I'm rsyncing between 2 USB external hard drives. One hard drive is FAT32 and one is NTFS. I've examined some of the files and believe that the difference is that there's a 1-second modtime difference developing in some of the files somehow.
Here's an example. These duplicity files were synced from /media/BACKUPHD (the NTFS drive) to /media/VIDEOHD (the FAT32 drive) only a few hours ago this morning. They have not been touched or changed since then, but that 1-second difference in their time stamps has appeared:
Code: tim@localhost:~> stat /media/BACKUPHD/backups/duplicity/duplicity-full.20110107T145955Z.vol10.difftar.gpg File: `/media/BACKUPHD/backups/duplicity/duplicity-full.20110107T145955Z.vol10.difftar.gpg'
I am trying to create a backup script that will back up a single folder for a class i am in. I was wandering if I could get some help. If possible I would also like to know how to write a script that can encrypt that same file . I will be putting the back up in my /home/usr/Backup directory. I am not trying to back up my whole system just a single folder. I am using Fedora 11
I have deleted the mime directory under /usr/share/icons/Humanity, and many files' icons have changed to the gnome style. Then I restored the mime directory back, and typed command sudo gtk-update-icon-cache /usr/share/icons/Humanity. But my action doesn't work. The files' icons remains the gnome style.
I run BIND9 in Debian on my server, and did a dist-upgrade last night. Prior to the dist-upgrade, everything worked fine. It does not appear that any of the configuration or zone files were changed. Right now, I am seeing ServFail replies to the A queries:
I'm using rsync to create a mirror of the data files on our main server every day. I've looked at the man page, and can't see it; can I get a listing of the files that have been changed on or added to the mirror when it's completed? Can it just log what it's doing to a file?
how rsync will handle disk images. Will rsync copy only the changed blocks of a vhd or a lun? This is what I've been told, but wouldn't this require overlaying a filesystem on the vhd? How would rsync handle copying a 500GB lun?
I changed a few files in a couple of folders that are installed from a synaptic package. How can I save the changes as an installation package or save the changes to the package and save the package as a file?
For the second time in a week, I have set up an unmanaged CentOS 5.5 Storm Server at StormOnDemand, only to discover a ton of unauthorized changes to binaries (updated file checksums and sizes) on the server shortly thereafter.The time stamps do NOT change.If the time stamps did change, I would be hunting down ahat was doing some auto-updates. But the time stamps are not changing.This leads me to believe that either these servers are suffering from:1. A virus or hacker is compromising the box.2. system corruption.3. Something else? To eliminate the possibility of number 1, I toasted the first server and started over with a new server and enabled their firewall from the start to only allow access for two IPs via SSH... my IP and my biz partner's.
Then, one of the first things we installed was a system we created that maintains a snapshot of most directories on the system so that it can be used to watch the live directories for changes. At 4:07am (server time) this morning, we received notice from this system that a massive number of files had changed in these directories. Again, no file time stamps changed.So, my question is this... is there any legitimate reason in a fairly standard CentOS 5.5 install that would cause so many files to change?
