Ubuntu Servers :: Rsync Backup - Transfer Files With Appropriate Permissions
Dec 4, 2010
I wrote a script to wake up my windows machine and do an rsync backup of some of my files. I wanted to make this command a accessible through local bin so I made it executable. However the problem is that when I copies files is copies them with root permissions and i can edit or delete them. How can I set the files so they transfer with the proper permissions for my Ubuntu user?
# Description: This script first wakes up the client machine and syncs the appropriate folders.
# Finally the script shuts down the client if it was off to begin with.
if [ "$(whoami)" != "root" ]; then
echo "Permission Denied"
I'm trying to learn how rsync works to backup my system. I tried: Code: rsync -azvv /home /media/Elements I get a folder called home on my external hard drive but when I use ls -l to see the permissions they are all wrong. On my /home folder the permissions for /nathan are drwxr-xr-x 48 nathan nathan The permissions on the backup /nathan folder are drwx------ 1 nathan nathan
I also tried using the long version of -a which is -rlptgoD and that didn't work either. What do the 48 and 1 mean when I used ls -l? When I look in the /nathan folder the permissions are all screwed up too. A lot of the files are backed up as executable and the permissions are all screwed up. I also ran it with sudo, and that didn't work either. The permissions were still screwed up and ownership is messed up too.
If transfer all files under a directory by rsync, what is the order that rsync determines to transfer the files one by one?At first it looked like rsync transfers files in alphabetical order, but later I found rsync skipped some files in the first sweep through the alphabetic order, and then went back to transfer files that were skipped in the first time and this time still in alphabetic order.
I have a Linux host acting as an ISCSI server for a Windows box. I want to keep an off site backup, so I figure rsync will keep the ISCSI server synced with an offsite Linux host. I understand that Rsync does block level incremental transfer to conserve bandwidth ok, awesome.The trick is, that I also want an archival copy kept. Say I want to go back to a revision of a file from 10 days ago, I need to be able to do that.
I was planning on using Backup Exec, since we currently have a licensed copy. Throw the archives from Backup Exec onto the ISCSI server as well, and have it keep a rotating 30 day backup, or something like that. The issue I see here is that this will be creating a deleting files as it does its daily backup rotation. I'm guessing RSYNC will see these as new files, and likely retransmit everything on a daily basis. The question then becomes, is this assumption correct, or will it still know to do a block level incremental transfer even when file names and such are changing?
I support a small business which has an Ubuntu server running as a file server. The server is running Ubuntu 10.4. There is one hard drive which is mounted as /media/hdd. Each night this is backed up to an external USB hard drive mounted as /media/backup. The backup is carried out using the command:
Code: rsync -av /media/hdd/ /media/backup/
Is there a way to encrypt this back-up so that if the USB hard drive is plugged into another machine it cannot be read?
I have a big iso image which is currently being downloaded by a torrent client with space-reservation turned on: that means, file size is not changing while some chunks in in (4 Mib) are constantly changing because of a download.
At 90% download I do the initial rsync to save time later:
$ rsync -Ph DVD.iso /media/another-hdd/ sending incremental file list DVD.iso
Then, when the file's fully downloaded, I rsync again:
total size is 2.60G speedup is 1.00
Speedup=1 says delta-transfer was not used, although 90% of the file has not changed, target dir is on another FS and copying takes several minutes. Why doen't it try to speedup the transfer?! How can I force rsync to use delta-transfer?
I need to create a script that will compare the differences between two folders and then to copy only the updated and new files only to another directory. I know I need to use rsync here, I can write scripts so really it not how to create a script it is how do I accomplish the transfer of only new or changes files between two folders to a new file. Do I need to link these two folders first and then use the "--compare-dest" switch.
well, i know ther are issues when using rsync to copy files to ntfs partition like file permission blah blah. the thing is, i need to backup my music files periodically onto a ntfs partition from ext4. i really dont care about file permissions or any other stuff. when i use rsync, it should update the mp3 files on my ntfs (external) disc with the new ones.can i give a go with this operation? i have lot more important files on the external disc and i dont want this rsync corrupt or delete those files coz they are highly important files.
I'm going to make a nightly backup copy from one server to another, using rsync. If I have a sufficiently large file, say 4+ GB or so, I'm not interested in copying the whole file if only a small change has been made. Can rsync detect small changes on block level and backup only those if needed?
I've been using Ubuntu for about 2 years now, but still have trouble with some of the finer workings of linux. I have a laptop that I use for general computing, and a desktop hooked up to a TV as sort of a remote backup/htpc. A problem I run into is when I transfer files, they get transfered with the owner set as the original computer's account, and I can't do anything until I open a remote viewer and gksudo nautilus to change the permissions of the file. I looked at articles about permissions and uid's, gid's, and umask but can't figure out how to apply it to my situation.
I thought about doing something with groups but am not sure exactly what, and anyway, default group settings only give read access and what I'm really looking for is the ability to manipulate files and folders across the entire /home dir on my desktop from my laptop. Desktop is running 8.04 and laptop is running 9.10. BTW I am currently sharing through smbfs. I read that this has been replaced by cifs, but at the moment I would prefer not the mess with things if I don't need to.
I'm currently learning to use rsync to backup my music collection. I have a Firefox tab open to the rsync manual page(s) and have been reading man rsync and running experimental rsync operations.I've been doing this for the last 3-4 hours. I've used rsync for this purpose in the past with disastrous results. What was and is once again (due to a month and a half of file pruning) a 9000 file music collection had mysteriously grown to over 25,000 music files and 80GB of data! This was likely due to the fact that I didn't really know what I was doing with rsync and had never spent too much time learning about all the parameters, what their functions are and how they may relate to my goal.Here are the particulars:
* Source drive is a 500GB disk, /media/sata500/music/.
* Destination drive is a 250GB USB disk, /media/FreeAgent/music, connected to the same computer that houses the 500GB disk.
* I want to copy or backup files from /media/sata500/music to /media/FreeAgent/music.
* I do not want to create ANY duplicates of files that exist.
* I only want to add files to the destination drive if they are new on the source drive, like if I rip a CD and add the contents to the source. I want them copied over next time I run rsync.
Here's the rsync command in it's most recently used form, and probably very immature at this point.
This appears to have copied all files and folders and I'm satisfied that my goal has been met with some success. To convince myself of this I ran the command and then once it was complete I added 2 new songs putting them in their respective folders on the source drive and ran the same command again. The resulting output was
Two files transferred. Exactly what I want.Both folders now house 20,931 files and use 40.6GB. Identical as far as I can tell.What I'm concerned about are time stamps and play count data, etc. Anything that changes the original file. I don't want this data to cause a file to be transferred as I'm afraid that the new file will be created along side the old file of the same name thereby starting this whole music collection expansion thing all over again. I've invested a lot of time and effort to get it pruned down to where there are virtually no duplicates and albums are correct in that they contain the proper songs in the proper order.
I would like to backup important files (totaling about 400GB) on my ext 4 RAID 5 array to an ext4 external hard drive over USB (external drive is mounted to /mnt. In the future I'd like to automate the process using rsync and cron so for now I'm using rsync to transfer the files. My problem is that using the rsync command like this: # rsync -Pr "/dir1" "/dir2" "/dir3" "/dir4" /mnt
rsync shows me the checks and transfers for awhile and then throws up an i/o error (wish I had a screenshot to show but I don't). When I ls /mnt I get a similar i/o error. I then check /dev for the drive and find that it no longer shows up. Originally the partition was /dev/sdc1. I tried unplugging the USB at this point, plugging it back in and mounting the drive back to /mnt, however it has now assigned it to (you guessed it) /dev/sdd1. I get the drive mounted and try the original rsync command again, hoping the first error was a fluke or some kind of one-time drive fart. This time it makes it quite a bit further and then throws up the exact same problem. Am I doing something terribly wrong here? As I said, I'm very new to bash so I'm not making some absolutely moronic, newbie mistake.
Problem: permissions for rsync and BackinTime. Setup: Ubuntu 11.04, Two internal HD, #1=main, single boot, #2=backup drive. Question: How do I set up my 2nd HD with correct permissions? Background: I had previously a dual boot XP+10.04 with a 2nd HD formatted as NTFS. With this I was able to use my rsync and backintime to my 2nd HD with no issue. My new set up is EXT4 on both HD.
(I even tried to reformat my 2nd HD as NTFS, but that didnt fix the issue) I followed [URL] to mount the 2nd HD and get permissions. But now when I run backintime i get this error: [E] Error: rsync: opendir "/home/myhome/.ssh" failed: Permission denied (13) I did my requisite reading for a newbie, and am stuck. I ran backintime as root, and it backed up ok. How do I run my user version of backintime? (i.e. How do I fix the permission issue?)
i am trying to transfer a file from my live linux machine to remote linux machine it is a mail server and single .tar.gz file include all data. but during transfer it stop working. how can i work and trouble shooot the matter. is there any better way then this to transfer huge 14 gb file over network,vpn,wan transfer. the speed is 1mbps,rest of the file it copy it.
[root@sa1 logs_os_backup]# less remote.log Wed Mar 10 09:12:01 AST 2010 building file list ... done bkup_1.tar.gz deflate on token returned 0 (87164 bytes left) rsync error: error in rsync protocol data stream (code 12) at token.c(274) building file list ... done code....
I've 15 web servers (in private network) running RHEL, Apache. Needs to sync web files between them. each server is accessible to each other via public key (with passphares).
1) Main server is web1 (where dev upload files initially). So I can make all other servers accessible by web1 without password/passphares and run rsync periodically to update files between them. But security is an issue here as all servers will become easily accessible.
2) Run rsync daemon in all other servers (except web1) on designated port and run rsync command from web1 to sync files. This will do the work but running daemon in all servers might increase overhead and making sure that daemon is running all the time etc. are my concern for this implementation.
I have recently rented a server which is running Ubuntu linux. I can connect to it through ssh. I've written a Java application that I would like to run on the server 24/7. How do I transfer the files from my computer to the server so I can run the program from the server?
Thought I'd post it here because it's more server related than desktop... I have a script that does:
This is used to sync my local development snapshot with the live web server. There has to be a more compact way of doing this? Can I combine some of the rsyncs? Can I make the rsync set or keep the user and group affiliations? Can I exclude .* yet include .htaccess?
I got two ubutnu servers running side by side, I need to transfer several dir's of files, incl sub dirs to the other server via ftp and retaining the file permissions and dir structure. Both servers have ftp access, I am assuming theres an ftp command I use on the destination server to connect to the source server and retrieve the files? All the files reside in the one dir and need to be copied to the same dir on the destination server.
I'm going to be launching my website soon, and I found a company to host it on one of their dedicated servers. I think I'm going to go with fedora as to OS, and well my problem is I'm having trouble finding a company to backup up my files, that both supports fedora and well is reasonably priced.
So i pulled some files off my buddy's computer via my wireless home network, i can access them but in the permission tab the owner is "nobody" so i can move the files. How can i change the permissions to enable me to move the files?
Currently have access to a VPS where we are running a small game server on ubuntu - the problem is that it is a multi-user environment, so when one person restarts the server process, all files it creates are owned by that users name and group. I have created a group called 'game' and added both users to it, but I need to know how to make all files in the game server's directory to be r/w/x for the group 'game'. Currently, I have a script that chowns and chmods all files recursively on startup, but I'd prefer not having to do this.
I would like to change the permissions for a directory and all files inside the directory how do I do this? The website is located only on my local network so I am not worried about security. Also what would be the optimal permissions for running wordpress.
I am researching how to make an effective backup on Ubuntu Server. This server will have Vsftp, VPN, Samba stuff , many other added packages +many printers, many users + data. I know I can use tar for the data /u no problem. 1. I was testing tar on the /home directory on a few user directories. I then created a new directory and did a restore of the users directories on it. I noticed the /home/user owner and group were root. The files in each directory remained the same. This gave me concern. If I had a crash and had to restore these to a new HD. I would have to change these, what else would I need to change? 2. Since I have many config files, how do I back up them? I know I can do a dump, but then users shouldn't be on the system. The system files will change as they add users, printers, etc, and asking users to not work, is not really an option while dump is running. I thought I could do a tar on whole system. (cron late at night .. not as many users) Then in event of crash of HD.
1. Boot from live cd 2. format the new drive 3. tar back in the whole system
Will this work right? Is there something I am missing?