I am performing a dry run using Rsync on 2 different boxes.While i'm doing that, Under destination directory, I want a specific directory x to be ignored for sync.Please let me know the exact pattern to ignore the directory.The current command I'm using is:rsync -avnc --delete $LOCAL_DIR $USERNAME@$DESTINATION_IP:$REMOTE_DIRunder DESTINATION_IP, I would want to ignore a particular directory under REMOTE_DIR.
View 1 RepliesIs there a way to force rsync to not make directories in its destination directory; ie, to simply dump all of the files from the source directory directly into the destination without copying any of the folders that the files were originally in? I tried --no-dirs, but that seems to only be for empty directories.
View 1 Replies View RelatedWe have an rsync cron job set up to mirror all the files in a "..dashtdocsdocs" folder to the same folder on another server. It copies all the files over correctly and deletes any files in the "docs" directory that aren't in the sending directory, but it also deletes any files we put in the target directory's parent folder (..dashtdocs or other subfolders like ..dashtdocsimages) even though they've been excluded in the .rsync-filter file.
Here's our current rsync .sh:
/usr/local/bin/rsync --stats -qPzrtpl --delete --password-file=/var/run/.appprodrsync --log-file=/export/home
/webuser/logs/rsync-extranet-log -FF /export/home/ appprodrsync@appprod::dprweb_extranet/ > /export/home/webuser/logs/rsync-extranet-output 2>&1
[code]....
So for example server A has ..dashtdocsdocs and ..dashtdocsimages. Server B has ..dashtdocsdocs but if I manually copy the images folder over to ..dashtdocsimages, the images folder gets deleted from the target directory every time rsync runs.
A: ..dashtdocsdocs
..dashtdocsimages
B: ..dashtdocsdocs
..dashtdocsimages (<-- gets deleted everytime rsync runs)
I'd like to keep just the docs directory synched and update other folders manually, but they keep getting deleted. It looks to me like it's running a delete-excluded option, but that option wasn't used.
I am using Rsync to backup files to a another machine, the users on my fileserver do not exist on the backup server so Rsync throws errors about the permissions. It copies the files fine but I want to get rid of the errors and have Rsync ignore the permissions when backing up.
/backup is a mounted ftp directory
Below is the current command and output:
Code:
root@Fileserver:~# rsync -av --delete /shared/fileshare/ /backup/backup
building file list ... done
created directory /backup/backup
./
manager/ .....
rsync: chown "/backup/backup/manager/.chironfs.txt.c6MbJ7" failed: Operation not permitted (1)
rsync: chown "/backup/backup/manager/.cronman.txt.hdBG4P" failed: Operation not permitted (1) .....
sent 211115 bytes received 274 bytes 7686.87 bytes/sec
total size is 210263 speedup is 0.99
rsync error: some files could not be transferred (code 23) at main.c(977) [sender=2.6.9]
I'm trying to set up rsync to only copy new songs from my computer to another. I'm using the "--ignore-existing" argument, but it appears to copy all files anyway. The client (source) is Windows 7 64-bit running DeltaCopy Client and the server (destination) is Synology DS410 (running rsyncd).
Here is the syntax:
I have recently purchased an external hard drive in order to backup my home partition. In my PC I have a "1.5T" drive with several partitions on it, containing OSes and the home partition. The home partition is 1.3T according to df, the external drive contains one partition that spans the entire disk,df reports it as 1.4T in size. Both partitions are ext3. When I use rsync to copy files from the home partition to the external partition, the external disk becomes full, despite the destination - supposedly - being larger than the source. I don't understand why copying files from one partition to a slightly bigger partition should need more space than on the source partition. Does anyone know what is happening ?
Details : I created the partition on the external drive with gparted; gparted reported it the already have several gigabytes in used space immediately after the partitions creation - I thought at the time that this must be normal. The home partition contains many files of all sorts, including lots of big audio and video files. If you are wondering, for all my important files this external disk is only secondary backup, as they are also backed up to the "internet".
These are the mount points :
/mnt/tmp/ : home partition, /dev/sdb6
/mnt/external/ : external partition, /dev/sdc1
I used rsync to copy the files, I know there are more efficient ways to do this, but I wanted to use the same command that I will subsequently run to sync the backup.
rsync -av --progress --stats --recursive --perms --links --delete /mnt/tmp/ /mnt/external/
Next I tried adding the --sparse switch, as I was wondering if the problem may come form sparse files. I don't know however if rsync would go back and shrink the sparse file by just adding the switch and executing the command. I also added --one-file-system, for good measure. Here is what I ran next :
rsync -av --progress --stats --sparse --one-file-system --recursive --perms --links --delete /mnt/tmp/ /mnt/external/
I tried an fsck on the home partition :
fsck -f /dev/sdb6
This is the output from the last rsync :
rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32)
rsync: write failed on "abcd.avi": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(302) [receiver=3.0.6]
[code]....
Looking at the destination after a partial copy seems to indicate that the problem is not symbolic links being "expanded". I have not checked the source filesystem for sparse files, nor the destination to see if these files could be larger there, as this does not seem trivial.
Here is some additional info :
$ df /mnt/tmp/
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdb6 1415342836 1414173740 369096 100% /mnt/tmp
[code]....
I have a script that users execute to copy files from one server to another 3 boxes.
I want to prevent rsync from trying to copy files with spaces.
Anyone have an example I can use?
The rsync just craps out when they try to sync files with spaces, I tell them that but....
how do I gunzip to a destination directory other than the current one?
this did not work:
gunzip *.gz /putthemhere/
I'm trying to install Slackware package into some specific locations, like for example, I want to put Linux base package into at / and put applications on /usr/local. However when I'm installing using "setup" program, I cannot find a part that let me to choose the installation destination.
At "setup install" option, it gives six different installation method like full, newbie, menu, expert, custom, and tag path. But none of them (I cannot find it) gives an option where to put the installation package to.
I'm using rsync for a backup-sript at the moment and want to keep all files. The files are always unique, so I want to rsync without delete any file on the destination.
I've tried with --no-delete and --max-delete=0 but nothing seens to work. Is there even a possibility to do so?
I would like to create a directory public_html/archive that is a mirror of public_html/images, except that deletions in the images directory don't cause corresponding deletions in archive.
I was going to manage all of this in code, but then I figured, maybe the system can handle it for me in a robust way.
Note that the website will reference archive in real time, so it has to be an instant mirror and not something done nightly, etc.
I have the following content on the source directory:
source/foo.txt
source/bar/another_file.txt
source/bar2/and_another.txt
I want copy those files to a destination directory which, after copy, shall look like this:
destination/foo.txt
destination/another_file.txt
destination/and_another.txt
How can I do this? It seems that "cp" lacks such an option
On Debian Standard System I would like to is change /tmp directory from it's original /dev/hda8 to destination /dev/hdd5
Code:
Disk /dev/hda: 6448 MB, 6448619520 bytes
255 heads, 63 sectors/track, 784 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
[code]...
If I change /dev/hda8 to /dev/hdd5 and reboot the startx command fails to run.
I'm currently learning to use rsync to backup my music collection. I have a Firefox tab open to the rsync manual page(s) and have been reading man rsync and running experimental rsync operations.I've been doing this for the last 3-4 hours. I've used rsync for this purpose in the past with disastrous results. What was and is once again (due to a month and a half of file pruning) a 9000 file music collection had mysteriously grown to over 25,000 music files and 80GB of data! This was likely due to the fact that I didn't really know what I was doing with rsync and had never spent too much time learning about all the parameters, what their functions are and how they may relate to my goal.Here are the particulars:
* Source drive is a 500GB disk, /media/sata500/music/.
* Destination drive is a 250GB USB disk, /media/FreeAgent/music, connected to the same computer that houses the 500GB disk.
* I want to copy or backup files from /media/sata500/music to /media/FreeAgent/music.
* I do not want to create ANY duplicates of files that exist.
* I only want to add files to the destination drive if they are new on the source drive, like if I rip a CD and add the contents to the source. I want them copied over next time I run rsync.
Here's the rsync command in it's most recently used form, and probably very immature at this point.
Code:
rsync -t -r -vv --stats -i --log-file=/home/glenn/rsync.log /media/sata500/music/* /media/FreeAgent/music/
This appears to have copied all files and folders and I'm satisfied that my goal has been met with some success. To convince myself of this I ran the command and then once it was complete I added 2 new songs putting them in their respective folders on the source drive and ran the same command again. The resulting output was
[code]....
Two files transferred. Exactly what I want.Both folders now house 20,931 files and use 40.6GB. Identical as far as I can tell.What I'm concerned about are time stamps and play count data, etc. Anything that changes the original file. I don't want this data to cause a file to be transferred as I'm afraid that the new file will be created along side the old file of the same name thereby starting this whole music collection expansion thing all over again. I've invested a lot of time and effort to get it pruned down to where there are virtually no duplicates and albums are correct in that they contain the proper songs in the proper order.
What would be the easiest way to go about rsyncing the n newest files in a directory to a remote server?
View 2 Replies View RelatedIf transfer all files under a directory by rsync, what is the order that rsync determines to transfer the files one by one?At first it looked like rsync transfers files in alphabetical order, but later I found rsync skipped some files in the first sweep through the alphabetic order, and then went back to transfer files that were skipped in the first time and this time still in alphabetic order.
View 1 Replies View RelatedI have a WD world book edition 1TB NAS drive, and just purchased an acomdata 1tb drive and connected it to the NAS via USB. If I recall I think the WD NAS has a ext_ or some type of linux filesystem on it, and the acomdata has a ntfs filesystem on it.
What I want to do is copy over certain directory trees of the NAS to the USB attached drive. I usually use MS synctoy to sync folders from my windows pc to the NAS drive, and MS richcopy to make the initial transfer from PC to NAS. For this operation though, since it is taking place entirely on the NAS and its connected drive, I thought that rsync would be the best option, and it is available on my NAS drive.
Last night I entered in rsync -avr /movies/* /usb1-1share1/ to copy the entire "movies" dir to the drive, which shows up as usb1-1share1 on the NAS drive. It copied most of the directory tree ok, but a lot of the folders were empty, so this morning I tried rsync -Carv --ignore-existing /movies/* /usb1-1share1/ to try and get all the files missed, without recopying the 24GB that did make it across. This also managed to copy a few more GB over, but not everything.
I am running the command from an ssh session on the NAS using putty on my PC, in as user "admin" which should have all rights over these folders. There is a bunch of errors in the command window like this: rsync: failed to set times on "/shares/usb1-1share1/movies/classics/fulldvd/First Blood DVD/.VTS_01_2.VOB.RxdjWZ": Operation not permitted (1)
I want to restart another session and get the files it missed, but I want to find out what I am doing wrong first. Should I be doing this as root user? am I missing some switches or just plain doing it all wrong?
Recently i am trying to check on the rsync speed for single file(2.4GB iso) directory ( 900MB directory with files inside ) When i run the rsync for single file: the speed i get is average 50MBps However, for a directory: average speed is 10MBps Is there any reason behind this ? i tried to google but unable to get the concept.
View 2 Replies View RelatedWhy would rsync insert a user's home directory path in variable expansion when run via cron, but not when run manually. The gory details... Red Hat Enterprise Linux AS release 4 (Nahant Update 6) Linux 2.6.9-67.0.20.ELsmp The script (parts anyway, and simplified)...
Quote:
#!/bin/bash
. /home/bea/.bash_profile
echo rsyncloc=${rsyncloc} >> ${log}
[code]....
i would like to find and backup all *.mp4 files from /Pictures and its sub-directories and move them to a single directory on a remote. I can find and move the files but I don't want the directory structure...just the files to be placed in the remote directory.
To find my files I use
rsync -r -a -v -e "ssh -l user" --delete --include '*/' --include '*.mp4' --exclude '*' /home/drew/Pictures/ remoteserver:/Users/drew/mp4
but this creates all the subdirectories
I also tried
find ~/Pictures -name "*.mp4" -exec rsync -r -a -v -e "ssh -l user" --delete {} remote:/Users/drew/mp4 ;
This works but takes forever
I'm trying to setup rsync to backup a remote directory to my local drive.
I cd to the directory that I want to pull the files to, then I enter:
rsync -vrtW account@remote.com:~/public_html
I enter the password then it starts running. I get all the files listed, but none of them actually transfer. What am I missing?
I have a file that has been created by exiftools and then parsed and cut to fit my needs and thrown into a file that looks like this:
Code:
I then make the file executable and then execute it. Trouble is, the destination directories don't actually exist, so naturally, the "ln -s" fails.
Is there a way to create the destination directories dynamically based on what is specified in the file above?
[URL]
This is step 2.2:
Quote:
Open a terminal and change the directory to the download destination.
I'm not entirely sure what that means! I've tried doing the other steps without doing that one and it doesn't seem to work.
I cannot change directory to a more than three folder tree destination folder from ~ in terminal. I've checked everything. No Typos or misspell. The destination folder was recognized by "ls" command but when I went to it, the terminal said, "no such file or directory."
View 7 Replies View RelatedI am new to deb package. I have read some docs, but did not know how to specify the destination for to-be-installed files. For example, I want to install under user's home directory. How to specify that?
View 7 Replies View RelatedI've been using rsync in a bash script to make hourly copies of jpeg files that are created every few minutes. The images are contained in a number of subdirectories, with the filenames using the date and time
Quote:
data/images/GARI/GARI_20100825_135403.jpg
data/images/GARI/GARI_20100825_135903.jpg
data/images/FRED/FRED_20100825_135407.jpg
At the moment, my source and target directories are identical, and rsync is easy:
Quote:
rsync -av data/images/ /mnt/data/images
Note that the source directory is purged at regular intervals to stop it filling up. So the target directory has all the images, but the source doesn't.
I need to change my script so that the target directory has a different structure from the source directory, like this:
Quote:
/mnt/data/images/GARI/2010/2010-08/GARI_20100825_135403.jpg
/mnt/data/images/GARI/2010/2010-08/GARI_20100825_135903.jpg
/mnt/data/images/FRED/2010/2010-08/FRED_20100825_135407.jpg
Is there any way I can do this with rsync in a bash script?
I want to use rsync to backup a directory on my local hard drive to a USB hard drive.
I created the initial backup with:
rsync -av /share/chris/docs/* /usb/docs
I went ahead and created this directory
mkdir /tmp/rsync-backup
and, ran this
rsync av /home /tmp/rsync-backup
this is the result
rsync: link_stat "/av" failed: No such file or directory (2)
skipping directory home
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1060) [sender=3.0.7]
I have several copies of a file set with different organizational structures, but the same files. i.e.
On client1 files can be found in ~foobarfile1, ~foobarfile2, ~foo-avernfile3, ~foo avernfile4
On client2 files can be found in ~foo-barfile1, ~foo-barfile2, ~foo-tavernfile5, ~foo avernfile6
On client3 files can be found in ~file1, ~file3, ~file5, ~file7
I have access to one client and the server where I'd like all the files to be synced. I'm not worried about conflicts, just having a complete copy of all files[1-7]. Is there a way to cause RSYNC to remove the directory structure, so that I get something like:
client1% rsync files server:backup
client2% rsync files server:backup
etc
where at the destination all files will be checked against the destination set regardless of the source directory structure?
How can I use rsync to copy ONLY the my home folder (and nothing inside of it, just the folder name) to another machine. I've tried things like
Code:
rsync -av /path/to/src /path/to/dest/
or
Code:
rsync -av -f"+ */" -f"- *" /path/to/src /path/to/dest/
This last option recursively (through the -a switch) copies only folders, including all subfolders. If I try
Code:
rsync -v -f"+ */" -f"- *" /path/to/src /path/to/dest/
nothing is copied (not even my home folder.