I was wondering if there is a way to tell rsync to only apply changes (delete, overwrite,create) only if all files in the file list transferred successfully.Just to clarify, this would essentially be putting a transaction around the transfer.
View 3 RepliesI have rsync error message for this command,
Quote:
rsync -avr --delete --exclude lampp/tmp/* /opt/* /backup/opt/
rsync -avr --delete --exclude data/session/* /var/www/html/dmcatholic/* /backup/dmcatholic/
The error message is here.
[code]....
sent 1917154 bytes received 384128 bytes 5731.71 bytes/sec total size is 12220966785 speedup is 5310.50 rsync error: some files could not be transferred (code 23) at main.c(892) [sender=2.6.8]
Say someone logs into a Centos Linux box and SCPs a file from /root/ called 'textfile.dat'
Is there any way to log into the Centos Linux box, later, and see that textfile.dat was SCP'd from the box?
I transfer avi files on my usb drive to watch on my xbox. I have Ubuntu 9.10. When I watch the movie their are parts that skip or look messed up and sometimes freeze. When I transfer the avi files to my usb drive I do have a lot of other programs running so that could create problems when transferring the files. But it shouldn't right?
View 1 Replies View RelatedI'm making my own yum repository - firstly so that all the machines I administer can be updatedvia the internal network, secondly so that I can test any updates on a spare machine before passing them on, and thirdly so that I can add my own repo for internal software.
I've created the necessary folders under my webserver, and used rsync to update them from my local CentOS mirror, following the instructions at [URL]
I notice it says to run "createrepo" on the base repository, created by copying the rpms from release DVD.
When I rsynced the updates repo, but I notice that the files in repodata are very small. In fact, having a look inside them, filelists.xml contains no file details. But if I run createrepo in the updates directory, filelists.xml gets lots of file details inside it.
I wondered if maybe the local mirror hadn't been updated properly, but checking against mirror.centos.org shows that has the same files.
how does the (real, live, CentOS) updates repo work when there is nothing in the filelists?
I just tried to sync files from one server to another. After the sync process, I found the files are bigger than original ones.
I looked up the web and found someone mentions the rsync daemon. So I have to run the daemon on one server before I run the rsync?
The command I used is rsync --partial --progress -r source destination
I just purchased an Ibiza Rhapsody the other day, and installed Gnomad2 to transfer my library of mp3's to it. Once the transfer finished, only about half the number of songs showed up on the Ibiza as there were songs I was transferring over. All of them are proper mp3 files. Also, none of the files that transferred over can be played by the Ibiza. Sometimes it pops up an error message that it can not play the audio, but mostly it sits on the song and does not play. It reads the title and artist out of the MP3, but I guess it doesn't get the real data, considering the lack of playing, coupled with the fact that it does not show a length/timeleft for the song when I go to play it. In adddition, other mp3 players I've used come up in the filesystem, mount, etc, and you can browse it, directly paste in mp3's, etc (Makes life much easier). With this, it does not show up on the desktop like the others do, and I do not see it in the filesystem whatsoever. Upon plugging in, Ubuntu does realize something is happening, but thinks it is a camera and tries to import pictures. It's weird, because apparantly Gnomad picked it up (considering it transferred some files).
View 3 Replies View Relateduse rsync to cp such files and dirs under /var/www/html/mydir directory but these two files(/dir4/1.html /dir4/2.html) cant rsync to dest mechine.
rsync configure file,below...
I'm syncing a server over the internet with rsync, but it only works for a few hours before the backup fails with a "No route to host". I can restart the job and it'll will pick up where it left off, but is there an automated way to do this, or protect against a connection failure? I have about 170GB to copy over initially, but I can only get through about 4-5GB before the connection drops--manually restarting the sync everytime it drops will make the initial backup take days...
View 2 Replies View RelatedOk I have used dropbox for a long time but never had an issue but since recently switching to Ubuntu it seems that when I download a file and drag it into my dropbox the file shows in dropbox but it doesn't seem to be applying ...after dragging it into dropbox in the upper right hand corner of the file there is a blue circle with two white arrows in it while all existing files that I out into dropbox prior to using Ubuntu are there and accessible and have a green checkmark in the upper right hand corner..
View 4 Replies View RelatedAs an example, I have two servers, sm-i222 and fileserv. sm-i222 is a Win2k3 system running cygwin. fileserv is a linux box running RHEL 4.7. sm-i222 maps /cygdrive/c to the c: drive and /cygdrive/d maps to the d: drive(actually a single 4TB RAID). from the sm-i222 server /cygdrive/c I call a small script from the crontab. The internal IP for fileserv is 10.0.0.7. See below.
cd /cygdrive/d/fileserv/home
rm -rf /cygdrive/d/fileserv/home/*
rsync -avz -e 'ssh -c blowfish' root@10.0.0.7:/home/* .
These three lines perform well in that they make a full transfer of the fileserv:/home/ directory on fileserv to the appropriate place on sm-i222 using rsync. I use rsync instead of scp because I have to traverse subdirectories and symbolic links in the /home/... filesystem on fileserv. What I'm looking to do is use rsync to do an incremental transfer/backup of only the files that have changed since the last full backup. I'll manage the times I do this manually or in crontab. A colleague says this is do-able, but not how. Rsync.org says this is do-able but not how. Cygwin says this is do-able... see rsync.org. I believe what I'm looking for is a single rsync line like I have above that only transfers the changed files on fileserv to sm-i222.
Code:
find "$SOURCEDIR" -type f -name "*$ITEM" -printf "%P
"
I want to apply some shell script to the files outputed by the find command.
How can i do this.?
There are multiple files directories and also multiple files.
I'm trying to using rsync to backup some files, about half a TB. It's now it a state where it keeps sending the same files everytime it runs. for example:
rsync -av /data/source/* user@host:/data/dest
sending incremental file list
source/file1.txt
source/file2.txt
I then verify those files are copied over. then the next time it runs it does the same thing
rsync -av /data/source/* user@host:/data/dest
sending incremental file list
source/file1.txt
source/file2.txt
any idea why it's getting stuck on these files? I've tried to wipe the whole dest directory out and start over but no luck.
I have two directories, dirA whicht contains N gb of data and dirB which is supposed to contain only the newest M gb of data from dirA. When files are added to dirA, they sould also be added to dirB, while the oldest files in dirB should be deleted.Is that possible with rsync? or any other software?
View 1 Replies View Relatedrsync -r -v -e ssh root@nn.nn.nn.nn:/usr/local/websites/* /usr/local/websites and each time I run it it copies everything - all files. I thought rsync was only supposed to copy files that had been added or modified.
View 3 Replies View RelatedWhat would be the easiest way to go about rsyncing the n newest files in a directory to a remote server?
View 2 Replies View RelatedI'm using the command below to sync two directories. Problem is insted of deleting the files on the target directory it simply appends a ~ character at the end of the file name. Not sure why this is happening?I'd like to have all deletes on the source replicated on target.
View 2 Replies View RelatedThis is a quick one, I don't have any problems yet, I just want to check that this is going to do what I think it is. I added the line
Code:
00 01 * * * rsync -avz --delete /local-storage /mnt/usbackup
to my crontab file, am I to understand that this will backup /local-storage to my external
[code]...
I was going to do a rsync -r -a -z -v -p -e sshto move some files frome server to another, but then realized all I really need are files which have dates starting June 1, 2008 to current. Is there a way to have rsync only sync those files?he directory structure that's my source goes all the way back to 2004.
View 5 Replies View RelatedIf transfer all files under a directory by rsync, what is the order that rsync determines to transfer the files one by one?At first it looked like rsync transfers files in alphabetical order, but later I found rsync skipped some files in the first sweep through the alphabetic order, and then went back to transfer files that were skipped in the first time and this time still in alphabetic order.
View 1 Replies View RelatedRecently i am trying to check on the rsync speed for single file(2.4GB iso) directory ( 900MB directory with files inside ) When i run the rsync for single file: the speed i get is average 50MBps However, for a directory: average speed is 10MBps Is there any reason behind this ? i tried to google but unable to get the concept.
View 2 Replies View RelatedFor backup purposes, I have been trying to find out a solution for Rsync -avr sourcefolder targetfolder with Skipping 0 bytes files option.
However it seems that they are no solutions. Would someone have an idea, to skip to source files into the sourcefolder that have no content, ie. 0 bytes?
I have recently purchased an external hard drive in order to backup my home partition. In my PC I have a "1.5T" drive with several partitions on it, containing OSes and the home partition. The home partition is 1.3T according to df, the external drive contains one partition that spans the entire disk,df reports it as 1.4T in size. Both partitions are ext3. When I use rsync to copy files from the home partition to the external partition, the external disk becomes full, despite the destination - supposedly - being larger than the source. I don't understand why copying files from one partition to a slightly bigger partition should need more space than on the source partition. Does anyone know what is happening ?
Details : I created the partition on the external drive with gparted; gparted reported it the already have several gigabytes in used space immediately after the partitions creation - I thought at the time that this must be normal. The home partition contains many files of all sorts, including lots of big audio and video files. If you are wondering, for all my important files this external disk is only secondary backup, as they are also backed up to the "internet".
These are the mount points :
/mnt/tmp/ : home partition, /dev/sdb6
/mnt/external/ : external partition, /dev/sdc1
I used rsync to copy the files, I know there are more efficient ways to do this, but I wanted to use the same command that I will subsequently run to sync the backup.
rsync -av --progress --stats --recursive --perms --links --delete /mnt/tmp/ /mnt/external/
Next I tried adding the --sparse switch, as I was wondering if the problem may come form sparse files. I don't know however if rsync would go back and shrink the sparse file by just adding the switch and executing the command. I also added --one-file-system, for good measure. Here is what I ran next :
rsync -av --progress --stats --sparse --one-file-system --recursive --perms --links --delete /mnt/tmp/ /mnt/external/
I tried an fsck on the home partition :
fsck -f /dev/sdb6
This is the output from the last rsync :
rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32)
rsync: write failed on "abcd.avi": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(302) [receiver=3.0.6]
[code]....
Looking at the destination after a partial copy seems to indicate that the problem is not symbolic links being "expanded". I have not checked the source filesystem for sparse files, nor the destination to see if these files could be larger there, as this does not seem trivial.
Here is some additional info :
$ df /mnt/tmp/
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdb6 1415342836 1414173740 369096 100% /mnt/tmp
[code]....
We have an rsync cron job set up to mirror all the files in a "..dashtdocsdocs" folder to the same folder on another server. It copies all the files over correctly and deletes any files in the "docs" directory that aren't in the sending directory, but it also deletes any files we put in the target directory's parent folder (..dashtdocs or other subfolders like ..dashtdocsimages) even though they've been excluded in the .rsync-filter file.
Here's our current rsync .sh:
/usr/local/bin/rsync --stats -qPzrtpl --delete --password-file=/var/run/.appprodrsync --log-file=/export/home
/webuser/logs/rsync-extranet-log -FF /export/home/ appprodrsync@appprod::dprweb_extranet/ > /export/home/webuser/logs/rsync-extranet-output 2>&1
[code]....
So for example server A has ..dashtdocsdocs and ..dashtdocsimages. Server B has ..dashtdocsdocs but if I manually copy the images folder over to ..dashtdocsimages, the images folder gets deleted from the target directory every time rsync runs.
A: ..dashtdocsdocs
..dashtdocsimages
B: ..dashtdocsdocs
..dashtdocsimages (<-- gets deleted everytime rsync runs)
I'd like to keep just the docs directory synched and update other folders manually, but they keep getting deleted. It looks to me like it's running a delete-excluded option, but that option wasn't used.
I want to find files containing the "$" char (ascii 0x24). 'Grep -irl $ *' would output the names of every file in path *, of course, because it means end of line (EOL). So giving grep the string "$" won't do. So I tried 'grep -irl $ *'. But this doesn't work either and I do not understand why. Am I not escaping the dollar sign? grep should interpret it literally. Neither 'grep -irl "$" *' will work. Fortunately, there's LQ, besides grep's man page.
View 9 Replies View RelatedI have a big iso image which is currently being downloaded by a torrent client with space-reservation turned on: that means, file size is not changing while some chunks in in (4 Mib) are constantly changing because of a download.
At 90% download I do the initial rsync to save time later:
$ rsync -Ph DVD.iso /media/another-hdd/
sending incremental file list
DVD.iso
[Code]....
Then, when the file's fully downloaded, I rsync again:
total size is 2.60G speedup is 1.00
Speedup=1 says delta-transfer was not used, although 90% of the file has not changed, target dir is on another FS and copying takes several minutes. Why doen't it try to speedup the transfer?! How can I force rsync to use delta-transfer?
i would like to find and backup all *.mp4 files from /Pictures and its sub-directories and move them to a single directory on a remote. I can find and move the files but I don't want the directory structure...just the files to be placed in the remote directory.
To find my files I use
rsync -r -a -v -e "ssh -l user" --delete --include '*/' --include '*.mp4' --exclude '*' /home/drew/Pictures/ remoteserver:/Users/drew/mp4
but this creates all the subdirectories
I also tried
find ~/Pictures -name "*.mp4" -exec rsync -r -a -v -e "ssh -l user" --delete {} remote:/Users/drew/mp4 ;
This works but takes forever
Quick question. I need to ensure that all my files within specific folders are marked as being read only if the folders name is 'Forms' for example. There may be many 'forms' folders on the server within many different directories.It is a requirement that everything within these folders are marked as being read only but people are not doing so for whatever reason, and would therefore like for it to be done automatically i.e. a script executed by a cron job every eve to esnure that all files within the forms directories are marked read only.
View 1 Replies View Relatedmake a simple script to copy files from my "hfiles " directory to a "www" folder at leats I think its a directory, it has the permissions as "lrwxrwxrwx " but its like a public branch So I proceeded to make a file entittled MkPublic to copy the "hfiles" directory to the "www" branch
here is the code i used:
#!/bin/sh
# COPIES FILES
echo "copying files"
[code]....
when i try to run the script it tells me that there is a syntax error and an unexpected end of file and nothing seems to get copied anywhere so clearly there is something wrong with one im doing, i tried multiple ways : using the quotes, spacing the equal signs, but I seem to still get the same results.
Kernel 2.6.21.5, Slackware 12.0
Code:
bash-3.1$ cat /etc/cron.daily/slocate
#!/bin/sh
[code]...