I have two directories, dirA whicht contains N gb of data and dirB which is supposed to contain only the newest M gb of data from dirA. When files are added to dirA, they sould also be added to dirB, while the oldest files in dirB should be deleted.Is that possible with rsync? or any other software?
View 1 Replies
What would be the easiest way to go about rsyncing the n newest files in a directory to a remote server?
View 2 Replies View RelatedI'm trying to keep a directory full of log files manageable. Nightly, I want to delete all but the 10 most recent. How can I do this in a single command?
View 2 Replies View RelatedI'm looking for a Bash script that will go into a list of directories and delete all but the four most recently created files.
How can I do this?
I just tried to sync files from one server to another. After the sync process, I found the files are bigger than original ones.
I looked up the web and found someone mentions the rsync daemon. So I have to run the daemon on one server before I run the rsync?
The command I used is rsync --partial --progress -r source destination
use rsync to cp such files and dirs under /var/www/html/mydir directory but these two files(/dir4/1.html /dir4/2.html) cant rsync to dest mechine.
rsync configure file,below...
I'm trying to using rsync to backup some files, about half a TB. It's now it a state where it keeps sending the same files everytime it runs. for example:
rsync -av /data/source/* user@host:/data/dest
sending incremental file list
source/file1.txt
source/file2.txt
I then verify those files are copied over. then the next time it runs it does the same thing
rsync -av /data/source/* user@host:/data/dest
sending incremental file list
source/file1.txt
source/file2.txt
any idea why it's getting stuck on these files? I've tried to wipe the whole dest directory out and start over but no luck.
rsync -r -v -e ssh root@nn.nn.nn.nn:/usr/local/websites/* /usr/local/websites and each time I run it it copies everything - all files. I thought rsync was only supposed to copy files that had been added or modified.
View 3 Replies View RelatedI'm using the command below to sync two directories. Problem is insted of deleting the files on the target directory it simply appends a ~ character at the end of the file name. Not sure why this is happening?I'd like to have all deletes on the source replicated on target.
View 2 Replies View RelatedThis is a quick one, I don't have any problems yet, I just want to check that this is going to do what I think it is. I added the line
Code:
00 01 * * * rsync -avz --delete /local-storage /mnt/usbackup
to my crontab file, am I to understand that this will backup /local-storage to my external
[code]...
I was going to do a rsync -r -a -z -v -p -e sshto move some files frome server to another, but then realized all I really need are files which have dates starting June 1, 2008 to current. Is there a way to have rsync only sync those files?he directory structure that's my source goes all the way back to 2004.
View 5 Replies View RelatedIf transfer all files under a directory by rsync, what is the order that rsync determines to transfer the files one by one?At first it looked like rsync transfers files in alphabetical order, but later I found rsync skipped some files in the first sweep through the alphabetic order, and then went back to transfer files that were skipped in the first time and this time still in alphabetic order.
View 1 Replies View RelatedI was wondering if there is a way to tell rsync to only apply changes (delete, overwrite,create) only if all files in the file list transferred successfully.Just to clarify, this would essentially be putting a transaction around the transfer.
View 3 Replies View RelatedRecently i am trying to check on the rsync speed for single file(2.4GB iso) directory ( 900MB directory with files inside ) When i run the rsync for single file: the speed i get is average 50MBps However, for a directory: average speed is 10MBps Is there any reason behind this ? i tried to google but unable to get the concept.
View 2 Replies View RelatedFor backup purposes, I have been trying to find out a solution for Rsync -avr sourcefolder targetfolder with Skipping 0 bytes files option.
However it seems that they are no solutions. Would someone have an idea, to skip to source files into the sourcefolder that have no content, ie. 0 bytes?
I have recently purchased an external hard drive in order to backup my home partition. In my PC I have a "1.5T" drive with several partitions on it, containing OSes and the home partition. The home partition is 1.3T according to df, the external drive contains one partition that spans the entire disk,df reports it as 1.4T in size. Both partitions are ext3. When I use rsync to copy files from the home partition to the external partition, the external disk becomes full, despite the destination - supposedly - being larger than the source. I don't understand why copying files from one partition to a slightly bigger partition should need more space than on the source partition. Does anyone know what is happening ?
Details : I created the partition on the external drive with gparted; gparted reported it the already have several gigabytes in used space immediately after the partitions creation - I thought at the time that this must be normal. The home partition contains many files of all sorts, including lots of big audio and video files. If you are wondering, for all my important files this external disk is only secondary backup, as they are also backed up to the "internet".
These are the mount points :
/mnt/tmp/ : home partition, /dev/sdb6
/mnt/external/ : external partition, /dev/sdc1
I used rsync to copy the files, I know there are more efficient ways to do this, but I wanted to use the same command that I will subsequently run to sync the backup.
rsync -av --progress --stats --recursive --perms --links --delete /mnt/tmp/ /mnt/external/
Next I tried adding the --sparse switch, as I was wondering if the problem may come form sparse files. I don't know however if rsync would go back and shrink the sparse file by just adding the switch and executing the command. I also added --one-file-system, for good measure. Here is what I ran next :
rsync -av --progress --stats --sparse --one-file-system --recursive --perms --links --delete /mnt/tmp/ /mnt/external/
I tried an fsck on the home partition :
fsck -f /dev/sdb6
This is the output from the last rsync :
rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32)
rsync: write failed on "abcd.avi": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(302) [receiver=3.0.6]
[code]....
Looking at the destination after a partial copy seems to indicate that the problem is not symbolic links being "expanded". I have not checked the source filesystem for sparse files, nor the destination to see if these files could be larger there, as this does not seem trivial.
Here is some additional info :
$ df /mnt/tmp/
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdb6 1415342836 1414173740 369096 100% /mnt/tmp
[code]....
We have an rsync cron job set up to mirror all the files in a "..dashtdocsdocs" folder to the same folder on another server. It copies all the files over correctly and deletes any files in the "docs" directory that aren't in the sending directory, but it also deletes any files we put in the target directory's parent folder (..dashtdocs or other subfolders like ..dashtdocsimages) even though they've been excluded in the .rsync-filter file.
Here's our current rsync .sh:
/usr/local/bin/rsync --stats -qPzrtpl --delete --password-file=/var/run/.appprodrsync --log-file=/export/home
/webuser/logs/rsync-extranet-log -FF /export/home/ appprodrsync@appprod::dprweb_extranet/ > /export/home/webuser/logs/rsync-extranet-output 2>&1
[code]....
So for example server A has ..dashtdocsdocs and ..dashtdocsimages. Server B has ..dashtdocsdocs but if I manually copy the images folder over to ..dashtdocsimages, the images folder gets deleted from the target directory every time rsync runs.
A: ..dashtdocsdocs
..dashtdocsimages
B: ..dashtdocsdocs
..dashtdocsimages (<-- gets deleted everytime rsync runs)
I'd like to keep just the docs directory synched and update other folders manually, but they keep getting deleted. It looks to me like it's running a delete-excluded option, but that option wasn't used.
I have a big iso image which is currently being downloaded by a torrent client with space-reservation turned on: that means, file size is not changing while some chunks in in (4 Mib) are constantly changing because of a download.
At 90% download I do the initial rsync to save time later:
$ rsync -Ph DVD.iso /media/another-hdd/
sending incremental file list
DVD.iso
[Code]....
Then, when the file's fully downloaded, I rsync again:
total size is 2.60G speedup is 1.00
Speedup=1 says delta-transfer was not used, although 90% of the file has not changed, target dir is on another FS and copying takes several minutes. Why doen't it try to speedup the transfer?! How can I force rsync to use delta-transfer?
I have rsync error message for this command,
Quote:
rsync -avr --delete --exclude lampp/tmp/* /opt/* /backup/opt/
rsync -avr --delete --exclude data/session/* /var/www/html/dmcatholic/* /backup/dmcatholic/
The error message is here.
[code]....
sent 1917154 bytes received 384128 bytes 5731.71 bytes/sec total size is 12220966785 speedup is 5310.50 rsync error: some files could not be transferred (code 23) at main.c(892) [sender=2.6.8]
i would like to find and backup all *.mp4 files from /Pictures and its sub-directories and move them to a single directory on a remote. I can find and move the files but I don't want the directory structure...just the files to be placed in the remote directory.
To find my files I use
rsync -r -a -v -e "ssh -l user" --delete --include '*/' --include '*.mp4' --exclude '*' /home/drew/Pictures/ remoteserver:/Users/drew/mp4
but this creates all the subdirectories
I also tried
find ~/Pictures -name "*.mp4" -exec rsync -r -a -v -e "ssh -l user" --delete {} remote:/Users/drew/mp4 ;
This works but takes forever
I'm working on a bash script that will go through a directory, find the sub-directories that have been created since the last time the script ran, count the results, and output that integer (will most likely be '1' or less per each instance run) to a file. Give the circumstances, my previous (and very limited) experience with bash is not sufficient for me to pull this off. since it probably has bearing, is that my mail server stores files that it flags as viruses in a folder. It creates a sub-directory for each virus that it quarantines .I want to count those subdirectories and graph them with MRTG. Hence the script. I'm going to post what I've got so far and the purpose of it, because I'm told I have a very odd and efficient way of doing scripting.
[Code]...
But then it dawned on me that it wouldn't work because I would have to not count the directories that have already been counted and count the ones that have not been counted. Given that the purpose of this is to generate a graph about every 5 minutes, using find won't work because, to my knowledge, that will only find things based on whole day values, I need it almost down to the minute.
I want to run rsync on server A to copy all files from Server B when they are newer than 7 days.(find . -mtime -7) I don't want to delete the files on Server B.
View 2 Replies View RelatedI've been trying to rsync only files with mca in their filename but it's sending the whole lot across.
Code:
rsync -dtv --include='mca*' /source/source/source/source/source/source/source/ /dest/dest/dest/
How can I install g++ 4.0.2, already have the newest one. but one of my make files says it needs 4.0.2.
View 2 Replies View RelatedRecently i downloaded the stable version of kernel from kernel.org then i extract the tarball to /usr/src path , then after i complile the kernel for fuse and ext4 modulescommand which i use is mention below
[root@localhost linux-2.6.36.1]# make menuconfig
[root@localhost linux-2.6.36.1]# make
[root@localhost linux-2.6.36.1]# make modules
[code]...
writing a bash script that looks in a particular directory and emails the contents of the newest file.
View 1 Replies View Relatedbut anyway I have a lot of stuff on my Laptop (I am currently using) and I am using Windows Vista, but I want to, I think it's called dual boot, where I can use both Linux and Windows. The only thing is, I am not entirely sure how to do this, I used to have it done on one of my other Computers, but I forgot how I did it, I am currently downloading the newest version of Ubuntu and I want to be able to run both.
View 1 Replies View RelatedBasically, I am trying to locate and copy the newest .json bookmark backup in my .mozilla/firefox/w987sdg9.default/bookmarkbackups directory.
I tried this
Code:
ls -t ~/.mozilla/firefox/b1ahb1ah.default/bookmarkbackups/ | head -1
which does return the newest file, but only the filename itself. I found readlink, but I haven't gotten that to output a full path which I can then feed to copy. So, it seems to me that find might work well here, and I know how to find based on absolute dates, but not relative.
My Source folder contains 424.8 GB in 502,474 files. My Destination folder was created fresh, and after the copy contains 394.0 GB in 486.514 files. I am running it as grsync with root authority. The only options are to preserve time, permissions, owner and group., and to produce a verose output and transfer progress. There are no exceptions specified to skip any files.
The rsync command is this:
rsync -r -t -p -o -g -v --progress -c -l -H -D -s /mnt/Backups/monthly.3/ /mnt/EX-Fantom/monthly.3/
I have run it again to give it a chance to get it right. Same result. The source is in an rsnapshot folder, but this is the first backup, the original, containing only whole files, not links.
So I just used rsync to backup about 400gb of data to my NAS. Look just over a day to complete, which is what I figured. I decided I should run rsync again to see how its going to handle comparing and only adding new files to the remote location. So I added a few new files and then ran the backup again. Well rsync is trying to do a complete copy of all of my original data, even though they have not changed.
Is there a way that I can tell rsync to compare the two directories and only add the new files and delete the ones that are no longer in the original location?
Here is command that I am running:
Code:
sudo rsync -azvv --progress --stats /media/sda1/multimedia/movies /home/codeblue/NAS_Share_Point
