General :: Rsync N Newest Files In A Directory?
Nov 8, 2010What would be the easiest way to go about rsyncing the n newest files in a directory to a remote server?
View 2 Replies
ADVERTISEMENT
General :: Rsync Only Newest N Gb Of Files?
Jul 27, 2011I have two directories, dirA whicht contains N gb of data and dirB which is supposed to contain only the newest M gb of data from dirA. When files are added to dirA, they sould also be added to dirB, while the oldest files in dirB should be deleted.Is that possible with rsync? or any other software?
View 1 Replies View RelatedGeneral :: Order Of Transfer All Files Under A Directory By Rsync
Sep 25, 2010If transfer all files under a directory by rsync, what is the order that rsync determines to transfer the files one by one?At first it looked like rsync transfers files in alphabetical order, but later I found rsync skipped some files in the first sweep through the alphabetic order, and then went back to transfer files that were skipped in the first time and this time still in alphabetic order.
View 1 Replies View RelatedGeneral :: Rsync Speed For Single Files And Directory?
Mar 23, 2010Recently i am trying to check on the rsync speed for single file(2.4GB iso) directory ( 900MB directory with files inside ) When i run the rsync for single file: the speed i get is average 50MBps However, for a directory: average speed is 10MBps Is there any reason behind this ? i tried to google but unable to get the concept.
View 2 Replies View RelatedGeneral :: Rsync Deleting Files From Destination Directory?
Jul 14, 2011We have an rsync cron job set up to mirror all the files in a "..dashtdocsdocs" folder to the same folder on another server. It copies all the files over correctly and deletes any files in the "docs" directory that aren't in the sending directory, but it also deletes any files we put in the target directory's parent folder (..dashtdocs or other subfolders like ..dashtdocsimages) even though they've been excluded in the .rsync-filter file.
Here's our current rsync .sh:
/usr/local/bin/rsync --stats -qPzrtpl --delete --password-file=/var/run/.appprodrsync --log-file=/export/home
/webuser/logs/rsync-extranet-log -FF /export/home/ appprodrsync@appprod::dprweb_extranet/ > /export/home/webuser/logs/rsync-extranet-output 2>&1
[code]....
So for example server A has ..dashtdocsdocs and ..dashtdocsimages. Server B has ..dashtdocsdocs but if I manually copy the images folder over to ..dashtdocsimages, the images folder gets deleted from the target directory every time rsync runs.
A: ..dashtdocsdocs
..dashtdocsimages
B: ..dashtdocsdocs
..dashtdocsimages (<-- gets deleted everytime rsync runs)
I'd like to keep just the docs directory synched and update other folders manually, but they keep getting deleted. It looks to me like it's running a delete-excluded option, but that option wasn't used.
General :: Rsync Files Recursively But Place In One Remote Directory?
Jan 29, 2011i would like to find and backup all *.mp4 files from /Pictures and its sub-directories and move them to a single directory on a remote. I can find and move the files but I don't want the directory structure...just the files to be placed in the remote directory.
To find my files I use
rsync -r -a -v -e "ssh -l user" --delete --include '*/' --include '*.mp4' --exclude '*' /home/drew/Pictures/ remoteserver:/Users/drew/mp4
but this creates all the subdirectories
I also tried
find ~/Pictures -name "*.mp4" -exec rsync -r -a -v -e "ssh -l user" --delete {} remote:/Users/drew/mp4 ;
This works but takes forever
General :: Mailing Newest File In A Directory?
Jan 16, 2010writing a bash script that looks in a particular directory and emails the contents of the newest file.
View 1 Replies View RelatedGeneral :: Bash - Finding Newest File In A Directory?
Aug 24, 2010Basically, I am trying to locate and copy the newest .json bookmark backup in my .mozilla/firefox/w987sdg9.default/bookmarkbackups directory.
I tried this
Code:
ls -t ~/.mozilla/firefox/b1ahb1ah.default/bookmarkbackups/ | head -1
which does return the newest file, but only the filename itself. I found readlink, but I haven't gotten that to output a full path which I can then feed to copy. So, it seems to me that find might work well here, and I know how to find based on absolute dates, but not relative.
Ubuntu :: Use Rsync To Copy ONLY Home Directory And No Files?
Jun 1, 2011How can I use rsync to copy ONLY the my home folder (and nothing inside of it, just the folder name) to another machine. I've tried things like
Code:
rsync -av /path/to/src /path/to/dest/
or
Code:
rsync -av -f"+ */" -f"- *" /path/to/src /path/to/dest/
This last option recursively (through the -a switch) copies only folders, including all subfolders. If I try
Code:
rsync -v -f"+ */" -f"- *" /path/to/src /path/to/dest/
nothing is copied (not even my home folder.
Slackware :: Rsync Exclude Directory With All Subdirectories And Files
Dec 26, 2010In reading the rsync man page and browsing a lot of websites, I ended up a bit confused, or maybe it was just too much eggnog. Anyway, to exclude a directory "videos" with everything in it, which is /home/user1/camera/videos and I'm rsyncing the whole user1 directory to an external drive
[code]...
General :: Delete All But 10 Newest Files?
Apr 8, 2011I'm trying to keep a directory full of log files manageable. Nightly, I want to delete all but the 10 most recent. How can I do this in a single command?
View 2 Replies View RelatedGeneral :: Delete All Files (except The Four Newest) Throughout Directories Using A Bash Script?
Aug 16, 2011I'm looking for a Bash script that will go into a list of directories and delete all but the four most recently created files.
How can I do this?
General :: RSYNC - Ignore A Directory On Destination Box?
Jul 28, 2011I am performing a dry run using Rsync on 2 different boxes.While i'm doing that, Under destination directory, I want a specific directory x to be ignored for sync.Please let me know the exact pattern to ignore the directory.The current command I'm using is:rsync -avnc --delete $LOCAL_DIR $USERNAME@$DESTINATION_IP:$REMOTE_DIRunder DESTINATION_IP, I would want to ignore a particular directory under REMOTE_DIR.
View 1 Replies View RelatedSoftware :: Run Rsync To Download Files From A Server Without Rsync Daemon?
Sep 18, 2009I just tried to sync files from one server to another. After the sync process, I found the files are bigger than original ones.
I looked up the web and found someone mentions the rsync daemon. So I have to run the daemon on one server before I run the rsync?
The command I used is rsync --partial --progress -r source destination
General :: Rsync To Copy Directory Tree From NAS Drive ?
Apr 10, 2010I have a WD world book edition 1TB NAS drive, and just purchased an acomdata 1tb drive and connected it to the NAS via USB. If I recall I think the WD NAS has a ext_ or some type of linux filesystem on it, and the acomdata has a ntfs filesystem on it.
What I want to do is copy over certain directory trees of the NAS to the USB attached drive. I usually use MS synctoy to sync folders from my windows pc to the NAS drive, and MS richcopy to make the initial transfer from PC to NAS. For this operation though, since it is taking place entirely on the NAS and its connected drive, I thought that rsync would be the best option, and it is available on my NAS drive.
Last night I entered in rsync -avr /movies/* /usb1-1share1/ to copy the entire "movies" dir to the drive, which shows up as usb1-1share1 on the NAS drive. It copied most of the directory tree ok, but a lot of the folders were empty, so this morning I tried rsync -Carv --ignore-existing /movies/* /usb1-1share1/ to try and get all the files missed, without recopying the 24GB that did make it across. This also managed to copy a few more GB over, but not everything.
I am running the command from an ssh session on the NAS using putty on my PC, in as user "admin" which should have all rights over these folders. There is a bunch of errors in the command window like this: rsync: failed to set times on "/shares/usb1-1share1/movies/classics/fulldvd/First Blood DVD/.VTS_01_2.VOB.RxdjWZ": Operation not permitted (1)
I want to restart another session and get the files it missed, but I want to find out what I am doing wrong first. Should I be doing this as root user? am I missing some switches or just plain doing it all wrong?
General :: Make Rsync Ignore Directory Structure?
Apr 19, 2011Is there a way to force rsync to not make directories in its destination directory; ie, to simply dump all of the files from the source directory directly into the destination without copying any of the folders that the files were originally in? I tried --no-dirs, but that seems to only be for empty directories.
View 1 Replies View RelatedServer :: Rsync Can Not Rsync Files With Include Filter
Jul 21, 2010use rsync to cp such files and dirs under /var/www/html/mydir directory but these two files(/dir4/1.html /dir4/2.html) cant rsync to dest mechine.
rsync configure file,below...
General :: Rsync Command Inserting User Home Directory?
Apr 7, 2011Why would rsync insert a user's home directory path in variable expansion when run via cron, but not when run manually. The gory details... Red Hat Enterprise Linux AS release 4 (Nahant Update 6) Linux 2.6.9-67.0.20.ELsmp The script (parts anyway, and simplified)...
Quote:
#!/bin/bash
. /home/bea/.bash_profile
echo rsyncloc=${rsyncloc} >> ${log}
[code]....
General :: Setup Rsync To Backup A Remote Directory To Local Drive?
May 24, 2010I'm trying to setup rsync to backup a remote directory to my local drive.
I cd to the directory that I want to pull the files to, then I enter:
rsync -vrtW account@remote.com:~/public_html
I enter the password then it starts running. I get all the files listed, but none of them actually transfer. What am I missing?
General :: Rsync Would Keep Sending The Files Over And Over Again?
Mar 2, 2010I'm trying to using rsync to backup some files, about half a TB. It's now it a state where it keeps sending the same files everytime it runs. for example:
rsync -av /data/source/* user@host:/data/dest
sending incremental file list
source/file1.txt
source/file2.txt
I then verify those files are copied over. then the next time it runs it does the same thing
rsync -av /data/source/* user@host:/data/dest
sending incremental file list
source/file1.txt
source/file2.txt
any idea why it's getting stuck on these files? I've tried to wipe the whole dest directory out and start over but no luck.
General :: Rsync With Scp - Copies Everything - All Files
Jul 12, 2011rsync -r -v -e ssh root@nn.nn.nn.nn:/usr/local/websites/* /usr/local/websites and each time I run it it copies everything - all files. I thought rsync was only supposed to copy files that had been added or modified.
View 3 Replies View RelatedGeneral :: Rsync Not Deleting Files On Target
Sep 8, 2010I'm using the command below to sync two directories. Problem is insted of deleting the files on the target directory it simply appends a ~ character at the end of the file name. Not sure why this is happening?I'd like to have all deletes on the source replicated on target.
View 2 Replies View RelatedGeneral :: Rsync, Crontab And Backing Up Files?
Apr 11, 2011This is a quick one, I don't have any problems yet, I just want to check that this is going to do what I think it is. I added the line
Code:
00 01 * * * rsync -avz --delete /local-storage /mnt/usbackup
to my crontab file, am I to understand that this will backup /local-storage to my external
[code]...
General :: Rsync Files With Dates After A Certain Date?
Jun 8, 2010I was going to do a rsync -r -a -z -v -p -e sshto move some files frome server to another, but then realized all I really need are files which have dates starting June 1, 2008 to current. Is there a way to have rsync only sync those files?he directory structure that's my source goes all the way back to 2004.
View 5 Replies View RelatedGeneral :: Rsync And Making Sure All Files Have Been Transferred Before Applying The Changes?
Nov 15, 2010I was wondering if there is a way to tell rsync to only apply changes (delete, overwrite,create) only if all files in the file list transferred successfully.Just to clarify, this would essentially be putting a transaction around the transfer.
View 3 Replies View RelatedGeneral :: Rsync With Skipping 0 Bytes Files Option?
Jun 10, 2011For backup purposes, I have been trying to find out a solution for Rsync -avr sourcefolder targetfolder with Skipping 0 bytes files option.
However it seems that they are no solutions. Would someone have an idea, to skip to source files into the sourcefolder that have no content, ie. 0 bytes?
General :: Files Seem To Take Up More Space In Destination After Rsync Copy?
Feb 9, 2010I have recently purchased an external hard drive in order to backup my home partition. In my PC I have a "1.5T" drive with several partitions on it, containing OSes and the home partition. The home partition is 1.3T according to df, the external drive contains one partition that spans the entire disk,df reports it as 1.4T in size. Both partitions are ext3. When I use rsync to copy files from the home partition to the external partition, the external disk becomes full, despite the destination - supposedly - being larger than the source. I don't understand why copying files from one partition to a slightly bigger partition should need more space than on the source partition. Does anyone know what is happening ?
Details : I created the partition on the external drive with gparted; gparted reported it the already have several gigabytes in used space immediately after the partitions creation - I thought at the time that this must be normal. The home partition contains many files of all sorts, including lots of big audio and video files. If you are wondering, for all my important files this external disk is only secondary backup, as they are also backed up to the "internet".
These are the mount points :
/mnt/tmp/ : home partition, /dev/sdb6
/mnt/external/ : external partition, /dev/sdc1
I used rsync to copy the files, I know there are more efficient ways to do this, but I wanted to use the same command that I will subsequently run to sync the backup.
rsync -av --progress --stats --recursive --perms --links --delete /mnt/tmp/ /mnt/external/
Next I tried adding the --sparse switch, as I was wondering if the problem may come form sparse files. I don't know however if rsync would go back and shrink the sparse file by just adding the switch and executing the command. I also added --one-file-system, for good measure. Here is what I ran next :
rsync -av --progress --stats --sparse --one-file-system --recursive --perms --links --delete /mnt/tmp/ /mnt/external/
I tried an fsck on the home partition :
fsck -f /dev/sdb6
This is the output from the last rsync :
rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32)
rsync: write failed on "abcd.avi": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(302) [receiver=3.0.6]
[code]....
Looking at the destination after a partial copy seems to indicate that the problem is not symbolic links being "expanded". I have not checked the source filesystem for sparse files, nor the destination to see if these files could be larger there, as this does not seem trivial.
Here is some additional info :
$ df /mnt/tmp/
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdb6 1415342836 1414173740 369096 100% /mnt/tmp
[code]....
General :: Perform A Long Directory Listing Of All Files In The /bin Directory That Have Exactly Three Characters In Their Name?
Jan 18, 2010How do you perform a long directory listing of all files in the /bin directory that have exactly three characters in their name?
View 1 Replies View RelatedGeneral :: Files Created In A Directory Owned By Directory Group?
Jan 29, 2010Is there a way, on Linux, to cause all new files created in a directory to be owned by the directory's group instead of the creating user's group?
View 2 Replies View RelatedGeneral :: Cronjob Directory / Send An Email If Directory Contains More Than Ten Files?
Oct 13, 2010I want to run a cronjob every 15 minutes that checks a directory for files. If the directory contains more than ten files I want it to send an email to me.
All I have is this...
*/15 * * * * ls -l | wc -l | [filename] | mail -s "This is just a test" [email address]
I would rather not write a bash script. Is there an easier way to do this? I was looking into some commands like find and grep.