I'm using rsync for a backup-sript at the moment and want to keep all files. The files are always unique, so I want to rsync without delete any file on the destination.
I've tried with --no-delete and --max-delete=0 but nothing seens to work. Is there even a possibility to do so?
When I run rsync --recursive --times --perms --links --delete --exclude-from='Documents/exclude.txt' ./ /media/myusb/
where Documents/exclude.txt is
- /Downloads/
- /Desktop/books/
the files in those directories are still copied onto my USB.
And...
I used fetchmail to download all my gmail emails. When I run rsync -ar --exclude-from='/home/xtheunknown0/Documents/exclude.txt' ./ /media/myusb/ I get the first image at url.
I am performing a dry run using Rsync on 2 different boxes.While i'm doing that, Under destination directory, I want a specific directory x to be ignored for sync.Please let me know the exact pattern to ignore the directory.The current command I'm using is:rsync -avnc --delete $LOCAL_DIR $USERNAME@$DESTINATION_IP:$REMOTE_DIRunder DESTINATION_IP, I would want to ignore a particular directory under REMOTE_DIR.
View 1 Replies View RelatedI have recently purchased an external hard drive in order to backup my home partition. In my PC I have a "1.5T" drive with several partitions on it, containing OSes and the home partition. The home partition is 1.3T according to df, the external drive contains one partition that spans the entire disk,df reports it as 1.4T in size. Both partitions are ext3. When I use rsync to copy files from the home partition to the external partition, the external disk becomes full, despite the destination - supposedly - being larger than the source. I don't understand why copying files from one partition to a slightly bigger partition should need more space than on the source partition. Does anyone know what is happening ?
Details : I created the partition on the external drive with gparted; gparted reported it the already have several gigabytes in used space immediately after the partitions creation - I thought at the time that this must be normal. The home partition contains many files of all sorts, including lots of big audio and video files. If you are wondering, for all my important files this external disk is only secondary backup, as they are also backed up to the "internet".
These are the mount points :
/mnt/tmp/ : home partition, /dev/sdb6
/mnt/external/ : external partition, /dev/sdc1
I used rsync to copy the files, I know there are more efficient ways to do this, but I wanted to use the same command that I will subsequently run to sync the backup.
rsync -av --progress --stats --recursive --perms --links --delete /mnt/tmp/ /mnt/external/
Next I tried adding the --sparse switch, as I was wondering if the problem may come form sparse files. I don't know however if rsync would go back and shrink the sparse file by just adding the switch and executing the command. I also added --one-file-system, for good measure. Here is what I ran next :
rsync -av --progress --stats --sparse --one-file-system --recursive --perms --links --delete /mnt/tmp/ /mnt/external/
I tried an fsck on the home partition :
fsck -f /dev/sdb6
This is the output from the last rsync :
rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32)
rsync: write failed on "abcd.avi": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(302) [receiver=3.0.6]
[code]....
Looking at the destination after a partial copy seems to indicate that the problem is not symbolic links being "expanded". I have not checked the source filesystem for sparse files, nor the destination to see if these files could be larger there, as this does not seem trivial.
Here is some additional info :
$ df /mnt/tmp/
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdb6 1415342836 1414173740 369096 100% /mnt/tmp
[code]....
We have an rsync cron job set up to mirror all the files in a "..dashtdocsdocs" folder to the same folder on another server. It copies all the files over correctly and deletes any files in the "docs" directory that aren't in the sending directory, but it also deletes any files we put in the target directory's parent folder (..dashtdocs or other subfolders like ..dashtdocsimages) even though they've been excluded in the .rsync-filter file.
Here's our current rsync .sh:
/usr/local/bin/rsync --stats -qPzrtpl --delete --password-file=/var/run/.appprodrsync --log-file=/export/home
/webuser/logs/rsync-extranet-log -FF /export/home/ appprodrsync@appprod::dprweb_extranet/ > /export/home/webuser/logs/rsync-extranet-output 2>&1
[code]....
So for example server A has ..dashtdocsdocs and ..dashtdocsimages. Server B has ..dashtdocsdocs but if I manually copy the images folder over to ..dashtdocsimages, the images folder gets deleted from the target directory every time rsync runs.
A: ..dashtdocsdocs
..dashtdocsimages
B: ..dashtdocsdocs
..dashtdocsimages (<-- gets deleted everytime rsync runs)
I'd like to keep just the docs directory synched and update other folders manually, but they keep getting deleted. It looks to me like it's running a delete-excluded option, but that option wasn't used.
I am trying to use an old box as backup server. I have tried a couple of possibilities along the lines of:
Quote:
rsync -a --delete --progress --log-file=/home/$USER/info.txt -e ssh /home /etc root@192.168.0.106:/mnt/back
The problem is it does not delete files that has been removed from my local system?
I run the command as root on the local system.
(I realize I should properly not ssh into the server as the server's root but I'm having trouble with the permissions and I want to make sure everything else works before messing around with it)
I'm currently learning to use rsync to backup my music collection. I have a Firefox tab open to the rsync manual page(s) and have been reading man rsync and running experimental rsync operations.I've been doing this for the last 3-4 hours. I've used rsync for this purpose in the past with disastrous results. What was and is once again (due to a month and a half of file pruning) a 9000 file music collection had mysteriously grown to over 25,000 music files and 80GB of data! This was likely due to the fact that I didn't really know what I was doing with rsync and had never spent too much time learning about all the parameters, what their functions are and how they may relate to my goal.Here are the particulars:
* Source drive is a 500GB disk, /media/sata500/music/.
* Destination drive is a 250GB USB disk, /media/FreeAgent/music, connected to the same computer that houses the 500GB disk.
* I want to copy or backup files from /media/sata500/music to /media/FreeAgent/music.
* I do not want to create ANY duplicates of files that exist.
* I only want to add files to the destination drive if they are new on the source drive, like if I rip a CD and add the contents to the source. I want them copied over next time I run rsync.
Here's the rsync command in it's most recently used form, and probably very immature at this point.
Code:
rsync -t -r -vv --stats -i --log-file=/home/glenn/rsync.log /media/sata500/music/* /media/FreeAgent/music/
This appears to have copied all files and folders and I'm satisfied that my goal has been met with some success. To convince myself of this I ran the command and then once it was complete I added 2 new songs putting them in their respective folders on the source drive and ran the same command again. The resulting output was
[code]....
Two files transferred. Exactly what I want.Both folders now house 20,931 files and use 40.6GB. Identical as far as I can tell.What I'm concerned about are time stamps and play count data, etc. Anything that changes the original file. I don't want this data to cause a file to be transferred as I'm afraid that the new file will be created along side the old file of the same name thereby starting this whole music collection expansion thing all over again. I've invested a lot of time and effort to get it pruned down to where there are virtually no duplicates and albums are correct in that they contain the proper songs in the proper order.
iam trying to sync file server data into backup server machine by command- rsync -avu path/of/data ipaddress-of-backup-server:/path/where/to/save after running it ask for root password and manually it is successful.but i want to make it automatic.for that i also tried cronjob and also generated authentication key but iam not successful in login automatically..anybody know how to authenticate root to login for storing data in backup server.
View 14 Replies View RelatedDon't ask me why, but I need to back up a website with complete structure to a windows machine (so no tar/gzip - just an identical copy). I'm experienced with rsync, so I thought to do it that way. However, in the process I'm bound to lose my ownership/permission settings for each file and that will give problems when placing back certain files. Is there a way to either:
1. save those settings on a windows machine?
2. have an easy way to save the filetree with relevant information and a shell script to attach the info back when uploading files again?
when i use rsync command to backup my image file , it shows the following error message.
bash: line 1: /usr/bin/rsync: Argument list too long
rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
rsync error: remote command could not be run (code 126) at io.c(463) [receiver=2.6.8]
The command which i used is rsync -avrl -e ssh cms@server:/data/cms/data/images/* /mnt/Backup/Intranet_cms_backup/images
I have a tiny shell script to rsync files between two servers and remove the source files.
This script works fine, when it has been initiated manually or even when the rsync command is executed on the command line.
But the same script doesn't work, when I try to automate it through crontab.
I am using 'abc' user to execute this rsync, instead of root, as root login to servers are restricted in all of our servers, by us.
As I mentioned earlier, manual execution works like charm!
When this rsync.sh is initiated through crontab, it runs the first command(chown abc.abc ...) perfectly without any issues. But the second line is not at all executed, and there is no log entry i can find at /mnt/xyz/folder/rsync.log.
I just tried to sync files from one server to another. After the sync process, I found the files are bigger than original ones.
I looked up the web and found someone mentions the rsync daemon. So I have to run the daemon on one server before I run the rsync?
The command I used is rsync --partial --progress -r source destination
I want to synchronize sets of files (e.g. from or to flash memory). rsync is powerful, but --delete option is dangerous. Anyone know whether there's a way to do --delete interactively, i.e. get rsync or some near equivalent to ask (y/n, in a console window) before deleting?
View 9 Replies View RelatedI would like to use rsync to keep the hard drive in my media server synced with my video collection on my linux server. The media server I believe is running some version of linux running samba. I mount the media servers share to a folder on my linux server & use the following command: rsync -a -vv --delete /home/shared/Videos/* /mnt/WDLive/Shearer Files/Movies/
However, it does not delete files on the media server that I delete on the source. I also created a new folder on the source & moved some of the files into it. When I ran rsync again, it created the new folder on the media server, but it recopied all the files from the source again, instead of moving the files which were already on the media server into the new folder, so no I have 2 copies.
I'm configuring an rsync between 2 machines, A_Server --> B_Server, using the following script:Quote:
#!/bin/bash
#
# Script de backup a trav�s de Rsync desde RMP-1 hasta RMP-2.
[code]....
use rsync to cp such files and dirs under /var/www/html/mydir directory but these two files(/dir4/1.html /dir4/2.html) cant rsync to dest mechine.
rsync configure file,below...
im using rsync to sync files across multiple machines.
using the following:
rsync -az -e "ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no"
--delete --delete-excluded --force --exclude=.git --exclude=.bundle
--exclude=tmp --exclude=log/* --exclude=*.log --exclude=*.pid
user@host:/path/to/src/ /var/build/dest
I want to exclude all log files from being transferred from the src to dest and delete all existing ones on the destination so im using --exclude=*.log with --delete-excluded which works great ...
but i want to keep a certain log file intact on the destination. I want a --exclude-from-delete option
Is this possible with rsync?
I built a script that downloads my podcasts using Gpodder into the directory /HOME/SHARED/PODCASTS/ (with a subdirectory for each podcast)The script then selects the latest episode and copies it over to a target directory (it empies the target directory first and copies over everything) I want to use RSYNC to make sure the 'not so fresh' episodes get deleted and the "fresh" episodes get copied over. Then dropbox can sync the "new" files over to the cloud where i can access them via my ipad/iphone (whole other story).The thing is : i've replaced the cp command with the RSYNC command and now the script is acting strangely.
It selects and sync's over the "newest" podcasts to the destination directory. Then it suddenly DELETES all the episodes in the destination directory and copies over the three last files.
I have just been bothered by a fairly small issue for some time now. I am trying to search (using find -name) for some .jpg files recursively. This is a Redhat environment with bash.
I get this job done though I need to copy ALL of them and put them in a separate folder BUT I also need to keep the order intact after copying.
For e.g - If I get a JPG file under /home/usr/new/1/ then the destination also needs to be /test/old/new/1/.
At the moment, I am simply putting all files under /test/old/ and I can't somehow get the later /new/1/ folder path created under /test/old/
I understand this could well be done using while OR if else loop, though if someone can just guide me with a hint, I would be really grateful.
I will complete the rest of the steps and was asking here since I am still not comfortable with the shell/bash scripts yet and planning to be really good at it over the next couple of months.
I had a situation in which the the path of the file to be copied is written in other file and I had to copy it using shell script..I can use cp $(cat /home/robert/location.txt) /media/sda1 on normal linux shell...But I am using buildroot script where $(cat /home/robert/location.txt) evaluate to nothing..is just blank..
View 1 Replies View RelatedI have 2 folders that are synced using rsync. Right now if I delete a file in the source folder, the destination folder still retains that file. Is there a way to get rsync to delete files if they were deleting at the source?I could not find anything in the man page.not sure if I'm missing something or if that feature just doesn't exist.
View 4 Replies View Relatedwrote a script to sync my netbook music with my server music and everything was fine.but if for some reason lets say i run it accidently and i'm not connected to my network, i just wiped my netbook music .here's what i got so far:
Code:
Sudo mkdir /media/ServerMusic/
mount -t cifs -o user=Justin,password=****** //192.168.1.108/Public-3/Music/ /media/ServerMusic/[code]....
i need to have the --delete option in rsync (say i sync it and realize i hate some music, when i delete it from the server i don't want it still on the netbook.i want an if statement in here somehow, but i'm not sure how to do it.something like, the first command in the script should be a ping test, if ping doesn't work end the script, if ping does work continue or if the server music gets mounted to the created directory continue, if the mount failed, or even in the mounted folder is empty stop the script
what can i do here to ensure i can keep the --delete and not have to worry about losing my music if the mount fails?
I installed cygwin with rsync on a Win XP Machine. My goal is to backup a folder from one hard drive to another (both on XP machine).
I run the following command from a batch file:
Works fine except the --delete flag is not working. Copies everything in source to destination, but doesn't delete some extra files that are present on the destination, but aren't on the source, which it's supposed to. I looked at the rsync man page, and I'm doing everything right... such as not using wildcard.
The same command works perfect on another computer (XP machine; source and dest both on XP machine).
That is the whole script:
Code:
#!/usr/bin/env bash
# backup device for me, m.tornow, probably:
# /dev/sde1
[Code].....
and BU_MOUNTPOINT being the mount-point where a device is being mounted (duh).
I'm using Ubuntu 10.04 LTS server and Postgresql 8.4. I have a .sh script that is run by cron every other hour. That works fine. The .sh script includes an rsync command that copies a postgresql dump .tar file to a remote archive location via ssh. That fails when run by cron; I think because it is (quietly) asking for the remote user's password (and not getting it). I set up the public/private ssh key arrangement. The script succeeds when run manually as the same user that the cron job uses, and does not ask for the password. I am able to ssh to the remote server from the source server (using the same username) and not get the password prompt (both directions), so why doesn't rsync work? I even put a .pgpass file in the root of that user's directory with that user's password, and the user/password are identical on both servers.
I think the problem is rsync is not able to use the ssh key correctly. I tried adding this to my script but it didn't help.
Code:
Here is the rsync command embedding in the .sh script.
Code:
Here is the cron entry:
Code:
i've been using php/fwrite to add text to a file using the below code but how do i remove text from a file? note that i don't know what line it's on as there will be other content in the file.
View 5 Replies View RelatedWe used to send files in the form of .jpg, .tif, and/or .pdf. Normally the file name will be in the form of 08072011IE01CTYHUB.PDF (DDMMYYYY - is the date, IE - publication, 01 - page number, CTY - edition name and HUB - destination in three characters). These files will be stored in a common folder (say SOURCE). I need a script to move these files to destination by reading the destination from the file name through FTP. At destination these files should be moved to a folder meant for CTY. Please note before the file is sent through FTP it should be compressed (zipped) At SOURCE folder the files will be as:
08072011IE01CTYHUB.pdf
08072011IE01CTYSMG.pdf
08072011IEdavanamCTYHUB.pdf
08072011IEversaVANSMG.tif
etc. where first 8 characters are date in the form of DDMMYYYY, next 2 characters are publication, last 3 characters are destination, previous 3 characters are edition and left over in the middle are page number in the form of NN or name. Presently I am zipping these files and send it through FTP to the destination. At destination my counterpart takes the file, stores in appropriate location (like folder name CTY) for use. To automate the above process, I want a script.
delete a file on a remote server using shell scripting.
View 14 Replies View RelatedI need to find a string in a file ... then delete the line it is on, as well as the next 6 lines. Or, delete the line the string is on and all subsequent lines until the search finds the character "["
example:
filename = test.txt
contents:
[foo]
test>test
test>test
test>test
[Code]....
so, in this example. I'd like to search the file for string 'foo' and delete all lines from that line until [bar] (not deleting the line with [bar])
Why is there no Delete when I right click like there is with Windows in ubuntu? Pretty much everything else is there like new folder and so on Is there some way to add it? Also why when i delete something does it not ask me if I am sure that i want to delete that file?
View 6 Replies View Related