Software :: Rsync Command - Script ?
Sep 20, 2010
I work for a company that rsyncs its media content to a datacentre daily for distribution via https to our clients. Unfortunately we've started to have problems with our rsync command/script.
It runs fine, but it seems to re-copy files that are already at the destination, of the same size and of the same date modified? We have hundreds of thousands of files, so this really cant happen. Our team here at the company use Windows XP to create the media, then put in on a the local suse server via SMB.
The code is
Code:
Both servers local and remote are Suse 10. The target/paths are irrelevant because they are correct and work.
Local server LS -L
Code:
For example
Remote server LS -L
Code:
As you can see local/remote files are the same, any ideas why this re-copy would happen? What's rsync playing at? We obviously just want it to sync new files and changes.
View 4 Replies
ADVERTISEMENT
Jun 9, 2011
The requirements are:
SystemA must be 100% Identical to SystemB no matter what rsync runs into: symlinks, same size files with different dates, directories that no longer exist... etc.
What is your perfect rsync command?
View 2 Replies
View Related
Feb 15, 2011
I'm trying to put a variable into the rsync command but I just get some BS error about escaping - whatever that is suppose to mean.
This is my command:
Code:
rsync -rvvuc --exclude '/Output/*' ${OPTARGS} --delete "${SOURCE}scripts/" "${DEST}scripts/"
${OPTARGS} has this value:
--rsh='ssh -p 34'
Wont work. If I type that text directly, then it works. Other then using some if statement to have two different command calls, how can I make this work?
View 3 Replies
View Related
Mar 23, 2011
I'm going to be using this command to back up my files:
Should I change anything or is it ok?
View 4 Replies
View Related
Oct 27, 2010
I'm trying to understand rsync filters .. but it isn't goig to well.I want to rsync only files with a specific extension (.gz.des3). Could someone please tell me how to add such filter to the rsync commandline (not a seperate filter filer).
View 2 Replies
View Related
Sep 14, 2010
I have the following rsync command for making some backups:
Code:
rsync -r --progress --delete -H --numeric-ids -a --exclude=.gvfs /source /media/Backup
If I paste that in my Terminal, it will perform a backup of all the files and show me what's going on in the Terminal window. But how can I make that into a launcher? I have made a launcher on my desktop with that code in its Properties, but double-clicking it starts the rsync process (I can see HDD activity) but a Terminal window won't open.
View 6 Replies
View Related
Oct 6, 2010
I'm trying to set up rsync backups on my ReadyNAS and I'm getting the following error: ERROR: The remote path must start with a module name not a / This error is accompanied by the following information:
[Code]...
View 1 Replies
View Related
Apr 7, 2011
Why would rsync insert a user's home directory path in variable expansion when run via cron, but not when run manually. The gory details... Red Hat Enterprise Linux AS release 4 (Nahant Update 6) Linux 2.6.9-67.0.20.ELsmp The script (parts anyway, and simplified)...
Quote:
#!/bin/bash
. /home/bea/.bash_profile
echo rsyncloc=${rsyncloc} >> ${log}
[code]....
View 4 Replies
View Related
Feb 14, 2010
I received the following output from an rsync (3.0.0) command that was executed:
sending incremental file list
sent 77214 bytes received 484 bytes 155396.00 bytes/sec
total size is 254531170 speedup is 3275.90
What does "sending incremental file list" mean?
View 9 Replies
View Related
Aug 10, 2010
I have rsync error message for this command,
Quote:
rsync -avr --delete --exclude lampp/tmp/* /opt/* /backup/opt/
rsync -avr --delete --exclude data/session/* /var/www/html/dmcatholic/* /backup/dmcatholic/
The error message is here.
[code]....
sent 1917154 bytes received 384128 bytes 5731.71 bytes/sec total size is 12220966785 speedup is 5310.50 rsync error: some files could not be transferred (code 23) at main.c(892) [sender=2.6.8]
View 8 Replies
View Related
Oct 6, 2010
when i use rsync command to backup my image file , it shows the following error message.
bash: line 1: /usr/bin/rsync: Argument list too long
rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
rsync error: remote command could not be run (code 126) at io.c(463) [receiver=2.6.8]
The command which i used is rsync -avrl -e ssh cms@server:/data/cms/data/images/* /mnt/Backup/Intranet_cms_backup/images
View 7 Replies
View Related
Nov 17, 2010
Thought I'd post it here because it's more server related than desktop... I have a script that does:
[Code]....
This is used to sync my local development snapshot with the live web server. There has to be a more compact way of doing this? Can I combine some of the rsyncs? Can I make the rsync set or keep the user and group affiliations? Can I exclude .* yet include .htaccess?
View 6 Replies
View Related
Jan 7, 2011
When I run rsync --recursive --times --perms --links --delete --exclude-from='Documents/exclude.txt' ./ /media/myusb/
where Documents/exclude.txt is
- /Downloads/
- /Desktop/books/
the files in those directories are still copied onto my USB.
And...
I used fetchmail to download all my gmail emails. When I run rsync -ar --exclude-from='/home/xtheunknown0/Documents/exclude.txt' ./ /media/myusb/ I get the first image at url.
View 9 Replies
View Related
Apr 12, 2011
I have a tiny shell script to rsync files between two servers and remove the source files.
This script works fine, when it has been initiated manually or even when the rsync command is executed on the command line.
But the same script doesn't work, when I try to automate it through crontab.
I am using 'abc' user to execute this rsync, instead of root, as root login to servers are restricted in all of our servers, by us.
As I mentioned earlier, manual execution works like charm!
When this rsync.sh is initiated through crontab, it runs the first command(chown abc.abc ...) perfectly without any issues. But the second line is not at all executed, and there is no log entry i can find at /mnt/xyz/folder/rsync.log.
View 6 Replies
View Related
Sep 18, 2009
I just tried to sync files from one server to another. After the sync process, I found the files are bigger than original ones.
I looked up the web and found someone mentions the rsync daemon. So I have to run the daemon on one server before I run the rsync?
The command I used is rsync --partial --progress -r source destination
View 1 Replies
View Related
Jul 21, 2010
use rsync to cp such files and dirs under /var/www/html/mydir directory but these two files(/dir4/1.html /dir4/2.html) cant rsync to dest mechine.
rsync configure file,below...
View 2 Replies
View Related
Dec 8, 2010
I'm using Ubuntu 10.04 LTS server and Postgresql 8.4. I have a .sh script that is run by cron every other hour. That works fine. The .sh script includes an rsync command that copies a postgresql dump .tar file to a remote archive location via ssh. That fails when run by cron; I think because it is (quietly) asking for the remote user's password (and not getting it). I set up the public/private ssh key arrangement. The script succeeds when run manually as the same user that the cron job uses, and does not ask for the password. I am able to ssh to the remote server from the source server (using the same username) and not get the password prompt (both directions), so why doesn't rsync work? I even put a .pgpass file in the root of that user's directory with that user's password, and the user/password are identical on both servers.
I think the problem is rsync is not able to use the ssh key correctly. I tried adding this to my script but it didn't help.
Code:
Here is the rsync command embedding in the .sh script.
Code:
Here is the cron entry:
Code:
View 6 Replies
View Related
Jun 21, 2011
iam trying to sync file server data into backup server machine by command- rsync -avu path/of/data ipaddress-of-backup-server:/path/where/to/save after running it ask for root password and manually it is successful.but i want to make it automatic.for that i also tried cronjob and also generated authentication key but iam not successful in login automatically..anybody know how to authenticate root to login for storing data in backup server.
View 14 Replies
View Related
Jan 27, 2010
I want to use rsync in order to have a folder synced at startup with my fat32 partition. I figured out how to mount the fat32 partition automatically at startup but I'm failing with the rsync command.I came across this script but it works only the first time, when there is a new file to sync it fails.
View 3 Replies
View Related
Sep 20, 2010
I've bought a NAS (Western Digital My Book World Edition 1TB) which I want to use to make backups (preferably incremental) of some important files.
After much deliberation in finding suitable backup software it seems like good old rsync is the best thing for the job (backintime struggles to copy to remote locations?)
I have enabled SSH on the NAS config. I'm using the following command to do a test run on a small folder:
Code:
sudo rsync -azvv -e ssh /home/matt/Careers/ admin@192.168.1.100:/public/AutoBackup/
And I get the following error:
Code:
admin@192.168.1.100's password:
[Code].....
Anyone know where I'm going wrong here? I'm sure it's probably something simple but I can't crack it. I've tried variations on the destination folder such as admin@192.168.1.100:/AutoBackup/ without success.
View 4 Replies
View Related
Jan 18, 2010
I want to use rsync to synchronize some folders on my LAN. I have this working with two scripts; one runs at the beginning of my work session and gets the latest directory tree from the server, and the other runs at the end of the session to put any local changes back on the server. My "get" script looks something like this:
Code:
rsync -avuzb --backup-dir=/home/user/rsync_backup_dir --delete my_server:/home/bak/common_data/ /home/user/data
This works well, and with the "b" option any file that has been deleted from the master directory tree on the server will be deleted from the local machine and moved to the local backup directory. This is a safety measure to prevent the loss of files through a mistake (on my part).
The problem is the "put" script:
Code:
rsync -avuzb --backup-dir=/home/user/rsync_backup_dir --delete /home/user/data/ my_server:/home/bak/common_data
I want to run both scripts from the local machine, but the "put" script will not save deleted files to a backup directory. I tried using a remote backup directory like "my_server:/home/user/rsync_backup_dir" but this did not work. Is there a way to backup files deleted from a remote server from an rsync script run locally?
View 5 Replies
View Related
Mar 2, 2010
I saw in a magazine reference to using rsync to have identical copies of folders. This looks like something I could find useful as I have a large number of items in need of safe backup.
I have the folders on an old system on a home network and would like to copy these over to a USB Hard Drive.
Currently the folders reside on SFTP xxx.xxx.xxx.xxx and I wish to sync them to a USB port on my laptop.
View 9 Replies
View Related
May 15, 2010
I setup keyed ssh between two of my computers on my lan. It was working great. I used Ubuntu's Passwords and Encryption Keys tool to generate the key.
Yesterday I tried rsync'ing a grip of files to the ssh server and couldn't get it to work, I figured I was just messing the rsync command up. Today I just tried to log in using ssh and it just hangs there after I type the ssh command.
With verbosity on it stops at checking the blacklist file. Does this mean it decided my key is one of the badly generated ones? Why would it decide this now I thought it was supposed to catch that when generating the key.
Also I just checked that file it's trying to check doesn't even exist. Should it?
Code:
View 1 Replies
View Related
Feb 14, 2011
I'm currently trying to have crontab to automatically backup files from ramdisk. It works perfectly when I run it myself by simply cd:ing to scripts directory and type ./save_world.sh.
The problem is, that crontab DOES (at least it looks like it) run that command every one minute. /var/log/syslog does show it executing that line every one minute without any errors. I'm currently very confused what I did wrong here. I have tried rebooting, fiddling with crontab line, tried sudo crontab -e but nothing seems to work.
My script is called save_world.sh and it is located in /home/phoe/minecraft/rpg/
Code:
My crontab -e has one line and it is following:
Code:
I haven't determined any specific time yet, because I'm just trying to get it work first.
Snippet from /var/log/syslog:
Code:
View 4 Replies
View Related
Aug 2, 2010
I have a samba share to a windows 7 computer I do not know if I will be able to use backintime or not so I want to know how to have rsync do my backup.I read the man but I'm not sure if I understand the it.on same computer different hard drive to run every hour in a script. Leanne is windows 7 share and backup is the other hard drive in the computer rsync -arvRzEP /media/leanne /media/backup.
View 1 Replies
View Related
Feb 24, 2011
I am trying to run a rsync backup Script to backup my data from my HD to my NAS Drive (currently mounted as Samba share, but can change to NFS if that would improve things, if I can work out how!), but when I run the Script I get the following error on every file it tries to copy
Code:
As I mentioned I am mounting my Share from my NAS as Samba and below is the fstab entry
Code:
The rsync command with options is as follows
Code:
The error seems to be with the -a option, but I have no idea why.
View 7 Replies
View Related
Feb 20, 2011
I have done the port forwarding to a production machine from our office machine it includes 3 hopes ie 3 machine in which I have done the portforwarding... but I am not able to rsync and directory or file even small fies with 10KB, but I am able to do do the rsync in to the same machine with 2 hopes
View 1 Replies
View Related
Nov 26, 2009
I was trying to rsync datas from one server to a remote server. There is a firewall between the two servers and the 22 port is open for the servers.
Attached the error that I got.
View 2 Replies
View Related
May 26, 2010
i have a cheap-arse employer who doesn't provide any data backup for us. so i've setup my own very simple server with ubuntu 10.04 (no raid). from my linux desktop i want to backup to my file server. from my file server i configured samba to allow me to access my data from the many Windows PCs in the place. I also have a USB drive for when i go home or to another building.
most of my work happens on my linux desktop. so i use rsync to 'backup' from there to the file server. can i use rsync to keep my usb drive also sync'ed? in the same command?
when i access files on my file server from a Windows PC via samba, and change something, will rsync pick up that change on the server, and migrate the changes back to my linux desktop?
View 4 Replies
View Related
Jan 8, 2011
I'm trying to do a sync of the root home directory to a folder call backup and excluding some files.
This is wat i executed:
rsync -a /root/ --exclude-from '/root/rules.txt' /root/backup/
My rules.txt is as below:
- anaconda-ks.cfg
Somehow it doesnt read the rules.txt and it will always include the anaconda-ks.cfg file.
View 3 Replies
View Related