I've had an interesting experience that is completely unprecedented in my history of using rsync. As I write this I'm listening to my server squeal like a stuck pig as the Adaptec RAID controller rebuilds my RAID-5 array (and doesn't lose any data, I HOPE!!!!) rsync is an integral part of my Disaster Recovery plan at the office. I have a NFS server that hosts all the business critical VMs that my company runs and a couple of VMware hosts that link thereto and attempt to make the best of things. I know NFS isn't the most optimal solution, but that's a conversation for another thread.
My rsync woes began tonight and took an unexpected turn for the worst as I was attempting to implement the most white-knuckled portion of this scenario.The Process:
1. A script runs that shuts down all VMs that access the NFS array
2. Post-shutdown, the server then rsyncs to a local HDD for fast-backup and to minimize downtime
3. During daily operation, another script will rsync the backed-up VMs to an offsite server that keeps them in case we have a disaster scenario
My plans never made it to step 3 since rsync started crashing during the local portion (step 2).My Hardware:
1. Adapted 5405 4-port hardware RAID controller
2. 3 x Western Digital Black SATA 2.0 HDDs (1TB each) in RAID-5 config
3. 1 x Western Digital Black SATA 2.0 HDD (2TB) in solo-mode on the same controller
I wanted to use an older computer again, so I tried to fire it up, but alas. I don't recall when the last time was that it was running, but I don't think it didn't run back then. I just had a new computer
5. Then the computer halts. No message, no warning. I am not even able to hit control+alt+delete (I say this, cuz I noticed this. I am able to run and alter the BIOS in an earlier state of booting up, with the keyboard, obviously). It just completely hangs/locks.
[WHAT I HAVE TRIED SO FAR]
1. Removing all hardware but monitor, RAM, keyboard. - Result: comp stops doing stuff after checking RAM, without any message whatsoever.
2. Cleared the CMOS (with the jumper, also by removing the battery). - Result: no changes.
3. There are two pieces of RAM. I removed them all. Result: POST-beeps. I put one back: loaded till after disks. See above. Exchanged the RAM-piece with the other RAM-piece (so: still one piece of RAM): same thing.
I have a tiny shell script to rsync files between two servers and remove the source files.
This script works fine, when it has been initiated manually or even when the rsync command is executed on the command line.
But the same script doesn't work, when I try to automate it through crontab.
I am using 'abc' user to execute this rsync, instead of root, as root login to servers are restricted in all of our servers, by us.
As I mentioned earlier, manual execution works like charm!
When this rsync.sh is initiated through crontab, it runs the first command(chown abc.abc ...) perfectly without any issues. But the second line is not at all executed, and there is no log entry i can find at /mnt/xyz/folder/rsync.log.
I'm using Ubuntu 10.04 LTS server and Postgresql 8.4. I have a .sh script that is run by cron every other hour. That works fine. The .sh script includes an rsync command that copies a postgresql dump .tar file to a remote archive location via ssh. That fails when run by cron; I think because it is (quietly) asking for the remote user's password (and not getting it). I set up the public/private ssh key arrangement. The script succeeds when run manually as the same user that the cron job uses, and does not ask for the password. I am able to ssh to the remote server from the source server (using the same username) and not get the password prompt (both directions), so why doesn't rsync work? I even put a .pgpass file in the root of that user's directory with that user's password, and the user/password are identical on both servers.
I think the problem is rsync is not able to use the ssh key correctly. I tried adding this to my script but it didn't help.
Code:
Here is the rsync command embedding in the .sh script.
iam trying to sync file server data into backup server machine by command- rsync -avu path/of/data ipaddress-of-backup-server:/path/where/to/save after running it ask for root password and manually it is successful.but i want to make it automatic.for that i also tried cronjob and also generated authentication key but iam not successful in login automatically..anybody know how to authenticate root to login for storing data in backup server.
We are running an adserver with mysql as the database. We backup the database every hour and lately noticed that ads doesn't get delivered at the time of backup. The database size is 1.2 GB in which a single table ifself is 1.1 GB. Googling around the problem says this may be due to lock table feature in mysqldump and disabling it might solve the issue but i'm afraid whether this would be a problem. Can anyone please explain what lock table means and will it be a problem if i disable it or may be any other alternative solutions/assitance (other than master/slave or replication since we cannot afford it at this time) would be really helpful.
Last night at 1:47 AM yum installed the current version of kernel-headers on my RHEL5 server:
Mar 17 01:47:47 linux1 yum: Updated: kernel-headers-2.6.18-164.15.1.el5.i386 Mar 17 13:47:22 linux1 syslogd 1.4.1: restart.
This locked the server up until I could physically get to it this afternoon and reboot the server. I then finished downloading and installing the new kernel, rebooted again, and the server appears to be running fine.
This is the second time this has happened on this server.
Has anyone else seen this problem when you setup yum to automatically update your servers? Other than manually updating servers, does anyone have any suggestions on how to keep this problem from happening again?
I'm trying to rsync files and directories from a RedHat linux host(v 4.5 & 4.7) to a Windows server 2003R2 Standard Edition with cygwin running. I'm executing the rsync command from the cygwin shell. The transfer involves rsync'ing approximately 1 TB of data from the linux server to the windows server. After about 280+GB of data transfer, the transfer just dies.
There seems to be no particular file or directory that the transfer stops at. I'm able to rsync GB's of data from other linux hosts to this cygwin server with no problem. Files and directories rsync fine.The network infrastructure is essentially the same regardless of the server being rsync'ed in that it is GB Ethernet running through Cisco GB switches. There appear to be no glitches or hiccups across the network path.
I've asked the folks at rsync.samba.org if they know of any problems or issues. Their response has been neutral in that if the version of rsync that cygwin has ported is within standards then there is no rsync reason this problem should happen.I've asked the cygwin support site if they know of any issues and they have yet to reply. So, my question is whether the version of rsync that is ported to cygwin is standard. If so, is there any reason cygwin & rsync keep failing like this?
I've asked the local rsync on linux guru's and they can't see any reason this should fail from a linux perspective. Apparently I am our company cygwin knowledge base by default.
I want to run rsync on server A to copy all files from Server B when they are newer than 7 days.(find . -mtime -7) I don't want to delete the files on Server B.
Has anyone had any experience on using SUA(Services for UNIX Applications) rsync to "pull" files down to the Win2k3R2 server from a linux rsync host?I was trying to use cygwin rsync before until I found out from cygwin that the cygwin port of rsync was "flakey" and would fail intermittently for no apparent reason. cygwin suggested I use SUA or SFU for rsync services.
I've looked for/ am looking for any experience using SUA rsync to copy files down from a linux rsync host to the Windows host via rsync on the Windows host. Also, if you have done this successfully, do you have any pointers/caveats you can share on how you got it working? What I am basically looking to do is copy files and subdirectories of files from a linux host using rsync to some static location on a Windows server on a scheduled basis so that I can backup the windows server to tape using Symantec's Backup Exec application.
I'm doing it this way to avoid deploying the Remote Agents for either linux or Windows on the target hosts. As an alternative I've seen reference to a product called DeltaCopy that uses a native Windows rsync port with the native linux port of rsync to do what I need also.I realize this is not a strictly linux question, but more of a hybrid as I'm moving data to and from Windows and linux hosts. So, if this is too Windows-y a question, please say so and I'll withdraw my question.
So basically what I am trying to do is install Ubuntu 10.04 on a Dell Poweredge 2600 server. Right now I dont care wither I get the server version or the desktop version, I just need to get it on their. this is what I have run into. When installing the regular Ubuntu 10.04 desktop for 32bit, the server just locks up on the very first ubuntu install screen and the keyboard lights just start blinking non stop. When I use the server version, the server get to the Detecting Network Hardware and locks up at 94% and the keyboard will also start blinking nonstop. I have looked all over for an answer to this problem and have yet to find one. This guy figured out how to make it work but he never mentioned what version he was using. [URL]..
On this server I have 6 HDs, 5 of them are RAID 0 with 1 Hot Spare. I have also tried using the ubuntu-10.04.1-alternate-i386.iso.torrent and still had the same problem. I have spent the last 3 days searching all over google and the forum to an answer but had no luck.
Thought I'd post it here because it's more server related than desktop... I have a script that does:
[Code]....
This is used to sync my local development snapshot with the live web server. There has to be a more compact way of doing this? Can I combine some of the rsyncs? Can I make the rsync set or keep the user and group affiliations? Can I exclude .* yet include .htaccess?
When I run rsync --recursive --times --perms --links --delete --exclude-from='Documents/exclude.txt' ./ /media/myusb/
where Documents/exclude.txt is
- /Downloads/ - /Desktop/books/
the files in those directories are still copied onto my USB.
And...
I used fetchmail to download all my gmail emails. When I run rsync -ar --exclude-from='/home/xtheunknown0/Documents/exclude.txt' ./ /media/myusb/ I get the first image at url.
I attempted today to Rsync data from 'serverold' to 'servernew' and also delete data on 'servernew' that has been deleted / removed from 'serverold'. When I run the Rsync command I get the following error:
[Code]...
rsync warning: some files vanished before they could be transferred (code 24) at main.c(1042) [sender=3.0.7] why this is not working? Is my command syntax wrong or missing switches? I read the 'Man' page and gathered this would be the correct command but I could be wrong.
I have two servers. #1 is the main server and #2 is only used in case the first one is down. Both of them are in the same LAN. Both are debian Lenny configured the same way. For instance, if #1 has a problem, I simply disconnect it, set #1's IPs to #2 and I have my system ready. The only matter here is that there are plenty of files that need to be syncronized between #1 and #2. I thought that rsync was the answer to this problem I have.
I wanted to create a bash script that runs every day (with cron) and syncs the files I need (.conf, and other data). I used ssh-keygen to generate a pair of keys in order to login SSH without a password. The problem is that the permitRootLogin is set to yes in my sshd_config in both servers. So I can't log in directly as root by ssh. But I need to log in as root to be able to rsync the files between the servers because some of them are .conf files and aren't accesible for non-privilege users (Only root).
I switched last summer from Windows (used it since Windows 95) to Debian. I'm using Debian Jessie for a couple of months now and I'm getting used a little.
There are problems here and there, but I can solved them with some reading on the web. Not really a big problem...till now
I run Debian 8.2 om my PC (PC1). Bought an older PC (PC2) that I want to use as a backup server.
I'm using PC2 only for making backups, after the backup I switch it off again.
So I installed Debian 8.2 (net-install without DE and with SSH) on PC2 and tried to configure it to let it work as my backup location. Made a public SSH key and exported it to the root account (no problem) and to the user account (sensdeb), but there was an error "Access Denied"
Gave the user (sensdeb) sudo-rights via visudo file
# User privilege specification root ALL=(ALL:ALL) ALL sensdeb ALL=(ALL:ALL) ALL
I installed rsync.
The problem is that Rsync only works when I use the root account.
I don know how to give the user sensdeb the rights so that I can use that account for my backup tasks. Now it's possible to sync with the root account, but that should not be the way to do it, I read many times.
This should be a quick one. I'm trying to backup a single directory and it's subdirectories on my Lucid Server to a freenas box across my network. This is what I'm using to do that rsync -r -a -v -z * --delete freenas: DSIBackups..It almost works perfectly except for one problem. When a file is deleted at the source, this command doesn't seem to delete it on the receiving end. I assumed that the --delete would do that but aparently not.
I'm trying to set up my trusty Fedora box, with rsync over ssh to backup my windows machines at home and I need help configuring the rsync server (I'm using DeltaCopy as the rsync client on the windows side)I tried a few dry runs but it seems windows can't see the linux box, the rsync job just hangs for ever and never does anything.I should mention that ssh works fine.
How I use rsync to setup server from any existing server, that I want to create server from my existing one server where I have installed and tune everything, I want everything replecate on new server excluding network (IP) and /proc
My rsync takes backup of everything from the differenct linux servers to my backup device which is 2 TB only .Since it takes almost full backup of source , it consumes space lot in the backupdevice. So i wanted to keep all my backup files of one month old latest files in backup device, it should remove all files more than one month data.
Is there a way to limit the amount of data that goes in to an rysnc log. The problem I have is that I email myself the log file to make sure it went ok but sometime the log is hugeHere is what I am doing with rsyncrsync -azHK --delete-after /home/ /mnt/usbbackup/home/ >/backup-log/backup.txt
I'm trying to set up rsync on a clean install of ubuntu server 9.10, with ubuntu-desktop installed.
In the end, I'd liek the 2 windows clients (running Vista and Win7) to syncronise with the server when they log on/off. I'd also like them to access a printer attached to the server.
I'm trying to set-up a cron job that uses rsync to transfer back files from a remote server. ssh on the remote server is running on port 12345 but this seems to be a problem. My script looks like this:
I've 15 web servers (in private network) running RHEL, Apache. Needs to sync web files between them. each server is accessible to each other via public key (with passphares).
1) Main server is web1 (where dev upload files initially). So I can make all other servers accessible by web1 without password/passphares and run rsync periodically to update files between them. But security is an issue here as all servers will become easily accessible.
2) Run rsync daemon in all other servers (except web1) on designated port and run rsync command from web1 to sync files. This will do the work but running daemon in all servers might increase overhead and making sure that daemon is running all the time etc. are my concern for this implementation.
I am setting up a CentOS mirror on my company LAN for the 1st time ever. I am wondering what the do and don't are for this. Obviously I would need to open the TCP port for Rsync on my Firewall so that my rsync server can properly communicate with the external mirror but is there things I should do or know before I just blindly open TCP 873 on my network? I was moving the server to the DMZ interface and then point all my internal servers to my rsync server on the DMZ.
