I have successfully backed up my files using a script to a remote server with a log file output.However the log file is appended each time.I wish to have a different log file each time with date and time and have yet to figure that part out.
View 2 Repliesiam trying to sync file server data into backup server machine by command- rsync -avu path/of/data ipaddress-of-backup-server:/path/where/to/save after running it ask for root password and manually it is successful.but i want to make it automatic.for that i also tried cronjob and also generated authentication key but iam not successful in login automatically..anybody know how to authenticate root to login for storing data in backup server.
View 14 Replies View RelatedI launched a script using rsync and the option -delete-before, however the destination folder was the wrong one. I noticed it only seconds after but it was still too late. In less than 5 sec rsync deleted over 200gb of data on my external hard drive... It is not in my Trash nor on the HDD trash but it's not possible that the data were ereased in less than 5 sec! (I dont know how rsync handle deletions.) I know I am really stupid but is there a way to get back all my data (mainly HD movies in mkv, mp4 and some avi)
View 1 Replies View RelatedI am backing up may data using Rsync via SSH. I am trying to find a solution that allows me to push my backups from windows and Ubuntu to a Ubuntu server. I don't like the solution that requires my windows machines to have shared drives. Not viable for laptop users that may piggyback on free wireless networks. I would also like a solution that has a GUI for the desktop user but not totally required. I will need a solution that allows me to restore files by date. I do a lot of changes to files on a daily basis and sometimes need an older copy of a file.
I love using Rsync as it is a very simple solution and extremely quick. The only thing I don't know how to do is recover different version of a file. It would be nice to see a visual representation of all the files and different dates of each file. I know I might be asking a little too much, but, it is after all, Linux.
My rsync takes backup of everything from the differenct linux servers to my backup device which is 2 TB only .Since it takes almost full backup of source , it consumes space lot in the backupdevice. So i wanted to keep all my backup files of one month old latest files in backup device, it should remove all files more than one month data.
View 2 Replies View RelatedI've been given an old rsync script to update some data files on a few different servers... I am trying to understand what directory on the server it gets it's files as the way it seems to refer to the data on the server is:
data_source='random-server.fqdominan.net::Firstlogic'
Not sure what the "::Firstlogic" part with the two :: means.
Here is the entire script:
Code:
#!/bin/sh
# Begin user-configurable options #
# Who can run this script
authorized_user='appdtools'
# From where we rsync the update data
data_source='random-server.fqdominan.net::Firstlogic'
# From where we rsync the rollback data
rollback_data_source='random-server.fqdominan.net::Firstlogic_rollback'
# Where updates will go
data_target='/Firstlogic/postware/dirs'
# How to do the rsync .....
# Here is where we do the "work".
splash# Check user and print warning
if [ "${rsync_test}" == "-n" ]; then
update_address_data# Test the monthly update
else
stop_address# Stop the address standardizer
update_address_data# Load the monthly update
clean_logs# Empty the old log files
start_address# Start the address standardizer
fi
I have not tested this script yet but evidentialy it used to work. I just need to find out where to stick the files on random-server.fqdomain.net to test it out. The server it points to no longer has the files on them...
shed some light on what I am doing. I am wondering if I just havehings back to front.Server (MESH):Fedora 13Firewall ports open tcp 22(ssh), tcp 873(rsync)sshd service started
View 5 Replies View RelatedI'm trying to do an rsync from one RHEL box to another, but when I run it in verbose mode to see why its not working, all it does is show the root folder, then one folder in, then it stops. There are about a dozen folders under that root folder where the rsync starts, with about 350GB data spread between them. How can I tell why this isn't working? the same command is setup to run as a cron job, which was working.
View 1 Replies View RelatedWindows computer continually generates images (AutoGrabnnn). All downstream analysis on linux computer. I need analog of rsync bringing files over as they arise. rdesktop seems to be the approach. Is this already described somewhere to bypass much trial-and-error?
View 2 Replies View RelatedI'm looking for a most possible, secure solution to transfer data using rsync over Internet between 2 linux server.
I have 3 option: SSH, IPSEC and Kerberos.
Which one in your opinion should be most secure solution?
We purchased a new database system at work last October, ditching the old system because of a lack of support from the vendor. This is a retail Point of Sale and Backoffice database system. I am not sure what system the new one runs on, but the system we replaced was a Firebird data base. The reason I am posting is because we are now in need of the information contained in the old database which was not completely imported into the new system.
Basically the problem is this: The database in on a Windows XP system and I found a copy of SQL Manager Lite 2008 on the system, which after quite a bit of studying, I figured out how to extract the database into a removable file. I have this file (178MB) on a USB stick in a file called Backoffice.fbd.
My studying suggests to me that I can get into this database with MySQL. I have never used this and have no clue how to do this. All I want to be able to do get into the database and create tab deliminated spreadsheet files for each of the database sections (Customers, Repairs, Sales History, stock files, etc.) Is it possible to do this with Ubuntu and MySQL and if so, can expert suggest one or two things to get me started. While a guided tutorial would be nice because I am not an expert, I am willing to learn on my own if someone could point me in the right direction.
So I am trying to learn a little about Cygwin and rsync. I'd like to rsync some data from a Windows machine to a Linux machine. I've got Cygwin installed but I can't figure out how to tell Cygwin were rsync will be pulling the data from; basically, how to set the directories that I want to be rsnyc'd. I've googled and googled but I can't seem to find the answer to this exact question.
View 4 Replies View RelatedThought I'd post it here because it's more server related than desktop... I have a script that does:
[Code]....
This is used to sync my local development snapshot with the live web server. There has to be a more compact way of doing this? Can I combine some of the rsyncs? Can I make the rsync set or keep the user and group affiliations? Can I exclude .* yet include .htaccess?
When I run rsync --recursive --times --perms --links --delete --exclude-from='Documents/exclude.txt' ./ /media/myusb/
where Documents/exclude.txt is
- /Downloads/
- /Desktop/books/
the files in those directories are still copied onto my USB.
And...
I used fetchmail to download all my gmail emails. When I run rsync -ar --exclude-from='/home/xtheunknown0/Documents/exclude.txt' ./ /media/myusb/ I get the first image at url.
I'm always getting this error when I run the rsync command:
Code:
when I try to navigate the file:
Code:
it gives
Code:
I have a backup sh file that I have been using for a long time. It has always worked. 2 Days back I switched to a different pc and now suddenly the script don't work.If I run it manually in the terminal it works. But when it execute with cron it doesn't copy any files to the backup destination. It starts but doesn't copy anything.Can someone help me as to why it works manually but not with cron ?
View 9 Replies View RelatedI have an Ubuntu machine running NFS4 server and a plugapps (arch linux) machine connecting as the client. The plugbox is running an rsync job to backup the home directory from Ubuntu to a local USB HDD.
All of the files in the destination have owner nobody and group nobody.
Ubuntu /etc/exports:
Code:
/home 192.168.2.1/24(rw,sync,no_subtree_check,no_root_squash)
plugbox /etc/fstab
[Code].....
how I can mantain the file owners. I have the UID's and passwords sync'd between the two machines for both root and the user who's home dir is being backed up.
Using Ubuntu Server 10.04 LTS. I'm new to Ubuntu and testing Rsync. I successfully copied 3TB of data from a Win7 machine to an MDADM Raid5 array. All appears to be fine. Used a Win app for the copy. I then deleted a 250GB folder on the Raid5 array and recopied the data using Rsync. Rsync was executed via a Putty session on a WinXP machine. The source was an eSata attached drive (same drive used for the big 3TB copy) and the destination was the same Raid5 array. That copied just fine. I bit verified it with a Win7 app. Perfect.
I then used the following Rsync script to copy a single 26GB file from that same eSata drive back to (what I intended to be) the Raid5 array:
Code:
neil@ANTECUBSV:/mnt$ rsync -r -a -v -e ssh --delete /mnt/disk1/Test/ /mnt/Test
sending incremental file list
created directory /mnt/Test
./ C_VOL-S300-b001.spf
sent 3020267622 bytes received 34 bytes 50760800.94 bytes/sec
total size is 3019898880 speedup is 1.00
neil@ANTECUBSV:/mnt$ cd raid
Note that only about 3GB copied. No error messages were posted to the putty session. I made a mistake in the Rsync command, creating the Test folder directly in the mount folder rather than the Raid array, as I intended. That is a little strange, yes, but I would not think it would cause a partial copy? The /mnt folder is on my system drive, which had about 34GB available space before the copy, so comfortably would have had 6GB or so after.
The eSata disk is mounted as /mnt/disk1
The Raid5 array is mounted as /mnt/raid
I then recopied the file to the correct intended destination on the Raid5 array, which has about 400GB free space (plenty).
Code:
neil@ANTECUBSV:/mnt/raid$ rsync -r -a -v -e ssh --delete /mnt/disk1/Test/ /mnt/raid/Test
sending incremental file list
./
deleting 2010-07-05 Backyard Birds/Thumbs.db
deleting 2010-07-05 Backyard Birds/
C_VOL-S300-b001.spf
sent 11105775462 bytes received 34 bytes 50366328.78 bytes/sec
total size is 11104419840 speedup is 1.00
neil@ANTECUBSV:/mnt/raid$ df -h
Note that only about 11GB was copied, and this was confirmed with an ls -l command. Now I am correctly copying the file to the Raid array but it is still incomplete. I then copied the file back to the /mnt folder to see if the problem reproduces:
Code:
neil@ANTECUBSV:/mnt/raid$ rsync -r -a -v -e ssh --delete /mnt/disk1/Test/ /mnt/Test
sending incremental file list
created directory /mnt/Test
./
C_VOL-S300-b001.spf
sent 26327927554 bytes received 34 bytes 56558383.65 bytes/sec
total size is 26324713984 speedup is 1.00
neil@ANTECUBSV:/mnt/raid$ cd /mnt/test
This time I got my full 26GB file. Why I might be getting inconsistent results? This is quite troubling of course. I'd also be interested in basic a command line Linux diff app (that does file directory as well as bit level checking) if one is available.
i am trying to transfer a file from my live linux machine to remote linux machine it is a mail server and single .tar.gz file include all data. but during transfer it stop working. how can i work and trouble shooot the matter. is there any better way then this to transfer huge 14 gb file over network,vpn,wan transfer. the speed is 1mbps,rest of the file it copy it.
rsync -avz --stats bkup_1.tar.gz root@10.1.1.22:/var/opt/bkup
[root@sa1 logs_os_backup]# less remote.log
Wed Mar 10 09:12:01 AST 2010
building file list ... done
bkup_1.tar.gz
deflate on token returned 0 (87164 bytes left)
rsync error: error in rsync protocol data stream (code 12) at token.c(274)
building file list ... done
code....
I have 2 different mounts. One points to a local windows share(NTFS ->Samba) and the other one points to a PPTP VPN connection sharing(I belive that is NTFS too). I use "cifs" scheme in my fstab to mount these. And I use my Debian box to copy between these 2 mounts. I have started using Rsync for that purpose, I think that it works fine for now. My main problem is that it looks like Rsync cannot figure out if the files are same or not in source and target folders when I use these mounts. Most of the time Rsync copies the same files and folders over and over again even though those files and folders are on the target.
I am wondering if there is a way to make this scheme work? Being on a Vpn connection(slow) a Windows box, Rsync could have save a lot of my time if it could have recognized the files and folders that are same on both ends
I'm running rsync in daemon mode on CentOS 5.4, and I'm seeing inconsistent times reported in the log file.
View 2 Replies View RelatedI've just noticed a small problem I am having with my company file server. When making backups to an external NTFS drive weekly I have noticed that the file names with thai characters are not getting backed up. I receive the below error:
rsync: recv_generator: failed to stat (to the file name...)
Invalid or incomplete multibyte or wide character (84)
There are thousands of files on the server that contain thai characters in there names so how do I get around this problem so it will back up all files and not just the English character ones. I read somewhere that each file would need to be converted to a different character set but this would take years as there are so many files.
I have a tiny shell script to rsync files between two servers and remove the source files.
This script works fine, when it has been initiated manually or even when the rsync command is executed on the command line.
But the same script doesn't work, when I try to automate it through crontab.
I am using 'abc' user to execute this rsync, instead of root, as root login to servers are restricted in all of our servers, by us.
As I mentioned earlier, manual execution works like charm!
When this rsync.sh is initiated through crontab, it runs the first command(chown abc.abc ...) perfectly without any issues. But the second line is not at all executed, and there is no log entry i can find at /mnt/xyz/folder/rsync.log.
I just tried to sync files from one server to another. After the sync process, I found the files are bigger than original ones.
I looked up the web and found someone mentions the rsync daemon. So I have to run the daemon on one server before I run the rsync?
The command I used is rsync --partial --progress -r source destination
I made a shell backup script that uses Rsync and I am trying to get rid of the password prompt because it will use a CRON to run. I have set my variable in my shell script at:
PASSWORD_FILE=rsync_password
And the password in that file only takes up 1 line.
However when I run (ignore $DESTINATION)
rsync -aRvz tmp $DESTINATION --password-file=$PASSWORD_FILE
It still gives me the prompt. How can I accomplish this? I cannot allow a prompt and I do not want to have to use keys.
I'm going to be using this command to back up my files:
Should I change anything or is it ok?
I'm using rsync for a backup-sript at the moment and want to keep all files. The files are always unique, so I want to rsync without delete any file on the destination.
I've tried with --no-delete and --max-delete=0 but nothing seens to work. Is there even a possibility to do so?
I am looking for a way to control the order in which files (or more specifically, certain subtrees) are updated for a very large rsync update done daily. There is a finite time frame and finite bandwidth to do these updates, and they usually do NOT complete. I want to control the order so that more important files are copied earlier.
What I have been doing is breaking up the rsync runs into a few steps where the first step is a very limited set of subtrees using --include and/or --exclude, and each step thereafter is increasing the amount, until the last step is the whole tree to be updated. This is not working because the time it takes to transfer the file list is large (varying from 10 minutes to 40 minutes depending on step). I really need to get this back down to doing a single rsync run to reduce that wasted time.
I cannot ultimately split things because there are a lot of hardlinks that need to be retained (or else the destination space and bandwidth would be exceeded).
I was looking at rsync's batch mode. My thought was if I could get a list of what needs to be updated, I could then sort that list and do just 1 more rsync run to do the updates in the order I want. But it turns out the batch mode file is in some cryptic binary format. I really have no idea if that could have worked.
The order selection is not simply based on the names in the top directory, so I cannot simply just list those in the preferred order on the command line (otherwise, yes, that would control the order). If I do list directory names from various depths, that list gets flattened into the destination directory, which would not work to replicate the whole tree exactly.
I am using rsync to backup data from a file server to a external device, the data is accessed from windows boxes via samba with their own usernames, user1, user2 etc... Rsync is failing to copy the file permissions when sending to the external device, I have tested sending to a ftp server and a usb hard drive and received the same error, see below:
Code:
root@Fileserver:~# rsync -avz --delete-after /shared/fileshare/ /backup/backup
building file list ... done
./
manager/
rsync: chown "/backup/backup/manager/cronman.txt" failed: Operation not permitted (1)
public/ .....
sent 339 bytes received 104 bytes 886.00 bytes/sec
total size is 4593 speedup is 10.37
rsync error: some files could not be transferred (code 23) at main.c(977) [sender=2.6.9]
When I backup to a local directory the permissions are copied correctly.
I am unable to transfer file using rsync. I have installed rsync and ssh package.. When I try to transfer file , its giving an error rsync -v -e ssh a usw3tvx@lalqau40:ssh:lalqau40: Name or service not knownrsync: connection unexpectedly closed (0 bytes received so far) [sender]rsync error: error in rsync protocol data stream (code 12) at io.c(463) [sender=2.6.8]I tried with simple format... nothing works...Even I couldnt ssh with hostname. However I can connect with Ip address... I
View 1 Replies View Related