I have a newly installed Dell Optiplex 755 with Ubuntu 10.04 (64-bit) and I am having serious issues with network copying. Whenever I start rsync och scp with larger amounts of data my system becomes practically unresponsive (all types of apps grey out) until those processes are completed. Cpu stays at below 10% and I have lots of free memory and such .
View 3 RepliesI ran Karmic until last week, and noticed that it seemed to be getting slower. I was hoping a clean install of Lucid would fix that. it's worse. If I leave my machine running for a few hours, something begins dragging down the performance to a crawl.
System Monitor sometimes won't come up - launches an empty grey window. Other times, System Monitor reports that gnome-system-monitor is consuming 86-96% CPU. This is a dual-core Dell e6500 laptop with 4 Gigs of RAM, it should fly. Any ideas on troubleshooting this?
I upgraded to 10.04 from 9.10.
Network accesses are now dramatically slower.
I upgraded a copy of my previous 9.10 installation so I can still use 9.10.
Network does work with 10.04 but is extremely slow. Website loading is initially quite fast but then slows to a crawl. Second and third tabs appear to stall completely, waiting for the first tab to finish.
All of this works just fine under 9.04.
Toshiba M45-S165
1.5 GB RAM
I've got a Lenny system on a quad core system with 4GB of ram. Currently Linux www 2.6.26-2-amd64 #1 SMP Thu Nov 25 02:05:44 UTC 2010 x86_64
I've had it for a couple of years, and its been fine, very fast and reliable.
Lately, its got a problem: after its been running for a day or two, access using SSH, or TLS/HTTPS gets really slow. Painfully slow. A reboot returns the system to normal fast operation.
I can't think of anything that has changed, other than I got a new SSL cert and installed it.
The system is mostly a vanity domain server, with Apache HTTPD running a half dozen very low volume services. Its running bind9, mysql, and a few other things, nothing big, nothing that would load down a quad core with lots of ram. The disk is only 14% used.
My system slows much down so that i cannot use it at all when am watching any online videos or movies am using firefox 3.6.13 .
My spec is:
intel celeron 2.53 Ghz
512 Mb ram
ubuntu 10.04 on 40 GB seagate ide harddisk
Windows Xp on 80 GB western digital ide harddisk in dual boot.
And also my browser cannot open login page of yahoo site and some time it waits saying waiting for s7.addthis.com, or waiting for ad.yieldmanager.com like some thing when i am trying to open some sites
I have a Actiontec m1000 modem from Qwest that seems to offer more speed to my brothers Windows PC then my Linux PC (same make and module) I checked the ping on all systems, and they all look fine. The only thing that seems different is the number of open sessions in the modem, Linux only uses 2, and Windows uses 80. My guess is that the modem uses sessions to control the bandwidth, but I don't know how to add sessions to Linux networking to compete for bandwidth. I did test in the same browser.
View 4 Replies View RelatedI'm having to rebuild my home server following a failed upgrade from 8.04 to 10.04 . All data lives in /shared on this server, the contents of which are mirrored weekly to a USB HDD which is mounted at /backup, using rsync:
sudo rsync -av --progress --delete /shared /backup
I recovered the contents of the backup drive to the rebuild server's /shared directory using the cp command with the archive flag set to preserve ownership, timestamps etc. Everything looks fine to me. However, when I do a test rsync (adding -n to the command above) then it looks like rsync wants to recopy everything, and I'm at a loss to see why. For example, below is a test on the subdirectory /shared/backgrounds. The file attributes look identical:
Code:
chris@quadra:~$ ls -la /shared/backgrounds
total 8112
drwxr-xr-x 2 chris chris 4096 2009-04-12 11:06 .
[code]....
Is there some condition that rsync can detect that I can't see? Is rsync sensitive to the way the HDD is mounted? (the USB HDD is actually mounted at /media/backup, and /backup is a soft link to this.)
I noticed that sometimes my Linux server will randomly start to lag really badly, to the point where even a http request takes forever. It is an Intel Core2Quad with 8GB of ram running FC9. This is my "Everything" server so it does file, email, DNS, web (for local stuff only), VMs and so on. There are about 5-6 VMs running on it at any given time. I manage it through a VNC session and have some SSH consoles within that session. This way if I reboot my PC I don't lose all my SSH consoles. If I need to SSH to any server I do it from there. I treat it kinda like a terminal server to some extent.
When this slowdown happens, top is not really useful because I also do F@H so that will always be to the top, but it's low priority. The VMs are also always near the top. This does not change whether it's slow or not, so when it's slow, I have nothing to go by on how to troubleshoot. The load does seem to skyrocket though. Right now it's doing the slowdown thing and it's at 8.09. Normally it's at around 3 which imo is good as it it's under 4. I have 4 cores so anything more than 4 means it's queuing. At least that's how I understand it. This is the output of top:
Code:
[root@borg ~]#
[root@borg ~]#
[code]....
I just installed a new HD on my system with multiple HD's already. I have a drive with two versions of Ubuntu & would like to copy the complete drive to the new drive along with all the contents & partitions of the Ubuntu drive.
1 - Could I partition the new drive & just copy the contents using rsync?
2 -If I copy all the contents over could I just reinstall Grub & edit fstab & be good to go?
Using Ubuntu Server 10.04 LTS. I'm new to Ubuntu and testing Rsync. I successfully copied 3TB of data from a Win7 machine to an MDADM Raid5 array. All appears to be fine. Used a Win app for the copy. I then deleted a 250GB folder on the Raid5 array and recopied the data using Rsync. Rsync was executed via a Putty session on a WinXP machine. The source was an eSata attached drive (same drive used for the big 3TB copy) and the destination was the same Raid5 array. That copied just fine. I bit verified it with a Win7 app. Perfect.
I then used the following Rsync script to copy a single 26GB file from that same eSata drive back to (what I intended to be) the Raid5 array:
Code:
neil@ANTECUBSV:/mnt$ rsync -r -a -v -e ssh --delete /mnt/disk1/Test/ /mnt/Test
sending incremental file list
created directory /mnt/Test
./ C_VOL-S300-b001.spf
sent 3020267622 bytes received 34 bytes 50760800.94 bytes/sec
total size is 3019898880 speedup is 1.00
neil@ANTECUBSV:/mnt$ cd raid
Note that only about 3GB copied. No error messages were posted to the putty session. I made a mistake in the Rsync command, creating the Test folder directly in the mount folder rather than the Raid array, as I intended. That is a little strange, yes, but I would not think it would cause a partial copy? The /mnt folder is on my system drive, which had about 34GB available space before the copy, so comfortably would have had 6GB or so after.
The eSata disk is mounted as /mnt/disk1
The Raid5 array is mounted as /mnt/raid
I then recopied the file to the correct intended destination on the Raid5 array, which has about 400GB free space (plenty).
Code:
neil@ANTECUBSV:/mnt/raid$ rsync -r -a -v -e ssh --delete /mnt/disk1/Test/ /mnt/raid/Test
sending incremental file list
./
deleting 2010-07-05 Backyard Birds/Thumbs.db
deleting 2010-07-05 Backyard Birds/
C_VOL-S300-b001.spf
sent 11105775462 bytes received 34 bytes 50366328.78 bytes/sec
total size is 11104419840 speedup is 1.00
neil@ANTECUBSV:/mnt/raid$ df -h
Note that only about 11GB was copied, and this was confirmed with an ls -l command. Now I am correctly copying the file to the Raid array but it is still incomplete. I then copied the file back to the /mnt folder to see if the problem reproduces:
Code:
neil@ANTECUBSV:/mnt/raid$ rsync -r -a -v -e ssh --delete /mnt/disk1/Test/ /mnt/Test
sending incremental file list
created directory /mnt/Test
./
C_VOL-S300-b001.spf
sent 26327927554 bytes received 34 bytes 56558383.65 bytes/sec
total size is 26324713984 speedup is 1.00
neil@ANTECUBSV:/mnt/raid$ cd /mnt/test
This time I got my full 26GB file. Why I might be getting inconsistent results? This is quite troubling of course. I'd also be interested in basic a command line Linux diff app (that does file directory as well as bit level checking) if one is available.
How can I use rsync to copy ONLY the my home folder (and nothing inside of it, just the folder name) to another machine. I've tried things like
Code:
rsync -av /path/to/src /path/to/dest/
or
Code:
rsync -av -f"+ */" -f"- *" /path/to/src /path/to/dest/
This last option recursively (through the -a switch) copies only folders, including all subfolders. If I try
Code:
rsync -v -f"+ */" -f"- *" /path/to/src /path/to/dest/
nothing is copied (not even my home folder.
This command would copy the files to the local directory,find /mnt/nas -type f -ctime 1 -iname '*.avi' -exec rsync -av {} /mnt/Mythbuntu
View 1 Replies View RelatedI'll often run iceweasel for watching videos via flash (hulu, ....., et cetera) while working, in fedora I never noticed any performance problems, watching the video while using chrome for browsing, gedit for editing, et cetera, everything was responsive. Though, now on debian running iceweasel for the same purpose hogs memory and processor time, which slows down the responsiveness of the system as a whole, making it difficult to use other programs.I'm curious how I should go about troubleshooting this issue, and what diagnostics might help me find the solution.
View 4 Replies View RelatedI have a WD world book edition 1TB NAS drive, and just purchased an acomdata 1tb drive and connected it to the NAS via USB. If I recall I think the WD NAS has a ext_ or some type of linux filesystem on it, and the acomdata has a ntfs filesystem on it.
What I want to do is copy over certain directory trees of the NAS to the USB attached drive. I usually use MS synctoy to sync folders from my windows pc to the NAS drive, and MS richcopy to make the initial transfer from PC to NAS. For this operation though, since it is taking place entirely on the NAS and its connected drive, I thought that rsync would be the best option, and it is available on my NAS drive.
Last night I entered in rsync -avr /movies/* /usb1-1share1/ to copy the entire "movies" dir to the drive, which shows up as usb1-1share1 on the NAS drive. It copied most of the directory tree ok, but a lot of the folders were empty, so this morning I tried rsync -Carv --ignore-existing /movies/* /usb1-1share1/ to try and get all the files missed, without recopying the 24GB that did make it across. This also managed to copy a few more GB over, but not everything.
I am running the command from an ssh session on the NAS using putty on my PC, in as user "admin" which should have all rights over these folders. There is a bunch of errors in the command window like this: rsync: failed to set times on "/shares/usb1-1share1/movies/classics/fulldvd/First Blood DVD/.VTS_01_2.VOB.RxdjWZ": Operation not permitted (1)
I want to restart another session and get the files it missed, but I want to find out what I am doing wrong first. Should I be doing this as root user? am I missing some switches or just plain doing it all wrong?
I have recently purchased an external hard drive in order to backup my home partition. In my PC I have a "1.5T" drive with several partitions on it, containing OSes and the home partition. The home partition is 1.3T according to df, the external drive contains one partition that spans the entire disk,df reports it as 1.4T in size. Both partitions are ext3. When I use rsync to copy files from the home partition to the external partition, the external disk becomes full, despite the destination - supposedly - being larger than the source. I don't understand why copying files from one partition to a slightly bigger partition should need more space than on the source partition. Does anyone know what is happening ?
Details : I created the partition on the external drive with gparted; gparted reported it the already have several gigabytes in used space immediately after the partitions creation - I thought at the time that this must be normal. The home partition contains many files of all sorts, including lots of big audio and video files. If you are wondering, for all my important files this external disk is only secondary backup, as they are also backed up to the "internet".
These are the mount points :
/mnt/tmp/ : home partition, /dev/sdb6
/mnt/external/ : external partition, /dev/sdc1
I used rsync to copy the files, I know there are more efficient ways to do this, but I wanted to use the same command that I will subsequently run to sync the backup.
rsync -av --progress --stats --recursive --perms --links --delete /mnt/tmp/ /mnt/external/
Next I tried adding the --sparse switch, as I was wondering if the problem may come form sparse files. I don't know however if rsync would go back and shrink the sparse file by just adding the switch and executing the command. I also added --one-file-system, for good measure. Here is what I ran next :
rsync -av --progress --stats --sparse --one-file-system --recursive --perms --links --delete /mnt/tmp/ /mnt/external/
I tried an fsck on the home partition :
fsck -f /dev/sdb6
This is the output from the last rsync :
rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32)
rsync: write failed on "abcd.avi": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(302) [receiver=3.0.6]
[code]....
Looking at the destination after a partial copy seems to indicate that the problem is not symbolic links being "expanded". I have not checked the source filesystem for sparse files, nor the destination to see if these files could be larger there, as this does not seem trivial.
Here is some additional info :
$ df /mnt/tmp/
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdb6 1415342836 1414173740 369096 100% /mnt/tmp
[code]....
On my Ubuntu box, I have a mounted windows share connected via gvfs called graphics. I want to backup everything on a nightly basis from graphics to backupserver/graphics . If I use rsync, it will not copy files that have parent directories with funky characters in them (but the directories themselves will be copied!). Everything else gets rysnced just fine.
graphics/test/macdir/picture.psd
...when rsynced over to ...
backupserver/graphics/
gives the error:
rsync: mkstemp "/home/administrator/.gvfs/drobo on x.x.x.x/linux_backups/graphics/test/ macdir/.picture.psd" failed: Operation not supported (95)
The directory macdir gets created but there is nothing in there. This happens for all files underneath dirs with funky names. cp -Rf works perfectly! Directory and child files all get copied over no matter how strange the characters get in the directory names.
I am trying to create a simple bash script to rsync some folders within a directory stucture. I am using wild cards, in the rsync source directory structure, but my command always fails. I believe it is the way I am using wild cards within my for loop. Here is my command ;
Code:
for seq in `cat test.txt` ; do rsync -nvP /folder/folder/folder/folder/folder/**/$seq /folder/folder/folder/ ; done This always fails, where if I do a ls to the destination, to test the path, it always works.
when I installed 13.37 I created a local copy of the entire stable tree (source/ and all the rest) just to have all that stuff around to browse offline.
Now, to instruct myself, I'm trying to use rsync to keep this stuff up to date. But I seem either to have misread the rsync man page or ... well, I don't know. I am issuing the following command and getting the results seen below:
Code:
I've got quite a decent rsync script setup, however I'd like to invoke it whenever there's change to a file. My initial idea was to use find, however this has two major flaws - the first being my particular unix veriant cant understand -print0 which means this doesn't work, the second is that I'm not 100% sure how to put variables into quotation marks so ls can understand the target:
Code:
for i in `find /shares/ -mtime -1 -print`; do ls -ltr $i;done
I want to run rsync on server A to copy all files from Server B when they are newer than 7 days.(find . -mtime -7) I don't want to delete the files on Server B.
View 2 Replies View RelatedOn Linux I do:
rdesktop remotepc
How do I copy and paste between my local system and the remote system?
Just tried to rsync /repositories/server:/monitoring for SLE 11.1 which is visible through http [URL], but when I browse through [URL] the SLE_11.1 directory isn't there.
View 3 Replies View RelatedI often have to transfer huge data over our LAN from one computer to another. The size of the files varies and can be somewhere from 2 GB to 50GB or more! The only accepted connection protocol between machines are ssh; and rsync and scp are the only options available for copying over network (unison is not installed). I usually use rsync with "-z" option to copy over network. if "rsync -z" is faster than "scp" for data transfer?
View 5 Replies View RelatedI believe it was rsync was the tool. I have a box running CentOS 5. It has a 250GB HD in it. I have another drive with the exact model. Currently it has a as a said a 250GB IDE drive. I want to shutdown the machine, install this other hard drive and set up a cron job that will backup my main drive at times that I set. This way if the main drive fails, I do not loose all the data and have to rebuild the server from scratch as I have been custom configuring it for years. I can't remember if there was an issue with the main drive being mounted to back it up or not. I have looked at some of the how's on rsync but they seem to only talk about using another server for this. If I shit down the box, install the new drive, and the box boots back up, is it going to ask about that drive or what do I need to do to get rsync going and does it partition the drive as such? And can I do it this way. This way if the main drive fails, I can just swap the drive and be on my marry way.
View 2 Replies View RelatedA bunch of hosts all connected to each other.I'd like to have a directory where a group of hosts can dump files and all access, synchronising with each other their changes automatically and transparent to the user.How can I have it so that rsync updates both ways?
* host A asks host B for a list of files, modification dates, checksums
* then applies the rules to select which files it will fetch from B
--> if A owns the file then do not get modifications from B
--> files differ, then select newer file
* then fetches them
* and so on for each host...
I have a Linux box, running rsync, only problem is, I have to turn it off at night. What I want to do is turn it on and when it turns on, rsync will already be running in daemon mode rather then having to run rsync daemon. Does anyone know the solution? I'm sure its simple, I just cannot figure it out
View 3 Replies View RelatedThought I'd post it here because it's more server related than desktop... I have a script that does:
[Code]....
This is used to sync my local development snapshot with the live web server. There has to be a more compact way of doing this? Can I combine some of the rsyncs? Can I make the rsync set or keep the user and group affiliations? Can I exclude .* yet include .htaccess?
When I run rsync --recursive --times --perms --links --delete --exclude-from='Documents/exclude.txt' ./ /media/myusb/
where Documents/exclude.txt is
- /Downloads/
- /Desktop/books/
the files in those directories are still copied onto my USB.
And...
I used fetchmail to download all my gmail emails. When I run rsync -ar --exclude-from='/home/xtheunknown0/Documents/exclude.txt' ./ /media/myusb/ I get the first image at url.
I have automated backups running on my ubuntu box using rsnapshot (rsync) basically following this tutorial. My concern is how to restore if everything is lost. Since it does not seem that I can backup the entire drive and that I have to choose individual folders and some like proc/ cause problems.
Currently in my rsnapshot.conf I have (see below). Is there a way to just clone the entire drive? Or should I not do this? Questions:
1, Can I backup in a way such that it is a clone of the drive so that it can be swapped with the current drive?
2, If not yes to clone, If I had a total drive failure would I install a basic ubuntu and then replace all the files with the backups?
3, Anything I might not know about that I should.
[Code]...
I regularly rsync directories to an external usb drive. Recently I've started getting errors which look:
blah blah:Read-only file system (30)
The external usb drive has two partitions (ext3 and fat32) and I rsync to the ext3 one. The rsync command is
rsync -ahi --delete /home/mike/some_dir /media/usb_drive
People have reported this error with faulty usb cables but everything was working fine until recently so it seem unlikely a usb cable would suddenly wear off.