Server :: Very Slow Raid Performance / Only Write Speed
Jan 5, 2011
I have recently migrated my file server over to a HP Microserver. The server has two 1TB disks, in a software RAID-1 array, using MDADM. When I migrated simply moved the mirrored disks over, from the old server Ubuntu 9.10 (server) to the new one 10.04.1 (server).I Have recently noticed that write speed to the RAID array is *VERY* slow. In the order of 1-2MB/s order of magnitude (more info below). Now obviously this is not optimal performance to say the least. I have checked a few things, CPU utilisation is not abnormal (<5%) nor is memory / swap. When I took a disk out and rebuilt the array, with only one disk (tried both) performance was as to be expected (write speed >~70MB/s) The read speed seems to be unaffected however!
I'm tempted to think that there is something funny going on with the storage subsystem, as copying from the single disk to the array is slower than creating a file from /dev/zero to the array using DD..Either way I can't try the array in another computer right now, so I though I was ask to see if people have seen anything like this!At the moment I'm not sure if it is something strange to do with having simply chucked the mirrored array into the new server, perhaps a different version of MDADM? I'm wondering if it's worth backing up and starting from scratch! Anyhow this has really got me scratching my head, and its a bit of a pain! Any help here would be awesome, e-cookies at the ready! Cheers
I wasn't sure where to post this question so administrators, feel free to move it.I have a media server I set up running Ubuntu 10.4 Server, and I set up a software raid 5 using 5 Western Digital Caviar Green 2TB 7200RPM 64MB drives. Individually they benchmark (using the Ubuntu's mdadm GUI (pali?somthing...) at about 100-120mb/s read write.I set the raid 5 up with a stripe size of 256kb, and then I waited the 20 hours it took to synchronize. My read speeds in raid are up to 480mb/s, but my write max is just under 60mb/s. I knew my write performance would be quite a bit lower than my read, but I was also expecting at least single drive performance. I have seen other people online with better results in software, but have been unable to achieve the results they have gotten.
My bonnie++ results are more or less identical (I used mkfs.ext4 and set the stride and stripe-width).The PC has 2048mb of RAM and a 2.93Ghz Dual Core Pentium (Core 2 Architecture), so I doubt think that's the bottle neck. These drives are on the P55 (P45*) South Bridge SATA controller.
I had a F14 samba server working flawlessly now for a quite time. Unexpectedly about a month ago the write speed slowed down around 64kbit/s and that's really slow. I haven't changed any configs, only updated the server. I've troubleshooted this from many angles: -It is not a network issue because sftp writes are good. -It is not a RAID1 issue because write speed from another directory (locally) are good. So basically samba is the only thing left.
Here's example from smb.conf: [MyShare] comment = My share path = /mnt/RAID_share writeable = yes browseable = yes writelist = +smbuser
Here's a snip from logs today: [2011/08/04 09:04:27.509273, 1] smbd/server.c:267(remove_child_pid) Scheduled cleanup of brl and lock database after unclean shutdown [2011/08/04 09:04:47.526336, 1] smbd/server.c:240(cleanup_timeout_fn) Cleaning up brl and lock database after unclean shutdown [2011/08/04 09:17:57.634660, 1] smbd/server.c:267(remove_child_pid) Scheduled cleanup of brl and lock database after unclean shutdown [2011/08/04 09:18:17.635196, 1] smbd/server.c:240(cleanup_timeout_fn) Cleaning up brl and lock database after unclean shutdown .....
I am new to Centos and linux in general. I have just got myself a Dell 1950 server with 2x 1T SATA2 hard disks in it. now the server comes with a PERC5i Raid card with 512Mb. Well I put these in raid 0 and the raid card initilzed the disks to 128 writeback and read ahead. When I loaded centos it did not recognize the dell layout so therefore wanted to initalize it again, so I done this as it wanted. Now I created a 80gb boot and o/s partition and a 100gb swap the rest was created into LVM space to run solus vmwear. But I found the raid 0 to be getting extreamly slow read and write speeds.
Example same disks Desktop PC max 214mb/s windows vista 64 Server with centos 104mb/s
Now I am not sure but I am told that I need to align the o/s with the raid card settings but I have no idea how to do this. How to do this in plain easy step by step instructions. I mean how to calculate it, how to format the disk this way, and what files to edit where if needed. I have spent hours trying to figure out why my raid 0 is slower than a single disk.
I'm new to openSUSE and my computer is quite slow although my computer isn't that bad. I opened up ksysguard and it appears that my CPU is the bottleneck. My CPU usage is usually 100%, then after a few seconds, it goes down to 20-60% and then it goes back up to 100% after another few seconds. It says I have 141 processes running (I don't know if that's normal or not).
My Specs are: CPU: AMD Duron (tm) processor 1.8GHz Graphics Card: NVIDIA GeForce 6200 Memory: 2GB RAM I'm using KDE.
I'm currently experiencing some serious issues with WRITE performance on a RAID-1 array. I'm running Ubuntu 10.04 64 bit server with the latest updates. To evaluate the performance ran the following test: [URL]... (great article btw!) Using dd to measure, write performance is only at 8.7 MB/s. Read is great though at 74.5 MB/s. The tests were ran straight after rebooting and I have not (YET!) done any kernel tuning or customization, running the default server package of the Ubuntu kernel. Here's the motherboard in the server: [URL]... with a beta bios to support drives over 300GB.
[code]...
As you can see from the bo column there is definitely something stalling. As per top output, the %wa (waiting for i/o) is always around %75 however as per above, writes are stalling. CPU is basically idle all the time. Hard drives are quite new and smartctl (smartmontools) does not detect any faults.
So I have been doing some RAID 5 performance testing and am getting some bad write performance when configuring the RAID with an even number of drives. I'm running kernel 2.6.30 with software based RAID 5. This seems rather odd and doesn't make much since to me. For RAID 0 my performance consistently increases as I add more drives, but this is not the case for RAID 5. Does anyone know why I might be seeing lower performance when constructing my RAID 5 with 4 or 6 drives rather than 3 or 5?
I started a thread in the hardware section, but I've narrowed it down to being a software issue. The problem is that all USB devices are written to at about 1MB/s, and is read at equally slow speeds. After testing my computer on an older Ubuntu release (8.04 32bit), I verified it wasn't a hardware issue, because the read and write speeds were like they were supposed to be. Then, I downloaded the 32bit release of the Ubuntu version I'm currently using (an up-to-date version of Natty) and booted the live cd of it. It was equally slow there as well.
The read/write speeds are equally poor on openSUSE and Arch Linux, so I don't think that it's an issue with Ubuntu, as much as it seems like a kernel issue (or maybe not, you guys can correct me if it seems like it isn't ).
I got a new machine with GA-p55A-ud3 mobo and a WDC WD10EARS 1T disk. When I tried to benchmark the disk IO, I was suprised by the low write speed:
[Children see throughput for 1 initial writers = 35962.63 KB/sec Parent sees throughput for 1 initial writers = 35962.63 KB/sec Min throughput per process = 35962.63 KB/sec
I am hoping someone can help me I setup a brand new ubuntu 11.04 64bit system with:-AMD Phoenom 840 (3.7GHz overclocked) 8GB DDR3 RAM.WD20EARS x 4madmin (RAID5) SetupFor some reason I cannot get the write speed above 15mb/s & I have tried modifying the cache_Stripe_size from 256 to 8192.My read speeds are great burst at 120mb/s & stable at about 60mb/s.The only other possiblility is it could be my HDs which I read causes slow writes not sure if thats true though. point me in the right direction so I can fix this issue as I don't want to move to the dark side (Windows) .
I just tried NFS for the first time after reading that it's considerably faster than SSHFS, which I currently use, but I'm experiencing slow write speeds and problems while copying files in nautilus.
I have a 4 drive RAID 5 array set up using mdadm. The system is stored on a seperate physical disk outside of the array. When reading from the array its fast but when writing to the array its extremely slow, down to 20MB/Sec compared to 125MB/Sec reading. It does a bit then pauses, then writes a bit more and then pauses again and so on.The test i did was to copy a 5GB file from the RAID to another spare non-raid disk on the system average speed 126MB/s. Copying it back on to the RAID (in another folder) the speed was 20MB/s.The other thing is very slow several KB/s write speed copying from eSATA drive to the RAID.
I just wanted to know if having my laptop set to ondemand, will this affect performance in any way? I realize it increases the clock speed to performance when the CPU is under load, but does the time it take to go from ondemand to performance affect speed? Will there be any noticeable difference between the two setups? I have a dual core intel at 2.2GHz when in performance. When ondemand is set with no load it downclocks to 800Mhz.
Compared to my laptop notebook with a HD of 5400rpm, the write performance of raid1 on an ubuntu lucid server is unacceptable. In the begining, I installed ubuntu 9.04 server(alternate) using raid1 with two WD 1TB HDs of 7200rpm(Green Power) and then performed dist upgrade to 9.10 and then to 10.04.
I guess the write performance initially was reasonable since the installation and data migration(copy from another computer over LAN) didn't take too much time. However, after upgrading the server to 9.10 or so, I found large file upload through samba or ftp tends to block and time out. It is of no use whether to change the daemon or the client program so that I tried to test the read/write performance on the server to figure out the situation.
To my surprise, using strace I found even a simple program like cp would easily get blocked eventually in a write() system call for decades of seconds. Hence, I perform another disk writing test using dd for data size ranging from 50MB to 1GB. Performance test commands are listed as follows:
if the data to write is equal or fewer than 150MB, the command returns immediately at very hight speed but the raid disks starts to sync and busy so that the terminal prompt seems to freeze. I think this behavior is normal under the raid1 configuration, isn't it?
But when the data size is equal to 200MB, the test command blocks for seconds and the write speed is measured at about 16.6MB/s. Of course, the raid disk still starts to sync and busy afterwards. Next, I test writing with data of size 1GB. The command blocks so long for about 770 seconds(<2MB/s) while the same test runs for only 17.49 seconds(60MB/s) on my laptop.
I also burn a Lucid LiveCD to boot the server and mount the raid device to run the test again but the results remain similar. Does that means even I re-install the system on the raid, the problem never disappears?
PS: the disks run under the mode of UDMA6 without change.
It stores all my important stuff, as well as some music and movies.I use a second linux box in my living room to "stream" content via NFS or SAMBA share.The streaming tends to stop several times during playback, and needs to fill up its buffer again before continuing to play.I also have some Windows XP and 7 based computers that connect to this file server.I have noticed that directory listing is VERY slow, and there is a huge lag when I want to save/read a file from/to my home directory.
This is my setup:Ubuntu Server 10.10 64 bit (I have the same problem with 32bit ubuntu) 3 RAID5 arrays with 4 hard drives eachLVM on top of the 3 raid5 arrays.The Logical Volume i use is about 6.5TB, and I use the ReiserFS file systemThis LVM has grown over the years, and has had som replaced disks. So I have used the pvmove, and extend commands a bit.I have tried using IOTop and top to check if there is not enough resources available, but that doens't seem to be the problem.I haven't been able to find out why streaming over the network stops, but I know it is the server that causes the problem.Does ReiserFS have any performance problems with large logical volumes? Would changing to EXT4 or some other FS give any performance gain?
I have just formated and did a fresh install to start all over. I installed Ubuntu 9.1 and everything went fine but the network. The computer serve as a NAS and MediaPlayer. When getting a file from the server, I get a speed of 40-60mb/sec, which are good for a Gigabit network. When I try to put a file in, the speed can't pass more than 1mb/sec and often stop.
Timing buffered disk reads: 12 MB in 3.01 seconds = 3.98 MB/sec
I have looked around but do not understand the complexities of rebuilding kernel etc so would need some assistance. I have checked IO_support and it is set to 0
Code:
Here are the results for hardware installed after a quick lspci -v
Code:
If there are any noticeable incompatibilities here and how I could work to speed this up. I only have SSH access and could possibly get a KVM on it as it is currently locked up in a DataCenter.
I have set up a file server (Ubuntu Server Edition) for our lab. People can connect to common Samba file shares from their personal laptops/desktops, which run either Windows Vista or Mac OSX. The guys with OSX have upload/download speeds of ~2 MB/s, while the Vista machines are slogging away at ~200kb/s for downloads and ~400kb/s for uploads. In both cases, the connection are through wired ethernet ports which should function identically. Since the Macs work fine on the same network, I assume this is a Vista issue.
I have tried troubleshooting one of the Vista machines by: 1. Turning off the Remove Differential Compression feature 2. Disabling autotuning following these instructions 3. Adding a registry key following the same link above.
But nothing has improved. Anybody have any advice on addition tweaks to the Vista machine? Is there a chance that this is actually a server-side/samba issue?
I have ClearOS (CentOS) installed. I have 2 x 2TB SATA HDDs (hda & hdc). At installation time, I configured a RAID 1 (and LVM) between the two HDDs. After a power problem happened, the two HDD were re-syncing and I checked it using: watch cat /proc/mdstat The speed didn't exceed 2100 KB/s I tried the following with no change:
When i try to copy files from my samba server (Ubuntu 9.10) to my windows seven media center, the speed is extremely slow. So slow that is better to download 100mb file from the internet, than from my lan. And on my lan, every card and switch is at 1000 mbps speeds on cat5 cables. And from XP or other linux machine
I'm looking to set up a server with attached mass storage device and tape autoloader to run linux. It's set up under Windows at the moment. Goal is to have users, connecting from individual workstations and laptops, backup their data to the linux server. On their personal machine, some users run linux, some MacOS, some Windows. I plan to set up the 5 500 GB drives as RAID5. I understand that if setting up as software raid the format is "physical volume for RAID". Under this setup, will Windows users be able to read/write and function as expected? I can't assume only linux user access.
I have recently upgraded to a M4A77TD motherboard and 8Gig of memory.I have a AMD Athlon(tm) II X4 645 Processor, and I cant say that I see much improvement over the one that died. I have noticed that the HDD light is constantly flashing so I wondered if I couldnt use some of the Ram to create a ramdrive to speed things up . So ..my question ... what should I put on my ramdrive? My own thoughts were the var and usr directories. I would load the ramdrive at startup (which would slow down my startup) And Rsync back again on shutdown.Has anyone done this? Ive had a look through the forum but cant see anything ..but my search terms may not be all that good.
suse linux 11.1 64 bit var=0.3G usr=5M lib=0.2G opt=0.6G sys=0.6G As far as I can see swop is never used
I have A dell vostro v13. I am have installed LMMS which runs ok if I put minimal demands on it but if I start putting to many demands on it like running More then three Zynaddsubfx plugins or multiple sample players the cpu moniter goes into the red and it starts to sound bad and slow down a great deal. I know a small laptop is probably not the best choice to run a DAW on unless it has a good processor but this is the only computer I currently have and I really like LMMS. Is there any way I can get more performance out of my computer? I looked a little into overclocking but dell Bios Does not allow for this. Is there any tricks that I can do to get LMMS to perform better given the limitations of my system?
I have an older PC running Ubuntu 10.04, and the system is slow, especially with Firefox running. I recently upgraded to 2 gig memory, but it didn't help. I have plenty of space on my hard-drive, using only 30 gig of 80 gig. My CPU is Intel Celeron 2.40GHz. I have broadband, and the speed seems good. What are the system requirements for Ubuntu 10.04? Any tips for speeding performance, esp on Firefox? I used to do a defrag when I had Windows; is there any need to do that on Ubuntu also?
My desktop tower is pretty old and runs horribly slow nowadays. I only use it for a little internet browsing and to upload photographs etc. It runs on 1Gb RAM and has the celeron(R) 2.4GHz processor. What is the best way to boost overall performance speed/response times? would a better processor be viable?
I built a RAID5 storage array using 'mdadm' on 3x WD Green 1TB hard drives. I used the Disk Utility GUI to create the array and it took about 24 hours to build. When I started copying files to it I noticed it performed at decent speeds for a while, then got really slow, the sped back up. Just for laughs I ran the Read-Only Benchmark function in the Disk Utility and got a graph that would even confuse stock brokers.Any thought on what the issue might be? I tried searching around for an answer, but most people are only affected by wirte performance issues not reading.
I've been having some problems copying files to USBs. If I'm copying a large (100MB+) amount of data, at random points the transfer will just stop for 30+ seconds before continuing. Sometimes it doesn't start up again at all. Consequently, the write speed drops to less than 1MB/sec, sometimes as low as 100 KB/sec. I do not have these problems on Windows 7, where I achieve speeds of ~16 MB/sec easily. I have had the same results with several USBs (2-32 GB) on several file systems (fat32, ext2) with several different computers running fully patched versions of Ubuntu 10.04, which suggests the problem is related to the way the OS accesses the hardware.
First of all, apologies if this is not the correct place to post this. I recently installed openSUSE 11.2 (Gnome, 32bit) on my newish PC. It is an Acer Aspire X3200 (AMD Phenom X4 Quad-Core, 1.8 GHz, 4096 MB, NVIDIA GeForce 8200 Integrated, can provide detailed specifications if needed). I've to say I am very disappointed with the results so far! Sorry to say this but I'm having a whole load of problems that I never had when I had Vista or with current Win 7. Here are the issues:
1. Slow network performance. I've disabled ipv6 but it is still the same. Not only when I use Firefox, it is also slow when I use console based programs. 2. Instability - Gnome system monitor crashed 6 (or more) out of 10 times I use it. System Monitor users more CPU resources than anything else! Also when I started the PC this morning, I only had the desktop wallpaper. No icons, no taskbar, no mouse movements or menus, nothing. I had to force shut down it after a while. I'm sure we all are familiar with this on XP but I've never seen something like that on Linux!
3. Annoying noise coming every time I move mouse (mainly when scrolling) and even if I'm not doing anything. I suspect it is coming from my HDD indicator as it always blink even when nothing is running. This can't be a hardware issue as I've never had this on Vista or Win 7 (it is virtually silent on Vista and XP but not on SuSE) This noise is driving me totally mad.
4. Slow performance in general. No matter what I do, even typing this on this text window, it seems everything is running so slow. I never had this problem when I was running 10.x on my 6 year old PC. CPU is always running around 20%-30%, when I don't run any other applications but just Firefox (I have Foxclocks, Delicious and FEBE as add-ons and I don't have any problem using them on Vista or Win 7). I am using Compiz, not sure if this has got anything to do with that.
I have a small SDD system disk and a large data disk on a server based on fedora core. It is organized this way so that the data disk can be spun down most of the time, reducing noise and heat.I'd like some way to periodically create a bootable backup of the system disk on the data disk, so that if the SDD goes belly-up, I lose minimal data and can very quickly bring the system up.Some thoughts:1. I know I can create a partition on the large disk and make the system boot from a raid0 mirror.However, the constant writes to the system disk (e.g. by the journal daemon) will ensure it never spins down.2. I can do a periodic format and cp -ax to the "copy of system" partition, but that doesn't leave the disk bootable because it needs changes to the /etc/fstab, and possibly a mkinitrd to become bootable.3. Or something like boot from a TFS (translucent filesystem) where base layer would be the raid, and the writable layer would be on the SDD, with periodic pushes of the writable part to the base layer (I'm not sure this is even possible).
4. In a virtual world this could be managed by appropriate location and use of snapshot images, but I'm not in a virtual world - as far as I know :0).Ideally I'd like something like raid that only brought the copy of system disk into sync with the running system infrequently (perhaps every few hours). Can anybody suggest how to do this?
