Ubuntu :: Random SATA Drive Is Always Busy / In Use (RAID 5 / Mdadm)
Dec 3, 2010
I have 4 SATA's in a RAID 5 array using mdadm. Yesterday when I started the computer the RAID did not build/mount. When trying to load the array manually I get the message "mdadm: cannot open device /dev/sd(a,b,c,d)1: Device or resource busy" The drives should not be mounted or in use. The output of the drives in mdadm (mdadm --examine /dev/sd_1) looks normal.
The weirdest part is that rebooting often changes which drive is marked as busy, it can be any of the 4 SATA drives. how to figure out why/what is being used and how to disable it? I have tried searching for similar threads here and in google and haven't found anything similar or that worked.
I have 4 SATA's in a RAID 5 array using mdadm. Yesterday when I started the computer the RAID did not build/mount.
When trying to load the array manually I get the message "mdadm: cannot open device /dev/sd(a,b,c,d)1: Device or resource busy" The drives should not be mounted or in use.
The output of the affected drive in mdadm (mdadm --examine /dev/sd_1) looks normal.
The weirdest part is that rebooting often changes which drive is marked as busy, it can be any of the 4 SATA drives. What is being used and how to disable it?
I am trying to set up a mdadm raid in a new PC that I am building for home theatre. the machine boot just fine from /dev/sdc running ubuntu 9.10 However in gparted /dev/sda and dev/sdb show to be part of /devmapper/sil_ajbicfacbaej Both dev/sda and /dev/sdb were drives that used to be part of a sil hardware raid on a previous machine. I would like to use them as a new mdadm raid on this new machine the old hardware card was really quite slow. the drives are now pluged into the MB and should bw much faster there.
I've recently started having an issue with an mdadm RAID 6 array that been operational for about 2500 hours.
Intermittently during write operations the array stalls, dropping to almost 0 write speed for 10-30 seconds. When this occur one or both of the 2 drives attached to a 2 port Silicon Image si3132 SATA-II controller "locks up" with its activity light locked on. This just started occurring within the last week and didn't seem to coincide with any update that i noticed. The array has just recently passed 12.5% full. The size of the write does not seem to make any difference and it seems completely random. Some times copying a 5 GB dataset results in no slow down other times a torrent downloading to the array at 50kb/sec does cause a slow down and vise versa.
The array consists of 8 WD 1.5TB drives, 6 attached to the ICH9R south bridge, and 2 attached to a si3132 based PCI express card. The array is formatted as a single ext4 partition.
Checking SMART data for all drives shows no errors. Testing read speed with hdparm reports what i would expect (100mb/sec for each drive, ~425mb/sec for the array).
The only thing i did notice is that udma6 is enabled for all the ICH9R drives while only udma5 is enabled for the si3132 drives. Write cache is enabled for all the disks. Attempting to set the si3132 drive to udma6 results in an IO error from hdparm.
The si3132 drive is using the sata_sil24 driver. Nothing of interest appears in the kern or syslog. During this time top shows very high wait time.
The s13132 controller appears to have the original firmware from 2006 loaded, there are some firmware updates available on the Silicon Image website for this controller that now appear to offer separate firmwares for RAID operation (some sort of hybrid controller/software thing the controller supports) and a separate firmware for standard IDE use.
Has anyone had similar issues with this controller? Is a firmware update a reasonable course of action? If so which firmware is best supported by the linux driver?
I know i'm not using its raid features but i've dealt with controllers that needed to be in raid mode for ahci to be active and for linux to work well with them. I'm bit ify at the idea of just trying it and finding out as it could knock 2 disks of my array out of action.
I received some errors while running a benchmarking script to determine my ideal raid chunk size. There are several errors in the kernel log regarding the sata link and eventually the two drives i have connected to a pci express x1 sata card were no longer present in /dev/
the script i was using is available here [URL]..system specs 1 500gb western digital drive (system drive)3 2tb samsung f4 drives (2 connected to pci x1 card (sata II) and 1 onto onboard sata port (sata I)) single core amd 64 on SiS chipset debian 64bit testing
[Code]...
I rebooted the machine and everything appears to be happy. What do these errors mean? What steps should i take to prevent them in the future so it doesn't end up corrupting the array?
I installed mdadm fine and all and proceeded to run:mdadm --create --verbose /dev/md0 --level=stripe --raid-devices=2 /dev/sda /dev/sdbWith sda being my primary hard drive, and sdb being the secondary.I get this error message upon running the command"mdadm: chunk size defaults to 64Kmdadm: Cannot open /dev/sda: Device or resource busymdadm: create aborted"I don't know what's wrong!
I experienced a strange random error that appears to have been my PC losing contact with the PC's 1.5 TB SATA drive.
This happened once (thus far only once - about two hours ago) on my Intel Core i7 920 PC with a Asus P6T Deluxe V2 mother board running openSUSE-11.4 with KDE-4.4.4. A reboot failed to fix the problem but a cold shut down restart did.
Some details ...
I had a zillion apps open, connected to my mother's PC in North America (from here in Europe) using vnc, had thunderbird, firefox, many konsole sessions, and various leafpad editor sessions.
Firefox appeared to hang, so I tried to close the Firefox page, but the PC appeared frozen (although the mouse would move). Then after about 60 seconds, the screen flashed, firefox closed (crashed may be more accurate) and I had my desktop back, ... sort of .... And I was able to go to the desktop area where vnc was running and access my mother's PC. But I was missing almost all the icons off my KDE destkop and most the icons off the panel at the bottom of the desktop.
I tried going to <CTRL><ALT><F1> and then <CTRL><ALT><F7> but that made no difference. I tried going to <CTRL><ALT><F1> to login but it would not let me login as a regular user, nor as root. Nor could I ssh into the PC. After <CTRL><ALT><F7> back at the crippled GUI, I typed 'shutdown -r now' (as root) but the command was not recognized. Neither was 'top' (as a regular user), nor 'fdisk' (as root). Dir was recognized as a regular user as was CD. But I did not believe the information those commands were giving me (it did not appear up to date).
Eventually I killed X with <CTRL><ALT><Backspace> (twice) and then forced a shutdown start with <CTRL><ALT><DELETE>, but then the PC hung during the shutdown, with the hard drive light on solid. That never happens. After a minute of that I decided to hit the 'hardware' reset. The PC restarted, but was incredibly slow to go through the motherboard boot menu's and it reported the hard drive not detected and would not boot. I did a <CTRL><ALT><DELETE> to restart, and pressed DEL to go to BIOS, and then noted in BIOS that the SATA drive was not detected.
I checked the temperature and the PC was a nominal cool temperature (38 to 40 deg c).
I then switched OFF the PC. I also turned the OFF switch on the back OFF for good measure. I waited a minute. And I turned ON again the PC.
The PC booted normally as if nothing had happened.
Most bizarre and a bit of a fright. My Linux is usually rock solid.
Anyway, ... no more investigation for now. I'm doing a massive backup of all data since my last backup. This will take a while as I have about 400 GB of home movies (at various stages of processing) from my home video camera to backup to an external hard drive.
I may pickup another 2TB external drive this weekend, just to ensure I have multiple backups.
I have 4 WD10EARS drives running in a RAID 5 array using MDADM.Yesterday my OS Drive failed. I have replaced this and installed a fresh copy of Ubuntu 11.04 on it. then installed MDADM, and rebooted the machine, hoping that it would automatically rebuild the array.It hasnt, when i look at the array using Disk Utility, it says that the array is not running. If i try to start the array it says :Error assembling array: mdadm exited with exit code 1: mdadm: failed to RUN_ARRAY /dev/md0: Input/output error
mdadm: Not enough devices to start the array.I have tried MDADM --assemble --scan and it gives this output:mdadm: /dev/md0 assembled from 2 drives - not enough to start the array.I know that there are 4 drives present as they are all showing, but it is only using 2 of them.I also ran MDADM -- detail /dev.md0 which gave:
root@warren-P5K-E:~# mdadm --detail /dev/md0 /dev/md0: Version : 0.90
I have a raid 5 array that appears to have died. I was just routinely looking at /var/log/messages and noticed that a drive in the array was complaining (via SMARTD).
This is the /home directory, and so is backed up, so it's not critical, but I'd like to get some things that changed after the last backup (the week before I noticed the failure)
Let me start by outlining what I know :
It's a 2TB array spread over three disks (mdadm software RAID5), here are the drives:
MDADM gives the drives :
Now, the array *WAS* up ok, but I umounted it. (in which /dev/md0 was mounted to /home) Yes,I know - I didn't want any changes being made to the array by anything - at least that was my thinking at the time. In hindsight... I would have killed any processes, locked out the server, backed up again and 'then' unmounted it.
But we are where we are, I'm sure there'll be time for recriminations later.
When I try to remount it, I get :
Ok - looks like it's lost the type - it's normally worked, maybe we'll give it a little hint - it's ext3 with a journal.
When I tell it it's an ext3, I get :
Now, before I go charging off specifying superblocks further along the disk, but I can't remember where they're stored.
Neither can I recall what the blocksize I originally created the array as (I have a feeling I specifed 4K, but I could be wrong).
debugfs is only telling me :
I should also point out that this server is hosted, so it's 150 miles away from me at the moment, so I can't just whip them out and dd a copy.
Something weird happened last night and my raid5 failed. I am trying to re activate it and see if my data is dead or what. When I run mdadm -Asv /dev/md0 I get
Code: mdadm: looking for devices for /dev/md0 mdadm: cannot open device /dev/dm-1: Device or resource busy mdadm: /dev/dm-1 has wrong uuid. mdadm: cannot open device /dev/dm-0: Device or resource busy mdadm: /dev/dm-0 has wrong uuid. mdadm: cannot open device /dev/sde2: Device or resource busy mdadm: /dev/sde2 has wrong uuid. mdadm: cannot open device /dev/sde1: Device or resource busy mdadm: /dev/sde1 has wrong uuid. mdadm: cannot open device /dev/sde: Device or resource busy mdadm: /dev/sde has wrong uuid. mdadm: cannot open device /dev/sdd: Device or resource busy mdadm: /dev/sdd has wrong uuid. mdadm: cannot open device /dev/sdc: Device or resource busy mdadm: /dev/sdc has wrong uuid. mdadm: cannot open device /dev/sdb: Device or resource busy mdadm: /dev/sdb has wrong uuid. mdadm: cannot open device /dev/sda: Device or resource busy mdadm: /dev/sda has wrong uuid.
Trying to complete a RAID 1 mirror on a running system and have run into a wall at the last part. I can't add the active physical disk to the mirror. This is on a Centos 5.6 x86_64 system. Anybody know where to go from here? I've tried adding the nodmraid line to the kernel boot line with no luck. Tried removing the logical volumes from LVM, but it won't let me. Not a Linux newbie, but haven't set up a RAID in a long time.
[root@blackbox-0-2-e3-23-72-c5 ~]# mdadm /dev/md1 --add /dev/sda2 mdadm: Cannot open /dev/sda2: Device or resource busy
I have a raid5 on 10 disk, 750gb and it have worked fine with grub for a long time with ubuntu 10.04 lts. A couple of days ago I added a disk to the raid, growd it and then resized it.. BUT, I started the resize-process on a terminal on another computer, and after some time my girlfriend powered down that computer! So the resize process cancelled in the middle and i couldn't acess any of the HDDs so I rebooted the server.
Now the problem, the system is not booting up, simple black with a blinking line. Used a rescue CD to boot it up, finised the resize-process and the raid seems to be working fine so I tried to boot normal again. Same problem. Rescue cd, updated grub, got several errors: error: unsupported RAID version: 0.91. I have tried to purge grub, grub-pc, grub commmon, removed /boot/grub and installed grub again. Same problem.
I have tried to erased mbr (# dd if=/dev/null of=/dev/sdX bs=446 count=1) on sda (ide disk, system), sdb (sata, new raid disk). Same problem. Removed and reinstalled ubuntu 11.04 and is now getting error: no such device: (hdd id). Again tried to reinstall grub on both sda and sdb, no luck. update-grub is still generating error about raid id 0.91 and is back on a blinking line on normal boot. When you'r resizeing a raid MDADM changed the ID from 0.90 to 0.91 to prevent something that happend happened. But since I have completed the resize-process MDADM have indeed changed the ID back to 0.90 on all disks.
I have also tried to follow a howto on a similar problem with a patch on [URL] But I cant compile, various error about dpkg. So my problem is, I cant get grub to work. It just gives me a blinking line and unsupported RAID version: 0.91.
ubuntu 8.04 server can not detect seagate sata hard drive 2tb or sata Lg dvdrw x22 sata drive .is it possible to install it without buying a pci ide sata card?is it possible to get a driver for sata driver and sata drive that can be recognise by ubunto 8.04 server ?or to get the files for 1.44 floppy diskdoes the late edition of unbutu recognise sate hdd and sata cdrw drive automaticly during the installation of the unbutu?
I rebooted my server and out of nowhere the RAID5 array won't assemble. I've tried everything I could think of to reassemble the thing. I fear that the array is ruined, but I can't imagine how. Here are various bits of information: The simplest failure (with and without partition numbers, which have not been needed in the past):
Code:
richard@nas:~$ sudo mdadm --assemble --verbose /dev/md0 /dev/sd[bcd] mdadm: looking for devices for /dev/md0 mdadm: cannot open device /dev/sdb: Device or resource busy
I have 4 WD10EARS drives running in a RAID 5 array using MDADM.Yesterday my OS Drive failed. I have replaced this and installed a fresh copy of Ubuntu 11.04 on it.I then installed MDADM, and rebooted the machine, hoping that it would automatically rebuild the array.It hasnt, when i look at the array using Disk Utility, it says that the array is not running. If i try to start the array it says : Error assembling array: mdadm exited with exit code 1: mdadm: failed to RUN_ARRAY /dev/md0: Input/output errormdadm: Notenough devices to start the array.I have tried MDADM --assemble --scan and it gives this output:mdadm: /dev/md0 assembled from 2 drives - not enough to start the array.I know that there are 4 drives present as they are all showing, but it is only using 2 of them.I also ran MDADM -- detail /dev.md0 which gave:
root@warren-P5K-E:~# mdadm --detail /dev/md0 /dev/md0: Version : 0.90
I'm writing a monitoring plugin for a home server RAID, mdadm on Ubuntu 10.4. code...
I'm looking for the possible values of "state" but can't seem to find it anywhere, neither man nor the online documentation I have found seem to have a list.
Does anyone know where to find a list of possible states?
I have done lots of searching and I haven't been able to find anyone else with the same problem. Whenever I create a RAID with 'mdadm', regardless of level (I've done linear, 0, and 5) the command I use is:
I have a previously defined RAID5 (4 disks). This worked in Ubuntu 8.04.. I recently moved to CentOS5.. and I cannot seen to get the drive back online.cat /proc/mdstat shows, no raid levels (personalities).. and no drives listed.mdadm --detail -scan returns nothing.mdadm --QE returned a UUID string.. and the ARRAY output.I can mdadm --examine all the members of the original array.I am not versed in mdadm enough to really understand what I can run and should not run that would erase the data on the drives. Please assist.. I will try to post exact output of commands.. but the system is kind of unreachable and being rebuilt... i just want to ensure my data on the array is not lost
Recently, one the SMART utility said that one of the drives had failed and another drive was about to fail. I downed the box and hooked them up to my windows machine to run sea tools on them (They are all seagate drives). Sea Tools said that the drives were fine, while ubuntu said they were failing/dead. Yesterday I decided to try to fix one of the drives in the raid. I turned the server off, took the failed drive out, and restarted. Of course the raid didn't work because only 2 of the 3 drives were there, however it had been working w/ only 2 of the 3 drives for a couple months now (I'm a lazy college student). I turned it back off and back on with the drive there just to see if I could get the raid up again, but I havn't been able to get it to go. So far I've tried:
Code:
mdadm --assemble /dev/md0 /dev/sd[b,c,d] mdadm: no recogniseable superblock on /dev/sdb mdadm: /dev/sdb has no superblock - assembly aborted
[code]....
I'm looking for a way to trick the raid into working with just 2 drives until I can warranty the seagate and buy an external 1.5 TB drive to use as another backup. how to remove the bad drive from the array and replace it with a fresh drive, without data loss.
I just had a whole 2TB Software RAID 5 blow up on me. I rebooted my server, which i hardly ever do and low and behold i loose one of my raid 5 sets. It seems like two of the disks are not showing up properly.. What i mean by that is the OS picks up the disks, but it doesnt see the partitions.
I ran smartct -l on all the drives in question and they're all in good working order.
Is there some sort of repair tool i can use to scan the busted drives (since they're available) to fix any possible errors that might be present.
Here is what the "good" drive looks like when i use sfdisk:
Quote:
sudo sfdisk -l /dev/sda Disk /dev/sda: 121601 cylinders, 255 heads, 63 sectors/track Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0 Device Boot Start End #cyls #blocks Id System /dev/sda1 0+ 121600 121601- 976760001 83 Linux /dev/sda2 0 - 0 0 0 Empty
I have a fileserver which is running Ubuntu Server 6.10. I had a RAID5 array consisting of the following disks:
Code: /dev/sda1 /dev/sdb1 /dev/sdd1 /dev/md0 -
the raid drive for the above three disks. The sda1 disk has failed and the array is running on 2 of 3 disks
/dev/sdc (OS disk) /dev/sde (new 2tb disk - unused) /dev/sdf (new 2tb disk - unused)
My plan was to rebuild the array using the two new disks as RAID1. Would the best way to do this be to create a new RAID1 disk on /dev/md1 then copy all data over from /dev/md0? Also - this may sound stupid but since all 3 drives in md0 are identical i'm not sure physically which disk is bad. I tried disconnecting each disk one-by-one then rebooting but the system doesn't appear to want to boot without the bad drive connected. I've already failed the disk in the array with mdadm but i'm unsure of how to remove it properly.
My home-backup server, with 8*2TB disks won't boot anymore. Two disks failed at the same time and i rebuilt the raid 6 array without any problem, but now i can't boot the os. I'm using ubuntu server, 10.10. I've made screens of the displays to don't copy everything here. The problem at the boot:
And the Grub config: It's not a production server, but i would like to have it online. I've tried for the lasts 2 days (just a couple hours a day) but without success. I was suggested to do "mount -o remount,rw /" and than edit /etc/fstab, but it get the file don't exist error.
I'm having trouble with Ubuntu 10.10 and stable device names. When I installed Ubuntu, the root drive was the only one in the machine; it obviously got /dev/sda.
After the base installation, I installed three additional 2TB drives to make RAID-5 array. Ubuntu renamed the root drive to /dev/sdd. While annoying I lived with it.
After creating a single partition set to "Linux raid autodetect" on each drive, I created the RAID-5 array:
Code:
All was going well until a reboot. When rebooting Ubuntu decided to make the root drive /dev/sda this time and now mdadm --detail /dev/md0 reports:
Code:
How to fix the array and make the device names stable?
I'm running a Debian homeserver, with a 3-disk (1GB each) raid 5 array using mdadm (the OS is on a separate disk).Now, smartmontools noticed some bad sectors on one of the disks, and I'm not sure what to do next (except for backup of valuable data).I found some articles on how to fix these sectors, but I'm unaware what the result on the whole array will be.
My only goal is to have a raid-5 that auto-assembles and auto-mounts. Hardware: 4*2TB sata (raid disks), 1*500GB IDE (OS disk), 1*DVD IDE all plugged direct into the motherboard (nForce 750i SLI).
Starting partitions on the raid disks: gpt ext4 The problem occurs when I restart my comp after building it for the first time. I am able to see it assemble, I am able to partition it, I even mounted it Once.This is the second time I've built it so I have watched everything that happened. I don't know if this has anything to do with my problem, but when I created the raid my drive designations were: sda - 500GB(OS), sd[bcde] - 2TB(raid). When I restarted: sd[abcd] - 2TB(raid), sde - 500GB(OS).
I HAD a fedora 11 server with md RAID 1 across two 1TB SATA drives. The md0 space was set up to be an LVM PV and the single LVM VG was carved up into 5 or 6 LVs. The MB on this system died and I wound up buying a new one.
Now I want to recover the data from the RAID1 setup on the new server. However, when I attach the two 1TB drives to a new fedora 13 setup, mdadm is only able to find one of the two drives. The partition on the second drive shows "busy" during an mdadm -A -s -v to scan for md volumes.
Well, one drive should be enough since this is RAID1, right? Well, when I do a pvscan -v, the other drive shows up as a "NEW" pv not allocated to a VG. In addition, vgscan does print "Invalid metadata header checksum" when it runs but it doesn't point at any particular PV. I'm afraid to go any further with LVM since I can't afford to lose the data on this system. It is backed up offsite, but the restore will take several days and I can't afford to be down that long.
Are there any tools or techniques where I can dig deeper into what each drive, in the RAID1 pair, has right and wrong with it and pick one that I can force into a usable VG so that I can recover the data?
a server that was running a hardware isw raid on the system (root) disk. This was working just fine until I started getting sector errors on one of the disks. So, I shutdown the system and removed the failing drive and installed a new drive (same size). On reboot I went in to the intel raid setup and it did show the new drive and I was able to set it to rebuild the raid. So, continuing the reboot everything came up just fine except the raid 1 on the system disk. I have tried many times to get the system to rebuild the raid using dmraid, but to no avail it would not start a rebuild. In order to get the system back up and make sure that the disk was duplicated I was able to 'dd' the working disk to the new disk that was installed.At present when I look at the system it does not show up with a raid setup on the system disk ( this comprises the entire 1TB disk with w partitions sda1 as / and sda2 as swap).Problem:I have decided to forego the intel raid and just use mdadm. I have a test system setup to duplicate (not the software, but the disk partitions) the server setup.
Code: [root@kilchis etc]# fdisk -l Disk /dev/sda: 1000.2 GB, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes
I have a 7-drive RAID array on my computer. Recently, my SATA PCI card died, and after going through multiple cards to find another one that worked with linux, I now can't assemble the array. The drives are no longer in the order they were in previously, and mdadm can't seem to reassemble the array. It says there are 2 drives and one spare, even though there were 7 drives and no spares. I know for a fact that none of the drives are corrupted, because one of the non-working RAID cards was still able to mount the array for a short period, but would loose the drives during resyncing (I later found out that the chipset on the card was had extremely limited linux support). I have tried running "mdadm --assemble --scan" and after the drive is partially assembled, I add the other drives with "mdadm --add /dev/md0 /dev/sdc1". These both return errors and will not complete on the new raid card.
Code: aaron-desktop:~ aaron$ sudo mdadm --assemble /dev/md0 mdadm: /dev/md0 assembled from 2 drives and 1 spare - not enough to start the array.
I have a 4 drive RAID 5 array set up using mdadm. The system is stored on a seperate physical disk outside of the array. When reading from the array its fast but when writing to the array its extremely slow, down to 20MB/Sec compared to 125MB/Sec reading. It does a bit then pauses, then writes a bit more and then pauses again and so on.The test i did was to copy a 5GB file from the RAID to another spare non-raid disk on the system average speed 126MB/s. Copying it back on to the RAID (in another folder) the speed was 20MB/s.The other thing is very slow several KB/s write speed copying from eSATA drive to the RAID.
