General :: 2 Disks Failed Simultaneously On A RAID 5 Array?
Apr 15, 2011
I have a home server running Openfiler 2.3 x64 with 4x1.5TB software RAID 5 array (more details on the hardware and OS later). All was working well for two years until several weeks ago, the array failed with two faulty disks at the same time. Well, those thing could happen, especially if one is using desktop-grade disks instead of enterprise-grade ones (way too expensive for a home server). Since is was most likely a false positive, I've reassembled the array:
Code:
# mdadm --assemble --force /dev/md0 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1
mdadm: forcing event count in /dev/sdb1(0) from 110 upto 122
mdadm: forcing event count in /dev/sdc1(1) from 110 upto 122
[code]....
Right. Once is just a coincident but twice in such a sort period of time means that something is wrong. I've reassembled the array and again, all the files were intact. But now was the time to think seriously about backing up my array, so I've ordered a 2TB external disk and in the meantime kept the server off. When I got the external drive, I hooked it up to my Windows desktop, turned on the server and started copying the files. After about 10 minutes two drives failed again. I've reassembled, rebooted and started copying again, but after a few MBs, the copy process reported a problem - the files were unavailable. A few retried and the process resumed, but a few MBs later it had to stop again, for the same reason. Several more stops like those and two disks failed again. Looking at the /var/log/messages file, I found a lot of error like these:
Quote:
Apr 12 22:44:02 NAS kernel: [77047.467686] ata1.00: configured for UDMA/33
Apr 12 22:44:02 NAS kernel: [77047.523714] ata1.01: configured for UDMA/133
Apr 12 22:44:02 NAS kernel: [77047.523727] ata1: EH complete
[code]....
The motherboard is Gigabyte GA-G31M-ES2L based on Intel's G31 chipset, the 4 disks are Seagate 7200.11 (with a version of a firmware that doesn't cause frequent data corruption).
View 4 Replies
ADVERTISEMENT
Feb 26, 2011
Using a fresh copy of server 10.04 im trying to simulate a failed raid array on a pair of 2tb disks. Here is the procedure i have been following so far:
- Remove the dead disk partitions from each of the raid 1 arrays (substitute the correct md devices and partitions)
- mdadm /dev/md0 -r /dev/sdb2
- mdadm /dev/md1 -r /dev/sdb3
[code]....
I get an error here that sfdisk does not support gpt (guid partition table). I thought sfdisk did support gpt? It says to use parted, but i cant find a command that copies a partition table over from another disk in parted documentation. Any suggestions? I suppose i could make the partitions manually, but im writing a procedure for people who arent that technical and i need it to be simple enough to be run in my absence. manually building the partitions would be too hard for them.
View 2 Replies
View Related
Sep 10, 2010
I have a 7-drive RAID array on my computer. Recently, my SATA PCI card died, and after going through multiple cards to find another one that worked with linux, I now can't assemble the array. The drives are no longer in the order they were in previously, and mdadm can't seem to reassemble the array. It says there are 2 drives and one spare, even though there were 7 drives and no spares. I know for a fact that none of the drives are corrupted, because one of the non-working RAID cards was still able to mount the array for a short period, but would loose the drives during resyncing (I later found out that the chipset on the card was had extremely limited linux support). I have tried running "mdadm --assemble --scan" and after the drive is partially assembled, I add the other drives with "mdadm --add /dev/md0 /dev/sdc1". These both return errors and will not complete on the new raid card.
Code:
aaron-desktop:~ aaron$ sudo mdadm --assemble /dev/md0
mdadm: /dev/md0 assembled from 2 drives and 1 spare - not enough to start the array.
[code]....
View 4 Replies
View Related
Mar 4, 2010
I'm trying to do some RAID managing with mdadm. I would like to sync my spare disk and then remove it from the array for making a backup out of it with dd command (the best way i can think of to get the current image of the whole system as it can't be done using the active RAID as source, because is constantly in use and changing). So, I have RAID1 array with 1 spare and 2 active disks (configuration listed below). Now I would like to force spare to sync and then remove it from array, although not faulty.
However, mdadm man page states:
"Devices can only be removed from an array if they are not in active use. i.e. that must be spares or failed devices. To remove an active device, it must be marked as faulty first."
So, I'd have to mark a disk as faulty (which it is not) to be able to remove it from array. There seems to be several people reporting that they can't remove this faulty flag accidentally given to a drive. And mdadm does not give direct for such operation. Isn't there a way I could remove and add disks whenever feeling like it?? One way would be open the cover and physically remove the disk. I'm not taking the risk, though. System is almost always in use, so there is not much chance for me to power off for temporary disk removal.
RAID CONFIGURATION:
~# mdadm --detail /dev/md0
/dev/md0:
Version : 00.90.03
Creation Time : Fri Aug 4 17:38:26 2006
Raid Level : raid1
Array Size : 238950720 (227.88 GiB 244.69 GB)
View 3 Replies
View Related
Apr 11, 2010
I wanted to merge my 1TB disks into and RAID 5 array, 4 of them in RAID 5 is above 2Terabytes limit of msdos partition tables which grub2 can boot from, so I decided to start up the system from scratch, by building it on GPT partitions, but seems grub2 won't boot from GPT partition because it drops to grub rescue and I can't really do anything from there.
here's my set up:
/dev/md0 (raid 1) - 100MB total:
- dev/sda1, /dev/sdb1, /dev/sdc1, /dev/sdd1
/dev/md1 (raid 5) - 45GB total:
- dev/sda2, /dev/sdb2, /dev/sdc2, /dev/sdd2
/dev/md2 (raid 5) - something bit lower than 3TB:
- dev/sda3, /dev/sdb3, /dev/sdc3, /dev/sdd3
any tips how to have this system up and running? Because I've spent like 3 days jumping over various problems
View 8 Replies
View Related
Jan 13, 2010
I'm looking to stock my SuperMicro P8SCi with two 1-2 TB SATA hard discs, for running backups and web hosting. There are reviews of certain disks stating that the low-power disks will get kicked out of the Raid due to their slow response time, and it also appears that there have been quality problems with these newer disks, as if the race to size has lowered their reliability.
Can someone recommend a good brand and specific disks that you've had experience with? I'd rather not need to replace these after putting them in, but I also don't want to pay significantly more for an illusion of quality.
View 2 Replies
View Related
Mar 25, 2011
I have been running a server with an increasingly large md array and always been plagued with intermittent disk faults. For a long time, I've attributed those to either temperature or power glitches. I had just embarked on a quest to a) lower case and drive temperature. They were running between 43 and 47C, sometimes peaking at 52C, so I've added more case fan power and made sure the drive cage was in the flow (it has it's own fan, too). Also, I've upgraded my power supply and made very sure that all the connectors are good. The array currently is a RAID6 with 5 Seagate 1,5TB drives.
When everything seemed to be working fine, I looked at my SMART logs and found that two of my drives (both well over 14000 operating hours) were showing uncorrectible bad blocks. Since it's RAID6, I figured, I couldn't do much harm, ran a badblocks test on it, zeroed the blocks that were reported bad, figuring the drive defect management would remap them to a good part of the disk and zeroed the superblock. I then added it back to the pack and the resync started. At around 50%, a second drive decided to go and shortly thereafter a third. Now, with two out of five drives, RAID6 will fail. Fine. At least, no data will be written to it anymore, however, now I cannot reassemble the array anymore.
Whenever I try I get this:
Code:
mdadm --assemble --scan
mdadm: /dev/md1 assembled from 2 drives and 2 spares - not enough to start the array
Code:
cat /proc/mdstat
Personalities : [raid0] [raid1] [raid6] [raid5] [raid4] [linear]
md1 : inactive sdf1[4](S) sde1[6](S) sdg1[1](S) sdh1[5](S) sdd1[2](S)
7325679320 blocks super 1.0
md0 : active raid1 sdb2[0] sdc2[1]
312464128 blocks [2/2] [UU]
bitmap: 3/149 pages [12KB], 1024KB chunk
Which is not fine. I'm sure that three devices are fine (normally, a failed device would just rejoin the array, skipping most of the resync by way of the bitmap) so I should be able to reassemble the array with the two good ones and the one that failed last, then add the one that failed during the resync and finally re-add the original offender. However, I have no idea how to get them out of the "(S)" state.
Code:
mdadm --examine /dev/sdd1
/dev/sdd1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x1
Array UUID : d79d81cc:fff69625:5fb4ab4c:46d45217 .....
View 2 Replies
View Related
Dec 2, 2010
Alright, I have this issue on both SystemRescueCD and Debian Squeeze. I have an ASUS P5Q Turbo board that supports hardware RAID. If I configure an array and then start the Linux installer or boot the rescue CD, I get /dev/sda and /dev/sdb instead of an array. What gives? I need to start installing within the hour so I am desperate for an answer!
View 1 Replies
View Related
Jun 7, 2010
I just had a whole 2TB Software RAID 5 blow up on me. I rebooted my server, which i hardly ever do and low and behold i loose one of my raid 5 sets. It seems like two of the disks are not showing up properly.. What i mean by that is the OS picks up the disks, but it doesnt see the partitions.
I ran smartct -l on all the drives in question and they're all in good working order.
Is there some sort of repair tool i can use to scan the busted drives (since they're available) to fix any possible errors that might be present.
Here is what the "good" drive looks like when i use sfdisk:
Quote:
sudo sfdisk -l /dev/sda
Disk /dev/sda: 121601 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
Device Boot Start End #cyls #blocks Id System
/dev/sda1 0+ 121600 121601- 976760001 83 Linux
/dev/sda2 0 - 0 0 0 Empty
[Code]....
View 2 Replies
View Related
Jun 30, 2011
I recognize that this isn't the typical question, but I have a problem with my OpenSUSE webserver, and I thought I would prevail on the community for some guidance. I have this webserver with an important MySQL db on it. The RAID array seems to have died while I was moving. (did someone drop it? dunno) Now it can't find any boot device. It has 4 old SCSI drives.
So, I know how to mount a IDE or SATA drive as a slave, in a Linux environment to read data off of it (to copy the MySQL files off of it.) But, how do I do that with a SCSI drive? Also, I have an additional (identical) server to the crippled one. What will happen if I just slide one of the scsi drives into the operating server? Is this second identical server going to help me at all? I don't even know what is on it. Can I reconfigure the RAID, so it's not using a drive, and then slide in a disk from the crippled server, and copy the data off of it?
View 2 Replies
View Related
Jan 9, 2011
I've got a raid5 array of 4 disks with ubuntu 8.04 runing on it that is currently still working:
/dev/sda
/dev/sdb
/dev/sdc
/dev/sdd
Smartmontools for /dev/sdc tell that there are 9 sectors pending for reallocation:
Code:
197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 9
198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 9
And /dev/sdd has increasing number of reallocated sectors (about 1 every couple of minutes):
Code:
5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail Always - 1735
/dev/sdc has failed a coulple of times this week (but I have always sucessfully readded it to raid5) . But the increasing number of reallocated sectores on /dev/sdd concerns me even more.
I'm affraid that during removal of /dev/sdd and adding new /devs/sdd disk, raid might fall appart. That's why I would try to do it in Ubuntu Live CD:If the raid falls appart (/dev/sdc fails) during the readding of new /dev/sdd disk, I might still remove the new /dev/sdd and return the previous one and assemble the raid with:
/dev/sda
/dev/sdb
/dev/sdd (old one that was previously removed)
Does assembling Raid in Ubuntu Live and adding new disk for /dev/sdd write anything on /dev/sda, /dev/sdb and /dev/sdc in the process of adding /dev/sdd into raid5?
View 2 Replies
View Related
Sep 27, 2010
I have a NETGEAR ReadyNAS NV+ with four 1TB drives in a RAID-5 array. This is our primary file storage. This has previously been backed up to a hardware RAID-0 array directly attached to our Windows server. The capacity of this backup array is no longer sufficient. So the plan was, take a bunch of 200GB to 320GB drives (And a 750) I had kicking around, chuck them in a couple of old SCSI drive enclosures I have collecting dust, attach them via IDA/SATA-to-USB adaptors to a USB hub, attach that to the server, create a JBOD array spanning the disks, and back up the NAS to that. Performance is not an issue as this is just to be used for backup, with the idea being as near to zero cost as possible (Spend so far = NZ$100�ish).
The first hurdle I struck was Windows not supporting Dynamic Disks on USB drives (Required to create a spanned volume). At first I resisted using another machine (i.e. a machine running Ubuntu) as I didn't want to dedicate a piece of hardware to backing up the NAS. I then decided it would be acceptable to do this via a VM, which is what I've done.So I have 10.04 running under VMWare Server 2.0.2 under Windows Server 2008 R2. The disks are all presented to the VM. I wasn't sure if I was going to end up creating the array under LVM or something else, but I noticed Disk Utility has an option to create an array, so I tried that. When I add two 250GB drives, the array size is 500GB. When I then add a 160GB drive, the array size drops to 480GB. Huh? If I keep adding disks (Regardless of order) the final array size comes out at 1.8 TB, as per the attached screenshot. Now with the following drives, I expected something more like:
160 + 250 + 250+ 750 + 250 +200 + 200 + 250 + 320 + 250 + 320 = 3.2TB
Am I missing something or making a false assumption somewhere?
View 4 Replies
View Related
Aug 17, 2010
I use slackware 13.1 and I want to create a RAID level 5 with 3 disks. Should I use entire device or a partition? What the advantages and disadvantages of each case? If a use the entire device, should I create any partition on it or leave all space as free?
View 4 Replies
View Related
Jun 20, 2010
Basically, I installed Debian Lenny creating two RAID 1 devices on two 1 TB disks during installation. /dev/md0 for swap and /dev/md1 for "/"
I did not pay much attention, but it seemed to work fine at start - both raid devices were up early during boot, I think. After that I upgraded the system into testing which involved at least upgrading GRUB to 1.97 and compiling & installing a new 2.6.34 kernel ( udev refused to upgrade with old kernel ) Last part was a bit messy, but in the end I have it working.
Let me describe my HDDs setup: when I do "sudo fdisk -l" it gives me sda1,sda2 raid partitions on sda, sdb1,sdb2 raid partitions on sdb which are my two 1 TB drives and sdc1, sdc2, sdc5 for my 3rd 160GB drive I actually boot from ( I mean GRUB is installed there, and its chosen as boot device in BIOS ). The problem is that raid starts degraded every time ( starts with 1 out of 2 devices ). When doing " cat /proc/mdstat " I get "U_" statuses and 2nd devices is "removed" on both md devices.
I can successfully run partx -a sdb, which gives me sdb1 and sdb2 and then I readd those to raid devices using " sudo mdadm --add /dev/md0 /dev/sda1 ". After I read devices it syncs the disks and after about 3 hours I see fine status in mdstat. However when I reboot, it again starts with degraded array. I get a feeling that after I read the disk and sync array I need to update some configuration somewhere, I tried to " sudo mdadm --examine --scan " but its output is no different from my current /etc/mdadm/mdadm.conf even after I readd the disks and sync.
View 1 Replies
View Related
Feb 3, 2011
i am trying to increase raid1 arraysize whcih is predefind,what i did i add two more drive in the raid but i could not increase raid array
exp:-
Creation Time : Wed Feb 2 18:14:56 2011
Raid Level : raid1
[code]....
View 1 Replies
View Related
Aug 14, 2010
I'm running a Debian homeserver, with a 3-disk (1GB each) raid 5 array using mdadm (the OS is on a separate disk).Now, smartmontools noticed some bad sectors on one of the disks, and I'm not sure what to do next (except for backup of valuable data).I found some articles on how to fix these sectors, but I'm unaware what the result on the whole array will be.
View 4 Replies
View Related
Feb 14, 2011
I've got a couple of commercial NAS boxes and I'm wondering if they (ReadyNas duo, DLink DNS-323) or any other NAS is suitable for having their RAIDed disks moved to a software-based NAS. To be specific, I'm a big fan of the (largely) Debian-based Ubuntu. Can the aforementioned NAS drives be migrated to Ubuntu (e.g. using the mdadm Linux command)?
Secondly, is there any commercial NAS that can be migrated over? Incidentally, here is a link to somebody who succeeded in a migration:URL...My specific scenario I'd like to prepare for, is the eventual (sudden) death of one of the NAS motherboards.
View 2 Replies
View Related
Apr 9, 2011
I have a 3ware controller that has a RAID 1 of two SATA disks.After an outage, the linux box (which is running ubuntu), restarted and the partition is now mounted read only. I only have the "/" mount point (this is a test server).Now, if I go to the 3ware controller by pressing ALT-3 while booting, I don't see any indication that there is something wrong with the disks.If I let the computer boot, I'm asked by fdisk if I want to fix/ignore/etc the inconsistencies found.
View 1 Replies
View Related
Feb 23, 2010
I'm renting a dedicated server with a company that claims that the server has 2 hard drives in a software RAID 1 array, but I need to make sure that the server really has the 2 HDD, and the size of the 2nd drive... how to do that ?? system is Centos 5.3
View 1 Replies
View Related
Jan 23, 2011
I'm currently using Windows Vista 32-bit on a RAID 1 array; I'm using the RAID provided by my motherboard so it's fakeRAID. Anyway, I'd like to do some C development under Linux but I'm not exactly sure how to go about installing it on a software RAID 1 array without messing up Windows. I'm not sure which Linux distro I'm going to install, so I'm hoping that information isn't important. Would I just resize my Windows partition and put Linux on the newly created partition? Do I have to worry about where Linux will put its bootloader or will it manage that on its own? I didn't mean software RAID, I meant fakeRAID.
View 1 Replies
View Related
Aug 3, 2010
We ran out of space on our server hard drive, so I installed 2 x 1GB drives, set them up as a software RAID1 array, copied the contents of /home to it, mounted it as /home for testing. Everything OK, so I unmounted it, deleted the contents of the /home folders (don't worry, we're backed up), then remounted the array. Everything was fine until we rebooted. Now I can't access the array at all; during booting the error "mount: special device /dev/md1 does not exist" comes up twice, and manually trying toe same issue. The relevant line from fstab reads:
/dev/md1 /home ext3 defaults 0 0
However, using webmin shows only md0, the RAID0 device on which the OSD was originally installed. There is no /dev/md1 device file. The mdadm.conf file reads as follows:
# mdadm.conf written out by anaconda
DEVICE partitions
MAILADDR root
ARRAY /dev/md0 level=raid0 num-devices=2 uuid=76fd4050:fb820568:c9bd3a59:ad3e70b0
So it's not listed; I'm assuming this is significant. Am I right, and whether I am or not, what can I do?
View 3 Replies
View Related
Jun 7, 2011
I recently upgraded a server from Fedora 6 to Fedora 14. In addition to the main hard drive where the OS is installed, I have 3 1TB hard drives configured for RAID5 (via software). After the upgrade, I noticed one of the hard drives had been removed from the raid array. I tried to add it back with mdadm --add, but it just put it in as a spare. I figured I'd get back to it later.Then, when performing a reboot, the system could not mount the raid array at all. I removed it from the fstab so I could boot the system, and now I'm trying to get the raid array back up.
I ran the following:mdadm --create /dev/md0 --assume-clean --level=5 --chunk=64 --raid-devices=3 missing /dev/sdc1 /dev/sdd1I know my chunk size is 64k, and "missing" is for the drive that got kicked out of the array (/dev/sdb1).That seemed to work, and mdadm reports that the array is running "clean, degraded" with the missing drive.However, I can't mount the raid array. When I try:mount -t ext3 /dev/md0 /mnt/fooI get:
mount: wrong fs type, bad option, bad superblock on /dev/md0,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
[code]....
View 1 Replies
View Related
Feb 2, 2011
I need to access each drive of a RAID array in order to update the firmware - I am completely new to linux. I think there are probably some potential challanges along the way to get this "working" For example, it seems to me that the array may present itself to application code as a single device. I need to drill down into the RAID and talk to each physical device in turn via some sort of pass-through mechanism. I'm currently pretty lost - can anyone point me in the "right" direction
View 2 Replies
View Related
Jan 6, 2011
1st I am relatively new to linux (but not to *nix). I have 4 disks assembled in the following intel ahci bios fake raid arrays:
2x320GB RAID1 - used for operating systems md126
2x1TB RAID1 - used for data md125
I have used the raid of size 320GB to install my operating system and the second raid I didn't even select during the installation of Fedora 14. After successful partitioning and installation of Fedora, I tried to make the second array available, it was possible to make it visible in linux with mdadm --assembe --scan , after that I created one maximum size partition and 1 maximum size ext4 filesystem in it. Mounted, and used it. After restart - a few I/O errors during boot regarding md125 + inability to mount the filesystem on it and dropped into repair shell. I commented the filesystem in fstab and it booted. To my surprise, the array was marked as "auto read only":
[Code]...
and the partition in it was not available as device special file in /dev:
[Code]...
View 1 Replies
View Related
Jun 24, 2009
I've tried to install Fedora 11, both 32 and 64 on my main machine.It could not install as it stops on the first install window. I've already filed a bug but really haven't seen any feed back yet.The bug has something to do with Anaconda and the Raid array but I really can't tell.
I have an Intel Board (see signature). I am running intel raid software under W7 currently.It works fine. But, I'm wondering, when I attempt to install F!!, is my current raid set-up causing problems? Do I need to get rid of the intel raid software and use a Fedor/Linux raid program to manage the raid array??
View 2 Replies
View Related
Jan 21, 2011
when I start my raid5, only 2 disks of 3 are active on md0. The 3rd disk is inactive on md_d0.When I do mdadm --examine, the two active disks report 2 active, 2 working, 1 failed. the inactive disk resports 3 active, 3 working, 0 failed.
View 2 Replies
View Related
Feb 17, 2011
I just experienced a HDD failure and while reorganizing the drives inthis machine I realized the benefits of UUID instead of /dev/sdX nomenclature. I am trying to determine the UUID of 2 disks that are assembled in a RAID1 array. right now they are /dev/sde & /dev/sdf with each only one partition. I tried ls -l /dev/disk/by-uuid but I get only the UUID of other disks, not the ones currently ID'd as sde & sdf. my mdadm.conf assembles several raid arrays all by UUID, but somehow, I cant recall how I got the UUIDs of the other HDDs at first...
View 14 Replies
View Related
Jun 4, 2010
I've been all afternoon trying to install Ubuntu Lucid on my fakeRAID 0 configured (2) HDDs and am unable to set GRUB up. The fake RAID setup is provided by Intel Matrix Storage Manager, it is correctly enabled and the BIOS is also correctly set up -- in fact, I've managed to install Windows 7 with no significant hitch. After struggling with partioning the drives (had to follow advice I found on a very helpful guide online [0]), creating the filesystems AND getting Ubuntu's installer to actually do what it is supposed to do, I now cannot seem to set GRUB up. My system, as it stands, is unbootable at all; via live CD only.
This is how the RAID0 dev is partitioned:
Code:
# fdisk -l /dev/mapper/isw_ecdeiihbfi_Volume0
Disk /dev/mapper/isw_ecdeiihbfi_Volume0: 1000.2 GB, 1000210694144 bytes
255 heads, 63 sectors/track, 121602 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 131072 bytes / 262144 bytes
Disk identifier: 0x6634b2b5 .....
View 2 Replies
View Related
Aug 16, 2011
I have built a couple RAID's, but I'm uncertain of how I should format the partitions of the raid. Should I format partitions on each disk, and then add them to a raid, or should I create a raid on unformated disks and then format the raid as a partition? Does it matter, and are there performance/reliability issues? I'm creating a RAID-5 using 3 SATA disks on RHEL for user data area.
View 5 Replies
View Related
Sep 25, 2010
I have implemented LVM to expand the /home partition. I would like to add 2 more disks to the system and use raid 5 for those two disks plus the disk used for /home. Is this possible? If so, do I use type fd for the two new disks and use type 8e for the existing LVM /home disk? Or do I use type fd for all of the raid disks?
View 1 Replies
View Related
