Ubuntu Servers :: Growing Software RAID5 With MDADM
May 13, 2011
My fileserver initially had 3 1TB drives in RAID 5 configured with mdadm as /dev/md1. (System root is a mirrored raid on /dev/md0) I went to go add a 4th 1TB drive to /dev/md1 and grow the raid 5 accordingly. I was initially following this guide: [URL] but ran into issues on the 3rd and 4th commands. I've been trying a few things to remedy the issue since, but no luck. The drive seems to have been added to /dev/md1 properly, but I can't get the filesystem to resize to 3TB. I also am not entirely sure how /dev/md1p1 got created, but it appears to be the primary partition on the logical device /dev/md1.
Relevent information:
The filesystem originated as ext3, I believe its showing up as ext2 in some of these results because I disabled the journal when doing some initial troubleshooting. Not sure what the issue is, but I didn't want to blindly perform operations on the filesystem and risk losing my data.
Created my own file server/nas, but get stuck in a problem after couple of months. I have a server with 4x 1,5tb disks, all connected to sata ports and 1 40gb ata133 disk running ubuntu 9.10 x64 amd. I've created a raid5 array using mdadm. It all worked great for couple of months but lately the raid5 array is degraded. disk sdd1 is faulting every few days. I have checked the drive but it is fine. If I re-add the disk and wait for 6 hours my raid5 array is all fine again, but after a few shutdowns, it is degraded.
my mdadm detail:
Quote:
root@ubuntu: sudo mdadm --detail /dev/md0 /dev/md0: Version : 00.90 Creation Time : Mon Dec 14 13:00:43 2009 Raid Level : raid5
I have ubuntu server 10.04 on a server with 2.8ghz 1gb ddr2 with the os on a 2gb cf card attached to the IDE channel and a software raid5 with 4 x 750gb drives. On a samba share using these drives I am only getting around 5 MB/s connected via wireless N at 216mbps and my router and server both having gigabit ports. Is a raid 5 supposed to be that slow? I was seeing speeds of anywhere from 20-50MB/s from other people and am just wondering what i am doing wrong to be so far below that.
Something weird happened last night and my raid5 failed. I am trying to re activate it and see if my data is dead or what. When I run mdadm -Asv /dev/md0 I get
Code: mdadm: looking for devices for /dev/md0 mdadm: cannot open device /dev/dm-1: Device or resource busy mdadm: /dev/dm-1 has wrong uuid. mdadm: cannot open device /dev/dm-0: Device or resource busy mdadm: /dev/dm-0 has wrong uuid. mdadm: cannot open device /dev/sde2: Device or resource busy mdadm: /dev/sde2 has wrong uuid. mdadm: cannot open device /dev/sde1: Device or resource busy mdadm: /dev/sde1 has wrong uuid. mdadm: cannot open device /dev/sde: Device or resource busy mdadm: /dev/sde has wrong uuid. mdadm: cannot open device /dev/sdd: Device or resource busy mdadm: /dev/sdd has wrong uuid. mdadm: cannot open device /dev/sdc: Device or resource busy mdadm: /dev/sdc has wrong uuid. mdadm: cannot open device /dev/sdb: Device or resource busy mdadm: /dev/sdb has wrong uuid. mdadm: cannot open device /dev/sda: Device or resource busy mdadm: /dev/sda has wrong uuid.
I'm trying to find out which one is safer when it comes down to recovery process in case of a drive failure
A RAID5 created in mdadm or a Stripe RAID created on pure LVM
the RAID is purely for data storage for a SAMBA server, the OS will reside on its own drive.Ideally the RAID physical hard drives should be re-build on another machine in case of catastrophic server failure (mother board problem, or any other random problems as an example)I can't decide which of the 2 software RAID method is more convenient and safest, don't care about performance, it'll be a dedicated server for mass storage, it's going to mirror other 3 file servers on fakeRAIDs (dmraid), it's simply a redundant backup for the backups
The important goal here is portability.from what've read it appears that LVM might be more portable?but according to some dated (2009) info the mdadm seems to be a bit buggy when it comes to rebuilding the array, yet LVM doesn't appear that safe either which one would you pick for ease to rebuild on catastrophic failures?
I am trying to build a media server for my home and still in the process of evaluating my OS options (Ubuntu Server, Fedora Core, or Win Server). I am planning to use four 1TB drives initially for the RAID5 array. Once it fill up i will add more 1TB drives.
My question is can Fedora Core create a RAID5 array and grow latter without having to back up data to external hard drive and re-create the array? I am looking for something that is easy to use and manage. If Fedora Core doesnt have this option, can you recommend other distributions that can do this?
I am trying to build a media server for my home and still in the process of evaluating my OS options (Ubuntu Server, Fedora Core, or Win Server). I am planning to use four 1TB drives initially for the RAID5 array. Once it fill up i will add more 1TB drives.
My question is can Fedora Core create a RAID5 array and grow latter without having to back up data to external hard drive and re-create the array? I am looking for something that is easy to use and manage. If Fedora Core doesnt have this option, can you recommend other distributions that can do this?
I cant seem to get my RAID 5 (consisting of 8 1tb hard drives) assembled for some reason and I have no idea why and cant find any solutions online. Ill go ahead and show what my problem is:
here is all my hard drives:
Code: server:~$ sudo fdisk -l Disk /dev/sda: 10.2 GB, 10242892800 bytes 255 heads, 63 sectors/track, 1245 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x0004f041
[Code]....
So as you can see the array for those last four look fine however for the first four it marks the last four drives as faulty for some reason. I am kind of clueless to do from this point on honestly, I have data on this array that I'd really like to save.
after a failed upgrade from 9.10 to 10.04 I had to format my computer and do a clean install of 10.04, and now my mdadm raid5 array wont start.my array is called "The Library", and i believe the space between "The" and "Library" is causing the command disk utility uses to start the array to fail.The exact error isAn error occurred while performing an operation on "The Library" (RAID-5 Array): The operation failed
Error assembling array: mdadm exited with exit code 1: mdadm: unrecognised word on ARRAY line: Library mdadm: unrecognised word on ARRAY line: Library
I am getting really frustrated with trying to get my RAID5 working again. I had a RAID5 array built with 4 of the Western Digital 1.5tb "Advanced Format" drives, WD15EARS. However, when copying 1.5gb dvd encoded files to the drive, I was getting speeds of ~2mb/s. When researching how to make this faster, I came across all the posts about the Advanced Format drives and how that was causing a lot of issues for a lot of people. It looked like the solution was simple enough: partition starting at sector 64 or 2048 or whatever and then recreate the RAID. However, this is not working for me.
Here are my computer specs: Motherboard: Gigabyte GA-EP43-DS3L LGA 775 Intel P43 ATX CPU: Intel Core 2 Duo E8400 Wolfdale 3.0GHz 6MB L2 Cache LGA 775 65W RAM: 4gb DDR2 1066 (PC2 8500) Video card: ASUS GeForce 9600GT 512MB 256-bit Linux: 10.04
i was adding another disk to my raid 5, all was going well it started the reshape, got past the critical zone, worked for 20mins, but now it seems to have crashed.When i cat /proc/mdstat, or mdadm -D /dev/md0, those programes hang and dont print anything or return.from my kern.log i can see that there was an error on a disk, the raid array removed it, was going to continue the reshape but finished immediately. Anyone know what i should do?
Mar 6 16:20:26 Aries kernel: [1931119.599107] md: reshape of RAID array md0 Mar 6 16:20:26 Aries kernel: [1931119.599107] md: minimum _guaranteed_ speed: 1000 KB/sec/disk. Mar 6 16:20:26 Aries kernel: [1931119.599107] md: using maximum available idle IO bandwidth (but not more
I've been playing with this for hours, and have been unable to figure it out. I tried to convert my RAID5 array of 4 active disks and 1 spare to a RAID6 with 5 active disks.
I did this:
Code: mdadm --grow /dev/md4 --raid-devices 5 --level 6 Here is what I have on /dev/md4:
Code: /dev/sde1 active /dev/sdg1 active /dev/sdj1 active /dev/sdf1 active removed /dev/sdh5 spare code....
but it tells me that /dev/sde is busy, and then that it has a bad superblock (From what I've read, I'm sure the bad superblock is just because of the "busy" message). I've tried this with the -f option, too, with no luck.
I have a problem with my mdadm RAID. I wanted to know if anyone had any experience with shrinking RAID5 arrays. I was growing the array from 5 to 6 devices however the grow got interrupted and it has recovered to 5 drives. The 6th drive is toast and I am unable to re add it to the system. I would like to drive the device listed as "removed". I have tried mdadm /dev/md0 --remove detached and failed with no success. I am running Ubuntu kernel 2.6.28-11 and mdadm is v3.1.1.
Here is output of a "mdadm -D dev/md0" /dev/md0: Version : 0.90 Creation Time : Wed Jan 12 00:46:41 2009 Raid Level : raid5 Array Size : 4883812480 (4657.57 GiB 5001.02 GB) Used Dev Size : 976762496 (931.51 GiB 1000.20 GB) Raid Devices : 6 Total Devices : 5 Preferred Minor : 0 Persistence : Superblock is persistent Update Time : Mon Feb 15 20:25:07 2010 State : active, degraded Active Devices : 5 Working Devices : 5 Failed Devices : 0 Spare Devices : 0 Layout : left-symmetric Chunk Size : 64K
UUID : 74fa5199:84b88e81:4ae0fbae:92643084 Events : 0.1331010 Number Major Minor RaidDevice State 0 8 16 0 active sync /dev/sdb 1 8 32 1 active sync /dev/sdc 2 8 48 2 active sync /dev/sdd 3 8 0 3 active sync /dev/sda 4 8 64 4 active sync /dev/sde 5 0 0 5 removed
Relatively inexperienced user using Linux/ubuntu. Not too savvy I admit and like to use GUI as much as possible. Not a great fan of the Terminal window... I have installed a couple weeks ago Ubuntu 10.10 (Desktop Edition) using Alternative install disk (don't ask why!) on 4Gb usb stick. Working fine except one thing with the raid array. I have created a raid5 array made of 6 drives using GUI (Disk Utility). After an expansion of the array (or was it a reinstall of the OS, I can't remember exactly?) the array does not autostart anymore. Of course nor does it automount anymore.
THE WEIRD THING is that I can still start it MANUALLY from the "Disk Utility" GUI after two tries. And it works just fine thereafter!!! The first time i try to start it gives an error (something about /dev/md0_127 being not ready or buisy). THE SECOND TRY ALWAYS WORKS like a charm, the array starts and i can mount it just fine. Here is a screenshot: I have also noticed that there is no entry in fstab for /dev/md0 although I can manually mount it using the same Disk Utility GUI. That is strange to me. Is it normal? i could easily add it manually but Ubuntu it won't boot anymore (i tried and failed, hence the reinstall). I tried for two weeks to find a solution browsing on different forums but the problem is beyond my expertise...
BELOW are further details about my configuration mdadm.conf, fstab, fdisk -l result and other info. I don=t want to loose my data but it would be nice to make this thing work and be able to access my fileserver via vnc instead of having to keep it connected to a lcd monitor as now. This is the blkid result:
I have an old Athlon XP 3000 machine that I keep around as a file server.It's currently got three 1TB drives which I had setup as mdadm raid 5 on FC10. The machine's original drive held the superblock for the raid array and it just had a massive heart attack. I've searched, my biggest source being URL...I can't tell if I can reassemble the superblock info lost with the original hard drive or if I've lost it all...
I've already asked this in the mythbuntu and didn't get an answer there so I'm trying here. OK just added another 2Tb WD drive to my mdadm controled RAID5 array, and the reshape is finished:-
Code:
Code: Mar 28 18:28:36 alpha2 kernel: [104357.343421] md: md1: reshape done. Mar 28 18:28:36 alpha2 kernel: [104357.525114] RAID5 conf printout: Mar 28 18:28:36 alpha2 kernel: [104357.525119] --- rd:4 wd:4 Mar 28 18:28:36 alpha2 kernel: [104357.525122] disk 0, o:1, dev:sda2
Have the following partitions, I've grown md1 from 3 drives to 4 drives with mdadm. It is now the size below of 531899712 from 354599292.
[Code].....
As you can see above however, the filesystem is not growing. What am I missing. I don't believe I need to do anything with the partitions. I should just resize my array as I have and xfs_grow and be finished.
3 new 1.5TB HD. 1 used 1.5TB hd with 980MB of data. I want to set up a raid 5 with a hot spare. I have music, pictures, videos, and movies (About 2.8TB worth). I have had a mismatch of drives previously, 250GB, 2 320GB, 500GB, 2 1TB and now a 1.5TB all with data. I have removed the one 250 and 2 320s and put the data on the 1.5TB that is currently installed.
What I would like to do is create a raid5 with the three new 1.5TB HD's, copy the data over from the currently installed 1.5TB and then grow or add that drive as a hot spare. Or just add it and then add another 1.5TB down the road as a hot spare don't know for sure.
In addition since I have 2 1 TB drives, I could add 2 more (Good deals on 1 TB drives right now) and have a total of 4 1TB drives. Could I have 2 raid5's (4-1TB's and 4-1.5TB's)in two separate arrays? I really do not know if that makes sense or not but here comes LVM. I am tired of managing my HD space and since i have multiple folders (Movies, music, pictures, videos) and within the movies folder I have R, G & PG folders for the ratings of the movies. (Pwd protect the R so the kids can't get to it) So with LVM installed with the Raid5 I should be able to create my folders and just keep adding data and not worry about moving folders around when I grow the storage by adding new drives. Is that correct? Maybe someone could point me to a how to.
Also, if I create 2 arrays (And I need to know so I can order the 2 additional 1TB drives), then I could put all the music, G and PG content on the one array and all the R and spicy stuff on the other and password protect it.
I'm setting up a DIY NAS system, running Ubuntu server from a CF and using 2 SATA drives. I only need RAID1. so that should do. Setting up RAID1 with mdadm is straightforward, and all of my tests with failure/recover scenario work fine in VirtualBox. Most of the tutorials on the net are talking about using mdadm in conjunction with LVM2. What is the reasoning behind LVM2 over mdadm?
I recently installed a new home backup server with Ubuntu 9.10 x86_64 using the alternate CD. I used the CD's installer to partition my disk and created a software RAID 5 array on 4 disks with no spares. The root file system is located outside the raid array.
At first the array performed nicely but as it started to fill up, the io performance dropped significantly to the point where I get a transfer rate of 1-2MB/s when writing!
I've built a server with (intentionally) very low-power components. The motherboard uses a Via C3 CPU running at 700MHz. The server has 512MB of RAM and I'm running 8.04 Server Edition (no GUI). This is purely a file server - not a lot of daemons started (except the defaults) -- no web server, etc. Just NFS, Samba and Open SSH (for remote administration). I'm not sure how much free RAM it has (it's down at the moment).
Is the RAM/CPU going to be inadequate for running software RAID5? I've done some big rsyncs and even without RAID, this thing is pretty slow. I'm not terribly concerned about the write speed, but if the read performance is going to be inadequate for playing (not streaming - just playing) a 720p MKV movie over my LAN, then I need to rethink this.
As the title says, I have a failed RAID5 hard drive. What's the easiest way I can go by replacing it? I've seen many ways to do this, but I would like to know what other people are saying about this, and see how you would do it.
I have software raid 5 array, each time I reboot my server, I have to rebuild array again. Rebuilding array takes too long. I am using ubuntu server 10.10.
I'm the user operating ubuntu 9.10 server. I made configuration with software mirroring(raid1). when I checked cron, I found the mdadm in cron.d dir. 57 0 * * 0 root [ -x /usr/share/mdadm/checkarray ] && [ $(date +\%d) -le 7 ] && /usr/share/mdadm/checkarray --cron --all -- quiet checkarray is supposed to be run on the first sunday of every month. so I just want to know
1. what does checkarray do exactly? 2. does it make a stress to system? 3. Is there any problem if I get rid of the script from cron?
I've got a strange problem. I have the following system:
[Code]...
After doing this install everything works fine as expected. I can reboot, shutdown and bootup as I much as I want to and the system will work. Now, I proceed to do the following (as root obviously - sudo bash)
[Code]...
When I try to restart the system now, I get to the grub boot loader and then it just breaks with the following message I've identified 'mdadm' as being the culprit here. Any idea why this would happen? Just a subnote. The reason I'm installing mdadm is to create a soft-raid as follows with the remaining space on each drive:
I also get sent to a Busybox (initramfs) shell with no text editor and don't know how to copy all the error messages and post them here. If there is a way, let me know. I've typed it out in the meantime:
Code: md0 : inactive sdxxxx Attempting to start the RAID in degraded mode... mdadm: CREATE user root not found mdadm: CREATE group disk not found
[Code].....
This is with a 3 disk RAID5 array. I turned off the system, pulled out a drive, and started it back up. Fresh install, all I've done so far is apt-get update and upgrade.
I've got a new Ubuntu 10.04 server install with a new 3 disk RAID 5. The boot disk is separate, not part of the RAID. I was trying to practice what I would do if a disk died to recover the RAID, so I unplugged one of the three disks. The machine now just hangs on startup. It shows fsck at the top of the screen but doesn't got anywhere from there. If you press a key it shows the Ubuntu splash screen. If I plug the disk back in, everything boots up normally. So, my question is, how do I get the machine to boot with one of the RAID members missing? I know I can recover it using the Live CD, but it would be nice to be able to get back into the machine without the CD.
