General :: Scsi RAID Jbod And Arrays - Disk Utilization And The Corresponding Low Data Transfer
Jul 6, 2010
So I have a system that is about 6 years old running Redhat 7.2 that is supporting a very old app that cannot be replaced at the moment. The jbod has 7 Raid1 arrays in it, 6 of which are for database storage and another for the OS storage. We've recently run into some bad slowdowns and drive failures causing nearly a week in downtime. Apparently none of the people involved, including the so-called hardware experts could really shed any light on the matter. Out of curiosity I ran iostat one day for a while and saw numbers similar to below:
Some of these kinda weird me out, especially the disk utilization and the corresponding low data transfer. I'm not a disk IO expert so if there are any gurus out there willing to help explain what it is I'm seeing here. As a side note, the system is back up and running it just runs sluggish and neither the database folks nor the hardware guys can make heads or tails of it. Ive sent them the same graphs from iostat but so far no response.
I have never preformed a rebuild of an RAID array. I am collecting resources, which details how to build an RAID 5 array when one drive has failed. Does the BIOS on the RAID controller card start to rebuild the data on the new drive once it is installed?
I am using an embedded platform in which I have connected an external harddisk (/dev/sda). The SCSI driver is present and I am using the SG_IO interface for performing the SMART commands with the Hard Disk. (Unfortunately not all the HDIO ioclts are present. So I opted for the SG_IO ioctl). But the data transfer (reading/write data from/to sector) is not working with the SG_IO ioctls. So I searched for some other options. Later in one of the places, I found that we can actually mount the /dev/sda to some mount point in /mnt and then make a XFS file system (mkfs.xfs) of this.
And then we can create the directories and do file operations on this mounted directory. Here the simple read/write systems calls can be used for this. I was thinking about this implementation. But I am confused how I can map the actual LBA (Logical Block Address) to the device file offset. I mean if I want to write to the sector 5, there will be a LBA for it. So I can do lseek on my device and then write the data there. So how the mapping between LBA and device file offset can be calculated.
I had used DELL 1950 with 300 GB raid disk. Now, I purchased Dell 2950 with 450 GB (6 disk - 3 pairs of raid). I wanted to pull out old 300 GB from 1950 and put it in 2950 (temporarily) to copy all contents to the new system. How do I know which HDDs I need to pull out from 2950 so that I can replace them with 300 GB HDD to mount. I do not know how raid setup (I know unix alone - not raid commands). Is this possible? How to do it?
I have created software raid 5 configurations on the second harddrive its working fine and i have edited fstab file for auto mounting when it reboot but when i reboot the computer raid doesn't work i have to re-create the arrays by typing "mdadm --create" command again and mount again manually ,is there anywhere i can do this once without retyping the commands again after rebooting and i am also using redhat 5
So I configured a couple of hard drives as JBOD in the BIOS. The boot up screen says they're in JBOD configuration. However when I boot into Ubuntu 10.10. I still see them as 2 separate disks and I mount them separately and write to each separately. I never used JBOD before but I would assume it would be like a RAID where the disks would show up as one volume.
I need to write a script to report useful information on disk utilization for each user's home directory.For each directory I need to show: 1. the long listing of that directory entry (but not the files in the directory), so that I can see the rights and owners of the directory.2. The amount of disk used by that directory, in human-readable format, including subdirectories. I need to have two lines for each user one after the other. For example:
/home/user1 directory info /home/user1 disk usage /home/user2 directory info /home/user2 disk usage
The script will assume that all users, except user root, have their home directories in the /home directory (no need to do anything with the /etc/passwd file). And if the administrator adds or removes users, the script should still work correctly (so the script shows the information for all current users).
Here's what I do know. The command "ls -ld /home/user's_name" will give me the info I need for #1. And the command "du -hs" will give me the info I need for #2. What I don't know is how to grab each individual directory in order to apply the above commands to each of them in order. ???
I am trying to access data that is on a Raid 5 array in Ubuntu... There are 4 installed disks (250gig disks) - 3 of which are setup as a Raid 5 array (the 4th is active but unused). These show up as one large drive (498gig). I have had an issue with the drive where it is no longer allowing Windows to boot - I receive a disk read error on boot (so the OS does not load, obviously!) - what happened was basically I unplugged then replugged in one of the disks which affected the array... I physically reconnected everything as it was, I then had to 'reactivate' the disk in the Raid BIOS... at that point the array seemed OK, was the right size, etc (and was listed as "Optimal" in the Raid BIOS) however, the problem with the disk read error persists.
I have started the machine using Ubuntu v9.10 from a CD (non-destructive mode) and it shows a disk of the right size (ie: on the desktop and in Nautilus it says "498gig Filesystem" ). However, in Nautilus, the disk appears empty with no folders or files on it (even with hidden files shown).... If I view 'Information" for the disk it shows 67gig used space and 399gig free space (which is correct). Also, if I view the disk in Gparted, it shows a disk with about 67gig of used space and 399 free space on a 464gig disk (with 8gig unallocated). One more thing.... when I try the command 'sudo dmraid -tay' it says that there is no raid disk (there are in fact, no drives plugged into IDE or SATA slots - all disk are plugged into the RAID controller card). Anyway, at this stage, I just want to copy the data to a single hard disk if possible and move on.
I've got a raid5 array of 4 disks with ubuntu 8.04 runing on it that is currently still working:
/dev/sda /dev/sdb /dev/sdc /dev/sdd
Smartmontools for /dev/sdc tell that there are 9 sectors pending for reallocation:
197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 9 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 9 And /dev/sdd has increasing number of reallocated sectors (about 1 every couple of minutes):
5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail Always - 1735 /dev/sdc has failed a coulple of times this week (but I have always sucessfully readded it to raid5) . But the increasing number of reallocated sectores on /dev/sdd concerns me even more.
I'm affraid that during removal of /dev/sdd and adding new /devs/sdd disk, raid might fall appart. That's why I would try to do it in Ubuntu Live CD:If the raid falls appart (/dev/sdc fails) during the readding of new /dev/sdd disk, I might still remove the new /dev/sdd and return the previous one and assemble the raid with:
/dev/sda /dev/sdb /dev/sdd (old one that was previously removed)
Does assembling Raid in Ubuntu Live and adding new disk for /dev/sdd write anything on /dev/sda, /dev/sdb and /dev/sdc in the process of adding /dev/sdd into raid5?
With mdadm was only able to add a new drive to the using the --force function. I do not feel comfortable with using the function that way though. When I remove a disk in VMWare, it perfectly says that the drive is lost and the array is degraded (mdadm --detail /dev/md0). Although after re-adding the drive, it immediately shows device as busy for both mdadm and sfdisk when I don't use --force.
Recovery and repairation of degraded array worked fine with sfdisk --force, mdadm --add --force, it automatically started recovering and took not so long. What are best practises to manage software raid-1 arrays?
I am in a situation where I am stuck with a LVM cleanup process. Although I know a lot about AIX LVM , but this is first time I am working with Linux LVM2. Problem is that I created two RAID arrays on storage, which appeared as mpath0 & mpath1 devices (multipath) on RHEL. I created logical volumes and volume groups and every thing was fine till I decided to clean the storage arrays and ran following script:
#!/bin/sh cat /scripts/numbers | while read numbers do lvremove -f /dev/vg$numbers/lv_vg$numbers vgremove -f vg$numbers pvremove -f /dev/mapper/mpath$numbersp1 done
Please note that numbers was a file in same directory, having numbers 1 and 2 in separate line. Scripts worked well and i was able to delete definitions properly (however I now think I missed one parted command to remove the partition definition from mpath device. When I created three new arrays, I got devices from mpath2 to mpath5 on linux and then I created vg0 to vg2. By mistake, I ran above script again for cleanup purpose and now I got following error message
Cant remove physical volume /dev/mapper/mpath2p1 of volume group vg0 without ff[/B]
Now after doing mind search, I now realize that I have messed up (particularly because mpath devices did not map in sequence to vg devices and mapping was like mpath2 --- to ---- vg0 and onwards). Now how I can cleanup the lvm definitions? should i go for pvremove -ff flag or investigate further? I am not concerned about data, I just want to cleanup these pv/vg/lv/mpath definations so that lvm can be cleaned up properly and I can start over with new raid arrays from storage?
Suppose during a script execution I am attaching one or two new disk having different vendor id to host machine, how do I know which disk corresponds to which file in /dev/ directory? i just want to perform some operation on those device from some script, how will i know which file in /dev directory correspond to which disk(having same size but different vendor id).
I'm trying to setup a RAID 5 array of 3x2TB drives and noticed that, besides having a faulty drive listed, I keep getting what looks like two separate arrays defined. I've setup the array using the following : sudo mdadm --create /dev/md01 --verbose --chunk=64 --level=5 --raid-devices=3 /dev/sdb /dev/sdc /dev/sde
So I've defined it as md01, or so I think. However, looking in the Disk Utility the array is listed as md1 (degraded) instead. Sure enough I get :cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md1 : active raid5 sde(F) sdc sdb 3907028992 blocks level 5, 64k chunk, algorithm 2 [3/2] [UU_]
So I tried getting info from mdadm on both md01 and md1 :user@al9000:~$ sudo mdadm --detail /dev/md1 /dev/md1: Version : 00.90 Creation Time : Sun Jan 9 10:51:21 2011 Raid Level : raid5 ......
Is this normal? I've tried using mdadm to --stop then --remove both arrays and then start from scratch but I end up in the same place. I'm just getting my feet wet with this so perhaps I'm missing some fundamentals here. I think the drive fault is a separate issue, strange since the Disk Utility says the drive is healthy and I'm running the self test now. Perhaps a bad cable is my next check...
what do I have:2x 150GB drives (sda) on a raid card (raid 1)for the OS (slack 13.37)2x 2TB drives (sdb) on that same raid card (raid 1, too)2x 1.5TB drives (sdc,sdd) directly attached to MoBo2x 750GB drives (sde,sdf) attached to MoBo too.if i got about it the normal way, i'd create softRAID 1 out of the the 1.5TB and the 750GB drives and LVM all the data arrays (2TB+1.5TB+750GB) to get a unified disk.If I use btrfs will I be able to do the same? mean I have read how to create raid arrays with mkfs.btrfs and that some lvm capability is incorporated in the filesystem. but will it understand what I want it to do, if i just say
With mdadm was only able to add a new drive to the using the --force function. I do not feel comfortable with using the function that way though.
When I remove a disk in VMWare, it perfectly says that the drive is lost and the array is degraded (mdadm --detail /dev/md0). Although after re-adding the drive, it immediately shows device as busy for both mdadm and sfdisk when I don't use --force.
Recovery and repairation of degraded array worked fine with sfdisk --force, mdadm --add --force, it automatically started recovering and took not so long.
What are best practises to manage software raid-1 arrays?
I have two SAS RAID controller cards in a Dell server in slots 2 & 3, both with an array hanging off them. I went to install a third card into slot 1, but then when it boots it says two of my sd's have bad magic number in the super-block and it wants me to create an alternative one, which I don't want to do. If i remove the new card, the server boots perfectly like it did before I added the new card. Is the new card trying to control stuff that isn't hooked up to it because its in slot 1, so its confusing RHEL?
I have a problem with arrays in awk. What i want is to take some data from a file (ssh log) and print it to a html table. I have managed to print some stuff (user logged in and how many times they have logged in) What i want more is to take all the ip that each user logged in from and print it in a row next to the username and times (in the code i typed blabbla where i want the ip to be shown. How do you think i should approch that, multidimensional arrys maybe?
First off I'd like to say I'm very new to Ubuntu, so I'm still trying to learn.I have a K8 motherboard with an adaptec U320 SCSI card with RAID ability.To that card I connected two 15k RPM 35 GB Maxtor SCSI drives.For some reason I'm not able to install Ubuntu 9.10 onto these drives with both drives in RAID 0.With both drives separately configured Ubuntu doesn't even see them.I have by the way run Windows XP and 2000 succesfully on these drives in Raid 0 configuration.I set up the array in the card's bios as bootable with write cache enabled.The system's bios sees the array as the array to boot from.Ubuntu (both standard and alternate) sees the array and I have tried to install Ubuntu on it by manually partitioning it or having me guide it with or without LVM.I tend to delete and rebuild the array between attempts so I have a clean slate to start from every time I try.
I have no other drives (except the CD of course) installed on this computer.The whole installation goes very well untill the end where I get a message that it could not install the boot loader (grub?).Every single time I've tried to install Ubuntu in all sorts of ways onto my RAID 0 array I have run into problems installing that boot loader, and I've tried that card and those disks in another computer as well.Tomorrow I'd like to try to manually set up the partitions with a small /boot partition on a standard hard drive with / on the array, but if somebody please has any idea's on how I might get it working without having to rely on another hard disk (which might not even work of course)
For example I am copying data to USB-flash drive using some file manager. When the file manager shows that transfer is complete, flash drive indicator continues to light. As far as I know this is some kind of caching system...
1. Is it OK to close file manager when transfer window is closed but the flash drive indicator continues to light (data is still being copied)?
2. Is it better if I turn off this caching technology?
I build a new test machine where i need to bring data from live machine. the data is kind of flat files and some propreitary application axigen mail server.
now what i am suffering from, which commands to do first practise. there is remote site with 1mps speed of wireless between live and test machine. on daily basis aprox 14gb .tar.gz files it need to move it.
i found scp,rcp,rsync,sftp etc. which is fastest way to replicate or copy to remote machines.
the data is on live machine /var/opt/application and on remote same directory too /var/opt/application
i try using scp it take aprox 8-10 hours to copy single 14gb file.
if possible where to see such commands logs results, if anything get down error discontinue while copying.
I configured non-anonymous ftp server in my Ubuntu 10.04.it's working downloading and uploading through thrid party software like filezilla.Now i think that without using any other software i want to upload and download the ftp content in browser it'self.i heard that using webmin i can upload and download ftp data sharing through browser.
I currently have a 1.5TB sata drive connected to my system, with data already on it. In the next few days I am planning to use mdadm to create a raid5 array with another 2 1.5TB drives along with the one I already have. Is it possible to create the raid without losing the data on my current hard drive? Obviously it will be backed up before hand just in case, but it would be nice not to have to restore nearly a TB of data after raid'ing the drives.