Server :: Finding A Distributed File System With RAID5 Like Features?
Jun 17, 2011
I am currently looking for a file system that needs to be distributed over several nodes and need redundancy, like RAID5, and the ability to grow if needed, like LVM. Also not all nodes are going to be located in the same data center, but I guess that's not that important as long as the connection between DCs is sufficiently fast. I am currently looking at AFS, Coda, GFS and OCFS to see if they have what I need.
Edit: I just figured that it may be better to have the RAID5-like setup within one location, and have RAID1-like mirroring between the locations. That would probably better for performance, right?
View 2 Replies
ADVERTISEMENT
Oct 6, 2009
is there a way to set-up a file system, where you have multiple nodes (PC's) linked to one storage pool? ZFS seems to support this idea with multiple disks on one node, but could you connect multiple nodes in this manner?
View 3 Replies
View Related
Jan 9, 2010
I have no drive failures but just need to recreate a raid5 set as the next free MD disk number. Originally I built a temp OS of debian on a single drive and had 4x2TB drives in a raid5 software array (MD0) this worked fine and allowed me to move all data to it, and remove our old fileserver. I have now pulled out the 4 x 2TB Raid 5 drives and created a new OS on two new 80GB drives, partioned as follows,
MD0 is now 250mb Raid1 as /boot
MD1 is 4GB Raid1 Swap
MD2 is 76GB Raid1 as /
If I turn off and push back in the 4x2TB drives I cannot see a MD3. I presume I would need to create a MD3 from these 4 drives but I dont want to mess things up as its live data. So im here asking for help, or a bit of hand holding to get it done right.
PS - Its a Debian Lenny 5.0.3 Raid1 fresh install replacing a Debian Lenny 5.0.3 on a single disk.
View 2 Replies
View Related
Jan 21, 2010
Last year I was looking into fault-tolerant distributed file systems and I recall one kernel-based system that required a physical partition on each machine in the cluster, but would treat it as a single volume - ie. a write on one server would appear on the disk on all the servers.Unfortunately I didn't bookmark the specific system I was looking at, and now a year later I can't remember the details.What I don't want is NFS - a single file server with a file system mounted on various machines. What I do want is mirroring - one disk shared among multiple servers, so that if one server dies, it doesn't make any difference to the rest of them.
A bit of investigation turned up Red Hat's GFS, which kind of looks like what I want, but looks more and more like an NFS model to me. I was wondering what everyone's opinion of the various options out there were.
View 3 Replies
View Related
Apr 30, 2010
I'm trying to understand how deb packages work. Are there source debs?I assumed there were, but from[URL].. I see that source is distributed in a GZipped file.
View 3 Replies
View Related
Jul 9, 2010
A bunch of hosts all connected to each other.I'd like to have a directory where a group of hosts can dump files and all access, synchronising with each other their changes automatically and transparent to the user.How can I have it so that rsync updates both ways?
* host A asks host B for a list of files, modification dates, checksums
* then applies the rules to select which files it will fetch from B
--> if A owns the file then do not get modifications from B
--> files differ, then select newer file
* then fetches them
* and so on for each host...
View 1 Replies
View Related
Apr 13, 2011
I have install centos distributed for server ..
linuxmans@yahoo.com
View 2 Replies
View Related
Mar 31, 2011
I'm using CentOS 5 and just found out that there are people using my Apache Web Server as proxy (version 2.2.3 the OS's default).
I've checked out that my mod_proxy is already disabled but the unauthorized incoming request is still very large.
So I tried to compile an Apache from raw (2.2.17) just enabling mod_rewrite and then found out that the 'proxy' request is still there.
Tried httpd -l and here are the modules inside the Apache :
Code:
# /opt/apachew/bin/httpd -l
Compiled in modules:
core.c
mod_authn_file.c
[Code]....
There should be no mod_proxy but why should it still accepting proxy request? I checked the access_log file and still found [URL] and so on.
My question is : - How to block all the proxy request and only accepting the normal web server request? I also have tried to enable proxy in the previous Apache and then setting up "ProxyRequest Off" and Proxy * only from localhost but the request is still there.
I tried by myself using the webserver as proxy in browser and all the URLs I opened including the false URL will be redirected to my index.php and this should show me that 'you-can't-use-this-as-proxy-anymore' right?
Insane people still believe they could use my server and they keep trying and trying.
I couldn't block the IP addresses because some of them are using the mobile data (GPRS/UMTS) and that is our market for the website.
View 5 Replies
View Related
Jun 14, 2011
I cant seem to get my RAID 5 (consisting of 8 1tb hard drives) assembled for some reason and I have no idea why and cant find any solutions online. Ill go ahead and show what my problem is:
here is all my hard drives:
Code:
server:~$ sudo fdisk -l
Disk /dev/sda: 10.2 GB, 10242892800 bytes
255 heads, 63 sectors/track, 1245 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0004f041
[Code]....
So as you can see the array for those last four look fine however for the first four it marks the last four drives as faulty for some reason. I am kind of clueless to do from this point on honestly, I have data on this array that I'd really like to save.
View 3 Replies
View Related
Jan 7, 2011
I'm planning to add 1tb sata disk to my lovely file-server under ubuntu 10.10,what i want is use this disk as additional storage for network user,indows and ubuntu?I mean when my ubuntu server down (worse case) I can easily take out the disk from ubuntu machine and plug in on windows machine
View 2 Replies
View Related
Jan 28, 2011
Can windows read files from a home file server with an ext4 file system? or do I have to partition the drive with the server (ext4) and an ntfs partition with the files on?
View 4 Replies
View Related
Oct 29, 2010
I've had software RAID 5 arrays for a while now, so they were set up before a RAID array could be partitioned. I had two separate RAID 5 arrays on the same set of drives. One was for / and the other for /home. I moved the / to an SSD and figured I'd expand the other RAID array by failing a drive, repartitioning it then adding it back in. After repeating for the remaining drives, I could then expand the RAID array to use the full size of the drives.
Partway through the second drive being added back in, the RAID array stopped with a kernel error. The drive I was adding and another drive both showed as failed. I couldn't restart the array so I copied the failed drive (Seagate's SeaTools did show it as faulty, but without SMART being tripped) to a new one and tried again. dd_rescue reported the drive copied correctly but I still couldn't restart the array.
So I tried the old standby of recreating the array. This allows me to start it but the ext3 file system won't mount. So I then tried my script (listed in another thread) to try every combination of drives to assemble the array and mount the file system. Still no luck.
View 2 Replies
View Related
Jun 8, 2010
I broke down and spent some $$ on a new server for home use. I mostly do technical research and testing, plus store movies and music. My interests are mainly in the IET iscsitarget performance.
Server system consists of an AMD Phenom II 550, 8GB RAM, 1x 80GB system partition, and a LVM-vg0 software raid5, running Ubuntu 10.04 server x64.
The vg0 consists of 3 x 500GB 7200RPM SATA drives (mdadm) sliced up with 100GB for VBox VMs, one slice is an iscsitarget for a Windows 2k3 server, and another slice iscsitarget for a desktop.
With this setup, the win2k3 server is booted from a .vdi image stored on the ext4 lvm raid5 vg0. Here are the DiskTT stats.
Code:
However, with the Win2k3 server and the MS iscsi initiator I get using DiskTT:
Code:
The speeds are incredibly slow considering a non iscsi connection is screaming fast in comparison. Any ideas?
All nics are 1GB.
Here is my ietd.conf
Code:
View 1 Replies
View Related
Mar 26, 2010
So here it is, I have a fileserver setup on Ubuntu Server 9.10 AMD64. In this system is 4 1.5TB SATAII drives. They are configured in RAID5 using MD. Now everything is working beautify, except when you need to create a directory. When you do, the system hangs for about 30 seconds. The I/O wait in top jumps up as well. Then the directory is created and everything works once again.
The odd part is it is only Directory creation that does this. I can copy, move, download and stream files off the server perfectly. I am baffeled as to what is causing it. It might be related to the fact that I expanded the array from 3 to 4 drives, it was after that, that I first noticed the problem.
View 2 Replies
View Related
Mar 10, 2010
For the troubleshooting of one server (having 73Gb 3HDD, Raid5 of 140Gb). When I check in the Array The Logical Vol Appears as One HDD not Online
0 HDD1
1 HDD2
2 HDD3
0 HDD not showing Online, When we set it for Oneline & save, After restarting it will go off.
View 1 Replies
View Related
Aug 11, 2010
how to recover A mounted RAID5 data???
View 5 Replies
View Related
Jun 30, 2011
I know you can fail and then remove a drive from a RAID5 array. This leaves the array in a degraded state.
How can you remove a drive and convert the array to just a regular, clean array?
View 9 Replies
View Related
Mar 23, 2010
I am kinda stuck while providing solution for the above problem. I have achieved the fail over using keepalived but not sure how can we replicate the data from one server to other seamlessly and have them in sync with each other. My prime requirement for this project is end user should not notice the fail over and replicated copy of data should be available on the secondary as well.
View 3 Replies
View Related
Feb 25, 2009
I have a weird performance issue with a centos 5 running a nfs server and a rh8 client. I think the fact that it is rh8 client should be downplayed. It is just that with rh8 client the performance degradation seems more clear. See test details below OS in server is Centos 5 x86_64 kernel 2.6.18-92.1.22.el5
1Gb connection between machines File to test over NFS is a 1GB file. First of all I wanted to measure how the network alone performs while using NFS. So in the server side I run a "cat" command on the 1GB file to /dev/null. Please note that the disk read speed is about 98MBs. At this point the file system has the 1GB file cached in memory. In the client side a "cat" on the same file gives me a speed of about 113MBs. It seems then that the bottleneck in this instance is the network and it is very close to nominal speed. So the network performance is really good. (BTW I know that the server got that file from cache because a vmstat or iostat shows no disk activity.)
The second test is reading from disk with no caching involve. In the server I flushed the 1GB file from the memory. For instance by reading another 5GB file and I repeat the same thing as above in the client (a cat on the 1GB file). Now, the server has to go to disk.(vmstat or iostat shows the disk activity). However the performance, now, is about 20MBs, I was expecting something closer so 90MBs. (since the reading speed in the server in the first test showed 98MBs).
This second test was repeated for ext2, ext3, xfs with no significant differences. A similar test using a RH8 NFS server and client gets me close to 60MBs for a 1GB file not cache by the file system in the serverSince network speeds and disk read speeds are not the bottlenecks ... what or where is the limiting factor then?
View 4 Replies
View Related
Jul 20, 2010
I've got a new Ubuntu 10.04 server install with a new 3 disk RAID 5. The boot disk is separate, not part of the RAID. I was trying to practice what I would do if a disk died to recover the RAID, so I unplugged one of the three disks. The machine now just hangs on startup. It shows fsck at the top of the screen but doesn't got anywhere from there. If you press a key it shows the Ubuntu splash screen. If I plug the disk back in, everything boots up normally. So, my question is, how do I get the machine to boot with one of the RAID members missing? I know I can recover it using the Live CD, but it would be nice to be able to get back into the machine without the CD.
View 9 Replies
View Related
Dec 5, 2010
I'm a light linux user over the last couple of years and I decided to built a HTPC/NAS device.
Setup:
40gb ide -> usb boot drive
3x2tb sata (4k Sector) drives
I've got another 2tb identical drive but it's holding data that is going to be copied to the raid after it's up and running and then be 'grown' into the raid array to yield a final 5+tb array. I tried doing a disk util raid array and it ended up failing after reboot due to it using the /dev/sd* designations and they swapped. I have no idea how to do the UUID version, my googlefu and practical guide to ubuntu. So I decided to do it manually in order to also fix the sector issue as disk util wasn't formatting them correctly and once formatted wouldn't let me create a raid array from the discs.
[Code]....
View 9 Replies
View Related
Jun 8, 2010
I have a 2-quad core Intel Xeon system that I have set to RAID-5. I then attempted to install Ubuntu Server. When the message came up about enabling SATA RAID, I said NO because I have hardware RAID installed on the box. So I completed the install, and when I go to view the RAID configuration using the Intel Matrix Storage Manager during boot, it says:Volume0, Level: RAID5(Parity), Status: DegradedWhen I install enabling SATA RAID, I get a message asking me to "configure iSCSI volumes". When I click on that, I get another message: Loginto iSCSI targets. When I click on that I get another message: "Enter an IP address to scan for iSCSI targets". I have no idea what to put as thye iSCSI target portal address!
View 5 Replies
View Related
Mar 23, 2010
I am setting up a new server and am in the midst of testing RAID. This is an Ubuntu 9.10 server. RAID1 (/dev/md1) is spread across 12 one-terabyte SCSI disks (/dev/sdi through /dev/sdt). It has four spares configured, each of which are also one-terabyte SCSI drives (/dev/sdu through /dev/sdx). I have been following the instructions on the Linux RAID Wiki ([URL]....
I have already tested the RAID successfully by using mdadm to set a drive faulty. Automatic failover to spare and reconstruction worked like a champ. I am now testing "Force fail by hardware". Specifically, I am following the advice, "Take the system down, unplug the disk, and boot it up again." Well, I did that, and the RAID fails to start. It outright refuses to start. It doesn't seem to notice that a drive is missing. Notably, all the drive letters shift up to fill in the space left by removing a drive. The test I did was to:
[code]....
Is removing a disk from the bus a reasonable test in the first place? Meaning, is this likely to happen in a production environment by other means than a human coming by and yanking out the drive? Meaning, is there a hardware failure that would replicate this event? Because, if so, then I don't know how to recover from it.
View 1 Replies
View Related
Jul 5, 2011
I have an Acer Altos EasyStore SATA NAS box that hung, the only way to reboot was to crash the system (unplug it). Upon reboot it was not recognising the hard drives (it wanted to do a destructive reinitialize). Most of the importent data was backed up, however some was overlooked and we'd quite like to get it back. Removing the disks and placing them in a PC with enough SATA bays to cope, and booting with a live linux distribution (System Rescue CD) I can see the 4 drives are not suffering hardware error and that the original partions exist. Using mdadm I can assemble the Arrays without error (seems to be three but the only one I am concerned with is the RAID5 array of about 3TB). /dev/m1p2 mounts as a loopdevice once an offset is entered. In turn this mounts as an XFS parition. However despite df showing the partition almost to be full. ls -l or ls -a on the mount point shows it to be empty!
I got thusfar using a translation from a German language forum, unfortunately I only speak a little German, and the only other English language post on a simlilar matter I found within that site had no replies. The next step was to unmount loop, then run xfs_chack and xfs_repair on the file system. xfs_check returns that there is are a few dir size and offset errors along with link count mismatches. This I would presume normal for a file system that has become slightly corrupted. xfs_repair (version 3.0.3) gets as far as Phase 3 it finds and corrects zerolength entries, offsets on directories and bogus inode numbers. However the final two lines are:
Code:
realloc failed in blkent_append (2671166480 bytes)
zsh: segmentation fault xfs_repair /dev/loop1
A search on the error missing out data size just returns code to generate it, is anybody able to explain what it means? Also remounting hard drive, ls and varients of still do not return anything. Am I missing some thing (root I am logged in with now would have different credentials presumably to root on the NAS box, so how do I get around this)?
View 5 Replies
View Related
Feb 8, 2010
We are trying to define an appliance for an application server so I would like to know which should be the best file system type for this kind of use, basically our web applications uses libraries of 50 KB and our web apps.creates temp and logs files not bigger than 3 MB.
View 1 Replies
View Related
Jun 7, 2011
I recently upgraded a server from Fedora 6 to Fedora 14. In addition to the main hard drive where the OS is installed, I have 3 1TB hard drives configured for RAID5 (via software). After the upgrade, I noticed one of the hard drives had been removed from the raid array. I tried to add it back with mdadm --add, but it just put it in as a spare. I figured I'd get back to it later.Then, when performing a reboot, the system could not mount the raid array at all. I removed it from the fstab so I could boot the system, and now I'm trying to get the raid array back up.
I ran the following:mdadm --create /dev/md0 --assume-clean --level=5 --chunk=64 --raid-devices=3 missing /dev/sdc1 /dev/sdd1I know my chunk size is 64k, and "missing" is for the drive that got kicked out of the array (/dev/sdb1).That seemed to work, and mdadm reports that the array is running "clean, degraded" with the missing drive.However, I can't mount the raid array. When I try:mount -t ext3 /dev/md0 /mnt/fooI get:
mount: wrong fs type, bad option, bad superblock on /dev/md0,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
[code]....
View 1 Replies
View Related
Jan 26, 2009
I'm looking for a tool that can directly convert mp3 file to a swf file. Tried ffmprg - 'ffmpeg -i master.mp3 -ar 22050 -ab 32 -f swf -y -vn new.swf', output is as follows:
Output #0, swf, to 'new.swf':
Stream #0.0: Audio: 0x0000, 22050 Hz, stereo, 0 kb/s
Stream mapping:
Stream #0.0 -> #0.0
Unsupported codec for output stream #0.0
I found a way to perform this task using ffmpeg to convert the mp3 file to a wav file, and converting the wav into a swf file using swftools (wav2swf), although this is not a good option because the output swf file size is very big - for a 12mb input 320kbps stereo mp3 file, the swf file is 25mb. Also downloaded a free direct mp3 to swf conversion software, which had an output swf file - with the same 12mb mp3 file - of ~400kb.
The requirements I have from the tool are:
1. Can be ran from the shell.
2. Direct conversion, or an indirect one that will produce a small output swf file.
3. Suport for multiple file conversion is a plus.
View 6 Replies
View Related
Aug 25, 2010
I have gotten my DNS servers up and running, and query against them using dig returns the proper information. How do I get the information to be distributed across the Internet now?
View 1 Replies
View Related
Sep 1, 2011
I'm planning on setting up a new Linux box expressly for distributed computing (BOINC, SETI@home, etc.). All things being equal, what's better- More clock cycles or more cores?
View 4 Replies
View Related
Oct 19, 2010
I'm looking for the best distributed filesystem to share my home directory among all my computers, and I'm interested in what advice I could get here.
Here are my requirements:
1) It must be completely POSIX compliant, because I've noticed that many programs will break if not.
1a) Must support file locking (i.e. flock()) preferably sharing lock advisories between the clients
1b) Must support hard links, and be able to recognize a hard link connection by looking at the number of references
2) It must have encryption, and security suitable for it to be exposed to the internet.
2a) I would like to use password-less authentication. Preferably SSL client certs (HTTPS style). Static keys (SSH style) are also fine. I would begrudgingly accept a password authenticated system if it used a fairly secure challenge/response algorithm if it met all of my other requirements. (Extra points if it supports PKCS#11 and thus supports smart cards)
2b) Encrypted traffic must be an option.
2c) No thanks to kerberos. It was too annoying to maintain when I tried it before and I don't like passwords.
3) Preferably, it should be stable and fairly easy to maintain.
4) Optionally, it should be tolerant of network drops, and reconnect automatically as needed, because my laptop wireless is a little flaky sometimes.
View 4 Replies
View Related
