Server :: NAS On Btrfs - Possibly Zfs/xfs - Software-getting Started ?
Jun 28, 2010
Currently have multiple small DNS-323 NAS-in-a-box that need to be replaced with a single storage server.
Since the setting is for an advanced home server (2-3 simultaneous clients max) that will hold movies and application/game ISO's, I was originally going to go with a simple RAID6 on freenas. As I learned about silent data corruption, and with a max of 3 simultaneous users, I don't see a need for a hardware raid processor. So I gathered that software RAIDZ (RAIDZ2) based on ZFS or BTRFS that prevent silent data corruption would be a better match (please correct me if I am wrong).
NEEDS: The needs (in addition to prevention of silent data corruption) are basically online capacity expansion (OCE), and 2+TB volumes (64bit LBA); with a preference for the ability to spin down idle drives. I'm going to make the assumption that any solution I go with will allow me to read/monitor actual SMART drive data.
STUMBLING BLOCK(S):
1. Would someone validate the (perhaps mistaken) assumption that since BTRFS is integrated into the latest Linux kernel, it is mature enough for NAS candidacy.
2. Please validate: Forward-compatibility of BTRFS has been stated (no need to reformat with subsequent BTRFS updates).
3. Am I missing any NAS software (open-source or otherwise) that would meet these requirements and provide a nice pretty graphical interface?
4. What software solution would you recommend? (btrfs on xxxxx? zfs-fuse? linux+xfs (+mdadm)? ..?)
5. What are my next steps?
Accumulated hardware for project:
SERVER: 3u chassis with 16hot swap trays. Tyan S5360G2NR i7520 mainboard. Single Intel 2.8Ghz CPU (dual-capable). 4GB ECC RAM (do i need more?). Dual hot-swappable 550W psu. CD-ROM
DRIVES: Eight 1.5Tb SATA drives, plus an additional four that are in-use in the DNS323 that I will migrate over using OCE. Four 750 SATA drives that are also in-use and can be migrated over via OCE. One 16Gb SATA SSD, intended for Boot OS and logs.
INTERFACE CARDS: Two 3Ware Escalade 9500S-8 8Port SATA RAID (PCI-X, 64bit) [can change these to something else].
UPS: 1500VA
I am working on a NFS server embedded on a PowerPC plateform (4650EX, 512Ram, 1 GB Ethernet) but i can't mount my exports folders from my client. Here are messages :
I'm looking for a write one, read many file synchronization, file system or similar technique. In fact I want a to imitate a zfs_send/zfs_receive similar behaviour on Linux. My requirements are pure local (or in terms of distributed file systems: rack-aware) file handling with remote synchronization. Some random features/requirements I want to achieve are (I use the terms file and "file system" here interchangeable, as I don't care wheter I have to synchronize file based or use full file system supporting that): Access on files from a specific host is pure local, i.e. the node has a full copy of a file, no parts are distributed. Read and write access is local, the remote side gets synchronized with write changes only Write changes on the remote side are non-blocking and asynchronous No need for concurrent file access, concurrent, distributed locking and so on Every file is changed by a single node only, other nodes have read access The file "owner", thus the node allowed to write files must be changeable Similar techniques (some as technology concept, not usable for me) exist, among those DRBD, MySQL Replication or even pure rsync.
I don't care whether I'm referring to a pure local solution (i.e. between two block devices) or a client/server mode. DRBD won't fit because its a peer to peer solution, I need to synchronize a lot of clients with a central storage through that way where DRBD just mirrors from peer to peer among two nodes and doesn't allow another node to take over the role of the primary node (i.e. node A is master, storage B secondary, I want to change to a thrid node C which becomes master and receives hence a full update from storage B.
Just as a guess, I'd make a local LVM volume on every node where write access happens. In the same time there exists a iSCSI target with multipath to a remote hosts which is shared with every node, again providing a clustered LVM. So I'd need to synchronize a local LVM volume with a remote LVM image. The question is just by which technique I could achieve that (i.e. as RAID 0, where every read and write goes to one node only and doesn't wait the other node to succeed). I could achieve this with rsync by a cyclic push to the remote host, but the problem is, that rsync isn't block based thus I'd need to synchronize every time the whole file, not only changes.
I recently installed KDE in my ubuntu 10.10. To access KDE, I want to use startx /usr/bin/startkdeBut using it disables sound in KDE (Sound is working fine in GNOME). It doesn't sound for anything like login sound, totem, mplayer or any other playerBut when I press Alt+Ctrl+F1 to change to virtual console, the playback resumes from where it was in time and when coming back Alt+Ctrl+F7 and the time in totem (or any other player) doesn't move. While log out also it doesn't play logout sound and doesn't logout, so I have to press Ctrl+Alt+F1 so that logout sound play then it exits.When starting KDE by kdm or gdm, the sound works normally. But I don't want to login again using kdm or gdm and not to use root user to start kdm or gdm.I don't know what is the difference between when KDE is started by startx or by kdm/gdm where the same user login in kdm/gdm as that for startx
Is the Server or Desktop version of Ubuntu a better choice for a home file (and possibly a media) server?
To explain a little, I currently have a WinXP Pro pc that provides file storage for the users in our home (spouse, kids, self). On this pc I have a windows program ACRONIS running to perform automatic backup of every ones stuff. This has saved me many hours of headaches when someone has a HDD or PC failure. Everyone in the household directs all of their downloading activities to a download partition where I have an anti-virus & anti-malware program monitoring all new entries. This too seems to have been a good decision over the years. I'm trying to sort out which version of Ubuntu will be the most logical entry point.
I expect the questions I am seeking insight for are the normal getting started stuff. Using several different search criteria, I did not hit upon existing threads for the topic If there is a more appropriate forum / sub-forum, please let me know and I will take this question (thread) there.
At our university server, I patched together a script (lets call it connect.py) that kind of glues together a series of calls to different other programs (the first one is in Python2.X, the second is a binary so I use os.system('/path/to/program input.pot') to start it, and the last is in Python3.X). It kind of seems to work well now when working under my user account, where the script has the permissions to call the programs, because I installed them.Now, when I want to start the connect script by pressing a submit button in an HTML form, naturally I start running into trouble. Now its no longer my user who is calling, but the server (Apache) with very limited rights. In the progress of writing the connect.py script, I had to do quite some environment adjustment so the various Python modules could be imported.
What is the easiest way to get the script started from an HTML form? I guess it would not be good practice to start extending the PATH variables of Apache, would it?
Evidently I need to run Jack Control in combination with Qsynth to play my midi keyboard. That was ok for about a week or so, then Jack developed a problem. This was solved the first time by me unticking the Realtime box in Jack setup, but that has not worked this time. Must I rely on Jack just to have my midi keyboard sound?
The errors messages I receive are: Could not connect to JACK server as client. - Overall operation failed. - Unable to connect to server. Please check the messages window for more info.
and 12:45:12.427 Patchbay deactivated. 12:45:12.480 Statistics reset. Cannot connect to server socket err = No such file or directory Cannot connect to server socket jack server is not running or cannot be started 12:45:12.533 ALSA connection graph change.....
For some reasons I need to configure ubuntu in such way that it would turn off the computer if X-server or GDM hasn't started (but user is able to log in). By the way, what are the reasons for such situation besides errors with video drivers?
I was told that I should write a script, but I have no idea how to write scripts and where I should place it (I suppose in init.d with some parameters).
I am having some problems with rsync,I put the rsync command in a shell script, and I am calling the shell from a 3rd server, but it stops midway through.s anyone else experienced these problems?
I am re setting up a server of mine running red hat enterprise Linux server 6 and I had all of this working befor but for some reason I had troubles getting sasl to work and now when I login my smtp server I get an error stating that my username or password is incorrect though I am sure I am entering both correctly. Would anyone know what could be happening? I have been spending days on the web looking for the solution and only went from sasl not working when started as a service to this. For some reason I can't use Pam with saslauthd and had to use shadow instead of which from what I hear I get to use better methods of secure authentication with smtp
I am about to setup a computer with Maverick. This will be for experiment use only, there will be nothing of any critical nature on it. Should I consider using the BTRFS, yes or no ? What would be the advantages/disadvantages (if any) on using it as opposed to EXT4 ?
I'm just diving in to figuring out how to partition/utilise BTRFS.I am used to just installing with EXT4 and carving out a / and a /homeBut, from what I understand, this isn't the case with BTRFS?I know you have to create a separate /boot as grub doesn't support the file system.But, with BTRFS, we just create a / and /home and others would then just be subvolumes?What happens if I want to reinstall? I have liked being able to just wipe / and reinstall the OS, leaving my personal files in tact. Does this still happen if the /home is just a subvolume? Hopefully that makes sense
Okay, so I found a couple articles about the new BTRFS for Ubuntu that became available on 10.10, and I was wondering if some people could clear some stuff up for me. I am going to be upgrading my desktop box from 8.04 soon (I know,long time coming, but I had some hardware compatibility issues with recent releases that have been addressed on my end) I'm pretty new to a lot of these terms, so a detailed explanation would be awesome if you don't mind.
1 - Is this file system ready for the home user?
2 - Are there significant performance gains over ext4 and is it stable? If not now, will it be in the future?
3 - When a new subvolume is created, does that act like a partition? For instance, would a home folder in it's own subvolume act like home on a separate partition in the event of a reinstall or upgrade to the root filesystem? If not, what would be the specific advantages to using a subvolume?
4 - When creating a snapshot, is it bit for bit or a compressed image?
5 - Does compressing the root filesystem save a significant amount of disc space?
6 - Is there anything else that would be important to know about this filesystem?
If I have some things confused or misunderstood, it is because I am just starting to understand how some of the foundational stuff in my OS works
I'm a big fan of ZFS on FreeBSD (I've been using it on my home server since before it got stable; bleeding edge, baby!) and I'd like to try out btrfs to see how that's evolving. Since it's still largely in development, none of the usual mainstream distros have btrfs as an option. I haven't used Linux in a bunch of years, so I don't really know what my best options are for giving btrfs a try.
I decided to try playing with btrfs. I think you could do lots of fancy things with that. One thing puzzles me though - I can make a snapshot or a subvolume the default volume to mount (using btrfs set-default , but how do I then remove that subvolume default so that the base filesystem is the default mount again.
I have installed Debian through the Debootstrap process using the ext4 fs for root and it worked without a problem. When I tried to install Debian mounted on btrfs subvolumes, there are problems mounting root while booting so it crashes... Any clue if Debian supports btrfs subvolumes?
I hear that the non-live version of F15 has an option to choose the btrfs filesystem. I only see the DVD version as a non-live version and it's 3.4 Gigs. Is there a non-live CD version available?
I'm currently running the XFCE Spin of Fedora 15 (Xedora, as I like to call it (; ) on 64 bit, so sometimes I need a 32 bit environment to try stuff out.
I recently installed Linux Mint on a partition formatted to btrfs, and this is where my problems start, as I have no idea how to add this installation to my existing GRUB configuration. The grub.cfg on the Mint partition looks like this (GRUB2 btw.):
Code: # # DO NOT EDIT THIS FILE # # It is automatically generated by grub-mkconfig using templates # from /etc/grub.d and settings from /etc/default/grub #
I upgraded my 9.10 installation to 10.04 and decided to try out btrfs on a some spare drives in the system.sudo mkfs.btrfs -m raid0 /dev/sdb /dev/sdc /dev/sddthe only way the system sees the btrfs array is by running btrfsctl -a and then mounting /dev/sdc. but that has to be done in userland, not at boot. if i try to mount it via fstab, ubuntu won't load because it can't find the mount point./dev/sdc /Images atasum,thread_pool=128,compress,rw,user 0 0so where am i going wrong? I tried mounting via the UUID also but that didn't seem to work for me either.
During the installation of Ubuntu 10.04 the partitioner was wrongfully configured to see a functioning btrfs partition as ext4 (without reformatting it). Thus the installation process got stuck at 5%.Installer was run again ignoring the btrfs partition.btrfs-tools was added to the new 10.04, but the btrfs partition is now recognized ast4 with lost+found folder on it.Tried to add the btrfs to etc/fstab as btrfs but t won't mount.Can the partition/filesystem type be changed so that this is actually recognized and mounted as btrfs, hoping my data is still on it somehow?
I'm a bit concerned about this. I've installed Kubuntu 10.10 tonight, and had tried it a week ago briefly without updates. Both times I've had this really loud sound which I'm guessing is the fan going a bit mad. Is this related to a newer kernel or something? Is it dealt with easily? This is a Toshiba Equium laptop from 2 1/2 years ago. I've got the Powersave setting on now in the hope this might stop it happening while I'm trying to read up on this and get advice.
After all the observed Linux progress since I've been using it I'd be quite annoyed if something botched in the OS kills my laptop. The computer got quite hot, even though its feet are rested on a wooden board that normally prevents this.
had 10.04 and one day the pictures on facebook stopped showing up. like the display pics. i figured it was an error so i finally got the error fixed, but the facebook picture problem was still there. i thought upgrading to 10.10 would fix it, but it didn't. anyone know why the pictures won't show up? this never happened using windows and they ppear on every other windows computer i use
I was interested in the idea of the btrfs subvolumes, so I made a virtual machine and installed Slackware as per the instructions here: [URL] It all went very well, but when I tried to switch from the huge kernel to the generic kernel and use the initrd.gz generated from step 29 (except that I used 2.6.37.4-smp instead of whatever's there) in lilo.conf, it failed to boot. I also noticed that in the instructions themselves, the poster doesn't actually add the initrd.gz to lilo.conf, so I'm guess the huge kernel has everything it needs to boot properly.
I got a full system crash each time I try to do anything with network. Been fighting with this problem for weeks now. What I did: -tried every driver for broadcom4312 and it turned out that it even crashes with cable net
-got rid of networkmanager ( Mar 11 08:01:43 geburah NetworkManager: <info> Activation (eth1) successful, device activated. Mar 11 08:01:43 geburah NetworkManager: <info> Activation (eth1) Stage 5 of 5 (IP Configure Commit) complete. ) used: ifconfig eth1 up
I have an OpenSSH server running on Ubuntu 10.04, and it works fine.
I'm concerned that my SSH key may have been compromised and would like to replace it.
I tried replacing keys before and reinstalling OpenSSH and SSH before but ran into terrible trouble so I'm asking for instruction before touching anything this time.
Code: laeg@skyrocket:~/.ssh$ ls authorized_keys id_rsa id_rsa.ppk id_rsa.pub known_hosts Code: laeg@skyrocket:/etc/ssh$ ls
[Code]....
So can I just synpaptic 'fully' uninstall SHH (although probably even less necessary than..) and OpenSHH, backup sshd_config, delete the two dirs referenced above, reinstall both packages, insert my sshd_config backup, and then start from scratch following the guides linked below?
I'm running Maverick on an older Stinkpad. I have a btrfs /home as a second partition on the internal HDDand a brand shiny new 3TB external USB western digital drive also formatted btrfs. I'm having a problem whenever I hibernate/suspend the laptop by closing the lid and start it back up later, the external drive won't remount unless I do a complete reboot. I have no entry in /etc/fstab for the external drive.Quote:Unable to mount "blah" DBus error org.gtk.Private.RemoteVolumeMonitor.Failed: An operation is already pending
I have an HP laptop. I had Ubuntu 9 running perfectly for months and now it won't boot at all. When I start the computer it begins with the BIOS screen (and option to enter startup/boot menu). Then, that's it! Black screen, no matter how long you wait. I am running Ubuntu live off of a USB stick so I know all my hardware is fine.
When it crapped out: I was doing basic things such as I had Gedit, FireFox and Chrome (for emails) running. I wanted to do some image, batch edits and after googling a bit decided to install Picasa. The install seemed fine and I had Picasa up and running. All of a sudden a text file I wanted to edit with Gedit gave me an insufficient permissions error warning.
I do not remember exactly what I did from there but basically I closed out everything and shut down the computer, it hasn't booted since.
I really don't want to have to reinstall/reconfig my whole computer again. However I am not all that savvy with computers let alone Ubuntu, so if it's going to be a long, drawn out process trying to recover...
