Software :: Layered Filesystems Reliably Abstracted From Each Other?
Jun 14, 2011
Can FUSE filesystems be layered/nested? (I would use the verb "stacked", but that might confuse the issue with kernel module file system stacking in the same process space).I've scoured the web an haven't been able to find an answer to this question. I've also tested emperically for the answer, with inconclusive results. I have numerous use cases in mind for this. E.g.:Mp3 tagging FS on top of ZFS-FUSE. Union FS on top of NTFS-3G. Caching FS on top of checksumming FS on top of compression FS on top of encryption FS on top of webdav FS. And so on. (Yes I realize the last one would be a stretch in any scenario.) Or basically, ANY special-purpose FS on top of ZFS-FUSE, or NTFS-3G. (My particular interest is the former.)
I have tested this out - specifically, encfs on top of ZFS-FUSE. It didn't work. I don't recall the exact problems nor did I document the results, but the result was a very unhappy filesystem. (It did actually all mount without error if I remember correctly.) I tried many workarounds, and the end result was a non-functioning file system (I know that's vague; as is my memory). So that got me thinking, is it even possible? Is FUSE even designed for this? And if it is, is it designed in such a way that layered filesystems are reliably abstracted from each other so that they don't care and can't know what the underlying FS is? I do realize there would be performance penalties with these use cases, many non-exact alternatives, etc. So it would be nice to not receive use-case/alternatives lectures that (some) in the Linux community feel compelled to provide
After installing Ubuntu to a flash drive, and customizing it heavily, I am running out of room (8 percent remaining on a 4GB flash drive). While I could go buy another flash drive, I already have plenty of storage between a 500GB external hard drive and a DLink NAS. What I would like to do is use unionfs to mount an image (around 6GB in size) over my 4GB flash drive. When it is mounted, all file changes should be synced to the unionfs image, thus "archiving" my flash drive. I have created the 6gb image, formatted it as Ext4 (same type as my flash drive), but when I mount it (using 'unionfs-fuse /media/SG_EHDD/devices/unionfs.dev -o cow /') any changes that I make are not made on the image. I think I am missing an option, namely the -o chroot=SOME_TEXT, but when I put -o chroot=/media/SG_EHDD/devices/, it says it cannot load /media/SG_EHDD/devices//media/SG_EHDD/devices/unionfs.dev. It also gives me the same errors when I cd to /media/SG_EHDD/devices and run unionfs-fuse with ./unionfs.dev for the path to the filesystem image.
I've just bought a 6-core Phenom with 16G of RAM. I use it primarily for compiling and video encoding (and occassional web/db). I'm finding all activities get disk-bound and I just can't keep all 6 cores fed. I'm buying an SSD raid to sit between the HDD and tmpfs. I want to setup a "layered" filesystem where reads are cached on tmpfs but writes safely go through to the SSD. I want files (or blocks) that haven't been read lately on the SSD to then be written back to a HDD using a compressed FS or block layer.
So basically reads: - Check tmpfs - Check SSD - Check HD
And writes: - Straight to SSD (for safety), then tmpfs (for speed) And periodically, or when space gets low: - Move least frequently accessed files down one layer. I've seen a few projects of interest. CacheFS, cachefsd, bcache seem pretty close but I'm having trouble determining which are practical. bcache seems a little risky (early adoption), cachefs seems tied to specific network filesystems. There are "union" projects unionfs and aufs that let you mount filesystems over each other (USB device over a DVD usually) but both are distributed as a patch and I get the impression this sort of "transparent" mounting was going to become a kernel feature rather than a FS.
I know the kernel has a built-in disk cache but it doesn't seem to work well with compiling. I see a 20x speed improvement when I move my source files to tmpfs. I think it's because the standard buffers are dedicated to a specific process and compiling creates and destroys thousands of processes during a build (just guessing there). It looks like I really want those files precached.....
Every time I right click on my CD/DVD with my DVD+RW popped in the drive, it freezes for like 10 minutes and then it unfreezes with nothing happening.I think I formatted it before using ImgBurn but when I did it went from 4.7GB capacity to 3GB. Is it possible that I accidentally turned my double-layered DVD to single-layer? If so is there a way to turn it back to double-layer?
I want to simplify some of my rules, so I want to create rules for certain services like xmpp, web, etc. since some of them use multiple ports, and I toggle them on/off a lot. Can I simply put the jump to rule clauses in the Input chain, and once the sub chains run, does it return to the input chain after the jump to rule clause? I want to do this so I don't have a ton of rules in the input chain. I think that if I simply make a list of all the rules to jump to in the input chain, it will work itself through all of them until it finds a matching filter in one of them correct?
Ok this is a problem that has dogged me for years but I was hoping it was fixed in 11.4. Using the acroreader plugin pdfs sometimes work but most often they result in a white or black blank screen and once one pdf has failed then they all fail until I restart firefox. It is a huge pain - under kubuntu I have used mozplugger to display pdfs with okular in the browser but this doesn't seem to be in the repos.
I don't think it is just me, all the people I know using suse seem to have this problem but it never gets fixed. how to get pdfs reliably displayed in firefox?
I can't get HDMI sound reliably when using Ubuntu. The strange thing is, if I repeatedly reboot the computer, eventually the sound will work. This can take anything up to 10 reboots. Once working, it will keep on working as long as the machine is not rebooted or shut down. HDMI sound works fine under Windows.
aplay -l will only list the Intel (analogue and S/PDIF) hardware, not the Nvidia, even when it is working. Selecting between analogue and digital output under Sound Preferences has no effect on the HDMI either way.
I've been having problems connecting to my MSN (Hotmail) account with Empathy. Empathy is in my start-up applications, but every time I start my PC MSN is disconnected. I have to run "killall telepathy-butterfly" then disable and re-enable the MSN account in Empathy to get it working again.I read today in another forum post that you can remove telepathy-butterfly. So I've tried that. Now I can't get it to connect to my MSN account at all.
I've tried using the "HTTP method" and not, restarting Empathy, recreating the account, but whenever it connects it immediately disconnects and now says "Disconnected - Connection has been replaced by a new connection using the same resource"I only have one MSN account in Empathy, although I also use it for Google Talk, Facebook, Yahoo!, and People Nearby.Any ideas how I can get MSN reliably working every time without interference?
I have a DSL internet connection. The DSL is just a cable that comes into my house, no routers under my control. For a long time the internet functioned fine in Ubuntu using the default Network Manager. Recently however, my ISP was bought by another ISP, and the service name had to be changed in the DSL settings. In Windows, it connects without any issue whatsoever, only the service name change was required... But in Ubuntu, the first time I connect I get a "Apache: Not Found" page, the next time, I get to the previous ISP's login page, it continues like this for some time until I get it working. I have to try connecting it again and again.
When I asked the customer service, they said that because of the takeover, they have switched servers. But they said, it should work properly in Windows, and they were clueless about Ubuntu. Is there anything I can do to solve this issue?
To replace my aging hard-disk, I bought a new, bigger disk to hold two NTFS partitions (system + data) to run XPSP3.
Unless I missed it, PartClone in Clonezilla doesn't resize a partition to use the extra space. So I now need to resize the second partition.
Since NTFS is a proprietary filesystem, before I go ahead and use Gparted or some alternative, can experienced users confirm that resizing NTFS partitions using Linux tools is rock-solid, or I better use a closed-source, Windows-based solution instead?
I decided to try kubuntu after reading over the webs. I got lastest 10.04 release of kubuntu installed (downloaded yesterday).
So far i dont seem to get any mediaplayer to work correctly with video files (avi's and Video_ts.ifo's). I've tried:
- built in Dragon - Vlc - mplayer and kmplayer/smplayer - xine
The result is as follows
- Dragon plays only Dvd's "vob" files but cannot switch audio or subtitles - vlc wont open ANYTHING at all - mplayer crashes on anything as well - xine will play some videos as dragon but wont play vob files (no "muxer")
At first i installed recommended stuff to play mp3's. When i tried to watch films or dvd's i couldnt do it with Dragon so i tried to install so advertised VLC. I did, and when i open ANY file in VLC, it closes itself. Just closes. Or on rare occasions (when opening some of dvd's files) it just opens a big black screen and reverts to starting small state w/o screen or file opened. I can even sometimes see the first picture from the video blink before it closes. I couldnt get ANY file to play on VLC not even an mp3... I then installed some packages that told they contain codecs to play alot of formats, and package for "encoded dvd's" - nothing.
After that i've seen that now i can play some videos (avi's and stuff) and even DVD's with built in Dragon video player, but dragon cannot understand subtitles or different audio streams. I tried mplayer and Xine with no luck as well.
Now since its not a problem of one player, but i cannot get to play video_ts.ifo files at all and cannot get vlc and mplayer to play anything at all, there must be something i am doing wrong.
I recently installed Lubuntu 11.04 on my aunt's old computer (Sony PCV-LX800); I had it running Ubuntu for a few years but the latest releases are too much for it to bear. Anyway, a while back I installed a DVD-ROM drive in her computer in order to play DVD movies. It plays CDs and non-commercial DVDs fine, but with the commercial DVDs, it doesn't work 80% of the time. The 20% it does work, everything is fine.
I've installed all the necessary packages with every new release. Currently using the following: libdvdcss, libdvdnav4, and libdvdread4 (via the Medibuntu repository).Here's some info from this system while trying to play the DVD LotR: Return of the King Extended Edition:
Code: gloria@gloria-PCV-LX800-U:~$ mplayer dvd://1 MPlayer 1.0rc4-4.5.2 (C) 2000-2010 MPlayer Team
I'm having trouble getting this computer to boot up a lot of the time. It's an older AMD Athlon(tm) 64 Processor 3200+. Motherboard is (apparently, I'm just copying from the Sysinfo app) AMD K8 Athlon64 Opteron. I have Ubuntu Jaunty and Windows XP installed, but I don't know how relevant that is. When booting, it will simply stop around the stage where it detects the HDD and DVD drive, or right after. The last thing I see is the line with the DVD drive being found. I can't get into the BIOS, it does not make it that far before freezing.
If I keep rebooting, sometimes it will boot normally allowing me to access BIOS settings or make it to GRUB and start up the OS. If I do manage to boot the computer successfully, it is extremely stable. I often go months without a reboot. The problem is it's nice to be able to reboot once in a while without fear of the computer crossing over into the land of the dead. I can't figure out what's wrong, but the problem seems to be very early in the booting stage.
[root@ncc1701 ~]# hostname ncc1701 [root@ncc1701 ~]# service httpd restart Stopping httpd: [ OK ] Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using 63.123.155.104 for ServerName [ OK ]
problem is that I have no idea where it is getting the 63.123.155.104 address from. Its not mine and its not the IP of the server..
How do I put the FQDN in the etc/hosts file, server ip is 192.168.2.101 hostname - promisedland fqdn - promisedland.dyndns.org I just want to set up a web based file server. Here is what i put in hosts (/etc)-gedit
I am facing an error:"Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using TestServer.localdomain.com for ServerName"When I am restarting the httpd service.Can anyone please tell me what is this error about & how to fix it.bare with me as I am a newbie, so kindly requesting for the easy explanation.steps to set the Domain name.
I have two computers that have a direct ethernet wire between them. The interface is set to a static ip address on each side, and under control of the NetworkManager.
About 1/10 the time the interface does not start. The messages log file will show:
Jun 22 13:01:22 owl10 NetworkManager[601]: <info> (p6p1): carrier is OFF Jun 22 13:01:22 owl10 NetworkManager[601]: <info> (p6p1): new Ethernet device (driver: 'r8169' ifindex: 2) Jun 22 13:01:22 owl10 NetworkManager[601]: <info> (p6p1): exported as /org/freedesktop/NetworkManager/Devices/0
[Code].....
So, I imagine there is a race condition where both boxes wait for the other to start the interface.
My question: Is there anyway to force NetworkManager to bring up an interface even if there is no Carrier?
BTW, this problem started recently on Fedora 13 after some upgrades. We upgraded to the latest Fedora 15 to try and solve it, and it now appears more often than it did on Fedora 13.
I want to know what file system there are for "/dev" directory??apparently different types was developed for managing devices on Linux.because I am a little confused about all file system on this directory.Another question is,which is file system sufficient for managing devices on Embedded Linux ?of course on our embedded system we do not have many pluggable device,so this directory can be static
I'm now using Ubuntu 10.04 with ext4 and for the second time in my life I experienced data loss (not for real: I got backups) and I'm assuming problems with the recent ext4 fs.
I want to restore all of my configurations (/etc and the like), data and home on reiserFS: is this possible? What to do in order to accomplish that?
I am tired of watching fsck check my filesystem when my eeepc 901 shuts down abruptly due to a crash. I know that with a journaling filesystem, I won't have to wait for a check. However, I am well aware of the poor I/O performance of the SSD, so I can imagine using a journaling filesystem being even more frustrating, since there will be constant writes to the journal? I will buy a new laptop without such a crummy ssd someday but, is there anything I can do now, on the software side of things?
i need to install windows to my computer, but i don't want to loose my linux either.So what i wonder is, since my filesystem is ext4, which filesystem can be used for both operation systems.Do i need to have fat32 or ntfs or can i still use under windows ext4? I have a large collection of movies on my one partition and if i have to convert it, then it will be kinda pain.
i have about 22 gb of music (mp3 & ogg) on my laptop harddrive.i also have an unused sony mp3 player with a 20 gb hard drive.what i want to do is back up the 22gb into a 20 gb space the music does not need to be playable on the sony player...just using it as a back up device.ok...2 issues:1. when i've tried compressing (tar.gz) mp3 files, little to no space is saved, i assume that a mp3 is pretty compressed already.is there another way to compress effectively ? i dont want to reduce bit rates of the individual music tracks.2. i formatted the sony hd using ext4, but this leaves me with only 16 gb usable space. tried fat32 and this left me with about 18gb.
I need to combine 6 different filesystems into one filesystem using rsync. I am so confused as to which parameters I need to use. The 6 fileystems are:
I have a 2TB USB drive which I use as a backup device - I dump two filesystems onto it, totalling around 1TB. However, doing the dump trashes my F11 system, making it basically unusable, not only during the dump but also afterwards. I have 8GB of RAM, all of which is needed and normally in use, but when dumping, the system starts hogging huge amounts of it as buffer space - up to 1GB of RAM is reported to be allocated. And rather than using free memory for buffer space, it seems to aggressively swap processes out to get it. The system tends to melt down as a result, and just switching virtual desktops can take 5 minutes.
But after the dumps finish, the problems continue - the system is currently trying to keep around 700-800MB free, and continually swapping out processes to do so, even after the buffer space in use has gone back to about 100MB. This seems like strange behaviour for a fairly common type of activity. Presumably a lot of the buffer space is used to store what is being read from the filesystem (which will never be needed again), and some is used to cache the writes to the USB drive which is slower than the internal hard drives.
I have spent a lot of time trying changes to some of the kernel parameters, after reading articles about them. Of all the ones I've tried, setting vm.dirty_ratio to 1 instead of 5 helps a bit, and setting vm.dirty_background_ratio to 5 instead of 20 makes some improvement I think. Setting vm.swappiness to 0 doesn't seem to help at all.
So my question (at last!) is - how can I back up my filesystems without my system dying? In particular, can I limit the space used for buffers somehow, or turn off buffering for the dump process? And why does dumping result in the system artificially keeping huge amounts of space free afterwards, so I have to reboot to make the system usuable again?
What are others' views and experience regarding automatically checking filesystems (running fsck) at boot time?To be more clear, I have left the ext3 filesystems on this machine set to require checking after a fixed number of mounts by using tune2fs with the '-c' option. I've done this mainly because of the following (from the tune2fs man page):
Code:Youshould strongly consider the consequences of disabling mount-count-dependentchecking entirely. Bad disk drives, cables, memory, and kernel bugs could all corrupta filesystem without marking the filesystem dirty or in error.e using journalingon your filesystem, your filesystem will never be marked dirty, so it will not normallybe checked. A filesystem error detected by the kernel will still force an fsck on the nextreboot, but it may already be too late to prevent data loss at that pointBut what does anyone else do? Is there really much risk to disabling this automatic checking
I have a mysterious filesystem issue with openSUSE on an old machine. Installation wasn't very pleasant but worked finally. It boots fine. However after 5 or 10 minutes, all filesystems vanish. Most programs (but not all) cannot be found anymore. "ls" output is empty everywhere. "top" used to work (it was working on a ssh connection that I just lost). Now it complains about unset TERM variable. I was able to set it again to xterm. Now "top" is working, displays the running processes : mysqld, Xorg, icewm (my WM), nothing strange there, nobody is eating up the resources. I have other linux installed on that machine : Ubuntu freezes occasionally, mostly with Gnome, but it seems to be a totally different problem (maybe related to the radeon driver) ; Mandriva works fine so far. The two IBM 120 GB harddrives are quite old but the manufacturer fitness test didn't find failures. I also checked for bad sectors before formating in openSUSE setup. It doesn't look like a physical I/O problem. In such a case, it would crash or at least complain about not being able to access the HD. Well ... I switched to console but I didn't get any login prompt and now I cannot switch back to X. I can still ping the machine but of course not ssh to it. SysReq keys (although enabled) doesn't work. I guess I will have to reboot brutally. Last time it took me about one houre of pressing 'y' tp repair the filesystem with Ubuntu. I'm sure I will be back soon. What do you guys think about that situation ? I suspect a kernel issue with that hardware. Have you heard about similar problems ? I'm afraid I will have to reinstall 11.1, which used to work fine, on that machine.
Some details about the hardware : The (onboard) Promise controller and the SATA controller are not used. The 2 HD are on the first IDE interface as master and slave.
