General :: Filesystem With Inodes Close On The Disk?
Jan 9, 2011
I'd like to make the ls -laR /media/myfs on Linux as fast as possible. I'll have 1 million files on the filesystem, 2TB of total file size, and some directories containing as much as 10000 files. Which filesystem should I use and how should I configure it?As far as I understand, the reason why ls -laR is slow because it has to stat(2) each inode (i.e. 1 million stat(2)s), and since inodes are distributed randomly on the disk, each stat(2) needs one disk seek.Here are some solutions I had in mind, none of which I am satisfied with:Create the filesystem on an SSD, because the seek operations on SSDs are fast. This wouldn't work, because a 2TB SSD doesn't exist, or it's prohibitively expensive.
Create a filesystem which spans on two block devices: an SSD and a disk; the disk contains file data, and the SSD contains all the metadata (including directory entries, inodes and POSIX extended attributes). Is there a filesystem which supports this? Would it survive a system crash (power outage)?Use find /media/myfs on ext2, ext3 or ext4, instead of ls -laR /media/myfs, because the former can the advantage of the d_type field (see in the getdents(2) man page), so it doesn't have to stat. Unfortunately, this doesn't meet my requirements, because I need all file sizes as well, which find /media/myfs doesn't print.Use a filesystem, such as VFAT, which stores inodes in the directory entries. I'd love this one, but VFAT is not reliable and flexible enough for me, and I don't know of any other filesystem which does that. Do you? Of course, storing inodes in the directory entries wouldn't work for files with a link count more than 1, but that's not a problem since I have only a few dozen such files in my use case.
Adjust some settings in /proc or sysctl so that inodes are locked to system memory forever. This would not speed up the first ls -laR /media/myfs, but it would make all subsequent invocations amazingly fast. How can I do this? I don't like this idea, because it doesn't speed up the first invocation, which currently takes 30 minutes. Also I'd like to lock the POSIX extended attributes in memory as well. What do I have to do for that?Use a filesystem which has an online defragmentation tool, which can be instructed to relocate inodes to the the beginning of the block device. Once the relocation is done, I can run dd if=/dev/sdb of=/dev/null bs=1M count=256 to get the beginning of the block device fetched to the kernel in-memory cache without seeking, and then the stat(2) operations would be fast, because they read from the cache. Is there a way to lock those inodes and/or blocks into memory once they have been read? Which filesystem has such a defragmentation tool?
I've added a second drive to a system and I need to extend the lvm and the filesystem to the second disk. Is there a way to do this online with centos 5.5? I specifically need extending the actual ext3 filesystem which seems to be the trick part.
I've just bought a new SSD hard drive:Kingston SSDNow V-Series SNV125-S2/128GB 2.5'' 128GB SATA/300The question is which filesystem whould you recommand and why?BTRFS vs NILFS2 or EXT4?If you choose ext4 would you enable jurnalling?I'm very close to choose Btrfs.Any experience with running any of these on your SSD?
Currently we use an iSCSI SAN as storage for several VMware ESXi servers. I am investigating the use of an NFS target on a Linux server for additional virtual machines. I am also open to the idea of using an alternative operating system (like OpenSolaris) if it will provide significant advantages.What Linux-based filesystem favours very large contiguous files (like VMware's disk images)? Alternatively, how have people found ZFS on OpenSolaris for this kind of workload?
My server started acting flaky this weekend and my Webmin interface was throwing strange errors. I finally tracked it down to the fact that I was out of inodes on my primary partition. I'm fairly certain that the /tmp folder has an outrageous number of files in it.I can't do an ls on the directory because the console just sits there forever after I issue the command. I also tried to do an rm -rf on the /tmp directory and it did the same thing.
Background: My mother's HP laptop had Ubuntu and Vista on it, Ubuntu my brother's doing. He decided he wanted to take off Ubuntu yesterday (he had forgotten the password), and deleted the partition that it was contained within. The computer now boots to this error.
Inventory: We no longer have the install disk for Windows Vista, he cannot tell me what version he used of Ubuntu, what partition it was on, any of the specs for the machine, or generally any information about the system. All I am aware of is that error on the boot-up screen. I have nothing else to work with.
I would like to remove Grub, and Ubuntu, and leave Windows intact (the request of the owner of the computer), but I have no idea what commands I could use to get rid of either when I can't access Windows, or how to properly remove them if I did access Windows.
i manage to delete some files from the system. now i need to recover them.. i know the inode # (through ext3undel) and also the size.Quote:Unfortunately, we cannot automatically obtain the name of a deleted filefrom Unix file systems - since the connection between the iNode (whichholds the MetaData, including the file namee real data is droppedon deletion. However, we can obtain a list of names from the deleted files.How can i use this information to recover the files?Also can i search the text from a partition? (file don't exists). As i need figures
The Linux File system uses the file path notation to abstract how data is accessed. Path really must be an environmental variable for the applcication that converts the path name to an inode so what is this application/Daemons name?
I have searched google for a couple of days, and I keep hearing about an INode limit on filesystems, but that doesn't seem to be the case.
Now whenever I try to download something, watch a ..... video, or listen to Pandora radio, it just stops playing after 2 seconds. Downloading says "No space left on device".I also get the error as root.I do have 5% and more free of HDD space. After reading the similar posts I checked all of this, so if I am overlooking something on the forum, I apologize for an extra post about.
I've had a look at some similar threads but as I'm very new to linux they're already a bit technical for me. Sorry, this calls for someone with patience. I gather from other threads that disconnecting an external drive without unmounting is a no-no, and this seems to be the likely cause. Now the disk is read only and I'm unable to change any settings through the usual control panel on ubuntu. I'm just not familiar with the terminal instructions. I tried to cut and past a few command lines from other threads but I got some warnings that proceding could damage data. Like this one: WARNING! Running e2fsck on a mounted filesystem may cause SEVERE filesystem damage.
I've setup a filesystem on a RAID 0+1 and am looking at moving root filesystem from a single disk to the new one. I could not install CentOS on mirrored filesystem as the RAID card did not have a pre-built driver for CentOS 5.3, so I had to compile the driver after installing the system.What I'm going to do now is:
1. Mount the new mirrored filesystem under /root1 2. use find | cpio to copy everything from the existing / to /root1 3. use grub to create a boot record on /root1 4. edit /root1/etc/fstab to point / to the new disk 5. reboot the system and keep my fingers crossed
Ive setup a filesystem on a RAID 0+1 and am looking at moving root filesystem from a single disk to the new one. I could not install CentOS on mirrored filesystem as the RAID card did not have a pre-built driver for CentOS 5.3, so I had to compile the driver after installing the system.
What Im going to do now is:
1.Mount the new mirrored filesystem under /root1 2.use find | cpio to copy everything from the existing / to /root1 3.use grub to create a boot record on /root1 4.edit /root1/etc/fstab to point / to the new disk 5. reboot the system and keep my fingers crossed
I have just formatted an external USB disk with a JFS filesystem. The partition shows up in 'Computer', and it mounts, but if I try and copy and files onto it, it will not do it. Clearly, Nautilus is mounting it read-only. How do I get this to behave like my USB Flash drive, where I plug it in, and its automatically mounted read/write?
I am trying to access a Magneto Optical disk written under OS/2 hpfs filesystem. I cannot locate a dang thing about support of CentOS for OS/2. I found a module for Linux OS/2 support that dated back to 1993, have no idea how to compile it, used alien to create an RPM package, said compiled succesfully, still can't mount the drive (/dev/sdb) with filesystem type -t hpfs. Can anyone out there help me, or does CentOS just not do this?
I recently installed Bio-Linux 5.0 as a dual boot system with XP for some bioinformatics applications, but Im having some problems with the amount of disk space which can be allocated specifically for the Ubuntu install.
I have been having problems with filesystem corruption on my eeepc 1000H for a long time now. I have tried using different filesystems, kernels and distributions (arch, slackware) to no effect. I am starting to grow suspicious that this problem lies somewhere else, as I haven't seen anyone else having similar problems in such a variety of scenarios.
I have tried testing my ram using memtest86+, didn't come up with anything after a full run through. I also have tried using e2fsck -c to check for bad blocks, it finds none. I had a go at using smartctl but wasn't really sure what I was doing. I did a long test and it came up with nothing anyway.
This problem is in addition to the problems I've been having with my intel graphics chip and KMS. A lot of the time there are lockups when booting into X, which can only be gotten out of by a hard reset. This is sometimes what causes the original filesystem errors. I've stopped messing around with KMS for now to eliminate this but my current system in unbootable. I'm guessing my disk is wrecked but have as yet seen no definitive proof. Can anyone recommend anything that I should do?
I am currently on ext4 with a custom kernel 2.6.33-rc6 (the stock kernel shipping with slackware does not have the elantech extension for psmouse included). When I was using arch, I was just using the stock kernels.
I am very new to linux, and I have a question regarding the filesystem check (fsck). The power recently went out and when I tried to restart linux the following error appears:
*/dev/sda1 contains file system w/errors, check forced it then goes on to say..
*An error occured during the file system check. Dropping you to a shell; the system will reboot when you leave the shell. Give root password for maintenance (or type Control-D to continue) I wasn't sure what to do, but checked some other online forums and they suggested running fsck manually - so I typed in the root password - and used the command, "fsck -A -V ; echo == $? ==" it then gave the following message
*WARNING!!! Running e2fsck on a mounted filesystem may cause SEVERE filesystem damage *Would you like to continue (y/n)
Again, I wasn't sure what to do so i just checked no. I then manually turned off the computer and was prompted at the beginning to press Alt-3. I was brought to another screen and it informed me one of the drives was degraded and suggested rebuilding the array. I tried doing this, but it still brings me back to the original error of, "/dev/sda1 contains file system w/errors, check forced," and the process continues.
Also, when I tried to rebuild the array, I didn't backup any of the data on our home directory before doing this (which was probably a big mistake). After being prompted to type the root password, I was able to give the ls command and look at all the directories...the home directory where our data was stored was empty and I am afraid I may have lost some information. Is there a possibility that data was lost when I was trying to rebuild using the old drives?
Several people have said that those of us who are having problems with Ubuntu (10.04) should ask some specific questions. Here is on below which I cannot get an answer to and never happens in Windows. Can any Ubuntu expert answer it for me? Would really restore my faith in Ubuntu (and go onto the other problems I have with it)I think I am running out of inodes on my eeepc701. It has happened before when I was using Xandros but now I am using Ubuntu 10.04.I get the following output:
Code: df -i Filesystem Inodes IUsed IFree IUse% Mounted on
My server started acting flaky this weekend and my Webmin interface was throwing strange errors. I finally tracked it down to the fact that I was out of inodes on my primary partition. I'm fairly certain that the /tmp folder has an outrageous number of files in it. I can't do an ls on the directory because the console just sits there forever after I issue the command. I also tried to do an rm -rf on the /tmp directory and it did the same thing. how I can clear out this directory?
I recently used up all my free inodes on my server. I had a bunch of mail messages that were sitting there using up a bunch, so I cleared the postfix queue. That gave me some room. What I'd like to do, is get a listing of the directories using the most inodes (or containing the most number of files), so that I can find the other culprits.Basically I want the output of "df -i" but to be able to do it recursively on a specific directory.
Each time I start my Ubuntu 10.10, I notice this messages in dmesg:
Each time the inode number is different. I made SMART tests on the disk, and all went fine. Do I have to worry? Could it be something related to a wrong shutdown? Update: I have just ran an fsck at boot, but when I logged in, the same orphan_cleanup was in dmesg.