Debian :: Cache A Directory Tree?

Nov 30, 2015

I have a home storage of many drives which are seldom accessed, and extremely seldom written to. So, I made several RAID1 arrays with "write-mostly" drives in each one, so that all the drives spin down after a while, and if they are accessed, only one drive in each array has to spin up. This way, I hope to minimize the risk of losing anything due to mechanical shock or wear.

But it was not so easy. First, the write-mostly drives did spin up on every read request due to a bug in the kernel, so I managed to get the fix accepted into the newer kernels (which is my largest contribution to the Linux development so far. See [URL] .....) Now, if I read directly from the array, it works as expected (only the first drive spins up); but if I create a filesystem there (ext4) and read from it, the second drive still spins up anyway. (Does ext4 write something when it reads a file?.. I have it mounted with relatime, so AFAIK it shouldn't...)

Well, I thought, I'll just mount the filesystems read-only and remount them if I need to modify the data there, which is not very often, to say the least. Also, this way the drives don't have to spin up every system shutdown to unmount the filesystems; during a power outage, it used to take a lot of time for them to spin up one by one just to unmount, while my UPS had to survive long enough for them to finish. That's one more problem solved.

But I decided to improve even further. Sometimes I just want to see the contents of those arrays without even accessing some files (for example, when I accidentally click the wrong network drive button in Windows, or when I just want to see if I have something there or not). If I read the directory trees during startup and keep them cached, then those drives won't even have to spin up at those moments! The question is, is there a way to do it? I remember Linus boasting about our kernel being the best at caching the filesystem trees, and I know there is vmtouch [URL] .... which allows us to keep some files cached, but what about the directory tree itself?.. If I simply "find" all the files there, it seems to work, but they get evicted from cache sooner or later...

View 3 Replies


ADVERTISEMENT

Ubuntu :: Convert Files, But Keep The Directory Tree?

Jun 10, 2010

I've got a directory structure full of files. I want to convert them all into some other format, but I don't know how to keep the directory structure, is there a way so I can "set" which command to use on files instead of `cp`?

View 4 Replies View Related

General :: Create Archives Of A Directory Tree On Ubuntu Box?

Oct 29, 2010

Are there any other archiving tools than tar that preserve Linux file permissions and user/group flags? I want to create archives of a directory tree on my ubuntu box. The directory trees are large, a first run with tar cvpzf archive.tgz /home/foo/bar yielded a 5GB archive. I also want to keep all permissions and other flags and special files.I'm fine with a 5GB archive, however to look inside that archive -- since it is a compressed tar archive -- the whole 5GB have to be decompressed first! (Or so it appears when opening it with the archive viewer -- I'm happy to be corrected.)So I need a way to "backup" a directory (tree) while preserving full filesystem attributes and right and creating an archive with an "index" that hasn't to be decompressed for browsing its contents. An archive is either a single file, or a (small) set of files that carries full and complete information within this file/s. That is, it can live on any filesystem (size permitting), it can be burnt onto a DVD, you can split it (after which the point of this question is really lost - but still), ...

View 3 Replies View Related

General :: Rsync To Copy Directory Tree From NAS Drive ?

Apr 10, 2010

I have a WD world book edition 1TB NAS drive, and just purchased an acomdata 1tb drive and connected it to the NAS via USB. If I recall I think the WD NAS has a ext_ or some type of linux filesystem on it, and the acomdata has a ntfs filesystem on it.

What I want to do is copy over certain directory trees of the NAS to the USB attached drive. I usually use MS synctoy to sync folders from my windows pc to the NAS drive, and MS richcopy to make the initial transfer from PC to NAS. For this operation though, since it is taking place entirely on the NAS and its connected drive, I thought that rsync would be the best option, and it is available on my NAS drive.

Last night I entered in rsync -avr /movies/* /usb1-1share1/ to copy the entire "movies" dir to the drive, which shows up as usb1-1share1 on the NAS drive. It copied most of the directory tree ok, but a lot of the folders were empty, so this morning I tried rsync -Carv --ignore-existing /movies/* /usb1-1share1/ to try and get all the files missed, without recopying the 24GB that did make it across. This also managed to copy a few more GB over, but not everything.

I am running the command from an ssh session on the NAS using putty on my PC, in as user "admin" which should have all rights over these folders. There is a bunch of errors in the command window like this: rsync: failed to set times on "/shares/usb1-1share1/movies/classics/fulldvd/First Blood DVD/.VTS_01_2.VOB.RxdjWZ": Operation not permitted (1)

I want to restart another session and get the files it missed, but I want to find out what I am doing wrong first. Should I be doing this as root user? am I missing some switches or just plain doing it all wrong?

View 3 Replies View Related

General :: Ftp Whole Directory Tree Using Vsftpd As Ftp Server In Fedora11

Feb 11, 2010

I am trying to ftp a whole directory tree using vsftpd as ftp server in fedora11

Code:

I am getting error while transfering a directory tree ...

View 1 Replies View Related

General :: Cannot Predict How Many Levels Deep Directory Tree Is

Sep 30, 2010

This little fragment does the opposite of what I want and I am trying to figure out how to reverse it. I want to strip off the directory and echo just the filenames. I cannot predict how many levels deep the directory tree is.

View 6 Replies View Related

General :: Overwriting Specific Files In Directory Tree Recursively

Jul 19, 2011

I would like to overwrite files in a directory tree, recursively. The ones I would like to overwrite match the filename "x_alpha*.png" and have a size exactly 456 bytes. Is there any way to search for these recursively in a directory tree, and overwrite them with a reference file, for example "e:mydirgood.png"

I am using Windows 7, but I have UnxUtils, so I can use those too. What I am looking for is something like this, generated automatically:
copy /y e:mydirgood.png e:mydiracx_alpha0023.png
copy /y e:mydirgood.png e:mydirefgx_alpha0045.png
copy /y e:mydirgood.png e:mydirhx_alpha0248.png

View 2 Replies View Related

Ubuntu :: File Copy Via 9.10 On Windows Not Updating Directory Tree ?

Aug 19, 2010

I have ubuntu 9.10 & windows 7 on my laptop. I have one 40GB Primary partition (C & 60 GB logical partition on my windows. I have given another 20GB for ubuntu.

I hibernated my windows7 & then started ubuntu 9.10. Then I copied a folder containing some PDF files from my ubuntu to that 60GB (D on windows. Then I rebooted the machine & choose windows from GRUB. the windows came up from hibernation but nowhere I was able to see that folder which I copied.

Since ubuntu supports (understands) NTFS file system it means when I copied that folder it should have updated the Directory Tree of NTFS on that 60GB (D but that folder is not shown.

When I restart the windows the folder appears (ofcourse because windows checks the file system again for consistency while in hibernation it does not).

View 3 Replies View Related

Software :: Perfoming A ./configure Script Outside The Original Directory Tree

Jan 18, 2011

I have a user account (no root access) on a Debian system. My disk quota for my home directory is limited to 250MB, of which I've used 150MB.

I would like to install a program that requires 95MB of space (let's call it prog-5.1) into $HOME/local/bin. Because the source files take up 1GB, I've had to place them in the /tmp directory (to stay under my limit).

Unfortunately when I run the config script, I receive the following:

Code:

The reason for this is that /tmp is mounted in noexec mode, meaning I can't execute code in /tmp directory:

Code:

I'm wondering if anyone has any ideas about running the config script in my home directory, and somehow pointing it to configure the files in the /tmp/prog-5.1/ directory.

View 5 Replies View Related

Programming :: Correct / Common Directory Tree For Software Packages?

Mar 16, 2011

I found it easiest to embed narp into the gtk+-2.0 installation because all I needed to do was tweak the Makefile.am and Makefile.in files and let configure do the rest.This is a hack. I've failed at getting my own makefiles to work.

Would anyone point me in the right direction on how to properly make an installer for Linux? Also, what is the correct/common directory tree for Linux software packages? Any more suggestions on proceeding with this project using community standards?

View 2 Replies View Related

Ubuntu :: Browsing A Directory Tree Directly In Gnome Panel - UPDATE

Jan 19, 2011

The problem was finding an applet to browse directory tree directly in the Gnome panel similar to that in "Recent Documents". The thread was closed without any solution posted there and also no solution available elsewhere (I searched for over 2 hours) in the forum.

Search for "file-browser-applet" in the Synaptic Package Manager. The applet with exactly that name is what you need. It allows to browse and open files on your computer from the GNOME panel. Once installed, "Add to panel".

View 1 Replies View Related

Programming :: Sort Oldest 5 Files In A Directory Tree Recursively Based On Timestamp?

Jun 3, 2010

I have a directory listing with many subdirectories having many files. I want to recursively search for the oldest 5 files starting from the base directory and not 5 from each subdirectory. I am writing a shell script which sorts them using ls -lRtur|egrep "txt|jpg" > /tmp/file1 Now from this /tmp/file1 file I want to sort the files same as what the ls -ltr command does that is oldest file time to newest file time first. How do I sort based on Linux time stamp? The files itself also have Linux timestamps embedded in them So I can sort based after extracting them as well if it is easier.
My /tmp/file1 has entries like below.

-rw-rw-r--. 1 usr1 usr1 705 2010-01-22 17:25 sample20100603173659.jpg

I want to get the 5 oldest files and then delete them.

View 1 Replies View Related

Red Hat / Fedora :: Makfile Which Can Compile A Tree For Source File And Create Output In Separate Directory

Jul 29, 2010

I am new to linux platform and my requirment is to write a makfile which can compile a tree for source file and create the output in separate directory .

View 1 Replies View Related

Server :: Clean Squid Cache Directory?

Mar 6, 2011

I am using squid on my cent os 5. The cache directory is almost full.

Code:
# cache_dir ufs /var/spool/squid 1000 16 256
kindly guide me how can i flush or clean the data inside the cache directory of squid using terminal

View 4 Replies View Related

Ubuntu :: Cannot Change Directory To A More Than Three Folder Tree Destination Folder From In Terminal?

Sep 18, 2010

I cannot change directory to a more than three folder tree destination folder from ~ in terminal. I've checked everything. No Typos or misspell. The destination folder was recognized by "ls" command but when I went to it, the terminal said, "no such file or directory."

View 7 Replies View Related

Ubuntu Servers :: Cache Directory And File Structure In Memory?

Aug 11, 2010

i am running Ubuntu Lucid x64 as a fileserver that shares its files via SFTP, NFS and Samba. Currently the hard disks are configured to go to standby if they are not needed. This works perfectly as long as no one browses the shares or my HTPC is running: That one repeatedly looks through the shares for new music or movies. In other words my problem is that the disks are spinning up a lot more often than they should have to. Additionally the spin-up time delays the response time while browsing. Since the machine has a lot of unused RAM i want to tell the kernel that it should keep the directory structure in memory. That way the disks would not need to spin up every time someone browses through the directories.

View 2 Replies View Related

OpenSUSE Install :: Get Info On The Opensuse Directory Tree?

Aug 17, 2010

I am looking for explanations on how Opensuse utilizes the root directories. Like what decides if a program goes in /bin vs /sbin vs /usr/bin etc...In other words, a breakdown of the directory structure and how each part is utilized. I am trying to get more familiar with it so it will be easier for me to locate things and I will know where to look for X part of a program that does Y.

View 3 Replies View Related

Fedora :: Error: Caching Enabled But No Local Cache Of //var/cache/yum/updates-newkey

Sep 24, 2009

I don't understand this error nor do I know how to solve the issue that is causing the error. Anyone care to comment?

Quote:

Error: Caching enabled but no local cache of //var/cache/yum/updates-newkey/filelists.sqlite.bz2 from updates-newkey

I know JohnVV. "Install a supported version of Fedora, like Fedora 11". This is on a box that has all 11 releases of Fedora installed. It's a toy and I like to play around with it.

View 12 Replies View Related

Software :: Redirecting APT Cache - Can Redirect Cache Of Apt To A Specified Folder Either On Command Line Or Via A Config Setting?

Jan 5, 2011

I was laughing about klackenfus's post with the ancient RH install, and then work has me dig up an old server that has been out of use for some time. It has some proprietary binaries installed that intentionally tries to hide files to prevent copying (and we are no longer paying for support or have install binaries), so a clean install is not preferable.

Basically it has been out of commission for so long, that the apt-get upgrade DL is larger than the /var partition (apt caches to /var/cache/apt/archives).

I can upgrade the bigger packages manually until I get under the threshold, but then I learn nothing new. So I'm curious if I can redirect the cache of apt to a specified folder either on the command line or via a config setting?

View 2 Replies View Related

Server :: Squid Cache System - Make It Cache All Files Like .exe .mp3 .avi

Mar 6, 2011

I installed squid cache on my ubuntu server 10.10 and it is work fine but i want to know how to make it cache all files like .exe .mp3 .avi ....etc. and the other thing i want to know is how to make my client take the files from the cache in the full speed. since am using mikrotik system to use pppoe for clients and i match it with my ubuntu squid

View 1 Replies View Related

General :: Difference Between Hardware Cache And Slab Cache?

Nov 22, 2010

I am reading slab allocator, it defines slab cache, i am quite confuse is it same as hardware cache?

View 2 Replies View Related

Debian Hardware :: Intel Soft Modem Configuration Error - Unable To Prepare Temporary Kernel Tree

Dec 4, 2010

I am runing debian-5 (lenny), the default kernel 2.6.26 was unable to start the x so i installed the new kernel linux-image-2.6.32-5-686_2.6.32-28_i386.deb, now x works, but there is problem building modules for the intel soft modem linux driver hsfmodem-7.60.00.00oem which is supplied in the modem cd by the intel company.

I installed these kernel packages for this purpose:
linux-base_2.6.32-28_all.deb
linux-image-2.6.32-5-686_2.6.32-28_i386.deb
linux-headers-2.6.32-5-486_2.6.32-28_i386.deb

[code]....

in all these cases hsfconfig failed with the above error,

View 2 Replies View Related

Debian :: Increase Apt-cache Limit?

Aug 21, 2011

There I was, trying to update the package lists in a fresh install of Knoppix, when:

View 8 Replies View Related

Debian Configuration :: Clear DNS Cache ?

Jan 18, 2011

A friend has a website. the site has been moved to another server. I am inside of a private network.

Problem: when trying to reach his website, my browser keeps connecting to the old server which is on another continent

Windows machines have no problem with this. They connect to the new server without issues.

NOW what is hard to understand: even the PING utility is pinging the old server.

I tried these and none works:
- computer restart
- Clear private data in Iceweasel (has ntg to do with PING, but tried...)
- installed nscd and restarted the service
- modified /etc/resolv.conf to point to another nameserver than modified it back

At the same computer, if I reboot and load Windows, the browser has no problem connecting to the new server.

View 8 Replies View Related

Debian :: Delete All Contents Of /var/cache/man?

Feb 13, 2011

I try to build live usb-hdd on my Lenny and get this problem:

Code:
# lh_build
..............................................................

[code]...

View 1 Replies View Related

Debian :: Output Of Apt-cache Policy

May 10, 2011

I'm using Debian Squeeze.

When I invoke apt-cache policy , for example , apt-cache policy zlib1g.

I get the output like:

Code:

And below the line "Version table:" , there is installed package version. I assume 1:1.2.3.4.dfsg-3 is version("epoch"+"upstream version"+"debian revision"), but what does the next "0" means?

View 2 Replies View Related

Debian Installation :: Kernel Liquorix And Nvidia Driver / Error Doesn't Found The Kernel Source Tree?

Oct 1, 2010

I installed the latest kernel liquorix (2.6.35) but when i want to install the Nvidia driver downloaded on the Nvidia website (256.53), i have an error message because Nvidia doesn't found the kernel source tree.

I install linux-image-2.6.35-6.dmz.2-liquorix-686_2.6.35-16_i386.deb, linux-headers-2.6.35-6.dmz.2-liquorix-686_2.6.35-16_i386.deb and build-essential. I don't understand why the installation doesn't works.

View 1 Replies View Related

Debian :: High Memory Usage In Cache?

Mar 12, 2011

I've installed my debian sid about one month ago (first xfce, next gnome) but noticed that it's kind of really slow. The upgrades take ages, launching (and using) firefox takes so much time,... In comparaison to my ubuntu, archlinux (on the same computer) or previous installation of debian there is clearly a problem somewhere.Today I tried to do a "top" sorted by mem usage : 3.5% xulrunner-stub, 2.1% dropbox, 1.4% aptitude (doing upgrade), 1.4% clementine,... nothing terriblebut still I've 2.7Gb or RAM used (more than 50%)

$ free -m
total used free shared  buffers cached
Mem:  3967 26851282  0 79 1938

[code]....

View 4 Replies View Related

Debian Configuration :: Raising The APT Cache Limit?

Nov 4, 2010

When adding repositories, I ran up past the default apt cache limit of 25165824. I found a couple of sources - one that treats apt.conf.d as an individual file, and another that treated it as a folder - which my system does.

Following those guidelines, I went into /etc/apt/apt.conf.d/debconf70 to add a line of code that sets the default to 2x and 4x that limit, and... well, apt doesn't seem to recognize the difference.

Here's the code I added at the end:

So, is there somewhere else that I need to change things? Am I completely off on this? I found a year-old thread on this error in the ubuntu forums, where a gentleman who is now a member of the staff simply suggested to the op "take out that debian repository you listed". It kinda negated the premise for me, you know? I'm kinda hoping there's more that I can do.

View 5 Replies View Related

Ubuntu :: Kept Getting "gtk-update-icon-cache: The Generated Cache Was Invalid" Fault

Feb 17, 2010

i was looking for a way to stop my menus taking a few seconds to load my icons when i first open them and found a few guides suggesting using the gtk-upate-icon-cache command, but with the any colour you like icon theme i'm using (stored in my home folder .icons directory) i kept getting a "gtk-update-icon-cache: The generated cache was invalid." fault i used the inbuilt facility in the acyl script to copy the icons to the usr/share/icons directory and tried the command again, this time using sudo gtk-update-icon-cache --force --ignore-theme-index /usr/share/icons/ACYL_Icon_Theme_0.8.1/ but i still get the same error. i tried with several of the custom icon themes i've installed and only 1 of the first 7 or 8 i tried successfully created the cache.

View 4 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved