Debian :: Cache A Directory Tree?
Nov 30, 2015
I have a home storage of many drives which are seldom accessed, and extremely seldom written to. So, I made several RAID1 arrays with "write-mostly" drives in each one, so that all the drives spin down after a while, and if they are accessed, only one drive in each array has to spin up. This way, I hope to minimize the risk of losing anything due to mechanical shock or wear.
But it was not so easy. First, the write-mostly drives did spin up on every read request due to a bug in the kernel, so I managed to get the fix accepted into the newer kernels (which is my largest contribution to the Linux development so far. See [URL] .....) Now, if I read directly from the array, it works as expected (only the first drive spins up); but if I create a filesystem there (ext4) and read from it, the second drive still spins up anyway. (Does ext4 write something when it reads a file?.. I have it mounted with relatime, so AFAIK it shouldn't...)
Well, I thought, I'll just mount the filesystems read-only and remount them if I need to modify the data there, which is not very often, to say the least. Also, this way the drives don't have to spin up every system shutdown to unmount the filesystems; during a power outage, it used to take a lot of time for them to spin up one by one just to unmount, while my UPS had to survive long enough for them to finish. That's one more problem solved.
But I decided to improve even further. Sometimes I just want to see the contents of those arrays without even accessing some files (for example, when I accidentally click the wrong network drive button in Windows, or when I just want to see if I have something there or not). If I read the directory trees during startup and keep them cached, then those drives won't even have to spin up at those moments! The question is, is there a way to do it? I remember Linus boasting about our kernel being the best at caching the filesystem trees, and I know there is vmtouch [URL] .... which allows us to keep some files cached, but what about the directory tree itself?.. If I simply "find" all the files there, it seems to work, but they get evicted from cache sooner or later...
View 3 Replies
ADVERTISEMENT
Jun 10, 2010
I've got a directory structure full of files. I want to convert them all into some other format, but I don't know how to keep the directory structure, is there a way so I can "set" which command to use on files instead of `cp`?
View 4 Replies
View Related
Oct 29, 2010
Are there any other archiving tools than tar that preserve Linux file permissions and user/group flags? I want to create archives of a directory tree on my ubuntu box. The directory trees are large, a first run with tar cvpzf archive.tgz /home/foo/bar yielded a 5GB archive. I also want to keep all permissions and other flags and special files.I'm fine with a 5GB archive, however to look inside that archive -- since it is a compressed tar archive -- the whole 5GB have to be decompressed first! (Or so it appears when opening it with the archive viewer -- I'm happy to be corrected.)So I need a way to "backup" a directory (tree) while preserving full filesystem attributes and right and creating an archive with an "index" that hasn't to be decompressed for browsing its contents. An archive is either a single file, or a (small) set of files that carries full and complete information within this file/s. That is, it can live on any filesystem (size permitting), it can be burnt onto a DVD, you can split it (after which the point of this question is really lost - but still), ...
View 3 Replies
View Related
Apr 10, 2010
I have a WD world book edition 1TB NAS drive, and just purchased an acomdata 1tb drive and connected it to the NAS via USB. If I recall I think the WD NAS has a ext_ or some type of linux filesystem on it, and the acomdata has a ntfs filesystem on it.
What I want to do is copy over certain directory trees of the NAS to the USB attached drive. I usually use MS synctoy to sync folders from my windows pc to the NAS drive, and MS richcopy to make the initial transfer from PC to NAS. For this operation though, since it is taking place entirely on the NAS and its connected drive, I thought that rsync would be the best option, and it is available on my NAS drive.
Last night I entered in rsync -avr /movies/* /usb1-1share1/ to copy the entire "movies" dir to the drive, which shows up as usb1-1share1 on the NAS drive. It copied most of the directory tree ok, but a lot of the folders were empty, so this morning I tried rsync -Carv --ignore-existing /movies/* /usb1-1share1/ to try and get all the files missed, without recopying the 24GB that did make it across. This also managed to copy a few more GB over, but not everything.
I am running the command from an ssh session on the NAS using putty on my PC, in as user "admin" which should have all rights over these folders. There is a bunch of errors in the command window like this: rsync: failed to set times on "/shares/usb1-1share1/movies/classics/fulldvd/First Blood DVD/.VTS_01_2.VOB.RxdjWZ": Operation not permitted (1)
I want to restart another session and get the files it missed, but I want to find out what I am doing wrong first. Should I be doing this as root user? am I missing some switches or just plain doing it all wrong?
View 3 Replies
View Related
Feb 11, 2010
I am trying to ftp a whole directory tree using vsftpd as ftp server in fedora11
Code:
I am getting error while transfering a directory tree ...
View 1 Replies
View Related
Sep 30, 2010
This little fragment does the opposite of what I want and I am trying to figure out how to reverse it. I want to strip off the directory and echo just the filenames. I cannot predict how many levels deep the directory tree is.
View 6 Replies
View Related
Jul 19, 2011
I would like to overwrite files in a directory tree, recursively. The ones I would like to overwrite match the filename "x_alpha*.png" and have a size exactly 456 bytes. Is there any way to search for these recursively in a directory tree, and overwrite them with a reference file, for example "e:mydirgood.png"
I am using Windows 7, but I have UnxUtils, so I can use those too. What I am looking for is something like this, generated automatically:
copy /y e:mydirgood.png e:mydiracx_alpha0023.png
copy /y e:mydirgood.png e:mydirefgx_alpha0045.png
copy /y e:mydirgood.png e:mydirhx_alpha0248.png
View 2 Replies
View Related
Aug 19, 2010
I have ubuntu 9.10 & windows 7 on my laptop. I have one 40GB Primary partition (C & 60 GB logical partition on my windows. I have given another 20GB for ubuntu.
I hibernated my windows7 & then started ubuntu 9.10. Then I copied a folder containing some PDF files from my ubuntu to that 60GB (D on windows. Then I rebooted the machine & choose windows from GRUB. the windows came up from hibernation but nowhere I was able to see that folder which I copied.
Since ubuntu supports (understands) NTFS file system it means when I copied that folder it should have updated the Directory Tree of NTFS on that 60GB (D but that folder is not shown.
When I restart the windows the folder appears (ofcourse because windows checks the file system again for consistency while in hibernation it does not).
View 3 Replies
View Related
Jan 18, 2011
I have a user account (no root access) on a Debian system. My disk quota for my home directory is limited to 250MB, of which I've used 150MB.
I would like to install a program that requires 95MB of space (let's call it prog-5.1) into $HOME/local/bin. Because the source files take up 1GB, I've had to place them in the /tmp directory (to stay under my limit).
Unfortunately when I run the config script, I receive the following:
Code:
The reason for this is that /tmp is mounted in noexec mode, meaning I can't execute code in /tmp directory:
Code:
I'm wondering if anyone has any ideas about running the config script in my home directory, and somehow pointing it to configure the files in the /tmp/prog-5.1/ directory.
View 5 Replies
View Related
Mar 16, 2011
I found it easiest to embed narp into the gtk+-2.0 installation because all I needed to do was tweak the Makefile.am and Makefile.in files and let configure do the rest.This is a hack. I've failed at getting my own makefiles to work.
Would anyone point me in the right direction on how to properly make an installer for Linux? Also, what is the correct/common directory tree for Linux software packages? Any more suggestions on proceeding with this project using community standards?
View 2 Replies
View Related
Jan 19, 2011
The problem was finding an applet to browse directory tree directly in the Gnome panel similar to that in "Recent Documents". The thread was closed without any solution posted there and also no solution available elsewhere (I searched for over 2 hours) in the forum.
Search for "file-browser-applet" in the Synaptic Package Manager. The applet with exactly that name is what you need. It allows to browse and open files on your computer from the GNOME panel. Once installed, "Add to panel".
View 1 Replies
View Related
Jun 3, 2010
I have a directory listing with many subdirectories having many files. I want to recursively search for the oldest 5 files starting from the base directory and not 5 from each subdirectory. I am writing a shell script which sorts them using ls -lRtur|egrep "txt|jpg" > /tmp/file1 Now from this /tmp/file1 file I want to sort the files same as what the ls -ltr command does that is oldest file time to newest file time first. How do I sort based on Linux time stamp? The files itself also have Linux timestamps embedded in them So I can sort based after extracting them as well if it is easier.
My /tmp/file1 has entries like below.
-rw-rw-r--. 1 usr1 usr1 705 2010-01-22 17:25 sample20100603173659.jpg
I want to get the 5 oldest files and then delete them.
View 1 Replies
View Related
Jul 29, 2010
I am new to linux platform and my requirment is to write a makfile which can compile a tree for source file and create the output in separate directory .
View 1 Replies
View Related
Mar 6, 2011
I am using squid on my cent os 5. The cache directory is almost full.
Code:
# cache_dir ufs /var/spool/squid 1000 16 256
kindly guide me how can i flush or clean the data inside the cache directory of squid using terminal
View 4 Replies
View Related
Sep 18, 2010
I cannot change directory to a more than three folder tree destination folder from ~ in terminal. I've checked everything. No Typos or misspell. The destination folder was recognized by "ls" command but when I went to it, the terminal said, "no such file or directory."
View 7 Replies
View Related
Aug 11, 2010
i am running Ubuntu Lucid x64 as a fileserver that shares its files via SFTP, NFS and Samba. Currently the hard disks are configured to go to standby if they are not needed. This works perfectly as long as no one browses the shares or my HTPC is running: That one repeatedly looks through the shares for new music or movies. In other words my problem is that the disks are spinning up a lot more often than they should have to. Additionally the spin-up time delays the response time while browsing. Since the machine has a lot of unused RAM i want to tell the kernel that it should keep the directory structure in memory. That way the disks would not need to spin up every time someone browses through the directories.
View 2 Replies
View Related
Aug 17, 2010
I am looking for explanations on how Opensuse utilizes the root directories. Like what decides if a program goes in /bin vs /sbin vs /usr/bin etc...In other words, a breakdown of the directory structure and how each part is utilized. I am trying to get more familiar with it so it will be easier for me to locate things and I will know where to look for X part of a program that does Y.
View 3 Replies
View Related
Sep 24, 2009
I don't understand this error nor do I know how to solve the issue that is causing the error. Anyone care to comment?
Quote:
Error: Caching enabled but no local cache of //var/cache/yum/updates-newkey/filelists.sqlite.bz2 from updates-newkey
I know JohnVV. "Install a supported version of Fedora, like Fedora 11". This is on a box that has all 11 releases of Fedora installed. It's a toy and I like to play around with it.
View 12 Replies
View Related
Jan 5, 2011
I was laughing about klackenfus's post with the ancient RH install, and then work has me dig up an old server that has been out of use for some time. It has some proprietary binaries installed that intentionally tries to hide files to prevent copying (and we are no longer paying for support or have install binaries), so a clean install is not preferable.
Basically it has been out of commission for so long, that the apt-get upgrade DL is larger than the /var partition (apt caches to /var/cache/apt/archives).
I can upgrade the bigger packages manually until I get under the threshold, but then I learn nothing new. So I'm curious if I can redirect the cache of apt to a specified folder either on the command line or via a config setting?
View 2 Replies
View Related
Mar 6, 2011
I installed squid cache on my ubuntu server 10.10 and it is work fine but i want to know how to make it cache all files like .exe .mp3 .avi ....etc. and the other thing i want to know is how to make my client take the files from the cache in the full speed. since am using mikrotik system to use pppoe for clients and i match it with my ubuntu squid
View 1 Replies
View Related
Nov 22, 2010
I am reading slab allocator, it defines slab cache, i am quite confuse is it same as hardware cache?
View 2 Replies
View Related
Dec 4, 2010
I am runing debian-5 (lenny), the default kernel 2.6.26 was unable to start the x so i installed the new kernel linux-image-2.6.32-5-686_2.6.32-28_i386.deb, now x works, but there is problem building modules for the intel soft modem linux driver hsfmodem-7.60.00.00oem which is supplied in the modem cd by the intel company.
I installed these kernel packages for this purpose:
linux-base_2.6.32-28_all.deb
linux-image-2.6.32-5-686_2.6.32-28_i386.deb
linux-headers-2.6.32-5-486_2.6.32-28_i386.deb
[code]....
in all these cases hsfconfig failed with the above error,
View 2 Replies
View Related
Aug 21, 2011
There I was, trying to update the package lists in a fresh install of Knoppix, when:
View 8 Replies
View Related
Jan 18, 2011
A friend has a website. the site has been moved to another server. I am inside of a private network.
Problem: when trying to reach his website, my browser keeps connecting to the old server which is on another continent
Windows machines have no problem with this. They connect to the new server without issues.
NOW what is hard to understand: even the PING utility is pinging the old server.
I tried these and none works:
- computer restart
- Clear private data in Iceweasel (has ntg to do with PING, but tried...)
- installed nscd and restarted the service
- modified /etc/resolv.conf to point to another nameserver than modified it back
At the same computer, if I reboot and load Windows, the browser has no problem connecting to the new server.
View 8 Replies
View Related
Feb 13, 2011
I try to build live usb-hdd on my Lenny and get this problem:
Code:
# lh_build
..............................................................
[code]...
View 1 Replies
View Related
May 10, 2011
I'm using Debian Squeeze.
When I invoke apt-cache policy , for example , apt-cache policy zlib1g.
I get the output like:
Code:
And below the line "Version table:" , there is installed package version. I assume 1:1.2.3.4.dfsg-3 is version("epoch"+"upstream version"+"debian revision"), but what does the next "0" means?
View 2 Replies
View Related
Oct 1, 2010
I installed the latest kernel liquorix (2.6.35) but when i want to install the Nvidia driver downloaded on the Nvidia website (256.53), i have an error message because Nvidia doesn't found the kernel source tree.
I install linux-image-2.6.35-6.dmz.2-liquorix-686_2.6.35-16_i386.deb, linux-headers-2.6.35-6.dmz.2-liquorix-686_2.6.35-16_i386.deb and build-essential. I don't understand why the installation doesn't works.
View 1 Replies
View Related
Mar 12, 2011
I've installed my debian sid about one month ago (first xfce, next gnome) but noticed that it's kind of really slow. The upgrades take ages, launching (and using) firefox takes so much time,... In comparaison to my ubuntu, archlinux (on the same computer) or previous installation of debian there is clearly a problem somewhere.Today I tried to do a "top" sorted by mem usage : 3.5% xulrunner-stub, 2.1% dropbox, 1.4% aptitude (doing upgrade), 1.4% clementine,... nothing terriblebut still I've 2.7Gb or RAM used (more than 50%)
$ free -m
total used free shared buffers cached
Mem: 3967 26851282 0 79 1938
[code]....
View 4 Replies
View Related
Nov 4, 2010
When adding repositories, I ran up past the default apt cache limit of 25165824. I found a couple of sources - one that treats apt.conf.d as an individual file, and another that treated it as a folder - which my system does.
Following those guidelines, I went into /etc/apt/apt.conf.d/debconf70 to add a line of code that sets the default to 2x and 4x that limit, and... well, apt doesn't seem to recognize the difference.
Here's the code I added at the end:
So, is there somewhere else that I need to change things? Am I completely off on this? I found a year-old thread on this error in the ubuntu forums, where a gentleman who is now a member of the staff simply suggested to the op "take out that debian repository you listed". It kinda negated the premise for me, you know? I'm kinda hoping there's more that I can do.
View 5 Replies
View Related
Feb 17, 2010
i was looking for a way to stop my menus taking a few seconds to load my icons when i first open them and found a few guides suggesting using the gtk-upate-icon-cache command, but with the any colour you like icon theme i'm using (stored in my home folder .icons directory) i kept getting a "gtk-update-icon-cache: The generated cache was invalid." fault i used the inbuilt facility in the acyl script to copy the icons to the usr/share/icons directory and tried the command again, this time using sudo gtk-update-icon-cache --force --ignore-theme-index /usr/share/icons/ACYL_Icon_Theme_0.8.1/ but i still get the same error. i tried with several of the custom icon themes i've installed and only 1 of the first 7 or 8 i tried successfully created the cache.
View 4 Replies
View Related