Ubuntu Servers :: Filesystem Lots Of Small Files?
Feb 15, 2010
i have a question about the best choice for a filesystem: We have in our company a productive webserver running debian with nginx webserver to serve images. The number is about 15 million, they are around 2kb or 3kb of size each and they are stored under some subdir in /home with a hashed subdirectory tree which is 2 leves deep. the images are copied with nfs from some other machine time to time. We get high load when this copying is running. Currently we are using ext3, and considering moving to jfs, cause its relativly low load impact, but i am still courios about the performance. I think serving lots of images is often practiced thing, so i would be interested what others are using as filesystem in this scenario, or if someone could make a recommendation for our case.
View 1 Replies
ADVERTISEMENT
Oct 26, 2010
I am currently in the process of moving around 20TB of data from one server to another. Security is not a concern, since the data are freely available to anyone on our network anyway. There are a couple of things that I'm trying to decide on:
(1) protocol choice
Of all the choices I have--nfs, ftp, scp, rsync, samba--has anyone done any benchmarking to show which would be the fastest/most robust transfer protocol? I know nfs has slow write speeds for synchronous transfers. Asynchronous would be faster, but less robust. I'm leaning toward rsync since it performs md5sums to confirm the file transfers. (Remember if there's a 1 in a billion chance that a byte will get corrupted, then I'll have 20,000 corrupt bytes in the transfer.)
(2) Nautilus emblems
We use emblems in Nautilus to categorize files. The old and the new server have the same directory structure.Is there any way to copy the Nautilus emblems from the old server to the new server. What I want is that if a user had marked a particular file with a star on the old server, then that file would be marked with a star on the new server when he/she logs in.
View 1 Replies
View Related
Oct 12, 2009
When I try to design a small filesystem, it always have the Segmentation fault.#: make makefile
Format will erase all context on the disk. Are You Sure? y
Program received signal SIGSEGV, Segmentation fault.
The question is,I cann't find the fseek.c anywhere,so I cann't solve the problem,can you help me?thanks very much.I think I should try my best to find the error in the code,code is too long,so the question i describe is not in detail.
View 3 Replies
View Related
Nov 28, 2010
I have been using ubuntu 10.10 32 bit. I thought of giving 64 bit a try. I downloaded the ubuntu-10.10-desktop-amd64 from the ubuntu website and burned it in to a cd. Then i boot from cd. But unfortuntely i am getting terminal instead of installtion or try options. It says : General errormounting filesystem" and then Lots of " Authentication failure". I typed startx in the terminal but i didnt get any gui.
I am using hp dv6 laptop with windows 7 preinstalled. Core i5 processer, nvidia g105m graphics card and 4gb ram. Ubuntu 10.10 32 bit works fine.
I don't know the whether the problem is because of my graphics card. But it works fine for me in 32 bit. The driver i am using is of version 260.19..06. I really want to use 64 bit OS.
View 4 Replies
View Related
Jan 26, 2011
I use nautilus and its awful. The only good thing are the tabs and the bookmarks on the cons:
* it freezes now and then when moving lots of files.
* Slow to navigate while large amounts of data are moving or displaying.
* Slow presenting of files where there are a lot of them (it should only render whatever is in screen and maybe render a properly sized navigation bar. Anything else shouldnt freeze the client. Render at its once pace if at all. I dont move the scroll bar it shouldnt freeze
* It doesnt queue (meaning it moves everything at the same time instead of queueing stuff, what makes moving stuff take longer) .
View 5 Replies
View Related
Mar 12, 2010
My family have three Linux operated ACER ONE netbooks, all of them need the new Firefox 3.6 download as the old format wont be supported supposedly by ..... etc. I have downloaded the SOFTWARE but cant get it to run, the Firefox explanation mentions lots of files etc that are cant access, nor can I excecute any files once they have been extracted. The same happens with any software but Iv ignored it till now, not having a "modern" browser will make things difficult. I need it explaining simply. On another subject can anyone use facebook chat on Linux
View 6 Replies
View Related
Jan 25, 2010
how to set up dns/bind but the issue I am hitting is that I want to just create a non internet domain setup ie t60.mysuperlan. desktop.mysuperlan. etc . Using a hosts file isn't the best as its pretty much about learning all the stuff to make it work.
View 2 Replies
View Related
Jan 5, 2010
i got an old server with a 64mb flash-disk in.this seems to be the boot-disk.would it be possible to get ubuntu server on this?there is also an ide-disk in.in my opinion it should be good to boot of the flash disk and put the /home or /srv on the ide disk.someone any ideas about this?
View 9 Replies
View Related
Nov 22, 2010
I've got a small web hosting service, for some of my friends. Where I want my friends to be able to have dedicated virtual servers just for a simple LAMP and ftp-server.
I've got quite limited hardware.
AMD Athlon II (Dual core)
4 GB 1333 MHz DDR3 RAM
100 Mbit Fiber-lan ( Dynamic IP )
What I would want to know is how to create the virtual servers in an truly optimal way. Should I use KVM or VirtualBox? How can I make the request to mydomain.org go to one of the virtual servers and the request to anotherdomain.com go to another virtual server?
If the virtual server is the best solution.
* How much memory should be sufficient for a simple LAMP/FTP-server?
* What would be the most optimal settings for my limited hardware capacity?
View 1 Replies
View Related
Sep 1, 2011
I am going to try and replace our crappy Windows/imail server with a new ubuntu 10.04 LTS based machine. It will end up with about 400 domains on it with around 3000 users total. 96% of the domains have very light use. 4% of the domains have heavy use but do not send any bulk mail.
Can someone here recommend a mail server software package that would be easy to administrate and easy to backup/replicate? All we need is POP with webmail. We do not need IMAP or want any group/collaborative functions.
What do you usually use for anti-spam on a linux mail server? We want to keep it all open source if possible.
For hardware we are looking at an HP DL360 G5 with dual Xeons, 4G of ram, and the P400 controller with 6 SAS 15k drives. Do you think a server like this will be able to handle the mail on linux? Volume is around 400k messages a day inbound with about 10% being legit/90% being spam. The server will be dedicated to mail hosting and will not host websites.
View 1 Replies
View Related
Mar 8, 2010
I am trying to setup a file server for a small office but have hit a couple of hurdles, is there a step by step guide how to setup a network for windows and mac computers to use it? I had setup a share etc but once I restarted the server all the files disappeared which I had in the home folder? Also when I setup users how can I use passwords that I select as everytime I set one it encrypts it and uses that instead of my one?
View 2 Replies
View Related
Jul 16, 2010
Small office... 20-30 computers all windows based. xp/vista/windows 7. 3 Servers running ubuntu 10.04... 1 domain controller, and two file servers with samba. We want to be able to login once in the morning, and then not have to worry about users/passwords at all throughout the day. So I was thinking that I could setup a PDC and then make the two filservers both BDC's and use ldap to authenticate everything. Would this even work out? and is there a better way to do it?
View 9 Replies
View Related
Dec 25, 2010
I installed it on very old machine P3 1200Mhz and it works ok but I have problem with text mode because I have 75x132 aprocs. text on screen and I don't see what I type or system replay. Is there simple way to change to 25x80 or something with greater letters. I don't have GDI.Someone sad to me to do:sudo dpkg-reconfigure console-setupand it's does the job but when I restart system it's gone.
View 3 Replies
View Related
Feb 7, 2011
Rather simple question, but is there a way to make an archive (simple tarball, no compression needed) out of a very large file and split it into parts? Basically I need a ~1GB file in 25MB pieces.
View 1 Replies
View Related
Jun 8, 2010
I try to make a small AVR-Board boot a Linux from my Lenovo laptop, which is configured as NFS server (running nfs-kernel-server). Everything is configured correctly and it should work, but it doesn't.I just installed another laptop (from ASUS) with the same configuration (OS, tools, settings) and there it works immediately.
So I wondered, what's going on on eth0 ? I checked with Wireshark and here are the things I found out.On both laptops, the following paket arrives:
[Code]...
View 1 Replies
View Related
Dec 26, 2010
I have several audiobooks that are each split into many small chapters, and I would like to string together about ten of them at a time, so that I don't have 60 4-5 minute mp3 files per audiobook.If I were to do this all by hand, with audacity or something similar, it would get very tedious, so I'd like to know if there's already a program that would do it for me when told which files to string together.
View 2 Replies
View Related
Jul 2, 2010
How to configure DNS server for my small network. In Fedora Linux operating system I want to use Linux as a DNS server for my small office. there are files " /etc/hosts" and "/etc/resolv.conf ".
what is the purpose of these files and in which situation we use these files? For configuring a new DNS server. how can i configure my own DNS server for my small office? Any blog or site in which they show steps to configure DNS server.
View 8 Replies
View Related
Mar 4, 2009
Hi am using winff, as converting video is a mystery to me. I have several avi files well about 26 of them. at about 8gb. now i have to convert them into avi again! so my divx dvd machine can read/play them.
I was wondering if I could compress them so they'd fit onto one dvd?
at present I select the 16:9, would the anamorfic be smaller? would it expand properly on my cheep divx dvd machine? I don't mind using the command line.
View 1 Replies
View Related
Mar 18, 2010
Samba seems to crash and come back after some seconds if I copy a lot of small files in a short period of time over the network. How do I fix it?
I have Ubuntu 9.10 Server 64bit running on a D945GCLF2 board sharing two 1TB ext4 formatted HDDs to my Windows PCs using samba. I've been having an issue with reading or writing files through samba. It happens during copying operations or checksumming, anything that reads or writes MANY small files in a small amount of time. I am pretty sure the problem has to do with my server because the server has run on two different LANs in different homes and will crash from activity with any of several other PCs. There is no crashing if I access the files through SSH, although when I do that the max transfer speed is less than 1MB/s.
When I induce the crashing, there is absolutely no output to the server terminal.
As an easy access example of something that will crash samba, extracting Cinebench R11.5 to the server will do the job. It always fails.
View 9 Replies
View Related
Sep 26, 2010
Here's my situation :
I had a samsung 1TB HDD that I used for storing data, on an xp machine, so it was formated as NTFS.I moved this HDD to another machine and installed Freenas on it, and the installation worked fine (fyi, I used the tutorial posted here :[URL]..During the installtion, Freenas installed it's system files to a new small UFS partion. After finishing the setup, I realised that I had changed the file system of the other partion (980gb, previously NFTS) to UFS and now I don'T know how to go back. I had about 400gb of data on it and I'm pretty sure it's still there, but don't know how to get it back.
I tried messing around with recovery software such as R-Studio, and I was able to see some of my files so I know they're still there. After quite a bit of googling around, the only solution I seem to find is using gparted which is a tool to modify partions file system without loosing data, but I'm afraid to use it.
So is there a way to browse NTFS data on a UFS partition and convert it so FreeNas can see my files ? Or is there a way to put the partition back to NTFS so I can back up my data to another drive before I lose something valuable ?
View 1 Replies
View Related
Oct 2, 2009
i want to make a script that change the names of files and folders into small letter because they are all in capital and more than 1000 fileit is impossible to do that with my hand
View 14 Replies
View Related
May 18, 2011
I am very new to linux, and I have a question regarding the filesystem check (fsck). The power recently went out and when I tried to restart linux the following error appears:
*/dev/sda1 contains file system w/errors, check forced it then goes on to say..
*An error occured during the file system check. Dropping you to a shell; the system will reboot when you leave the shell. Give root password for maintenance (or type Control-D to continue) I wasn't sure what to do, but checked some other online forums and they suggested running fsck manually - so I typed in the root password - and used the command, "fsck -A -V ; echo == $? ==" it then gave the following message
*WARNING!!! Running e2fsck on a mounted filesystem may cause SEVERE filesystem damage
*Would you like to continue (y/n)
Again, I wasn't sure what to do so i just checked no. I then manually turned off the computer and was prompted at the beginning to press Alt-3. I was brought to another screen and it informed me one of the drives was degraded and suggested rebuilding the array. I tried doing this, but it still brings me back to the original error of, "/dev/sda1 contains file system w/errors, check forced," and the process continues.
Also, when I tried to rebuild the array, I didn't backup any of the data on our home directory before doing this (which was probably a big mistake). After being prompted to type the root password, I was able to give the ls command and look at all the directories...the home directory where our data was stored was empty and I am afraid I may have lost some information. Is there a possibility that data was lost when I was trying to rebuild using the old drives?
View 2 Replies
View Related
May 26, 2010
I need to process billions of small files using bash shell commands with limited memory size (256MB). If any of those files contain certain "keywords", the file will be removed. I tried with command:
find . -type f -exec grep -i -l -H "keyword" '{}' + | xargs rm -rf
where all files are located within the current directory. But the command above failed in "out of memory".
View 1 Replies
View Related
Jun 30, 2010
I just finished getting a semi-old desktop computer up and running with ubuntu server edition. It's running subversion and samba for my office of 5 people. I was wondering if there was a simple online/cloud based backup service to automatically back up my subversion repository and samba shares in case of any hardware problems?
View 1 Replies
View Related
Apr 7, 2011
I'm installing Ubuntu 10.04 server on a HP MediaSmart server. Attached is an eSATA multi-bay with four hotswap disks.Since these disks are rotating on a daily basis I'm looking for the best filesystem for hot swapping purposes. If there is no choice I'll just stick to ext4 with safe unmount. But if its possible I'd like something so I could pull the disk out without damage. In my case it would be safe to assume that the disk hasn't been written to in the last 5 minutes before rotating.
View 6 Replies
View Related
Jul 8, 2011
I want the users of the other machines, which have accounts in my server, to mount their home directories in the server. I managed to do everything, except that for the moment I can only mount their home directories by being the superuser of the server, a privilege that I don't want to give to the users. Also, I don't want their home directories to be mounted automatically. Thus, from a "normal" filesystem share, I want to: 1-The home directories of user in other machines be mountable in the user areas of the server (I can do that already).
2-I want that the users be able to mount by hand their directories, so that the directories are not permanently mounted. Currently, I can only mount and umount being the superuser of the server. I don't want to give superuser privileges to all server users.
3-I don't want their directories to be mounted on startup (otherwise I could simply add the mounts to /etc/fstab). Thus, does anyone knows how can I give the users the privilege only to mount a specific filesystem?
View 4 Replies
View Related
Mar 1, 2010
My filesystem is almost full and the problem is in the proc folder.What does that folder contain and can I delete something from it?
View 1 Replies
View Related
Mar 31, 2010
I'm currently running a small server using 9.10 and I wondered if using groups was a possible route in order to keep users away from the bulk of the file system and keep them in locked their home directories.
What I planned to do is use a group named 'allowsystemfiles' to be added to admin accounts, then to set parts of the file system to that group, along with the permissions 0760 to keep non-admin users out.
Is is a good idea or will this hose my system?
View 7 Replies
View Related
Oct 21, 2010
What is the recommended filesystem to use when creating a home server/nas?
I'd be sharing files using SAMBA, DLNA Server or some sort of streaming. I'll have two win7 laptops, 2 ubuntu desktops and ps3 accessing the files. Most of the time the server will just 75% read from 25% writing.
Would ext4 be an ideal?
View 4 Replies
View Related
Feb 9, 2011
I have a very specific requirement outside the typical use case of "enterprise server". If possible, I need to run a FUSE filesystem on a headless server, and share it via Samba.Physical security is a non-issue.I see two challenges with this, that Google and I have been unable to figure out: I'm assuming that in order to mount a FUSE system, a user must be logged in. (Hence the "U" in "FUSE".) So the first challenge is how to get Ubuntu to automatically login, on the physical console, to a user terminal session. (I know this can be trivially done in GNOME, but this won't have X-Windows or GNOME installed.) This has to happen automagically after boot-up; manually logging in locally or via SSH isn't an optin.Second challenge (?), how to share a FUSE file system via Samba?
View 2 Replies
View Related
