Server :: Calculate Memory Available If Needed On A System?
Jul 16, 2010
I have set swappiness to 0:
# sysctl vm.swappiness
vm.swappiness = 0
According to various sources, this should mean that applications have priority over file chaches, and swap should only be used when the applications themselves need more memory than is physically available. So I naively took the value free provides in the '-/+ buffers/cache' line as 'free' as the amount of memory to be available on the server. Unfortunately this is not even close to true: On a server with 20GB RAM, memory utilization by this measure never reached 50%, yet the system swaps.
I then figured out that I could use 'sync; echo 3 > /proc/sys/vm/drop_caches' to drop cached stuff. I was very surprised to still have 12.5 GB cached data after doing that. I am figuring that it's those 12.5 GB which force the system to start swapping. I also tried to use /proc/meminfo to figure out how that cached memory is used (by comparing its content before and after dropping caches). However, I don't see the correlation between the numbers provided there and what part of the cache can be dropped.
The closest match seems to be the 'Mapped' line, which was 10GB. I am pretty sure that being mmapped keeps the kernel from dropping cache. However, the value is 2.5 GB less than the cache which can't be dropped. So it is not the whole answer. What I am looking for is some way to determine how much memory the kernel could provide by dropping stuff if he needs to because of memory pressure. Is there maybe a way to simulate drop_caches without actually doing so?
The amount of potentially available memory does not have to be scientifically correct, but the number should at least be always in the right ballpark, which right now, it ain't... The point here is that it's a productive system. Sy doing stuff like dropping caches or filling memory until the system starts to swap is not a permanent solution to figure out the value. By the way, it turned out that postgres was the culprit in this concrete case, stopping it made dropping all caches possible, but that does not answer the general question of how to estimate available memory...
1. Is my assumption correct that I can subtract 'Mapped' from the freeable cache memory completely?
2. Where could the other 2.5 GB be used?
3. Is there a way to get a better guess of how much memory the system can free if necessary, before swap has to be used?
As per the above calculation 81% of memory is used.Is this correct? and if so Am I running out of memory?what is the limit in % that I should maintain for a better performance?
how to calculate (if possible) the end address of an image file in a flash memory. I'm trying to create a checksum and checkheader function and the info that I got is the file's offset, how many sector it consumes and its size. I kinda need the end address, sad thing is, I don't know how to calculate it.test.img's start address is 0, the size is 0x20000 and consumes 3 sectors.
I am trying to write a script to calculate the total amount of installed memory to use during an anaconda kickscript, so the swap file is created at 2 x the installed memory. I so far have the amount of installed RAM DIMMS but need a way to total them up and produce a varible I can use in the pre section of the install.
Note: on some servers there could be from 1 DIMM up to 16 DIMMS installed so the script needs to be able to handle this. I also can not use bc as it does not exist during the install stage. I am guessing I need a while loop to do this and use expr but do not know where to start for this logic.
I tried to update to Ubuntu 11.04 from 10.10, and I got a strange error like "Unable to calculate changes, package ubuntu-desktop is needed but not going to be installed". Then the sistem claimed to return back to the previous state, but now ALL the package are marked as "upgradable", and the Software Updater is suggesting me to do a "partial upgrade". What happened? How I can get back with a coherent system? I don't installed the xubuntu-desktop, BTW.
i have an application that launches several pthreads, i know that the default size used by Linux is 8Mb for each pthread. However i would like to optimize the total memory usage by my application by decreasing the default stack size of each pthread to the needed resources. My questions:
- Are there any rules to set the pthread stack size. - How to compute the memory needed by each thread. - Is the malloc call inside a thread counted from the stack size of the same pthread?
I am working one project and i want to know that is it possible to find out the DNS (BIND) QPS (query per second). I mean how many QPS BIND can handle.
we found that if we use 'top' to show the memory usage of a server (SuSe Linux 10), we can get virtual memory usage as well as 'Resident memory' usage. For virtual mem or a particular process, it is around 1.1GB, which is large but for resident memory, it only consumes 300MB. Are there anyone who knows what the differences are? I would also like to know whether the difference (1.1GB - 300MB) = 800MB are actually available for use by other applications in the system.
when ever i try to install updates i get shown an error & the updates stop the error i get says "librpmio.so.0 is needed by package abrt-1.0.3-1.fc12.i686 (updates) librpm.so.0 is needed by package abrt-1.0.3-1.fc12.i686 (updates)"
I have been setting up a vps I got out with bhost.net, with CentOS installed. I've been learning and have set up everying I need with the exception of ftp/sftp.
Using yum I installed vsftpd and ran into problems, thinking it was something I might of done I did a fresh install of CentOS and I still recieve the same problem on a fresh install so it is nothing I have done to the server.
The problem is when connecting via a sftp client I get an out of memory error. This error is listed in the putty faq ( url ) under A.7.5, there is a brief explaintion of the cure under A.7.6.
there is mention of a login script but I don't know where this is located. I'm a novice at Linux but by no means incompotent when it comes to computing.
Is there a system admin tool for RHEL on montioring and logging system memory used and released that can dump to a log file?I'm having an issue with memory not being released when an application is closed. I need to have a tool monitor and log so I can troubleshoot to verify that it's an application not the OS.
I was just thinking about installing an operating system like ubuntu or ubuntu server on an external harddrive. this being possible , i would like to know if we can use the external drive to plug into any machine and run my ubuntu by just changing the needed bios mods in that system. and without further having to install the necessary hardware specific devices into my os.
Because each machine would have its own hardware set, how would the os handle it or would it have to install the necessary drivers and so on everytime it comes across a different system from the immediate previous hardware it was used with. and i know this was why laptops were invented even maybe to have that portability to use with but this without a laptop, just an external hard drive that can make up and help us use the hardware at hand with ease without any installations of any kind.
I work for an organization with a few hundred employees. We need a very simple CMS for posting basic organizational documentation such has step-by-step instructions, as well as pdf and MS Office documents.We would prefer something that would organize content hierarchically, so we can structure the info into departments and functional areas. A MySQL or similar database is preferred, but flat file is okay as long as it does not limit content/structure too much.We would probably have about 15 to 20 content authors/editors. No workflow, authentication (except authors/editors) or secure content areas are required.
The only limitation would be that the content could only be accessed from a particular range of IPs. (I assume that would be set in Apache.)
I have Bind on the machine at present, but I was wondering how much disk would be needed to make it a full DNS server that could act in place of a dead upstream service.
I have a query regarding top & virtual memory. When we run top it show VIRT (Virtual Mem), RES (Resident Mem) & SHR (Shared Memory). The total virtual memory of my machine is 4 GBs (2 GB RAM + 2 GB Swap), but still I am able to see a process showing 4000m virtual memory column. what it means, as its show VIRT Mem more than actual available VIRT memory
What are the bare minimum configuration files that would be needed to rebuild a RHEL server?We are thinking about creating a generic base image and then just copying over the necessary files (fstab, hosts, networking, etc) to get a failed system back up and running in the least amount of time possible. I am fairly new to Linux and have suggested that we have a share on a redundant server that is /server_configs/Svr_name/*.* (names are subject to change and *.* would be all of the pertinent config files to make a fresh build customized enough to emulate the failed server).Is this even possible and/or plausible?
the 4 port ethernet US Robotics, NAM USR RTR model 8000 1.0, router and printer port works in windows is there any support in linux to make printer driver work ? It is configured in windows xp as local printer through usr server driver.
I have Mythbuntu 10.04 installed on an exclusive HTPC and working great... until tonight. After letting the system update some packages (161 packages if I remember right), I suddenly have an issue where the graphical system won't start. After researching I found three error messages that might be causing that.
1. At the start of splash screen I see "UUID=xxxxxxxCD7 not ready yet or not present" I checked in /etc/fstab and found that this is the swap partition. I don't remember seeing this before so this could be the culprit.
2. I'm not at the computer in question right now but I saw a Plymouth error about "mountall" and then the message "plymouth command failed". Not sure if this could be the main error.
3. after a while (usually ca. 1-2 min) I receive thousands of errors of the kind "out of memory"... "kill process XXXX" (process vary wildly e.g. dbus-daemon, mysql, etc)... "process killed"..."respawning"
After error 3, I'm not able to switch to graphical console (ctrl-alt-F7). If I was in the graphical console at this moment, I simply can't switch to the CLI console. I'm always afraid of updating my system since I've seen lots of things breaking afterwards (usually the proprietary graphic drivers) but this is really strange.
I just installed Fedora 13 onto a "new" (to me) box, and everything seems mostly fine, except that I only seem to be able to use about 900.2 MiB of 991.4 MiB of RAM memory, according to System Monitor. I haven't set up a swap partition yet, so the problem became very, very, very obvious. When I get to about 890 MiB usage, everything goes very quickly downhill, locking up, etc. The peak I saw was 900.2 MiB. What's going on? Does this have anything to do with the system being 64-bit? Might the memory be bad?
Can I add memory to the Linux box without rebooting the system? I think Linux check the hardware at booting time. But if I need to add more memory to the server while I do not want to stop the service. Is there some method to make Linux use the new added memories?
I have a new lenovo thinkserver hardware. I was configuring my newly installed kernel (slackware 13.1 - kernel 2.6.33.4) maybe to weeks now. I was testing the configuration of the kernel parameters.Suddenly out of the blue my BIOS started to report memory error. I have 6 Fully buffered memories in pairs. Four has no advanced ECC and two supports ECC.It is possible to damage the memory by some wrong kernel configuration???I cannot believe that this is possible but I cannot see any other explanation how can a two weeks old hardware get wrong.
I have 4G memory, but linux can only detect 3.5G. The graphics card is a 128M one, even if this is shared there should still be more than 3.5G memory. What is the problem in here? code...
Last weekend i've set up my first headless ubuntu home file server and torrent downloader with ubuntu 9.10.Very cool but CPU is way too fast for a home server: P4 HT 2.8Ghz, unfortunatly it has only 256Mb of ram, so no X server and no VNC (old HP office pc) At the moment memory usage is only 40Mb without X server. Besides SSH works just fine Few questions i can't seem to find answers to on google:What is a good CL network monitoring program?mething similar to htopUbuntu 9.10 has a lot, about 20-30, console-kit deamon instanses running after boot each using some memory that i can't spare.
Last week my server crashed. I'm trying to diagnose the cause.
This is the relevant error message in /var/log/messages:
Code:
I'm assuming that I can conclude, then, that apache/httpd was the cause of the memory leak?
Next, I've been tracking my memory usage. Using top, this is an average memory load level for my server:
Code:
I'd like to confirm if my understanding of this data is correct, because Plesk indicates that my memory usage is only 50% or less. (Though I have read a number of reports indicating that Plesk's measurements are frequently wrong.)
Top says: Of the 2,073,156K total memory, 1,982,572K (95.63%) is being used, 90,584K (4.37%) is free. Of that sum, 421,948K (20.35%) are being used as buffers. Additionally, of the 2,096,472K of Swap, 60K is used, and 887,700K (42.34%) is cached.
My questions: Is my memory actually being 95% used? Or is the buffered quantity (20.35%) not a use of physical/virtual memory? (i.e. is it disk usage?) Does the amount of cached Swap influence the percentage of physical/virtual memory being used?
In other words, who is correct? Plesk says I'm using 40-50% of my memory, whereas top says 85-95%.
