Programming :: Quick Measurement Of Free System Memory From Within C Program
Jul 21, 2010
I need to monitor the amount of free physical memory on Linux from within a large C program. The sampling will occur very frequently, so the measurement cannot be performance intensive. The fact that Linux uses much of the theoretically free memory for cache and buffers means that just measuring the free pages is not sufficient. Using free + cache + buffers gives an overestimate as not all cache/buffers can be freed, but I could get a rough idea of how much generally can't and subtract that from the answer.
Possible options that I've come across so far are: Parsing /proc/meminfo - but that involves reading from file which is slow. Extracting the free, cache and buffers values from the output of the Free command - but is there a quick way to do this? Parsing the /proc/freemem file produced by the API here - but this is again reading from file. Is there a way to get that output directly? Speed is an extremely high priority, and the answer it must accurately represent the amount of memory that my program could expand into (to within a few Mb).
I am writing a script to do a speed measurement from my ISP's test ftp server. To do this interactively I connect to the server, enter credentials, enable hash and download a test file. The process concludes with something like
Quote:
################################################################ 226 Transfer complete. 1048576 bytes received in 2.87 secs (356.6 kB/s)
So I entered the credentials for the server in my .netrc file and created a little bash script as follows
Quote:
#!/bin/bash cd ~/Desktop # poiuytrewq is the limit string of the "here" document
[code]...
The script executes as expected EXCEPT that I do not see the statistics at the end of the transfer. It terminates thusly
I have tried adding another ftp command such as ls to the end of the "here" document (in case the results were clipped by the bye command) but to no avail. The ls result immediately follows the ### - no stats I am at a loss.
I am using malloc and frees a lot in my program. It shows its allocated but when i remove it doesnt show as the memory is removed(I am using the top command to view VIRT memory usage). If this continously grows what would happen to my program (Will it go out of memory?)
I am looking for free database that has low memory usage and innodb and memory like engins that has C API and support trigger and client/server support for using in embedded linux systems.
I went to an interview last week and there was this guy who asked a simple question that i have been trying to solve for a couple of days. I tried google but i just cant get the search keywords right. The result were just useless. Well, the question is : "How can we allocate a limited memory to a process before we start its execution" well, the question is related to an X11 system so may be some flags must be set to limit its memory.
I wrote a program in lcc in windows and I have to write it in gcc in unix. In lcc there was an option to use more memory than the default for the stack. The following code is working in lcc but in gcc it gives segmentation fault:
I would like to know if there is a linux command to verify the amount of memory used by a program. The programs I am using were compiled with gfortran.
I wrote a multithread program(approx 1000 thread have to run) and each thread has to parse a file(for each thread there is one file, ex:thread1 has to parse file1 and thread2 has to parse file2 like this....). I wrote "parse" program as follows. It is working well, if i create 50 threads. but if i run more than 200 thraeds Im getting doublefree corruption as follows:
And some time I am getting parsing problem and error af follows:
Code:
powersetting.6607:1: parser error : Start tag expected, '<' not found (where powersetting.6607 is file name, when i check this file it is started with '<').
I have a system with 1 GB RAM. I'm running KDE 4. I created a tab to look that the Physical Memory in the System Monitor program, which I assume appears to look at the same stats that "top" looks at. In that Physical Memory tab I have 3 tables: Used Memory, Free Memory, and Application Memory.The Used Memory table shows that the system is using .94 of .98 GiBytes. The Free Memory table shows that the system has .5 GiBytes of RAM free.
However the Application Memory shows that only 339 M-Bytes of RAM is being used.Note that "top" shows the same info.So where is the other .6 GiBytes of RAM that the Used Memory table shows as being used?If I look at the process table which is supposed to encompass all of the processes running, including the ones for the OS, then it appears to add up to the 339 M-Bytes being used in the Application Memory table. Is the rest of the memory being held in reserve by the OS to be used as needed? If so, then why when another application is opened the Free Memory goes down instead of staying constant?I also noticed this memory "black hole" when I was running 11.0 on a system with 4 GB of RAM. The OS appeared to "take up" a large chunk of memory that was NOT being used by any applications and making it "disappear" - meaning that the applications were using about 1.3 GiBytes of RAM and Free Memory was showing only .7 GiBytes instead of the over 2 GB of RAM that should be free.
I am trying to run a simple perl program that requires getting stock price data from yahoo for just 1 ticker symbol, and it was running fine till this morning, wherein it froze and displayed the message: Out of memory!
I cleared my cache by running the following:
Code: $sync $sudo echo 1 |sudo tee /proc/sys/vm/drop_caches $sudo echo 2 |sudo tee /proc/sys/vm/drop_caches $sudo echo 3 |sudo tee /proc/sys/vm/drop_caches but it hasn't helped.
Even Firefox has been freezing, so I basically cannot do anything on my computer.
I have a java program that runs on Debian as a background processor. Yesterday the Java program stopped running. I looked at the memory usage, the system only had 5MB memory left, so my guess is that the java program ran out of memory to use.
However, after we restarted the java program, we could see that the free memory count started to go up. It kept going up from 5MB to over 400MB. The increase of memory happened slowly, when I measured it, I could see that with each minute passing by, there were a bit more memory added into the free memory pool, and meanwhile, the java background process was running.
I wonder why this would ever happen. It's as if our java program first brought the machine done because it consumed all the memories, then after restart, it starts to give back memories.
Basically I have a machine with 16GB of RAM and have just discovered that using all of it can crash the whole system over one process. How could I run a process on the system in such a way that if more than 90% of system memory is used, the process immediately crashes?
I need to allocate a % of the total system memory for a buffer but what is the best method to determine how much memory is in the system? So far the only way I have found is to get the pages of memory:
Code: long sysconf(_SC_PHYS_PAGES) Is that the only option?
I have now almost completed my aim of using Debian and PClinuxOS on different installed hard disks. For PCLOS I use an On-Disk set of disks which I have installed on a separate hard disk which serves as a repository from Synaptic, which makes program install/remove very quick. Is it possible to do the same thing with the 5 DVD's for Debian lenny on another hard disk?
I've got a question on free disk space. I'm currently running CentOS 5.5 on in Xenserver virtual environment. We've had an issue with disk space. My question is as follows: - from a ssh connection i run df -h this gives the value of 90% used leaving me with 9GB. If I use system monitor via a VNC connection the free disk space value is 20GB free on the same volume. Which one is correct? I do use SNMP to monitor the same volume and should alert me when < 10% is free I know this works as I set the alert threshold to < 90% I get an alert.
I would not exactly consider myself a newbie but this is a bit of a newbie question so here it goes.
When I run top I see used memory and free memory. When I first boot the system there is lots of free memory in about a day or two that memory whittles down from 2GB of Free mem(of 4GB) to 20MB and then maintains 20-50MB of free memory. This seems to be consistent across different servers. At first I thought there had to be some major memory leaks but it just seemed to prevalent to not have been noticed before.
I found a better way to gauge memory usage is the use of the free command. Now I am confused though, what is Free memory as it is defined and monitored in top?
What is the code snippet required to get an image to repeat vertically up a page? I can only find info relating to repeating *background* images in my search attempts, and this is just a regular image file which I want to repeat over the pre-existing page background. I'd like it to repeat all the way up the extreme left hand edge of the page. Say the image is called abd.jpg and is in the same folder as the index.html.
I want to add my C program's path to the environment variable PATH. My C program called "md5". So that I can execute it from anywhere (i.e any directory). My md5 program is located at "/home/ahuq/MappingServer/md5_program". So what I did was to put: "export PATH=$PATH:/home/ahuq/MappingServer/md5_program" in the running SHELL. This only makes temporary changes and lets me run the "md5" program from anywhere temporarily.
root@ahuq-kitchen:/home/ahuq# md5 -sanis MD5 ("anis") = 38a1ffb5ccad9612d3d28d99488ca94b But I want to make this change permanent. I tried to put the line "PATH=$PATH:/home/ahuq/MappingServer/md5_program" at the end of "/home/ahuq/.bashrc" and "/home/ahuq/.profile" files. I logged out of the SHELL and went in again. But it didn't work:
root@ahuq-kitchen:/home/ahuq# md5 -sanis No command 'md5' found, did you mean: Command 'cd5' from package 'cd5' (universe) Command 'mdu' from package 'mtools' (main) Command 'mdb' from package 'mono-debugger' (universe) md5: command not found
Maybe I am putting the PATH statement in the wrong place of the above mentioned files. There are a lot if-else-fi structures inside those two files. Do I need to put the PATH statement inside any of those structures? Do I just logout or do I have to restart the system to make the changes active?
I have a very weird memory issue (as you may have guessed from the title). There's a discrepency between the memory accumulated by the various applications (using "ps -eF" to collect that) and the memory reported by system monitor/"free" Basically, I have around 0.5GB of memory that is not occupied by any application, but is not free as well. (It is not used as cache as well)
When we have Oracle 10g in RedHat, the free memory goes down soon after bootup of the system and never increases. I learnt from various forums that Linux does cache the disk files for the better performance. And when any application requires memory they get the memory and the caching does not block any memory. We want to capture the free memory available for one the harware planning activity.
1) When we use vmstat command, it gives very less free memory. since it caches large size files, the free memory is very low.
2) We tried free command, but how do we know how much actual free memory is available. i.e. excluding the cache usage
When I use top to see memory usage, I have 65gb ram but only 1.3gb of it free and remaining is shown as used. When I ran my program It gives memory insufficiency error. Although no other program is using the remaining 63.7gb ram it is hold. How can I get free the unused ram?
im trying to collect some data. Is there a way to get the free memory stats over the past few days from some file... i looked into the sar log files but they dont have date stamps in them!
