General :: Get The Physical Memory Used By A Couple Of Processes?
Sep 3, 2010
I am doing a test to get the memory used by apache`s apache2 processes. I used a script to get VmSize and VmRss from /proc/pid/status, and loop through that to get the sum of VmSize and VmRss of all the apache2 processes.
I found the VmSize (about 4GB) and VmRss (about 3.4GB) are much larger than the physical memory (1GB) when apache server was saturated. It was said because of the multi-counted libiraries size used by many processes simultaneously. Then , how to get the physical memory used by apache2 processes? Or how to get a more reasonable memory data?
I found from command 'top' that 8GB memory are used. However, using command 'ps' with some options to grep the running processes and then summing up the memory used by the running processes are less than 2 GB. Where has the used memory gone ?
I am trying to build and bring-up Linux (embedded) for a piece of hardware which have MIPS 74K proccessor 16MB Flash, 128MB DDR and network/usb support. How to configure/set into the kernel the exact addresses of the physical memory map? How does the kernel know where is the system ram, i/o memory, root FS? I have read some book and I found how the applications can go and read some special files like /proc/iomem to find out info about memory but what I need is how to set those addresses at the beginning when I build the kernel and FS in order to boot the kernel on my h/w.
I am monitoring physical memory in a server I administer, and my hardware provider told me they had increased physical memory size to 4Gb... However, using several tools (free -m; top; dmesg | grep Memory; grep MemTotal /proc/meminfo I discovered that I actually have 3Gb, not 4... But, my doubt comes from the fact that dmesg | grem Memory tells me I have 3103396k/4194304k available The first number is effectively 3Gb, but the second one, is 4! so, why I am looking at this two different numbers?
When we want to setup a linux system, there is a common a suggestion like set the swap space as twice as big than your physical memory, I want to know why do we need this and how is this suggestion come from?
A process is trying one access to memory, for example through an array (ex.: vect[0]=123. What happens?
Here below what I guess but I'm not sure and accept any comment (please, distinguish between "the system" and "the CPU" in case).
Let's suppose swapping to disk disbled.
We have two scenarios: without and with cache.
If no cache is present in the system: 1. The CPU must discover the phys addr of vect[0] virtual addr. To do that, has to read from 3 (or 2 depending on the system?) pages tables, stored in memory as well. 2. The CPU writes to the final address.
These mean 4 memory accesses.
If cache is present: 1. Like above but, if the pages tables are in cache, we have 3 accesses to that. 2. If the req. page is not in cache, it's reads from ram and transferred to it. Afterwards, cache is written. In the best case we have 4 cache accesses.
I have a 32 bit Ubuntu installed and my Laptop has 4GB RAM, but only 3GB is considered by Linux. My question is: what is the reason for the upper limit on physical memory ?
Code: dmesg | grep Memory [0.000000] Memory: 3052428k/3112960k available (4673k kernel code, 56364k reserved, 2121k data, 656k init, 2200904k highmem) I am familiar with the virtual memory concept where linux splits upper 1GB for kernel and lower 3GB for user processes. In total, linux 32bit can address 4GB virtual addresses. Does this meant that 1GB of physical memory is already mapped to 1GB of kernel space and Linux only shows the remaining 3GB physical memory left for the user in the above command.
I did some searching on the internet and found some articles related to this, but it only confused me further since some articles suggest 4GB is the upper limit with mentioning whether it's virtual or physical memory, some bring in the concept of PAE, etc. I'm relative new to Linux's memory management, so it'd be really helpful if someone could answer this.
The following code is for monitoring the memory used by apache processes. But I got a problem that the data I got by this script is much larger than the physical memory. It was said that there are some libraries used simultaneously by many processes, so the data I got has some double counted part. Because apache has many httpd processes.
Anyone have an idea of getting the multi-processes memory used?
I work on Linux for ARM processor for cable modem. There is a tool that I have written (as the job demands) that sends/storms customized UDP packets using raw sockets. I form the packet from scratch so that we have the flexibility to play with different options. This tool is mainly for stress testing routers.I actually have multiple interfaces created. Each interface will obtain IP addresses using DHCP. This is done in order to make the modem behave as virtual customer premises equipment (vcpe).
When the system comes up, I start those processes that are asked to. Every process that I start will continuously send packets. So process 0 will send packets using interface 0 and so on. Each of these processes that send packets would allow configuration (change in UDP parameters and other options at run time). Thats the reason I decide to have separate processes. I start these processes using fork and excec from the provisioning processes of the modem. The problem now is that each process takes up a lot of memory. Starting just 3 such processes, causes the system to crash and reboot.
I have tried the following:-
1-I have always assumed that pushing more code to the Shared Libraries will. when I tried moving many functions into shared library and keeping minimum code in the processes, it made no difference to my surprise.
2-I also removed all arrays and made them use the heap. However it made no difference. This maybe because the processes runs continuously and it makes no difference if it is stack or heap?
3-I suspect the process from I where I call the fork is huge and that is the reason for the processes that I make result being huge. I am not sure how else I could go about. say process A is huge -> I start process B by forking and excec. B inherits A's memory area. So now I do this -> A starts C which inturn starts B will also not help as C still inherits A?. I used vfork as an alternative which did not help either. I do wonder why.
As i undertsand - out of 1GB of the virtual Address space for Kernel from 3GB to 4GB of the process address space, Kernel image (code, data, bss, stack, heap) resides staring @0x0 address. Vmalloc area starts either at the end of Physical ram size or at 896M. This 896M cap is mandated to ensure that minimum of 128MB is reserved as vmalloc_reserve for vmalloc,kmap etc.
Is the understanding correct? Now trying to map Physical Zones into this 1GB address space
Initial 16MB is mapped to ZONE_DMA 16MB - 896MB is mapped to ZONE_NORMAL 896MB - 1024MB is mapped to ZONE_HIGHMEM
Does this mean that Kernel image is residing in ZONE_DMA area? Any call to vmalloc() in kernel code will return address beyond 896M? insmod of any LKM will internally invoke vmalloc() to obtain contiguous area - where will this code physically located along with rest of kernel code in ZONE_DMA or in ZONE_HIGHMEM?
I am facing an issue where the process starts hanging. When I closely look at the logs I come to know that some of the child processes that are forked by the parent process are not finished.
1) Is it possible that the child processes that are not finished occupy the socket memory of the parent process and ultimately a point is reached where no socket memory is available to fork new child processes.
2) What is the standard limit of socket memory in linux?
3) What is the fate of such child processes (as I have mentioned above)?
4) How to debug such cases so that the exact problematic area is identified?
I wrote a program that multiplies 2 matrices using multi-threads and another one using multiple processes and shared memory. Both in C.I need to find the total memory usage of these programs. I know of the top command, but when my matrices are relatively small they don't even show up on top because they complete so fast, how can I find the memory usage for these instances?Also, how can I find the total turnaround time of my programs
I have a system with 2G of memory and swap memory of 4G.
This is the output from :
PHP Code:
How could they do to the memory cache to be used as much? Because, occasionally, swap is used and note that the system could use the memory cache does not swap ...
I have a system with 1 GB RAM. I'm running KDE 4. I created a tab to look that the Physical Memory in the System Monitor program, which I assume appears to look at the same stats that "top" looks at. In that Physical Memory tab I have 3 tables: Used Memory, Free Memory, and Application Memory.The Used Memory table shows that the system is using .94 of .98 GiBytes. The Free Memory table shows that the system has .5 GiBytes of RAM free.
However the Application Memory shows that only 339 M-Bytes of RAM is being used.Note that "top" shows the same info.So where is the other .6 GiBytes of RAM that the Used Memory table shows as being used?If I look at the process table which is supposed to encompass all of the processes running, including the ones for the OS, then it appears to add up to the 339 M-Bytes being used in the Application Memory table. Is the rest of the memory being held in reserve by the OS to be used as needed? If so, then why when another application is opened the Free Memory goes down instead of staying constant?I also noticed this memory "black hole" when I was running 11.0 on a system with 4 GB of RAM. The OS appeared to "take up" a large chunk of memory that was NOT being used by any applications and making it "disappear" - meaning that the applications were using about 1.3 GiBytes of RAM and Free Memory was showing only .7 GiBytes instead of the over 2 GB of RAM that should be free.
I allocated a chunk of memory using kmalloc in a Device Driver. Kmalloc provides a pointer to the allocated memory. This is one of my first few drivers.
I assume that the address returned is a Virtual address. I need to find the physical address of the memory location. I am working on an Intel 64 bit Fedora machine. I used the virt_to_phys() routine present in <asm/io_64.h>. I found that this routine returns an unsigned long value (32 bit) instead of an unsigned long long value (64 bit). Moreover, it seems that it simply returns the address - OFFSET instead of extracting the value in the page tables.
So is there any function / system call in Linux which will allow me to see the actual physical address on the Intel 64 arch.
I recently built a new server, running 64 bit Slackware 13.0 with the following specs:
MSI 785GTM-E45 AMD Phenom II X2 550 2GB DDR2 Onboard video from AMD 785G chipset 2x 80GB IDE system drives using software RAID with 2GB swap partition
I only include these because I'm not convinced my problem is not hardware related at some level. Basically, when I first start up the system, the memory usage is anywhere from 60 to 200MB. Then it starts to gradually climb until there is only 12-15MB free. This can take anywhere from a few hours to a few days.
The only thing I really use this for is to serve Samba shares and the occasional SSH login. I've set up Samba shares to be accessed by my Windows machines as well as a Mac. Initially I just explored the network if I wanted to see the share from these machines, but that was too slow and unreliable (the server would not always show up in Windows), so now I automatically mount the share as a network drive at startup (from Windows). So I don't know if this would have anything to do with the steadily increasing memory usage. These systems are not on/connected all the time, but the memory usage seems to rise anyway.
When I run top, it reports that nearly all of the physical memory has been consumed (after a while of uptime), but none of the swap space has even been touched. This is a typical output of the first several lines, sorted by swap size:
I can't be sure, but I believe the message just repeated itself before this part. MS-7549 is one of the components of the mainboard (MSI), maybe the northbridge or something (it seems to be associated with more than one model), so I thought for sure this meant a hardware problem, maybe with the storage controller or memory. But Memtest86 passes for more than a day without any errors, and I loaded Windows on one of the disks and ran it as hard as I could with prime95 and nothing so much as flinched. So I've got to think this is some kind of issue with linux and/or how it is handling my hardware configuration.
I work on Linux for ARM processor for cable modem. There is a tool that I have written (as the job demands) that sends/storms customized UDP packets using raw sockets. I form the packet from scratch so that we have the flexibility to play with different options. This tool is mainly for stress testing routers.
The details are here.
I actually have multiple interfaces created. Each interface will obtain IP addresses using DHCP. This is done in order to make the modem behave as virtual customer premises equipment (vcpe).
When the system comes up, I start those processes that are asked to. Every process that I start will continuously send packets. So process 0 will send packets using interface 0 and so on. Each of these processes that send packets would allow configuration (change in UDP parameters and other options at run time). Thats the reason I decide to have separate processes.
I start these processes using fork and excec from the provisioning processes of the modem.
The problem now is that each process takes up a lot of memory. Starting just 3 such processes, causes the system to crash and reboot.
I have tried the following:- 1-I have always assumed that pushing more code to the Shared Libraries will help. So when I tried moving many functions into shared library and keeping minimum code in the processes, it made no difference to my surprise.
2-I also removed all arrays and made them use the heap. However it made no difference. This maybe because the processes runs continuously and it makes no difference if it is stack or heap?
3-I suspect the process from I where I call the fork is huge and that is the reason for the processes that I make result being huge. I am not sure how else I could go about. say process A is huge -> I start process B by forking and excec. B inherits A's memory area. So now I do this -> A starts C which inturn starts B will also not help as C still inherits A?. I used vfork as an alternative which did not help either. I do wonder why.
reduce the memory used by each independent child processes.
I'm running several SHOUTcast server instances and a WowzaMediaServer instance on a CentOS machine. I'm experiencing a memory leak problem, but I can't figure out which processes are eating memory.
TOP command reports as follows:
[Code]...
Something misterious to me (I'm still a Linux newbie) is that TOP reports a total of 7.5GB used ram but very small percentage for single process (0-1%). Memory consumption starts at 1GB/8GB after reboot and in three days running gradually increases up to 8GB. I'm practising with Linux, but I still miss a lot to understand what's happening on my system. For instance, are there linux kernel logs saved somewhere that I can look at?
I have two processes that share a piece of memory, and i want to use the shared memory to send data from one process to the other. it's like a simple consumer-producer problem. when the producer fills the shared memory, it waits until the consumer can consume some data in the memory; the consumer needs to wait if there is no data in the memory. The thing gets complicated when both threads are allowed to sleep and wait for the other to wake it up.
i wanted to use condition variable of pthread for synchronization, but it doesn't work in multiple processes. i tried semaphore, but it's quite complicated and i still cannot make it right. I believe it's a common problem and someone should have written similar code before, or maybe the code is even wrapped in a library, but when I search for it on Internet, I only found information about how to share memory between processes. Does anyone know where I can find this kind of code or library?
Whenever I'm running my application process, I've 1M physical memory usage is increasing for every 2 hours.This I observed using 'free -m' command.But 'top' command did not showing any increase 'RSS' size.It is same as it was started initially.Even though I stopped my process,the increased memory was not released back. If I start my application process then again memory usage start increasing by 1M for every 2 hours. increase of memory usage observer with 'free' and that too when my application is running, but top command is not showing any change in the RSS sizeIf my application is leaking any memory which is allocated by new/malloc, that should be released back whenever my application exit and the size increase will be show through top command for that process, right? This is not happeningThis proves that there is no potential leaks in my process.But why physical memory is increasing when only my process is running?
