Programming :: Memory Optimization For Child Processes?
Sep 6, 2010
I work on Linux for ARM processor for cable modem. There is a tool that I have written (as the job demands) that sends/storms customized UDP packets using raw sockets. I form the packet from scratch so that we have the flexibility to play with different options. This tool is mainly for stress testing routers.
The details are here.
I actually have multiple interfaces created. Each interface will obtain IP addresses using DHCP. This is done in order to make the modem behave as virtual customer premises equipment (vcpe).
When the system comes up, I start those processes that are asked to. Every process that I start will continuously send packets. So process 0 will send packets using interface 0 and so on. Each of these processes that send packets would allow configuration (change in UDP parameters and other options at run time). Thats the reason I decide to have separate processes.
I start these processes using fork and excec from the provisioning processes of the modem.
The problem now is that each process takes up a lot of memory. Starting just 3 such processes, causes the system to crash and reboot.
I have tried the following:- 1-I have always assumed that pushing more code to the Shared Libraries will help. So when I tried moving many functions into shared library and keeping minimum code in the processes, it made no difference to my surprise.
2-I also removed all arrays and made them use the heap. However it made no difference. This maybe because the processes runs continuously and it makes no difference if it is stack or heap?
3-I suspect the process from I where I call the fork is huge and that is the reason for the processes that I make result being huge. I am not sure how else I could go about. say process A is huge -> I start process B by forking and excec. B inherits A's memory area. So now I do this -> A starts C which inturn starts B will also not help as C still inherits A?. I used vfork as an alternative which did not help either. I do wonder why.
reduce the memory used by each independent child processes.
View 1 Replies
ADVERTISEMENT
Sep 6, 2010
I work on Linux for ARM processor for cable modem. There is a tool that I have written (as the job demands) that sends/storms customized UDP packets using raw sockets. I form the packet from scratch so that we have the flexibility to play with different options. This tool is mainly for stress testing routers.I actually have multiple interfaces created. Each interface will obtain IP addresses using DHCP. This is done in order to make the modem behave as virtual customer premises equipment (vcpe).
When the system comes up, I start those processes that are asked to. Every process that I start will continuously send packets. So process 0 will send packets using interface 0 and so on. Each of these processes that send packets would allow configuration (change in UDP parameters and other options at run time). Thats the reason I decide to have separate processes. I start these processes using fork and excec from the provisioning processes of the modem. The problem now is that each process takes up a lot of memory. Starting just 3 such processes, causes the system to crash and reboot.
I have tried the following:-
1-I have always assumed that pushing more code to the Shared Libraries will. when I tried moving many functions into shared library and keeping minimum code in the processes, it made no difference to my surprise.
2-I also removed all arrays and made them use the heap. However it made no difference. This maybe because the processes runs continuously and it makes no difference if it is stack or heap?
3-I suspect the process from I where I call the fork is huge and that is the reason for the processes that I make result being huge. I am not sure how else I could go about. say process A is huge -> I start process B by forking and excec. B inherits A's memory area. So now I do this -> A starts C which inturn starts B will also not help as C still inherits A?. I used vfork as an alternative which did not help either. I do wonder why.
View 1 Replies
View Related
Jul 22, 2010
I am facing an issue where the process starts hanging. When I closely look at the logs I come to know that some of the child processes that are forked by the parent process are not finished.
1) Is it possible that the child processes that are not finished occupy the socket memory of the parent process and ultimately a point is reached where no socket memory is available to fork new child processes.
2) What is the standard limit of socket memory in linux?
3) What is the fate of such child processes (as I have mentioned above)?
4) How to debug such cases so that the exact problematic area is identified?
View 2 Replies
View Related
Oct 14, 2010
the wrong part of the forum but basically im working on a project and getting no where what so ever and i was wondering if i could get your help. Basically i have to create two scripts do:
The parent script which is going to:
o spawn several child processes.
o keep track of the progress of the child processes.
[code]...
View 1 Replies
View Related
Oct 13, 2010
If a process forks its child and communicate with the child using pipe, do closing the write end of the pipe and terminating the writing process have the same effect?
View 3 Replies
View Related
May 26, 2010
I'm in the process of writing a program that is a server- it will accept connections and stuff, and spawn a child process for each. However, i've run into a small problem. I do NOT want to bother with keeping track of the processes unless i need to. So, i set SA_NOCLDWAIT (#ifdef) on a SIG_IGN to the SIGCHLD handler through sigaction interface. The standard says that it the kernel will then keep track of reaping zombie processes for me (a HUGE plus). However, upon receiving a SIGINT signal, i want to stop the server from accepting new connections (done), and then wait for there to be no new connections. I was thinking of just putting a loop like so:
Code:
while((wait(NULL) != (pid_t)-1) && errno != ECHILD);
However, I'm not *sure* that this will work, especially with SIGCHLD still ignored. So how can i tell if there are still child processes? I can't find any call like int getnumchld(pid_t proc); (i wish). Plus it would be inefficient to spin on that function anyway. OTOH, i would rather *NOT* have to do the same thing in a loop with a system("ps |...>file"); read(file); etc. either. Is there a way i can portably implement this feature (I was hoping i could run it on linux and the major BSDs, at least).
TO SUM IT UP:
How can i tell if a process has no child processes if i've SIG_IGN'd SA_NOCLDWAIT'd the SIGCHLD? Is there a _reasonably_ portable way to do so? I *don't* want to manually wait for EVERY process. Maybe only those still active at the time of SIGTERM, but that requires keeping track of the number of connections and whether those have terminated...
EDIT: Does anyone know if the above code *would* work, even with SIGCHLD ignored and the kernel cleaning up zombies *for* me? I checked the manpage and it doesn't say much.
EDIT1: Note that all of the processes are in the same process group and session. SO i can find them through this as well. Perhaps even setting the uid/gid and finding all processes run by that group?
EDIT2: i have an idea if the above isn't feasible. If there is no "elegant" way to do it, i could reduce the complexity by sending a SIGUSR1 to the whole process group. Each process would then set a flag telling it to send a SIGUSR1 in reply and send a SIGUSR2 when it is done executing. Then i could keep a count of signals. Maybe that would be *easier*. Or perhaps a count of all child processes and just a termination signal to decrement the counter.
View 2 Replies
View Related
Sep 3, 2010
I'm looking for a way in Perl to be able to take a list of servers, ssh multiple commands to it and store the results. If I do this process serially, sometimes one server will hang the whole script and if it doesn't, it still takes hours to complete.
I'm thinking what I need to do is make a parent loop that calls out a separate process that passes the server name to the child sub process and then executes all the commands I have defined in its own process. If one server 'hangs', at least that won't stop the script from doing all the other servers in the list.
I'm guessing using the fork() command would serve me best, however, all the online descriptions I have found have been vague at best.
View 4 Replies
View Related
Apr 11, 2011
I have written a simple script which has to find required patterns from a bunch of files ( where each file is around 2 GB each,which contain the output of seq 1 10000000000000) on an 8 core machine.I am current forking 6 child processes which run simultaneously on 6 cores of the processor & have to search for the required pattern in 6 different files & inform the parent process when a pattern is found using a PIPE.
The problem is,when a child process is done reading a text file looking for a pattern,it is becoming a zombie process.It exits cleanly when i put a $SIG{CHLD} = "IGNORE"; in the script.Can any one tell me whats going on & how do i improve the communication between child and parent processes?
Code:
#!/bin/perl
use strict;
[code]...
View 3 Replies
View Related
Feb 27, 2011
I have two processes that share a piece of memory, and i want to use the shared memory to send data from one process to the other. it's like a simple consumer-producer problem. when the producer fills the shared memory, it waits until the consumer can consume some data in the memory; the consumer needs to wait if there is no data in the memory. The thing gets complicated when both threads are allowed to sleep and wait for the other to wake it up.
i wanted to use condition variable of pthread for synchronization, but it doesn't work in multiple processes. i tried semaphore, but it's quite complicated and i still cannot make it right. I believe it's a common problem and someone should have written similar code before, or maybe the code is even wrapped in a library, but when I search for it on Internet, I only found information about how to share memory between processes. Does anyone know where I can find this kind of code or library?
View 2 Replies
View Related
Jul 19, 2015
I think linux must has something like RAM or memory optimization , when system can not find enough RAM .some process must be control and control some process for eat RAM .
View 4 Replies
View Related
May 23, 2011
I have a doubt about signals in C programming. I have done this little program to explain it. It creates a child process with fork and, when the child ends, receives the SIGCHLD signal and wait for its termination.Ok, quite easy, BUT when I execute this code the SIGCHLD signal is received twice, first as an error (returns -1) and the second one to finish the child process.I don't understand the meaning of the first received signal. Why is it generated? Is the code wrong? (if you add the SIGINT and press Ctrl+C during the execution it also receives two signals instead of one)
Code: #include <stdio.h>
#include <unistd.h>
#include <string.h>
[code]....
View 4 Replies
View Related
Nov 17, 2010
I want to check all the child processes attached to a specific process. Say for a example; my process is a java process. Then I wanna know what are the processes attached to that java process. At the moment I use the command # ps aux|grep java; this gives the details about the currently running java process. So I can kill the whole java process using #kill -9 {pid} ...but there are sub processes attached (I guess child processes) to java process. I wanna view all of them & kill whatever the process I like not the whole thing. I'm using Red hat 5 enterprise edition. and currently a weblogic application server is running on that.
View 2 Replies
View Related
Aug 25, 2010
I have a situation where I have several screen (/usr/bin/screen) sessions running. 2 of the screen sessions (ps1 and ps2) run a script that launched SipP with specific parameters. 1 script starts SipP and has it make 50 calls where the other makes only 20 calls. However The script is configured where we can change how many calls it makes if needed.
So the problem is, due to issues with SipP, we must restart everything every 12 hours (at maximum). So I am trying to work out scripts to stop the SipP processes cleanly. In order to do so I need to figure out which SipP process is spawned by which screen. i.e. which sipp was started by screen session ps1, and which one was started by screen session ps2.
Now I can do ps -ef | grep <number of calls configured> to find out but then I would have to change my stop script every time we reconfigure how many calls are made, and have a separate stop script for each screen session. I would much rather be able to send the screen name as a parameter to the stop script and have it work no matter how many calls SipP is configured to make.Also your standard kill -1 <PID> does not shutdown SipP cleanly. So working out those details is a bit more tricky. Anyone know how I can determine what processes are spawned from a specific screen session?
View 2 Replies
View Related
Jul 20, 2010
Im wondering if there is a simple way to protect environment variables to be changed by child processes?
View 1 Replies
View Related
Feb 15, 2011
well i have just started with shell scripting...how to find all child processes of a parent process given to script as argument.
View 10 Replies
View Related
Jan 17, 2010
I found from command 'top' that 8GB memory are used. However, using command 'ps' with some options to grep the running processes and then summing up the memory used by the running processes are less than 2 GB. Where has the used memory gone ?
View 7 Replies
View Related
Dec 7, 2009
Consider the following code:
Code:
int main()
{
int i=0;
pid_t pid;
for(i=0;i<2;i++)
[code]....
I get the following output:
Parent: chid_pid=4356 i=0 parent's pid=4355
This is child 4356 i=0
This is child 4357 i=1
[code]....
I can observe instead of two children(as I expect) processes there are three. This is because child process 4356 creates its own child. Why all the messages of the type "This is child X i=Y" are concentrated one under another? How exactly fork works? Is affected by the fact that I have a dual-core processor?
View 3 Replies
View Related
Mar 31, 2011
Though I have developed some web applications (automation tools) using PHP which have been primarily meant for Intranet (within the organizations I have worked for), I have paid little attention to SEO (it was not required until now). But now I want to understand how do we go about implementing SEO and where do we fit this mechanism in a system / website?
Do we need any other tools or learn some specific programming /scripting language to achieve it?Will implementing a good SEO yield identical results irrespective of what search engine (be it Google or Yahoo!) is being used by users?
View 1 Replies
View Related
Sep 18, 2010
I want to move files from a $SOURCEDIR to a $DESTBASE/$DESTDIR. Under $DESTBASE there are many directories, and I need to test beforehand if a file from $SOURCEDIR already exists in any of them.
This is obviously extremely slow, and the real use case involves dozens of dirs and thousands of files. Creating a temporary "index" file for the find command (instead of running it every iteration) speeds it up a little, but it's still very clumsy.
View 14 Replies
View Related
Sep 2, 2010
The following code is for monitoring the memory used by apache processes. But I got a problem that the data I got by this script is much larger than the physical memory. It was said that there are some libraries used simultaneously by many processes, so the data I got has some double counted part. Because apache has many httpd processes.
Anyone have an idea of getting the multi-processes memory used?
Code:
#!/bin/sh
#G.sh 20100813
USAGE="Usage: $0 processName"
[Code].....
View 14 Replies
View Related
Jan 25, 2010
I recently built a new server, running 64 bit Slackware 13.0 with the following specs:
MSI 785GTM-E45
AMD Phenom II X2 550
2GB DDR2
Onboard video from AMD 785G chipset
2x 80GB IDE system drives using software RAID with 2GB swap partition
I only include these because I'm not convinced my problem is not hardware related at some level. Basically, when I first start up the system, the memory usage is anywhere from 60 to 200MB. Then it starts to gradually climb until there is only 12-15MB free. This can take anywhere from a few hours to a few days.
The only thing I really use this for is to serve Samba shares and the occasional SSH login. I've set up Samba shares to be accessed by my Windows machines as well as a Mac. Initially I just explored the network if I wanted to see the share from these machines, but that was too slow and unreliable (the server would not always show up in Windows), so now I automatically mount the share as a network drive at startup (from Windows). So I don't know if this would have anything to do with the steadily increasing memory usage. These systems are not on/connected all the time, but the memory usage seems to rise anyway.
When I run top, it reports that nearly all of the physical memory has been consumed (after a while of uptime), but none of the swap space has even been touched. This is a typical output of the first several lines, sorted by swap size:
Code:
top - 07:45:48 up 2 days, 12:33, 2 users, load average: 0.00, 0.00, 0.00
Tasks: 105 total, 1 running, 104 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0%us, 0.2%sy, 0.0%ni, 99.8%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
[code]....
I can't be sure, but I believe the message just repeated itself before this part. MS-7549 is one of the components of the mainboard (MSI), maybe the northbridge or something (it seems to be associated with more than one model), so I thought for sure this meant a hardware problem, maybe with the storage controller or memory. But Memtest86 passes for more than a day without any errors, and I loaded Windows on one of the disks and ran it as hard as I could with prime95 and nothing so much as flinched. So I've got to think this is some kind of issue with linux and/or how it is handling my hardware configuration.
View 5 Replies
View Related
Sep 3, 2010
I am doing a test to get the memory used by apache`s apache2 processes. I used a script to get VmSize and VmRss from /proc/pid/status, and loop through that to get the sum of VmSize and VmRss of all the apache2 processes.
I found the VmSize (about 4GB) and VmRss (about 3.4GB) are much larger than the physical memory (1GB) when apache server was saturated. It was said because of the multi-counted libiraries size used by many processes simultaneously. Then , how to get the physical memory used by apache2 processes? Or how to get a more reasonable memory data?
View 7 Replies
View Related
Nov 6, 2010
I'm running several SHOUTcast server instances and a WowzaMediaServer instance on a CentOS machine. I'm experiencing a memory leak problem, but I can't figure out which processes are eating memory.
TOP command reports as follows:
[Code]...
Something misterious to me (I'm still a Linux newbie) is that TOP reports a total of 7.5GB used ram but very small percentage for single process (0-1%). Memory consumption starts at 1GB/8GB after reboot and in three days running gradually increases up to 8GB. I'm practising with Linux, but I still miss a lot to understand what's happening on my system. For instance, are there linux kernel logs saved somewhere that I can look at?
View 4 Replies
View Related
Jan 9, 2010
Here's what i have with Ubuntu 9.10:
Code:
ruslan@hplaptop:~$ ps -Ao%mem,pid,cmd
%MEM PID CMD
[code]...
View 14 Replies
View Related
Jun 29, 2009
I know of /etc/security/limits.conf and that can be used to limit all sorts of good things, but I haven't found anything that talks about using this when the users come from LDAP. Would I be able to do something like
@"Domain Users" soft nproc 25
@"Domain Users" hard nproc 40
where Domain Users is the group all users belong to in our system.
View 3 Replies
View Related
Nov 2, 2010
Are there any tools to view/edit user space memory of running processes on Linux?
It would be a great learning tool.
View 1 Replies
View Related
Sep 22, 2010
I wrote a program that multiplies 2 matrices using multi-threads and another one using multiple processes and shared memory. Both in C.I need to find the total memory usage of these programs. I know of the top command, but when my matrices are relatively small they don't even show up on top because they complete so fast, how can I find the memory usage for these instances?Also, how can I find the total turnaround time of my programs
View 1 Replies
View Related
Jul 30, 2009
I am troubleshooting something and I got this problem.
If I do "pstree -p"
It shows,
Code:
[code]....
However, it does NOT show up in "ps -elf"
Code:
ps -elf | grep soffi
0 S whho 7734 1 0 80 0 - 36435 - 11:14 pts/2 00:00:03 /usr/lib/openoffice/program/soffice.bin -splash-pipe=5
0 S whho 7833 7759 0 80 0 - 751 - 11:21 pts/3 00:00:00 grep soffi
I was wondering if 7735, 7736, 7737, 7743 were really processes. Then I checked /proc, I could cd to /proc/7735, /proc/7736, etc, but I could not ls them out. I looked at the man page of "pstree", it says,
Code:
Child threads of a process are found under the parent process and are shown with the process name in curly braces, e.g.
icecast2---13*[{icecast2}]
So, what does all this mean? Does it mean that 7735, 7736, 7737, 7743 are just threads but not processes? If so, why could I cd to /proc/<id> but not see them in "ps -elf".
View 10 Replies
View Related
Mar 21, 2011
I am new to C and linux. My code below does arbitary writes but I cant figure out where or how it does it.
I am calling the insertNode() function with seq = 'MISSISSPPI$' and alphabets = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ$'
Code:
Weird behaviour I should mention is that when I check for NULL pointer in node->child[index], the unassigned values are not null anymore, they point to arbitary memory.
View 12 Replies
View Related
Apr 10, 2011
Now,I created one parent and two children. I must create some value as random in child1 and child1 has to send these values to child2.Child2 must read them.. now,I cant create some random values and i can send them.But child2 doesnt work. I have two functions. One them is writing, another one is reading. Child2 uses reading function,but it doesnt work (child2 cant call it, because writing function in endless loop) What i must do? I used wait,usleep... No way.. I attached my file and also there is code..
PHP Code: #include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <signal.h>
#include <stdarg.h>
#include <sys/types.h>
[Code].....
View 2 Replies
View Related
