Programming :: Perl - Multiple Child Processes In Parallel?

Sep 3, 2010

I'm looking for a way in Perl to be able to take a list of servers, ssh multiple commands to it and store the results. If I do this process serially, sometimes one server will hang the whole script and if it doesn't, it still takes hours to complete.

I'm thinking what I need to do is make a parent loop that calls out a separate process that passes the server name to the child sub process and then executes all the commands I have defined in its own process. If one server 'hangs', at least that won't stop the script from doing all the other servers in the list.

I'm guessing using the fork() command would serve me best, however, all the online descriptions I have found have been vague at best.

View 4 Replies


ADVERTISEMENT

Programming :: Perl Child Processes Become Zombie On A Multi-core Processor?

Apr 11, 2011

I have written a simple script which has to find required patterns from a bunch of files ( where each file is around 2 GB each,which contain the output of seq 1 10000000000000) on an 8 core machine.I am current forking 6 child processes which run simultaneously on 6 cores of the processor & have to search for the required pattern in 6 different files & inform the parent process when a pattern is found using a PIPE.

The problem is,when a child process is done reading a text file looking for a pattern,it is becoming a zombie process.It exits cleanly when i put a $SIG{CHLD} = "IGNORE"; in the script.Can any one tell me whats going on & how do i improve the communication between child and parent processes?

Code:
#!/bin/perl
use strict;

[code]...

View 3 Replies View Related

Programming :: Memory Optimization For Child Processes?

Sep 6, 2010

I work on Linux for ARM processor for cable modem. There is a tool that I have written (as the job demands) that sends/storms customized UDP packets using raw sockets. I form the packet from scratch so that we have the flexibility to play with different options. This tool is mainly for stress testing routers.

The details are here.

I actually have multiple interfaces created. Each interface will obtain IP addresses using DHCP. This is done in order to make the modem behave as virtual customer premises equipment (vcpe).

When the system comes up, I start those processes that are asked to. Every process that I start will continuously send packets. So process 0 will send packets using interface 0 and so on. Each of these processes that send packets would allow configuration (change in UDP parameters and other options at run time). Thats the reason I decide to have separate processes.

I start these processes using fork and excec from the provisioning processes of the modem.

The problem now is that each process takes up a lot of memory. Starting just 3 such processes, causes the system to crash and reboot.

I have tried the following:- 1-I have always assumed that pushing more code to the Shared Libraries will help. So when I tried moving many functions into shared library and keeping minimum code in the processes, it made no difference to my surprise.

2-I also removed all arrays and made them use the heap. However it made no difference. This maybe because the processes runs continuously and it makes no difference if it is stack or heap?

3-I suspect the process from I where I call the fork is huge and that is the reason for the processes that I make result being huge. I am not sure how else I could go about. say process A is huge -> I start process B by forking and excec. B inherits A's memory area. So now I do this -> A starts C which inturn starts B will also not help as C still inherits A?. I used vfork as an alternative which did not help either. I do wonder why.

reduce the memory used by each independent child processes.

View 1 Replies View Related

Programming :: Creating Child Processes In A Script?

Oct 14, 2010

the wrong part of the forum but basically im working on a project and getting no where what so ever and i was wondering if i could get your help. Basically i have to create two scripts do:

The parent script which is going to:
o spawn several child processes.
o keep track of the progress of the child processes.

[code]...

View 1 Replies View Related

Programming :: Ipc Between Parent And Child Processes Using Pipe?

Oct 13, 2010

If a process forks its child and communicate with the child using pipe, do closing the write end of the pipe and terminating the writing process have the same effect?

View 3 Replies View Related

Programming :: Find Number Of Child Processes (C / POSIX)?

May 26, 2010

I'm in the process of writing a program that is a server- it will accept connections and stuff, and spawn a child process for each. However, i've run into a small problem. I do NOT want to bother with keeping track of the processes unless i need to. So, i set SA_NOCLDWAIT (#ifdef) on a SIG_IGN to the SIGCHLD handler through sigaction interface. The standard says that it the kernel will then keep track of reaping zombie processes for me (a HUGE plus). However, upon receiving a SIGINT signal, i want to stop the server from accepting new connections (done), and then wait for there to be no new connections. I was thinking of just putting a loop like so:

Code:

while((wait(NULL) != (pid_t)-1) && errno != ECHILD);

However, I'm not *sure* that this will work, especially with SIGCHLD still ignored. So how can i tell if there are still child processes? I can't find any call like int getnumchld(pid_t proc); (i wish). Plus it would be inefficient to spin on that function anyway. OTOH, i would rather *NOT* have to do the same thing in a loop with a system("ps |...>file"); read(file); etc. either. Is there a way i can portably implement this feature (I was hoping i could run it on linux and the major BSDs, at least).

TO SUM IT UP:

How can i tell if a process has no child processes if i've SIG_IGN'd SA_NOCLDWAIT'd the SIGCHLD? Is there a _reasonably_ portable way to do so? I *don't* want to manually wait for EVERY process. Maybe only those still active at the time of SIGTERM, but that requires keeping track of the number of connections and whether those have terminated...

EDIT: Does anyone know if the above code *would* work, even with SIGCHLD ignored and the kernel cleaning up zombies *for* me? I checked the manpage and it doesn't say much.

EDIT1: Note that all of the processes are in the same process group and session. SO i can find them through this as well. Perhaps even setting the uid/gid and finding all processes run by that group?

EDIT2: i have an idea if the above isn't feasible. If there is no "elegant" way to do it, i could reduce the complexity by sending a SIGUSR1 to the whole process group. Each process would then set a flag telling it to send a SIGUSR1 in reply and send a SIGUSR2 when it is done executing. Then i could keep a count of signals. Maybe that would be *easier*. Or perhaps a count of all child processes and just a termination signal to decrement the counter.

View 2 Replies View Related

Programming :: Perl Script (Both Parent And Child) Dieing

Apr 13, 2010

I am writing a perl script where forking a child and the child is waiting for some response from C++ exe. Here is the code snippet:

Code:
my $retPid = fork;
if ($retPid) {
# I am parent ...
# Install signal handler
$SIG{CHLD} = &REAPER; ...
# sleep for a week (6 hours at a time)
for ($cnt = 0; $cnt < 28 ; $cnt++) {
select(undef, undef, undef, 21600);
last if ( ! -e "/proc/$retPid" ); } ...
} else {
# I am child
($iid, $rc, %data) = $cpp_exe->getResponse(604800); # 1 week timeout ... }

If the cpp_exe takes very long time (around 30 hours) to respond then the perl script (both parent and child) is dieing. The SIG{CHLD} handler defined in parent is not getting called. It seems parent is dieing first then child is dieing (or may be both dieing at the same time). Why the script is dieing?

View 1 Replies View Related

Programming :: Perl To Kill Off Some Processes?

Nov 18, 2010

how to even start this Perl script. I have the following processes:

Command used to get this info:

ps aux --forest | grep -e process_name -e ksh | awk '{if ($1 == "user1" && $1 != "root" && $1 != "UID" && $1 != "xfs" && $1 != "mfg" && $1 != "mfgnet") print $0}'

Processes

user1 2819 0.0 0.0 4272 612 ? S Nov17 0:00 \_ -pksh-ksh
user1 2820 0.0 0.0 64956 1584 pts/833 Ss+ Nov17 0:00 \_ -ksh

[code]....

I need a way to kill off the pids 2819, 2820 because they do not have a process tied to them like pids 2918, 2922 and 6657. The way it works is peek shell (pid 2918)is opened then it starts a ksh (pid 2922) session then from there the end user runs a command (pid 6657).

View 7 Replies View Related

Programming :: Creating Multiple Processes Using Fork

Oct 23, 2010

As an assignment i was doing a program to create two process using fork and pass messages between them using message queue.Did it worked well until my friend tried to copy it using scp.suddenly all hell broke loose as processes without ran syncronisation ie. in tech terms the process just wont wait wen a message queue is empty.it keeps on executing randomly.but after a reboot .. everything worked fine. until again i tried to do scp on my system on purpose. and again the program just went mad.

View 3 Replies View Related

Programming :: Creates A Child Process With Fork And, When The Child Ends, Receives The SIGCHLD Signal And Wait For Its Termination?

May 23, 2011

I have a doubt about signals in C programming. I have done this little program to explain it. It creates a child process with fork and, when the child ends, receives the SIGCHLD signal and wait for its termination.Ok, quite easy, BUT when I execute this code the SIGCHLD signal is received twice, first as an error (returns -1) and the second one to finish the child process.I don't understand the meaning of the first received signal. Why is it generated? Is the code wrong? (if you add the SIGINT and press Ctrl+C during the execution it also receives two signals instead of one)

Code: #include <stdio.h>
#include <unistd.h>
#include <string.h>

[code]....

View 4 Replies View Related

Debian :: Running Boot Processes In Parallel?

Apr 5, 2011

Does Debian 6 "Squeeze" automatically run boot processes in parallel if not how do I

configure it to do so. Here is the quote from my /etc/init.d/rc :
# Specify method used to enable concurrent init.d scripts.
# Valid options are 'none' and 'makefile'. Obsolete options

[code]...

View 2 Replies View Related

Programming :: Indexing Multiple Drives With Perl?

May 7, 2010

I have 4 separate drives, all un-RAIDed.Each drive has almost the same root directories, but different contents in each folder.

For example:

/mnt/sda/TV Shows/Lost/
/mnt/sdb/TV Shows/Your_Favorite_Show/

The goal is to have a single folder that has symlinks to all the files in each of the drives. Pretty much a poor man's JBOD. Previously, I had problems with conditions like 2 drives having the same sub folder contents, but I ended up solving that with the current script I'm using now.What I'm looking for now is speed. I'm very new to Perl and the script takes about 12 minutes to complete with the current drives.

Basically, the script makes a list of all directories and files in each drive. First, it makes the directories. I didn't use any validation because if a directory already exists, it simply won't make one. However, with the files, I used a hash to only keep the unique files. Then I use the key/value pairs with ln to create every link to the files only, not directories.

Code:

#!/usr/bin/perl
use warnings;
my @drives_to_sync = qw ( /mnt/sda/ /mnt/sdb/ /mnt/sdc/ /mnt/sdd/);

[code].....

View 2 Replies View Related

General :: Finding All Attached Child Processes

Nov 17, 2010

I want to check all the child processes attached to a specific process. Say for a example; my process is a java process. Then I wanna know what are the processes attached to that java process. At the moment I use the command # ps aux|grep java; this gives the details about the currently running java process. So I can kill the whole java process using #kill -9 {pid} ...but there are sub processes attached (I guess child processes) to java process. I wanna view all of them & kill whatever the process I like not the whole thing. I'm using Red hat 5 enterprise edition. and currently a weblogic application server is running on that.

View 2 Replies View Related

General :: Memory Optimization For Child Processes?

Sep 6, 2010

I work on Linux for ARM processor for cable modem. There is a tool that I have written (as the job demands) that sends/storms customized UDP packets using raw sockets. I form the packet from scratch so that we have the flexibility to play with different options. This tool is mainly for stress testing routers.I actually have multiple interfaces created. Each interface will obtain IP addresses using DHCP. This is done in order to make the modem behave as virtual customer premises equipment (vcpe).

When the system comes up, I start those processes that are asked to. Every process that I start will continuously send packets. So process 0 will send packets using interface 0 and so on. Each of these processes that send packets would allow configuration (change in UDP parameters and other options at run time). Thats the reason I decide to have separate processes. I start these processes using fork and excec from the provisioning processes of the modem. The problem now is that each process takes up a lot of memory. Starting just 3 such processes, causes the system to crash and reboot.

I have tried the following:-

1-I have always assumed that pushing more code to the Shared Libraries will. when I tried moving many functions into shared library and keeping minimum code in the processes, it made no difference to my surprise.

2-I also removed all arrays and made them use the heap. However it made no difference. This maybe because the processes runs continuously and it makes no difference if it is stack or heap?

3-I suspect the process from I where I call the fork is huge and that is the reason for the processes that I make result being huge. I am not sure how else I could go about. say process A is huge -> I start process B by forking and excec. B inherits A's memory area. So now I do this -> A starts C which inturn starts B will also not help as C still inherits A?. I used vfork as an alternative which did not help either. I do wonder why.

View 1 Replies View Related

Programming :: Delete Multiple Lines In A File Using Perl?

Apr 15, 2011

I have a file looks like the following:

digraph topology
{
"192.168.3.254" -> "10.1.1.11"[label="1.000", style=solid];
"192.168.3.254" -> "10.1.1.12"[label="1.000", style=solid];

[code]...

Order of these lines are random... So I cannot delete line #19, for example... And you can see that top four lines I want to delete are pairs. So there might be some clever way to detect the lines, if a line has both "1.9" and "1.11", then delete the line... I am new to perl language. The following is the code I have now... I think I just need to write some code inside the while loop checking if I want to delete the line $dotline before I write to a NEW file.

Code:

#!/usr/bin/perl -w
$TOPPATH = "/tmp";
$NAME = "topology";
$FILENAME = "$TOPPATH/$NAME.dot";

[code]....

View 16 Replies View Related

Programming :: Unable To Match Across Multiple Lines In Perl

May 11, 2011

I'm trying to split a text file into various parts. Everything in between "123" and "break" (including linebreaks) goes into the splitted file.

e.g. using this text file:

This should split into 4 files. However I'm only getting 2 files: one for the line "123break" and one for "123 blah break". The two occurrences that contain linebreaks are being ignored. The .* part of my match should capture linebreaks seeing that I'm using the /s modifier shouldn't it? Even when I use the match /(123 break)/gs it still doesn't capture the first occurrence. I'm using Perl v5.12.3 (from ActiveState) on Windows XP. The text file is also in Windows format.

Code listed below.

The above code generates two files Output_1.txt and Output_2.txt which contain "123break" and "123 blah break" respectively. I want it to generate four files.

View 4 Replies View Related

Server :: Finding Child Processes Spawned From Screen?

Aug 25, 2010

I have a situation where I have several screen (/usr/bin/screen) sessions running. 2 of the screen sessions (ps1 and ps2) run a script that launched SipP with specific parameters. 1 script starts SipP and has it make 50 calls where the other makes only 20 calls. However The script is configured where we can change how many calls it makes if needed.

So the problem is, due to issues with SipP, we must restart everything every 12 hours (at maximum). So I am trying to work out scripts to stop the SipP processes cleanly. In order to do so I need to figure out which SipP process is spawned by which screen. i.e. which sipp was started by screen session ps1, and which one was started by screen session ps2.

Now I can do ps -ef | grep <number of calls configured> to find out but then I would have to change my stop script every time we reconfigure how many calls are made, and have a separate stop script for each screen session. I would much rather be able to send the screen name as a parameter to the stop script and have it work no matter how many calls SipP is configured to make.Also your standard kill -1 <PID> does not shutdown SipP cleanly. So working out those details is a bit more tricky. Anyone know how I can determine what processes are spawned from a specific screen session?

View 2 Replies View Related

Programming :: Convert Multiple File In Directory - Ascii To Hex In Perl ?

Apr 9, 2011

I have found a perl script that can convert single file: ascii to hex.

However I have thousand of file that I want to convert from ascii to hex.

Here is the perl script that convert single ascii file to hex in single line:

Quote:

So I would like to read multiple file from a directory.

Then the file will be have same name file with hex data.

Here is sample of the read and write directory file.

Quote:

View 3 Replies View Related

Programming :: Perl Regex Not Matching Across Multiple Lines Despite Ms Flags / Fix It?

Aug 16, 2010

I have written a regular expression (tested in regexpal and regextester alpha something) with which I want to replace something like code...

but it only matches functions which occupy one line only, despite my tests showing multiple line matching in javascript testers online and using the m and s flags (which should make it multi line no?)

View 14 Replies View Related

Programming :: Perl's Length() Counts Umlauts Multiple Times

Dec 14, 2010

I'm programming some skript to get statistical information about some texts. This includes calculating the mean of word lengths.Unfortunately, Umlauts count as two characters. In the example below the output is 9, it should be 6.

sincercly, Max

Code:
#!/usr/bin/perl
use POSIX;
use locale;
my $test = length("ABC���");
print $test;

View 4 Replies View Related

Software :: Protect Environment Variables To Be Changed By Child Processes?

Jul 20, 2010

Im wondering if there is a simple way to protect environment variables to be changed by child processes?

View 1 Replies View Related

Programming :: [Perl] Fail To Sort A File With 300,000 Lines By Multiple Columns?

Nov 10, 2009

Each line of the file I am sorting is in the following format:

<url> <month> <day>

For example:

[URL]

I wrote the following to sort:

Code:

#!/usr/bin/perl
$in = shift;
chomp($in);

[code]....

The script worked fine for my small testing files, but failed in my input file. The input file is 18MB and containing more than 300,000 lines. The output will contains some lines like that:

url_one 10 1
url_two 10 1
url_three 10 3
url_four 10 1

Is that because my file is too big for perl to handle ?

View 10 Replies View Related

General :: Child Processes Of A Parent Process Occupying Socket Memory?

Jul 22, 2010

I am facing an issue where the process starts hanging. When I closely look at the logs I come to know that some of the child processes that are forked by the parent process are not finished.

1) Is it possible that the child processes that are not finished occupy the socket memory of the parent process and ultimately a point is reached where no socket memory is available to fork new child processes.

2) What is the standard limit of socket memory in linux?

3) What is the fate of such child processes (as I have mentioned above)?

4) How to debug such cases so that the exact problematic area is identified?

View 2 Replies View Related

General :: Find All Child Processes Of A Parent Process Given To Script As Argument?

Feb 15, 2011

well i have just started with shell scripting...how to find all child processes of a parent process given to script as argument.

View 10 Replies View Related

Ubuntu :: Accessing Parallel Ports From Perl Program

Dec 11, 2010

I am trying to write to the parallel port pins from a Perl program running Ubuntu Linux, to light up led's on a bread board. I've already accomplished this on this same box using a C program, but I cannot get my code below to work. I believe the problem is with the line:
$parport = Device::ParallelPort->new('auto:0');
I've tried different variables between the ('') ticks, but nothing has worked. I use Perl on my web pages for forms, but this has stumped me.

Code is below:
#!/usr/bin/perl require "subparseform.lib"; &Parse_Form; print "Set-cookie:
cart_id=1234; user_id=123;
"; print "Content-type: text/html
"; use strict; use CGI; use Device::ParallelPort;
# use Device::ParallelPort::drv; # use Device::ParallelPort::drv::linux # use Device::ParallelPort::drv::parport
# Set up your parallel port object and tell it what driver to use.
#my $parport = Device::ParallelPort->new('auto:0'); print "It works!!!";

View 1 Replies View Related

Programming :: Under Which Circumstances A Child Process Creates Another Child Process Using Fork

Dec 7, 2009

Consider the following code:

Code:

int main()
{
int i=0;
pid_t pid;
for(i=0;i<2;i++)

[code]....

I get the following output:

Parent: chid_pid=4356 i=0 parent's pid=4355
This is child 4356 i=0
This is child 4357 i=1

[code]....

I can observe instead of two children(as I expect) processes there are three. This is because child process 4356 creates its own child. Why all the messages of the type "This is child X i=Y" are concentrated one under another? How exactly fork works? Is affected by the fact that I have a dual-core processor?

View 3 Replies View Related

Programming :: Split A File To Multiple File Using Awk Or Perl?

Feb 27, 2011

I have a single file that contain multi-text something likes this:

Quote:

No. Time Source Destination Protocol Info
185 27712.068199 192.168.18.23 192.168.18.191 SMTP S: 250 2.1.5 Ok
No. Time Source Destination Protocol Info
186 27715.068293 192.168.0.50 192.168.5.2 TCP suncacao-jmxmp > 44693 [ACK] Seq=1 Ack=1 Win=64807 Len=1380

[Code].....

View 14 Replies View Related

Server :: Parallel Multiple Ssh Tool Pseudo Tty Support

Jan 24, 2011

I use mpssh or pssh to issue commands to multiple servers. Problem is that some of the commands require pseudo terminal and won't work with these tools. This will work just fine: "ssh -t user@server command" as stated under SSH man: -t = Force pseudo-tty allocation. This can be used to execute arbitrary screen-based programs on a remote machine, which can be very useful, e.g., when implementing menu services.

Multiple -t options force tty allocation, even if ssh has no local tty. So, is there a workaround for the above tools? Or is there any other tool for multi server administration that will work? If you are not sure what I am talking about just try executing "ssh user@server top" and "ssh -t user@server top" to know exactly the difference.

View 13 Replies View Related

General :: How To Create Multiple Children Before Any Child Executes

Aug 27, 2010

Iv got an assignment to complete and I'm stuck at the basic level.

A part of the assigned problem is :

The main process will read the file, and will create N number of child processes (Where N is taken as input) as early as possible before all the children starts its execution. Before creation of each child, the main process should read the file to store all the required data in then corresponding data structure. Child processes should not read the file for getting their information.

As far as my knowledge about this , the child executes before the parent. How do I make the parent not pass the control to it's children before it finishes creating all the children?

View 3 Replies View Related

Server :: How To Check Multiple Processes?

Jun 13, 2011

I have running license server on my server. Right now I would like to write small status script and check if software is running.My software include 3 deamons:

1) daemonA
2) daemonB
3) daemonC

My script should check, if each of this deamon is running. If all deamons are running then script should print short output: "License server is running" if one of this daemons is not running, output should "License server is not running". Is it possible to write small loop to check it ? Let say, loop will take new daemon name from deamons pool and will check if its running. Sometimes I need to check more than three daemons of one Program and I dont know how to write good script for this. Maybe somebody could help me with this loop that in the future I could also use; daemonD, daemonE, daemonF.etc.etc. if all daemons from pool is running then..."Software is running"

View 3 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved