Programming :: Run Parallel Jobs With Make -j4?
Nov 27, 2010
I use an application called redbutton-browser to access some of the things available on the redbutton digital tv channels. It compiles fine if I use a simple make but fails if I try to run parallel jobs with make -j4. I'd like some help altering the Makefile so that it does a few commands sequentially before it does the rest of the Makefile in parallel.
The relevant bit of the Makefile looks like this:
Code:
ISO13522-MHEG-5.c:xsd2c.c ISO13522-MHEG-5.xsd add_instance_vars.conf add_rtti.conf
make xsd2c
./xsd2c ISO13522-MHEG-5.xsd
./add_instance_vars ISO13522-MHEG-5.c ISO13522-MHEG-5.h
./add_rtti ISO13522-MHEG-5.c > rtti.h
[Code]....
View 7 Replies
ADVERTISEMENT
Mar 18, 2010
I need to spawn 2 processes in parallel and each takes an hour or so to finish. Is the following one of the correct ways of using `at` in a script run by crontab?
Code:
#!/bin/bash
# define the env var, cd, etc... assume everything ok up to this point
date +"The start time is %H:%M:%S"
rm -f a.fin
at now <<END_OF_AT
do_a &> a.log
[Code]....
View 2 Replies
View Related
Apr 29, 2011
I have a script which basically does many tasks and most of them can run in parallel. The server it runs on has 4 cores, so I changed parts of it to a makefile, and then run them with "make -j 4" which helps a lot. But looking at "top" while it runs, I see it could be improved. Sometimes tasks wait for other servers, so a higher -j value would help. And sometimes multiple tasks are waiting on the same disk, then I think a lower -j value would be better.
The kernel is fairly good at sharing CPU time among the cores, but resources like memory and disks are not shared very well. As an experiment, I tried compiling the kernel with "make -j" with no number. It didn't go very well.
So I wondered if anybody know of a better tool for things like this. I think it should be possible to write something that could change the -j value while it runs.For example:
- Start as many tasks in the background as there are cores. When tasks finish, start another one until done.
- After a little while, check CPU and memory usage to decide if more tasks could run in parallel. Or maybe less?
- If different tasks iowait a lot, check if they wait on the same disk. If so, renice/ionice/suspend some of them. In these situations I think it's better to have them run one after another.
- Other ideas?
I thought of making a script like that, but I doubt I'm the first one with that problem. Also, there are probably lots of things I haven't thought of.
View 5 Replies
View Related
Jun 27, 2010
I need to make some C program under linux to make some scripts that can talk to my Server's Parallel port by sending some quare signal as high or low, so what i can do ?
View 4 Replies
View Related
Jun 28, 2010
in my linux box i have lot of cron jobs. i need to make all cron jobs in script files. is it possible to make this. like[URL]Now how i change this in a script file
View 4 Replies
View Related
Oct 1, 2010
I have a bash script with a line like this in it:
Code: mysqldump -uusername -ppassword --master-data=2 --single-transaction --quick --add-drop-table --add-drop-database somedatabase | gzip > ./somedatabase.sql.gz The problem is, I have about 16 databases I'm dumping like this, but it's dumping
[Code]....
So, does putting in a & after a command work in a bash script at all? The thought was, if I want to run them in parallel, should I do this?
Code: #!/bin/bash
./scripts/dumpthis &
./scripts/dumpthat &
./scripts/dumpsomemore & Where "dumpthis" is just the first snippet of code above.
View 3 Replies
View Related
Aug 30, 2010
why when I use "jobs -l >> tmp" in a script file it does nothing, and when typing it at the prompt it creates the tmp file I expect it to...?
View 9 Replies
View Related
Feb 25, 2011
I am doing a project where 2 clients connect to server and communicate (chat) and transfer data one after other using sockets. I have working code for this in C language. Now our main aim is to create a communication link where two clients transfer multiple streams data parallely. To be more precise i want to transfer images files and audio files parallel at same time, so is it possible to send data parallel using one socket connection?
View 3 Replies
View Related
May 29, 2011
i want to know whether we can write a shell script which can utilize 2 CPU's of a cluster...I heard its possible through C Shell [URL] But im familiar with bourne shell only....So is there a possibility to write Parallel Scripts.
View 5 Replies
View Related
Jul 29, 2011
I came across some problems. I need to conduct parallel computing on CUDA. I input "uname -a"and read these: "Linux BXSJC 2.6.38.8-35.fc15.i686 #1 SMP Wed Jul 6 14:46:26 UTC 2011 i686 i686 i386 GNU/Linux". I use CUDAtoolkit 4.0. when I input: cd /home/bxsjc NVIDIA_GPU_Computing_SDK/C and "make -i ",the computer returns
[Code]...
View 7 Replies
View Related
Aug 8, 2010
I have a little hardware project and I would like to use the parallel port. I basically want to display status code on top of other custom peripheral on an headless server. I only use output as the display (7 segment) is not part of the device itself. The problem is, the parallel port only have 8 standard I/O pins. I have read somewhere that there is a way to change the mode of the port to enable more data pins using a specification not backward compatible with the "classic" mode (so not enabled by default). I didn't find any other information about this. As I would like to have all 17 non-ground pin to avoid using more circuitry than necessary, how to turn this mode on?
View 3 Replies
View Related
May 4, 2009
I'm looking for a general pointer, or label for what I am trying to do. Not a specific line of code. I've written a couple of small sh script that gets some user input, then calls several programs to run in the background with the &. My problem is, they all run at the same time. Is there some way to get these jobs to run sequentially rather than consecutively? If yes, what is this process called? I'm thinking there should be a way to line up background jobs in a job queue, similar to how a print queue works, one job at a time. After searching for a couple of hours, I'm thinking there must be a name for this, but I don't know what it is.
View 11 Replies
View Related
Jul 22, 2014
I am trying to replicate what is happening on this page under the tcsh shell, but using the bash shell found in Wheezy. Here is the page I am referring to:[URL] The command I am trying to replicate is on page 6 under figure 2.4. The command is "prompt> ./mem &; ./mem &".
I would like to run the same program twice, concurrently, but do not know how. Note that I am not trying to use a bash script, but rather by simply using syntax on the command line.
View 4 Replies
View Related
Sep 3, 2010
I'm looking for a way in Perl to be able to take a list of servers, ssh multiple commands to it and store the results. If I do this process serially, sometimes one server will hang the whole script and if it doesn't, it still takes hours to complete.
I'm thinking what I need to do is make a parent loop that calls out a separate process that passes the server name to the child sub process and then executes all the commands I have defined in its own process. If one server 'hangs', at least that won't stop the script from doing all the other servers in the list.
I'm guessing using the fork() command would serve me best, however, all the online descriptions I have found have been vague at best.
View 4 Replies
View Related
Oct 15, 2010
Never done it before, don't know how but its due next week. I am a computer systems student and have been required to built a hardware that will be operated by a program in G++ through the parallel ports. The hardware, thats a piece of cake, the software, now that's were the real problem is. I don't even know were to start.
View 2 Replies
View Related
Apr 7, 2011
I am trying run audio conversion on my server that I want limited to a certain number of processes based on process name. I am using the following script but it isnt limiting the number of job like I want it to.
Code:
#!/bin/bash
$num_jobs = 13
while [ $(ps -A | grep -v grep | grep -c pacpl) -ge $num_jobs ]
do
sleep 1
[Code]...
View 3 Replies
View Related
May 27, 2010
I want to write a shell script for submitting multiple jobs sequentially. For example,
#!/bin/bash
qsub job1.sh
# when job1.sh is done, run the next one
qsub job2.sh
how to make the shell script know that job1.sh is done and now I can run job2.sh?
View 4 Replies
View Related
Mar 2, 2011
If multiple threads operate on a single shared resource, we can lock it using pthread_mutex_lock. Uptill that it is fine, but why again pthread_cond_wait?
int shared=0; // global
.
.
// Thread 1
pthread_mutex_lock(&mutex);
shared=1;
pthread_mutex_unlock(&mutex);
When thread1 invokes the mutex lock, no other threads can access it in parallel. So why again and what for we use, pthread_cond_wait( ).
View 2 Replies
View Related
Apr 13, 2011
I just can't find... why is my code seg-faulting?
PHP Code:
/*
Code name: Assignment #7.
File name: slice.c
Program name: Matrix multiplication (parallel version).
Version: 1.
[Code]...
Is giving my a seg-fault right at the ending point where I do the final gather from the MASTER processor.
View 9 Replies
View Related
Sep 15, 2010
I want to gave much details as possible. working directory (~/a1/shell) in the shell directory i have Makefile. also in the shell directory i have subdirectory's (obj, src, include)
My current Makefile
Quote:
#What needs to be built to make all files and dependencies
clean:
# End of Makefile
I wanted it so: all .o files are created in the obj subdirectory, and my application, sshell, is created in the shell directory.
I am getting this error when i run the make run: No rule to make target 'shell.h', needed by 'shutil.o'. stop
View 1 Replies
View Related
Dec 16, 2010
I was using ubuntu before i over to fedora, in ubuntu i had the commandline "hd + nameOfFile". what is the parallel command in fedora??
View 2 Replies
View Related
May 22, 2011
I have this in my crontab (along with another line before it):
Code:
* * * * * /bin/echo "Does this work?" > ~/test
...and the file doesn't show up. I have a blank line after that one in the crontab. I have no /etc/cron.allow or /etc/cron.deny files. I'm running Ubuntu 11.04. /var/log/syslog shows "BEGIN EDIT" and "END EDIT" etc. when I run crontab -e, but doesn't say anything about trying to run commands. Anything I might be missing?
View 9 Replies
View Related
Feb 4, 2010
I just reinstalled linux on one of my computers, and now it seems that any scripts in the rc.d folders don't run at boot. I have to go in manually and start them all after I can get to a terminal. I have webmin running on this computer, and no matter what I do, it always says that nothing is supposed to start on boot, even though the scripts are clearly in their respective rc.d folders. What can I do to fix this?
View 1 Replies
View Related
Feb 18, 2011
I want use two computer to run OpenFOAM or Fluent software.These two computers are connected via a cross cable. I have access to another by "ssh maysam@192.168.1.1" OR "ssh 192.168.1.1" The problem is i should have access to another PC by "ssh maysam-desktop" command which can run parallel. maysam-desktop is name of anothe PC and maysam is name of its user.
View 1 Replies
View Related
Mar 2, 2011
I'd like to connect a parallel port printer to my new desktop and have got a Connectland C36 cable that shows up as: Bus 003 Device 002: ID 067b:2305 Prolific Technology, Inc. PL2305 Parallel Port.I've tried the URI /dev/usb/lp0 but I don't mange to set the printer working. May I need any driver?
View 8 Replies
View Related
Feb 23, 2010
My new Shuttle runs Debian fine, but like most new systems, the old serial and parallel ports are long gone. I have no problems with the serial port - there is a Sourceforge driver to use with most PL-2303 type USB to Serial adapter cables. But, I am having trouble googling up the USB to Parallel connection. There are no lack of USB/Parallel cables, but all that I have found come with canned software for Windows and only for connecting a printer. I have several non-printer devices that connect via the P-Port that I access using a Perl driver from CPAN. Anybody know of an open source *nix driver for any particular USB/PP cable? Or had any experience in moving from the straight PP to a USB/PP connection to a non-printer device?
View 1 Replies
View Related
Jan 10, 2010
I have been trying to install an old HP Laserjet 4100 printer using an USB to Parallel Cable (Cables Limited USB IEEE 1284 printer cable). The driver provided with the cable worked on windows XP and I was able to print. The cable is also sold as supporting Linux but it seems that is was tested for fedora core 4 and Moschip Support Team wrote me that ubuntu 9.04 was not supported. When trying to install the driver (makefile) I got a lot of errors (see annex) . I have a System 76 PC with ubuntu 9.04, kernel 2.6.28-17-generic, intel(R) Core(TM). Whwn connected the printer is not recognized.
How to install the cable ? Or any trade / model of printer cable really able to work with ubuntu ?
Additional information: lsusboutput, linux dirver file list, errors whwn carrying out the make and dmesg output
Additional information: lsusb indicates Bus 002 Device 002: ID 9710:7705 MosChip Semiconductor Printer USB to Parallel
The linux driver files are:
Errors when doing make:
View 7 Replies
View Related
Mar 17, 2010
I have a shell script to refresh some databases. Currently I open 5 sessions and to execute the five scripts.
Today, as test, I tried to see if I could run two refreshes simultaneously from one session using the '&':
. /refresh.ksh db1 &
. /refresh.ksh db2
The first one ran and the second one started only after the first one was finished. Could it be that the script refresh.ksh contains many commands and those are not pushed to the background? If so, what's the best approach to getting 5 db's refresh simultanously from on session?
View 6 Replies
View Related
May 6, 2011
I have 5 HP workstation connected with HP ML370 servers. My software "Synopsys" is node locked (run only on server)the problem is that when i run simulation its take too much time. I want to share server load on all 5 workstations on my network.how to utilize these workstations resources e.g RAM, CPU etc...
View 8 Replies
View Related
Aug 13, 2011
I am trying to print to an HP LaserJet 6L via a USB->parallel printercable, from a CentOS 5.3 machine using CUPS 1.3.7, but I cannot get itto work.When I plug the cable into the USB port on the CentOS computer, I seethat the following devices are created:
crw------- 1 root root 442, 6145 Aug 13 08:28 /dev/usbdev4.2_ep00
crw------- 1 root root 442, 6145 Aug 13 08:28 /dev/usbdev4.2_ep02
crw------- 1 root root 442, 6145 Aug 13 08:28 /dev/usbdev4.2_ep81
[code]....
View 9 Replies
View Related
