Programming :: Running Many Tasks In Parallel / But More Clever Than 'make -j'?

I have a script which basically does many tasks and most of them can run in parallel. The server it runs on has 4 cores, so I changed parts of it to a makefile, and then run them with "make -j 4" which helps a lot. But looking at "top" while it runs, I see it could be improved. Sometimes tasks wait for other servers, so a higher -j value would help. And sometimes multiple tasks are waiting on the same disk, then I think a lower -j value would be better.

The kernel is fairly good at sharing CPU time among the cores, but resources like memory and disks are not shared very well. As an experiment, I tried compiling the kernel with "make -j" with no number. It didn't go very well.

So I wondered if anybody know of a better tool for things like this. I think it should be possible to write something that could change the -j value while it runs.For example:

- Start as many tasks in the background as there are cores. When tasks finish, start another one until done.
- After a little while, check CPU and memory usage to decide if more tasks could run in parallel. Or maybe less?
- If different tasks iowait a lot, check if they wait on the same disk. If so, renice/ionice/suspend some of them. In these situations I think it's better to have them run one after another.
- Other ideas?

I thought of making a script like that, but I doubt I'm the first one with that problem. Also, there are probably lots of things I haven't thought of.