General :: RS232c And OpenGL In Same Program

I am trying to write a program that accepts user input from the keyboard and a joystick and then sends out the data using the rs232 port of my system to other hardware. I need to send out about 10 bytes every 20 milliseconds. The thing is I need to also keep on reading data from the same port and display it on an opengl screen. This visualizes the hardware data coming in.

So the way I did it was by spinning everything around opengl..I setup GLUT - as follows :
/* register call back functions */
glutDisplayFunc(redraw);
glutReshapeFunc(resize);
glutIdleFunc(visualize);
glutKeyboardFunc(getkey);
glutSpecialFunc(getspecialkey);

And in the idle function visualize(). I currently do the joystick reading and rs232 packet sending and recv stuff. I also call redraw after I am done so if I have received new data above, then it gets drawn on the screen. Everything works fine except that calling redraw() from visualize() is logical in a PC simulation where the simulation state needs to be rendered on screen after a simulation step. But in my case it takes up too much time, much greater than 20 ms. So I can no longer guarantee I can send out packets through the rs232 port every 20 ms. What are my options here if I want to send out packets and visualize data at a reasonable frame rate - packets within 20 ms is more imp of course.