Ubuntu Security :: Ssh Keys Can't Be Found

Scenario 1. I am doing this from /home/deploy directory I am trying to set up ssh with github for capistrano deployment. this has been an absolute nightmare. when I do ssh git@github.com as the deploy account I get Permission denied (publickey). so may be the key is not being found, so If I do a ssh-add /home/deploy/.ssh/id_rsa Could not open a connection to your authentication agent. (i did verify that the ssh-agent was running) If I do exec ssh-agent bash and then repeat the ssh-add then the key does get added and I can ssh into github. Now I exit from the ssh connection to my server and ssh back in and I can't ssh into github anymore! Scenario 2 if I login to my remote server and then cd into my .ssh directory and ssh into github then it all works fine I guess there is a problem with locating the key and for some reason the agent isn't funcitoning correctly.