How should I of responded to my boss yesterday when he felt I didn't do something fast enough for him?

You should have said "Well, if I were eating my lunch correctly I would not have time to do my job."