Thank you for posting. It seems that you two have been together for many years. Do you take medication for your bipolar disorder?
I wish we had the answer to your relationship problems. However, relationships are to be worked on by two people-- not just one side. So, if you are having issues in the bedroom and this is a major problem in your relationship then this needs to be addressed to him. If this doesn't change are you ready to end your relationship with your boyfriend? would you stray again to find a man who will sleep with you?
It's important to understand that he is also older now and may simply want your company as opposed to wanting to have sex all the time. It could be hormone related, health related, a compilation of the behavior that turns him on accompanied by stress, etc. There is definitely more than one factor involved. Perhaps he would be more willing to talk about this with a counselor or therapist?
One thing I do feel the need to point out is that the straying and the casual sex need to stop if you expect this man to be with you forever and if you really do love him. You work on what you have instead of going out of your relationship for meaningless sex with men that have no intention in loving you the way this man does.
Wishing you the best,