but even as kids they knew pain in ways that I feel James (with his charmed life) and Peter did not.

Exactly. I used this same argument with my friend who kept bugging me about Sirius/James. I think it is because James and Sirius are so much alike that they won't be good together like that.

Plus, Sirius and Remus finish each other's sentences even in the afterlife. How much more married can anyone get? XD