If you sacrifice your relationship with your own children to be with her then you will end up resenting her and I can't imagine a relationship working under that kind of strain.
How old are the children? How often could you see them if you took advantage of every opportunity? How often do you see them?
What if your W never changes, even a little bit? You have to work under that assumption. Will you never then see your own children much? How will you keep your kids from growing up with the belief that they weren't important enough for you...that you chose another wife over them?
It's well documented what can happen to girls growing up with an absent father. What can be done to avoid this?