I'm thinking do I slip away quietly or do i make a scene of 'I'm sorry we cant be in contact anymore
Either of these can trigger fears of abandonment. Especially the latter. Actually, this can trigger fears of abandonment in you both. Have you heard of Controlled Contact? (Especially since you have a daughter with your ex)
This essentially puts you in control of any contact.

If he calls a lot you can say things such as "I have 5 (or 10 or 20, whatever you are comfortable with) minutes to talk. Then when you end the call you can say "I can talk again on Tuesday" or whatever day you chose. This draws out the time between calls, he knows you will be available at the time you say you will be (be available for the time you chose). This can ease both of you away from excess contact.
You may want to check out the co-parenting board also since you have a child together here:
Co-parenting after the SplitThere will likely be contact of some sort throughout her childhood.