Can he fight me for the car?

I got a car loan my ex boyfriend is the co signer for the loan & his name is also on the title & he's starting to be a real (you know what) can he take me to court for the car or can he legally take the car from me even if it is more my car because he is the CO-signer & i'm the only one on the insurance?