Both my cards (mainstream Visa) require a PIN entry every now and again anyway. So contactless doesn't just go on forever amnd a stolen card will only allow a limited number of contactless before needing a PIN (at least for my cards).
I'm uncertain how phone contactless affects the periodic PIN as in effect phone uses a PIN (your face or fingerprint).
I’ve one card that currently requires me to put the card in and enter the PIN every 5 uses or so. Another has yet to ask me to do that in 12 months.
In reality that could mean, under current rules, a maximum loss of £400. Removal of that cap increases the risk too much for my liking.
Phone payments are different due to increased security and, certainly in the case of iOS devices, don’t even disclose your real card details.