I'd ban the taxis, then it's only buses. Then just remove the crossing, there'll be so little traffic (presumably) that people will let the bus go past when necessary.
But I agree in general, especially when there's an urban dual carriageway to cross, which requires waiting twice.