You can pick up used GPS devices on
ebay for under £50 if you keep your eyes open. There are thousands of pre-planned routes available online which you could use on such a device.
As for not getting lost when randomly exploring... Most devices will let you record where you have been and then backtrack the way you came. The only time that would catch you out is if you went down any one-way streets on the outward leg, in which case you would have to find another way round that on the return leg (or walk the bike through it against the flow of traffic?).
I agree that paper maps can be a faff out on the road, especially if it is wet and windy, but there is no reason not to consult digital maps in comfort at home before you set off. I didn't know my way round Yorkshire and Lancashire when I first moved here. In those days paper maps were all that were available to me so I worked with them and learned my way around , but nowadays I use
cycle.travel for ideas,
Memory Map to finalise new routes, and Streetview to check for unexpected problems.