Good question.
Definitely choose a radio that accepts an optional AA battery tray if you have that option!
Eg for this current model the Amazon.ca blurb says
Standard Horizon HX380 5W Commercial Grade Submersible IPX-7 Handheld VHF Radio w/LMR Channels Marine
Optional FBA-40 Alkaline Battery tray available
You’re better to buy both at the same time from someplace like Steveston Marine because I don’t think Amazon bothers to sell the bits and pieces but I could be wrong. also this might be available for some models but not others. This stuff usually is discounted at Boat Show time in Feb.
The Li-ion batteries are however getting more compact all the time and maybe some of the models can no longer offer bulky AA packs which need to generate around 7 or 8 V to work so that means 6 AA batteries – a bigger pack than the Li-ion.
Radios are notorious for working fine until you actually need them so it is not redundant to carry a backup power pack.
In addition, I would personally recommend a PLB for more on-water security.