I have removed my plastic levers. I had to make a custom tool.
After doing it once - I decided to avoid doing it again.
First, I removed the guage console from the car so that I could get at the levers from the backside. I could not remove the lever box (attached on the back of the guage console) without removing the levers. There is the conundrum. DO NOT Pry the levers off. They will surely crack or break (depending on how old the plastic is).
If memory serves, the lever just slides and then snaps into place on the metal blade. You need to slide a thin piece of metal, from the backside, between the plastic and the metal blade, to unspring the clip that holds the lever. Have a good look and make yourself the custom tool (with the thin blade) you will need.
Once you get the levers off, you can service the lever box and associated bits.
If you are trying to replace a broken lever and you have a replacement lever, just go ahead and remove the broken plastic bits and anything plastic left over. This can be done without fully removing the guage panel or lever unit. Try not to damage the plastic lever bezel. You should not damage the mounting blade. Then, as you say, slide on the replacement lever until it snaps into place.
HTH
B.G.