I know it's an old thread, but I found a simple way to pull the plastic inner pin out, without harming it; leaving both pieces usable for later reattachment. I've done this three times and am still using the original fasteners.
Screw a drywall screw about half an inch into the plastic center pin. Goes in real easy, with the fake Phillips head providing a good start. You might need to hold the pin from spinning by applying pressure with a finger to the plastic pin while you screw the drywall screw in.
Once screwed in ... you guessed it ... grab the drywall screw head with a pair of pliers or vice-grips and give a tug. The center should pull out leaving the outer plastic flange-piece to be pushed out from behind, or pried out with a suitable flat-blade screw driver.
Hope this helps someone out there...
Doxx