First of all, all 3D printing is composed of layers (either of deposited or sintered material, for the most part) so smooth layer transitions will always be a matter of perspective. - The cheapest and probably one of the easier ways to go about this is to get the part printed by a high quality FDM machine, that can do at least 100 micron layers and is well calibrated; and then going ahead and sanding the finished part (or using something like XTC-3D to smooth out the layers… and then sanding it down, more than likely.)
As far as materials go, on the FDM (FFF) side you have a ton of options, including expensive metal-filled and carbonfiber filled materials, but for a belt buckle, especially a proof of concept model, I doubt you will need that. PLA might be a bit on the fragile side, as it doesn’t like to bend, but well printed PETG is really strong stuff, and of course there is the classic option of ABS (which does give off noxious fumes, so hopefully people won’t be printing it in the house.)
Another option would be to go the route of SLA (DLP) which is laser cured resin. This will give you an initially smoother piece, though the resin printing process needs a lot more clean-up, post-processing and treatment to give you a finished product. SLA printers are also not as common, the material and running costs are higher… In short, this method is more expensive and probably won’t give you any different a result in something like a belt buckle than FDM printing would.
