Xorg most probably does not leak by itself, but applications leak into Xorg.
If applications use server-side pixmaps to store graphics, they appear in Xorg's memory statistics, instead of the application's.
So it's not Xorg to kill, but the misbehaving application.