gPodder only sends a message to your audio player to play the file and doesn't attempt to track playback position itself. However if you use the Panucci player this does support 'bookmarking' the audio playback and you can configure gPodder to open your podcasts with that player instead of the default - as long as you have Panucci installed that is.

If Panucci is installed you can set it as the player to use in gPodder's settings.