Two comments:
- According to what I saw in the profile, it seems you are forcing the frame rate to 25. Why not leave it as source frame rate?
- The target bitrate for video (512 kbps) seems to me a little bit low. I would target 784 kbps.

I will make some tests with various content I have. I very interested to see how you manage video size / aspect ratio transformation.