I don't think it's a bug in that case. The double-press activates caps-lock, not shift-lock, so the punctuation keys remain unshifted. It makes less sense happening with the Fn lock though, so I'd call it a bug there.