About 20 years ago I saw construction plans for an analogue telephoner scrambler in the local radio-amateur magazine. It worked on the principle of frequency spectrum reversal - low frequencies became high and vice-versa. Primitive and highly ineffective against government snooping but good enough against your kids/parents/spouse/great aunt eavesdropping on the parallel line. The voice would become an incomprehensible screeching noise that, even for those knowing how it was generated, was impossible to understand without a descrambler. Yet it stayed in the same frequency spectrum as the original voice and as such was transmitted over a normal telephone line without any additional equipment. Another advantage was that the scrambling was symmetric, i.e. the scrambler and the descrambler were the same thing.

I could not find the plans in a 2 minute duckduckgo (Google for the clueless ones) session but I found at least two dozen commercial products, some of them looking like they are based on the same design.

I imagine something like that should be easy to make a pulseaudio plugin for. The other party could then use a similar plugin or a simple black box.