Thread: [announce] ejpi rpn calculator
View Single Post
jkq's Avatar
jkq is offline jkq
2009-10-28 , 03:10
Posts: 251 | Thanked: 131 times | Joined on Oct 2009 @ USA
#7
Originally Posted by epage View Post
Calculations are done in floating point which can have a loss of percision, for example "e**(j * pi)+1" might not produce exactly 0
Have you seen the decimal module (http://docs.python.org/library/decimal.html)? It might be able to remove or reduce your precision "quirk".

-jkq