6

I'm computing sin of pi radians in emacs using this function:

#+begin_src emacs-lisp :eval never-export :exports both
(sin pi)
#+end_src

#+RESULTS:
: 1.2246467991473532e-16

The manual docs mentions that the argument is passed in radians. Just to be sure, I confirmed that sin pi radians is infact zero: https://en.wikipedia.org/wiki/Sine#Special_values

But it seems to be working fine for cosine functions:

#+begin_src emacs-lisp :eval never-export :exports both
(cos pi)
#+end_src

#+RESULTS:
: -1.0

I guess I'm missing something here. Can somebody point what I'm doing wrong ?

Drew
  • 75,699
  • 9
  • 109
  • 225
Sibi
  • 3,603
  • 2
  • 22
  • 35
  • 7
    "Floating point numbers are like piles of sand; every time you move one you lose a little sand and pick up a little dirt." Kernighan & Plauger, The Elements of Programming Style, p.117, attribute it to a "wise old programmer". – NickD Jan 21 '21 at 16:14
  • 3
    What you're missing is having read [What Every Programmer Should Know About Floating-Point Arithmetic](https://floating-point-gui.de). – Gilles 'SO- stop being evil' Jan 22 '21 at 13:56
  • Might come in handy: https://xkcd.com/2295/ – Arch Stanton Jan 22 '21 at 14:26
  • https://stackoverflow.com/questions/588004/is-floating-point-math-broken – Barmar Jan 22 '21 at 14:29
  • 2
    While these references about floating-point may come in handy, they don't actually answer the specific questions about why `(sin pi)` and `(cos pi)` gave the values they did (both are very good approximations to sin(`pi`) and cos(`pi`), in fact). If you just throw up your hands and say FP is noisy and stop there (like floating-point-gui.de) you'll fall into traps like continuing to evaluate sin at approximate points near , where the _function itself_ will magnify errors irrespective of how you implement it, with or without FP. – Sūrya Siddhānta Jan 22 '21 at 15:21
  • Try to compute (cos pi)+1, you won't get 0, but rather a numerical approximation, similar to what you got for (sin pi) – quarague Jan 23 '21 at 09:45
  • @quarague Did you try that? `(+ 1 (cos pi))` returns zero, because although `pi` is not exactly , cos(`pi`) is close enough to −1 that `(cos pi)` is rounded to −1. Emacs does abbreviate some floating-point numbers to shorter unambiguous decimal expansions (like `3.141592653589793` as an unambiguous abbreviation for the nearest floating-point number 3.141592653589793115997963468544185161590576171875 in IEEE 754 binary64), but −1 is not among them: when Emacs says `(cos pi)` gives `-1.0`, that really is −1, because −1 already is a floating-point number, unlike 3.141592653589793. – Sūrya Siddhānta Jan 23 '21 at 14:40
  • @SūryaSiddhānta I didn't try it. I don't know how Emacs computes internally. If it rounds the result after computing cos pi it will return 0. I would have thought that it evaluates cos(pi) to something like -1+1.22e-16 or something similar and just displays that as -1. If you then add 1 you get to see the numerical inaccuracy term. – quarague Jan 23 '21 at 15:51
  • The two floating-point numbers nearest to cos(`pi`) are −1, which differs from the true value by less than 10⁻³² parts of the true value, and −0.99999999999999988897769753748434595763683319091797, which differs by more than 10⁻¹⁶ (and, specifically, more than 2⁻⁵³). For cos, most math libraries will give a bit worse than 0.5ulp error (which would be correctly rounded like crlibm) but much better than 1ulp error, so in cases like this they will give the much better result, −1. You have to move quite a ways away from , say to `3.1415926430530807`, before `cos` gives anything but −1! – Sūrya Siddhānta Jan 23 '21 at 16:19
  • 1
    This isn't just an accident of high-quality math library implementation, though: it's also because, unlike the sine function, the cosine function is _well-conditioned_ near : even if your inputs are _badly approximated_ near , so you ask for cos(⋅(1 + )) when you want cos(), what you will get is cos()⋅(1 + ) where is a relative error _much smaller_ than the relative error in the input. (The condition number of cos approaches zero at .) – Sūrya Siddhānta Jan 23 '21 at 16:22
  • Welcome to floating-point rounding errors. Please check your expectations at the door. – Bob Jarvis - Слава Україні Jan 23 '21 at 19:22
  • 1
    @BobJarvis-ReinstateMonica Actually, floating-point rounding errors are only a small part of what's going on here. _Any_ system of numbers and arithmetic that can represent only approximately will yield similar results, even if sine and cosine are evaluated exactly with no rounding errors! – Sūrya Siddhānta Jan 23 '21 at 20:17

3 Answers3

20

The Emacs Lisp constant pi is not actually the number . Rather, the Emacs Lisp constant pi is an approximation to , good to about 16 digits -- pi is exactly 3.141592653589793115997963468544185161590576171875, which is the IEEE 754 binary64 floating-point number closest to . This is also the floating-point number closest to 3.141592653589793, which is why, when you evaluate pi, Emacs will show the much shorter 3.141592653589793 as an unambiguous abbreviation for it.

So when you pass pi to the sin function, you are not asking it to compute sin(), which is zero. Instead, you are asking it to compute sin(3.141592653589793115997963468544185161590576171875), which is slightly above zero because 3.141592653589793115997963468544185161590576171875 is slightly below .

The Emacs Lisp function sin may also return some additional error, but the error is very small: the value of Emacs Lisp (sin pi) differs from the true value of sin(3.141592653589793115997963468544185161590576171875) by less than .000000000000002% (that is, 2 × 10⁻¹⁷) of the magnitude of the true value. In particular, the true value of sin(3.141592653589793115997963468544185161590576171875) is irrational and so doesn't have a finite decimal expansion, but it is roughly

1.2246467991473531772260659322749979970830539012998 × 10⁻¹⁶,

whereas Emacs Lisp returns

1.22464679914735320717376402945839660462569212467758006379625612680683843791484832763671875 × 10⁻¹⁶,

which is wrong only after the sixteenth digit. (Again, this is the nearest floating-point number to 1.2246467991473532 × 10⁻¹⁶, which is why Emacs Lisp prints it unambiguously and more concisely as 1.2246467991473532e-16.)

This number returned by (sin pi) is not simply some random error! It turns out that it is actually a good approximation to the error in Emacs Lisp's approximation of ! This happens because when is near , sin() = ( − ) + (( − )³); this is the Taylor series of sine about . So if pi = − with absolute error , then (sin pi) computes an approximation to sin( − ) ≈ − ( − ) = , good to about 16 digits.

If you add the value of the Emacs Lisp constant pi ( − = 3.141592653589793115997963468544185161590576171875) and the value of (sin pi) (1.22464679914735320717376402945839660462569212467758006379625612680683843791484832763671875 × 10⁻¹⁶), using exact arbitrary-precision arithmetic, what you get is

3.141592653589793238462643383279505878966979117714660462569212467758006379625612680683843791484832763671875,

in which, if you are a Standard Nerd, you will start to recognize all the digits up to the 32nd or so, since pi + sin(pi) ≈ pi + = − + = . The familiarity ends there (or earlier if you did not spend an inordinate time as a Standard Nerd Child memorizing digits of ) because pi and (sin pi) each only have about 16 digits of precision, and so together they have only 32 digits of precision.


You should be aware, though, that the mathematical sine function is said to be ill-conditioned near (and near any odd multiple of ). What this means is that it magnifies errors in inputs near . Specifically:

  • Suppose you want to find the sine of some number near .
  • Suppose you don't actually know itself, but only an approximation ⋅(1 + ), with relative error .
    • For example, perhaps is a physical dimension, and you made a small error in measurement because you don't have an infinitely precise ruler or compass to measure with in the real world.
  • If you then compute sin(⋅(1 + )), you will get, at best, an approximation sin()⋅(1 + ) to sin(). The relative error of the result may be arbitrarily large even if the relative error of the input is very small, and even if the sine function is evaluated exactly.

The last part means that sine's being ill-conditioned near is a property of the function, not of the system of floating-point arithmetic. Even if you used some kind of infinite-precision transcendental arithmetic system with no rounding error whatsoever (arbitrary-precision rational arithmetic wouldn't cut it because sine is transcendental), the sine function itself could magnify your small input error near into a large output error.

Example: Suppose the true value of is the floating-point number nearest to 3.14159000000001, but the approximation you have owing to measurement error is the floating-point number nearest to 3.14159 -- this is in error by only a fraction ≈ 3 × 10⁻¹⁵ of the true value.

Then sin() ≈ 2.653589783138678 × 10⁻⁶ but your approximation to it will be sin(⋅(1 + )) ≈ 2.65358979335273 × 10⁻⁶ in which about half the digits are wrong.

So while you have storage for 16 digits of precision, and your input was good to nearly all of them, only 9 digits of the output are good and your output error in sin()⋅(1 + ) is ≈ 4 × 10⁻⁹, a million times worse than the input error, because of the question you asked (and not because you used floating-point arithmetic to answer it).

The lesson is that if you find yourself trying to answer the question of the sine of an input near , you should consider trying to rephrase the question another way -- for example, perhaps instead of measuring directly near , you can measure the much smaller = − , and then use the approximation sin() = + (³).

This phenomenon is, incidentally, related to why the historic trigonometric functions with crazy-sounding names like versine and exsecant exist -- it wasn't to torture students in school, but rather to allow navigational and astronomical questions to be phrased so they admit computational approximations without magnifying errors.


Why does it seem to work with cosine? The true value of cos(3.141592653589793115997963468544185161590576171875) is an irrational number: −0.9999999999999999999999999999999925012010866907120… Emacs Lisp returns a floating-point number (always a rational number), and the two nearest floating-point numbers are −1 and −0.99999999999999988897769753748434595763683319091796875.

Although Emacs Lisp doesn't always guarantee to return the nearest floating-point number to the true answer for transcendental functions like cosine, in this case it happens to do so, giving −1. (Usually when the nearest floating-point number is that much closer (relative error below 10⁻³²) than the next-nearest (relative error above 10⁻¹⁶), math libraries will return the nearest one.) So Emacs Lisp happens to return −1 as an approximation to cos(3.141592653589793115997963468544185161590576171875), even though the question wasn't to compute cos().

This isn't just an accident of high-quality math library implementation, though! It's also because, unlike the sine function, the cosine function is well-conditioned near : even if your inputs are badly approximated near , so you ask for cos(⋅(1 + )) when you want cos(), what you will get is cos()⋅(1 + ) where is a relative error much smaller than the relative error in the input.

Example: Suppose you have a much worse approximation to given by 3.14159264305308116860260270186699926853179931640625 (abbreviated 3.141592643053081), which is ⋅(1 + ) whose relative error from is more than 10⁻⁹.

If you ask for the cosine of this approximation when you really want cos(), it turns out that cos(⋅(1 + )) = cos(3.14159264305308116860260270186699926853179931640625) = −0.99999999999999994448884937843286910504136142018913… = cos()⋅(1 + ) whose relative error from the result you wanted (−1) is < 10⁻¹⁶.

That is, the relative error of the result is a million times better than the relative error of the input.

So cosine is much more tolerant to bad approximations of inputs near than sine is.

In more formal terms, the condition number of the function cos() is ⋅cos′()/cos() = −⋅tan() which approaches zero at (well-conditioned), whereas the condition number of the function sin() is ⋅sin′()/sin() = /tan() which approaches infinity at (ill-conditioned).

In general, you should avoid trying to ask about functions at approximations to points where they are ill-conditioned, and phrase your questions in terms of functions at approximations to points where they are well-conditioned -- the condition number is roughly the factor by which the function itself magnifies errors (before any additional rounding errors in the computation take effect).

6

I think this is simply the expected answer. As a float, 1.2246467991473532e-16 is not different from zero, given the floating point error Emacs Lisp can handle. One get the same answer in R for instance:

sin(pi)
#> [1] 1.224647e-16

and also in Python, as shown in this thread.

Actually, this is said in the Emacs Lisp manual:

Floating-point computations often involve rounding errors, as the numbers have a fixed amount of precision.

Most programming languages, other than computer algebra systems (e.g., Maxima or Maple) will give such approximate answers.

Philopolis
  • 1,094
  • 8
  • 14
  • 3
    Emacs' `calc` returns 0 for `sin(pi)` (if you've set `calc-radians-mode` with `m r`). [`(info "(Calc) Trigonometric and Hyperbolic Functions")`](https://www.gnu.org/software/emacs/manual/html_node/calc/Trigonometric-and-Hyperbolic-Functions.html) – npostavs Jan 22 '21 at 16:10
5

You are doing nothing wrong. This function is working with floating point numbers which have good, but limited, accuracy. 1.2246467991473532e-16 is pretty close to 0 (0.0... with 15 zeros before the 1224... kicks in).

Fran Burstall
  • 3,665
  • 10
  • 18