Skip to content

N does not respect precision values below 17 #1160

Open
@alldaygooning

Description

@alldaygooning

Description.
When not-evaluated number (e.g. Pi or 1/7) is passed toN(numeric, precision) as numeric argument, function does not evaluate it to specified precision, if precision is less than 17.

1. Example. Precision = 15

ExprEvaluator evaluator = new ExprEvaluator();
IExpr result = evaluator.eval("N(1/7, 15)");
System.out.println(result); 
System.out.println(result.fullFormString());

Output:

0.142857
0.14285714285714285`

Expected output:

0.142857142857143
0.142857142857143`

2. Example. Precision = 3

ExprEvaluator evaluator = new ExprEvaluator();
IExpr result = evaluator.eval("N(1/7, 3)");
System.out.println(result); 
System.out.println(result.fullFormString());

Output:

0.142857
0.14285714285714285`

Expected output:

0.143
0.143

3. Example. Precision = 17

ExprEvaluator evaluator = new ExprEvaluator();
IExpr result = evaluator.eval("N(1/7, 17)");
System.out.println(result); 
System.out.println(result.fullFormString());

Output:

1.4285714285714285*10^-1
1.4285714285714285`17*^-1

Expected output:

1.4285714285714285
1.4285714285714285 (basically, exactly what was expected).

I believe this is a bug, since matheclipse.org and NCalc+ both produce Expected outputs on the same inputs:

Image
Image

Summary
precision of less than 17 in N() evaluates number to actual precision of 16 (as seemingly evident by .fullFormString() and even more confusingly, toString() representation has precision of 5 or sometimes 6.

Prerequisites

  • Java 17
  • Matheclipse 3.1.0-SNAPSHOT

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions