Opened 9 years ago

Closed 9 years ago

#484 closed defect (fixed)

lamellarPC is precision limited

Reported by: pkienzle Owned by: richardh
Priority: critical Milestone: SasView 4.0.0
Component: SasView Keywords:
Cc: Work Package: SasModels Redesign

Description

Adapt the Lamellar PC calculation to use less precision limited math.

That means replacing ww = exp(…) with ww-1 = expm1(…), which can be calculated without loss of precision. Similarly, 1 - cos(x) for cos(x) near zero can be replaced with 2*sin2(x).

The alternative is to force lamellarPC to use double or quad precision even if the rest of the calculations are in single precision.

Use compare -longdouble to evaluate a model with quad precision.

Change History (4)

comment:1 Changed 9 years ago by butler

  • Priority changed from minor to critical

probably should be part of the core model conversion process. As I understand it this "works" so maybe only critical and not blocker for the release?

comment:2 Changed 9 years ago by ajj

  • Owner set to richardh
  • Status changed from new to assigned

comment:3 Changed 9 years ago by smk78

Is this ticket still valid now model has changed?

comment:4 Changed 9 years ago by ajj

  • Resolution set to fixed
  • Status changed from assigned to closed

Forcing double precision. Model is still a bit cranky, but can't be helped.

Note: See TracTickets for help on using tickets.