Decimal precision lost in assertion messages

[mikepotjer]

I have part of a test that checks if the value coming from a Currency field in a cursor is being rounded to 2 decimal places. Because of the default setting of SET DECIMALS TO, and the way TRANSFORM() formats a currency value, the failure message from FoxUnit looks like this:

-------------------------------
------Assertion Failure
-------------------------------
Pay Rate 00004 calculated wages are incorrect
Values Not Equal
Expected Value: $250.00
Actual Value: $250.00
-------------------------------
------Assertion Failure
-------------------------------
Pay Rate 00005 calculated wages are incorrect
Values Not Equal
Expected Value: $250.00
Actual Value: $250.00

I don't want to take the time to figure out where to plug in this code right now, but I did come up with a replacement function for TRANSFORM() which preserves decimal precision on all numbers, though you do lose the currency formatting. Here's the code that does that:

LPARAMETERS tuValue

LOCAL lcSetFixed, ;
  lcString, ;
  lnSetDecimals

DO CASE
  CASE VARTYPE( m.tuValue ) = "C"
    lcString = m.tuValue

  CASE INLIST( VARTYPE( m.tuValue ), "N", "Y" )
    * Force all relevant decimal places to be displayed.
    lcSetFixed = SET("Fixed")
    SET FIXED OFF
    lnSetDecimals = SET("Decimals")
    SET DECIMALS TO 18

    lcString = TRANSFORM( IIF( VARTYPE( m.tuValue ) = "Y", MTON( m.tuValue ), m.tuValue ) )

    SET FIXED &lcSetFixed.
    SET DECIMALS TO m.lnSetDecimals

  CASE ISNULL( m.tuValue )
    * In case SET NULLDISPLAY is something other than the default.
    lcString = ".NULL."

  OTHERWISE
    lcString = TRANSFORM( m.tuValue )
ENDCASE

RETURN m.lcString

[eselje]

Ok, if I'm understanding correctly we need to use your replacement function when we're outputting the Expected and Actual values to show more precision. Is that correct? Eric

…

On Mon, Feb 18, 2019 at 2:20 PM Mike Potjer ***@***.***> wrote: I have part of a test that checks if the value coming from a Currency field in a cursor is being rounded to 2 decimal places. Because of the default setting of SET DECIMALS TO, and the way TRANSFORM() formats a currency value, the failure message from FoxUnit looks like this: ------------------------------- ------Assertion Failure ------------------------------- Pay Rate 00004 calculated wages are incorrect Values Not Equal Expected Value: $250.00 Actual Value: $250.00 ------------------------------- ------Assertion Failure ------------------------------- Pay Rate 00005 calculated wages are incorrect Values Not Equal Expected Value: $250.00 Actual Value: $250.00 I don't want to take the time to figure out where to plug in this code right now, but I did come up with a replacement function for TRANSFORM() which preserves decimal precision on all numbers, though you do lose the currency formatting. Here's the code that does that: LPARAMETERS tuValue LOCAL lcSetFixed, ; lcString, ; lnSetDecimals DO CASE CASE VARTYPE( m.tuValue ) = "C" lcString = m.tuValue CASE INLIST( VARTYPE( m.tuValue ), "N", "Y" ) * Force all relevant decimal places to be displayed. lcSetFixed = SET("Fixed") SET FIXED OFF lnSetDecimals = SET("Decimals") SET DECIMALS TO 18 lcString = TRANSFORM( IIF( VARTYPE( m.tuValue ) = "Y", MTON( m.tuValue ), m.tuValue ) ) SET FIXED &lcSetFixed. SET DECIMALS TO m.lnSetDecimals CASE ISNULL( m.tuValue ) * In case SET NULLDISPLAY is something other than the default. lcString = ".NULL." OTHERWISE lcString = TRANSFORM( m.tuValue )ENDCASE RETURN m.lcString — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#20>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHy8TGi4SV4bwds8sOESH4sqL4Q3wg-Gks5vOwsrgaJpZM4bBjdp> .

[mikepotjer]

Yes (or something equivalent). Using my replacement, the message from my initial post should look like the following:

-------------------------------
------Assertion Failure
-------------------------------
Pay Rate 00004 calculated wages are incorrect
Values Not Equal
Expected Value: 250
Actual Value: 250.0049
-------------------------------
------Assertion Failure
-------------------------------
Pay Rate 00005 calculated wages are incorrect
Values Not Equal
Expected Value: 250
Actual Value: 249.995

I wrote a little .PRG to demonstrate the problem and show the solution. If you would like that, I can send it to you later. I forgot to push it to a repo, so I have to wait until I get home.