Skip to content

Issue with example in #11573 #11578

@sethvs

Description

@sethvs

Prerequisites

  • Existing Issue: Search the existing issues for this repository. If there is an issue that fits your needs do not file a new one. Subscribe, react, or comment on that issue instead.
  • Descriptive Title: Write the title for this issue as a short synopsis. If possible, provide context. For example, "Typo in Get-Foo cmdlet" instead of "Typo."
  • Verify Version: If there is a mismatch between documentation and the behavior on your system, ensure that the version you are using is the same as the documentation. Check this box if they match or the issue you are reporting is not version specific.

Links

#11573

Summary

I believe that this example
https://github.com/sdwheeler/PowerShell-Docs/blob/b00ec932fc27ee5d1f6f512d19a3c60a46f537d8/reference/7.5/Microsoft.PowerShell.Core/About/about_Type_Conversion.md?plain=1#L403-L418
is not fully correct.

Result of the [Int64]::MaxValue + 1L is of type [Double].

PS C:\> ([Int64]::MaxValue + 1L).GetType().FullName
System.Double

So, when we convert it to [Decimal] it is rounded to 15 significant digits.

As in the Remarks section of the Convert.ToDecimal Method:
The Decimal value returned by this method contains a maximum of 15 significant digits. If the value parameter contains more than 15 significant digits, it is rounded using rounding to nearest. The following example illustrates how the Convert.ToDecimal(Double) method uses rounding to nearest to return a Decimal value with 15 significant digits.

So, either we add L to 1 or not, the result will be the same:

PS C:\> ([Int64]::MaxValue + 1L).GetType().FullName
System.Double

PS C:\> [decimal]([Int64]::MaxValue + 1L)
9223372036854780000
PS C:\> ([Int64]::MaxValue + 1).GetType().FullName
System.Double

PS C:\> [decimal]([Int64]::MaxValue + 1)
9223372036854780000

On the other hand, if we leave the result as [Double], adding L to 1 will also not change the result:

PS C:\> '{0:N0}' -f ([Int64]::MaxValue + 1L)
9,223,372,036,854,775,808

PS C:\> '{0:N0}' -f ([Int64]::MaxValue + 1)
9,223,372,036,854,775,808

Details

No response

Suggested Fix

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions