-
Notifications
You must be signed in to change notification settings - Fork 238
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix #878 #907
Fix #878 #907
Conversation
Do you know how the expected value of 21.33 was produced back then? Which data type and format specifier was used? What is the exact value that is formatted, and is is read somewhere or calculated? If we would change the F17 format specifier to F14 then this specific case would again be formatted as 21.33. Maybe even less precision is enough? How much precision does the PDF standard even support? |
If I read this correctly then even F5 would be enough. |
Yes you are right, that's the theoretical precision. I'm going for 9 (Microsoft recommends F7 for single precision. I still have an issue with the change as the behaviour is not the same between Framework (some tests are failing) and dotnet8 (all tests are good) |
Yeah, there seems to be a difference in number formatting between .NET Framework and .NET 8: But this only affects numbers with more than 15 digits. Is this actually a problem? How big would a PDF page need to be so that this could be a real coordinate/size? edit: I found this about the problem: https://stackoverflow.com/a/1658420/631802 Btw, the |
@cremor thanks for the help here, now updated with your feedback |
This is in order to fix #878
Unfortunately some tests are now failing because we are now to able to assess to proper precision to use. We either can fix the test to make it pass, or we need to review the logic
Leaving it here as a draft for the moment. @cremor feel free to have a look