Consistent mapping from special double values to integers

thomas_braun
Posts: 586
Joined: 2009-10-07
Location: Germany

The guy sitting in front of my computer introduced a bug due to passing a NaN where an integer should have been.
And that NaN mapped to a -1 which was kind of hard to track down.

It would be nice to have a fixed way of converting from special double values to integers.
Currently that seems a bit unspecified.

Function DoStuff()
 
	int a
	int64 b
	uint64 c
 
	print "Mapping NaN to int"
 
	a = NaN
	b = NaN
	c = NaN
 
	printf "%x, ,%x, %x\r", a, b, c
 
	Make/FREE/B a1 = {NaN}
	Make/FREE/B/U a2 = {NaN}
	Make/FREE/W b1 = {NaN}
	Make/FREE/W/U b2 = {NaN}
	Make/FREE/I c1 = {NaN}
	Make/FREE/I/U c2 = {NaN}
	Make/FREE/L d1 = {NaN}
	Make/FREE/L/U d2 = {NaN}
 
	print/D a1[0], a2[0], b1[0], b2[0], c1[0], c2[0], d1[0], d2[0]
 
	print "Mapping inf to int"
 
	a = Inf
	b = Inf
	c = Inf
 
	printf "%x, ,%x, %x\r", a, b, c
 
	Make/FREE/B a1 = {Inf}
	Make/FREE/B/U a2 = {Inf}
	Make/FREE/W b1 = {Inf}
	Make/FREE/W/U b2 = {Inf}
	Make/FREE/I c1 = {Inf}
	Make/FREE/I/U c2 = {Inf}
	Make/FREE/L d1 = {Inf}
	Make/FREE/L/U d2 = {Inf}
 
	print/D a1[0], a2[0], b1[0], b2[0], c1[0], c2[0], d1[0], d2[0]
 
	print "Mapping -inf to int"
 
	a = -inf
	b = -inf
	c = -inf
 
	printf "%x, ,%x, %x\r", a, b, c
 
	Make/FREE/B a1 = {-inf}
	Make/FREE/B/U a2 = {-inf}
	Make/FREE/W b1 = {-inf}
	Make/FREE/W/U b2 = {-inf}
	Make/FREE/I c1 = {-inf}
	Make/FREE/I/U c2 = {-inf}
	Make/FREE/L d1 = {-inf}
	Make/FREE/L/U d2 = {-inf}
 
	print/D a1[0], a2[0], b1[0], b2[0], c1[0], c2[0], d1[0], d2[0]
End


•dostuff()
Mapping NaN to int
8000000000000000, ,8000000000000000, 8000000000000000
-1 255 -1 65535 -1 4294967295 -9223372036854775808 -9223372036854775808
Mapping inf to int
8000000000000000, ,8000000000000000, 8000000000000000
-1 255 -1 65535 -1 4294967295 -9223372036854775808 -9223372036854775808
Mapping -inf to int
8000000000000000, ,8000000000000000, 8000000000000000
0 0 0 0 0 0 -9223372036854775808 -9223372036854775808

IMHO the best solution would be to throw a RTE with rtFunctionErrors=1 but I would expect that this is too costly performance-wise.


Back to top