Hey CJ, check this out ...
This is interesting, but I also think it's another instance of Musk believing he's smarter than everyone else when it comes to topics that he doesn't specialize in.
For example, he posted the following:
Like what does that mean, "Floating point is not real?" Of course it's real, and no, it's not the result of emulation with "2 integers." Instead, it's the result of the IEEE 754 standard.
As for Tesla focusing more on integer (i.e. fixed-point) math over floating-point, I'm skeptical. Yes, CPUs can process fixed-point math more efficiently than floating-point, but what's the cost to the AI inference model when all the data points are represented by fixed-point numbers instead of floating-point?
I actually don't know that much about AI training and AI inference. But the mathematic operations, I believe, are similar to that of 3D graphics, and I know a thing or two about that. The whole "integer vs. floating point" thing was already resolved back in the late 90's when the 3D video game Quake was getting popular. Floating point is better, period.
It was also the topic of debate between the Intel Pentium, with its vastly superior FPU, and the AMD K5. AMD had to beef up its FPU with the K6 in order to compete (and edge out) the Pentium II.
Back to the Muskperger. This guy may be intelligent, but part of intelligence involves knowing what areas you're not that smart person in.
And I guarantee anyone that Musk doesn't know WTF he's talking about WRT integer vs. floating-point processing. Either that, or he's collectively smarter than every single engineer in nVidia and AMD's GPU departments, which is a load of crap.
Tenchusatsu |
