SI
SI
discoversearch

We've detected that you're using an ad content blocking browser plug-in or feature. Ads provide a critical source of revenue to the continued operation of Silicon Investor.  We ask that you disable ad blocking while on Silicon Investor in the best interests of our community.  If you are not using an ad blocker but are still receiving this message, make sure your browser's tracking protection is set to the 'standard' level.
Strategies & Market Trends : ajtj's Post-Lobotomy Market Charts and Thoughts
 Public ReplyPrvt ReplyMark as Last ReadFilePrevious 10Next 10PreviousNext  
To: Sun Tzu who wrote (94509)8/27/2025 10:41:33 AM
From: Sun Tzu1 Recommendation

Recommended By
ajtj99

  Read Replies (1) of 97576
 
Since we discussed nVidia, DeepSeek, and models in the same breadth, the post below in my feed seems appropriate:

It's actually functionally close to something that I have been working on as a side project.

It is interesting that the new DeepSeek AI v3.1 is trained using the UE8M0 FP8 scale data format, which is nothing but the logarithmic number system (LNS), meaning it has only exponent and no mantissa. Our multiplicative weights update (Madam) for training in that format was done several years ago while at NVIDIA It yields maximum hardware efficiency with no accuracy loss lnkd.in

Logarithmic number system achieves a higher computational efficiency by transforming expensive multiplication operations in the network layers to inexpensive additions in their logarithmic representations. In addition, it attains a wide dynamic range and can provide a good approximation. Also, logarithmic number system is biologically inspired, and there is evidence that our brains use such a format for storage.

However, using standard SGD or Adam optimization for training in logarithmic format is challenging, and requires intermediate updates and optimization states to be stored in full precision (FP32). To overcome this, we proposed Multiple Weights update (Madam) that instead updates directly in the logarithmic format and leads to good training outcomes.

Our LNS-Madam when compared to training in FP32 and FP8 formats, LNS-Madam reduces the energy consumption by over 90% and 55%, respectively, while maintaining accuracy.
Report TOU ViolationShare This Post
 Public ReplyPrvt ReplyMark as Last ReadFilePrevious 10Next 10PreviousNext  

HomeMailHotSubjectMarksPeopleMarksKeepersSettings
Terms Of UseContact UsCopyright/IP PolicyPrivacy PolicyAbout UsFAQAdvertise on SI
© 2025 Knight Sac Media.  Data provided by Twelve Data, Alpha Vantage, and CityFALCON News