Silicon Investor (SI) -- The First Internet Community

STOCKTALK

We've detected that you're using an ad content blocking browser plug-in or feature. Ads provide a critical source of revenue to the continued operation of Silicon Investor. We ask that you disable ad blocking while on Silicon Investor in the best interests of our community. If you are not using an ad blocker but are still receiving this message, make sure your browser's tracking protection is set to the 'standard' level.

Pastimes : Computer Learning -- Ignore unavailable to you. Want to Upgrade?

To: Frank Sully who wrote (109402)	2/28/2022 12:46:36 AM
From: Frank Sully	Respond to of 110581

Introduction (Addendum) I realized in reviewing the Introduction material I posted that the “Coming Singularity” post I linked gave only the material from John Pitera’s original 2017 Introduction Header to the “Singularity, AI, etc.” message board which he moderated. But it omitted the material from John’s Introduction Header on NVIDIA’s GPU (“Graphics Processing Unit”) chips and how they would lead to a revolution in AI and NVIDIA’s worldwide dominance, due to their ability to do parallel processing, as well as how the future of AI would rely on ASIC (“Application-Specific Integrated Circuits”), specifically Google’s TPU (“Tensor Processing Unit”) chips. John was prescient enough to realize that NVIDIA’s markets, GPU chips (and CUDA AI software) would lead to AI NVIDIA’s dominance of the world’s AI chip and software markets, but he was wrong about ASIC chips (specifically, Google’s TPU chips) supplanting NVIDIA and the AI GPU chips. The reason for this is that not only has NVIDIA been revising its GPUs biannually with vastly more powerful chips but NVIDIA has also been supplying the AI software necessary to implement training of DNN AI models (NVIDIA has half of its engineers developing AI software). even Google uses NVIDIA AI GPU chips. Recently, Meta Platforms (formerly FaceBook) announced it is using NVIDIA A100 AI GPUs in building the world’s fastest AI supercomputer powered by 16,000 NVIDIA A100 AI GPUs. Message 33676960 Atos recently announced the availability of the most advanced AI supercomputer based on NVIDIA AI GPU chips. Atos built one of Europe’s first supercomputers to employ the NVIDIA Ampere architecture, the JUWELS Booster at the Jülich Supercomputing Center. It uses 3,744 NVIDIA A100 Tensor Core GPUs to deliver 2.5 exaflops of mixed-precision AI performance.To provide a deeper understanding of climate change, Atos and NVIDIA researchers will run AI models on the system, currently ranked No. 8 on the TOP500 list of the world’s fastest supercomputers. Julich researchers used the system in April to conduct a state-of-the-art quantum circuit simulation. Last year, Atos led deployment of BerzeLiUs, a system built on the NVIDIA DGX SuperPOD and Sweden’s largest supercomputer. The company also has delivered supercomputing infrastructure in Europe, India and South America based on NVIDIA DGX systems. Next up, Atos is building Leonardo, a supercomputer at the Italian inter-university consortium CINECA. It will pack 14,000 NVIDIA A100 GPUs on an NVIDIA Quantum InfiniBand network and is expected to become the world’s fastest AI supercomputer, capable of 10 exaflops of mixed-precision AI performance. With the first glimpse of the BullSequana XH3000, it’s clear there’s much more to come from the collaboration of Atos and NVIDIA. Note that an “exabyte” is one quintillion bytes, or one billion gigabytes or one million terabytes. From John Pitera’s Original Introduction Header ****************************************************************************************** I am thinking that this thread head shall under, numerous revisions as we gain clarity from the technology innovations which will continue to provide continuing greater insights as to how we get from here to there, and also as the key companies involved ... NVDA is my initial Dark Horse as a main player in this evolution** .... There are several companies that are at Ground Zero of the intersection of all of this .... and several are the usual suspects, the GOOGL's, FB, MSFT, AAPL, BIDU, BABA, AMD, a series of names.... one of the companies that is as the vortex of much of this is NVDA and it is the initial company that has maneuvered itself into High-end Gaming ... from their legacy business ... which feeds into Virtual reality, Data storage Centers and Cloud Computing companies that are all currently using NVIDIA GPU's in their Clouds... Autonomous driving where NVIDIA's GPU's with their visualization and triangulation capabilities are helping to rapidly advance this game-changing initiative. The Kuda Programming Language, the architecture that it lays upon and the ability of to engage in "Deep Learning” has opened up a potential competitive edge to NVIDIA the past year or so. (Note from Frank “Sully”: recall that this was written in 2017.) The key is to see why the GPU market separated from the CPU and turned into such a huge phenomenon. It is all about thread-counts and parallel processing. The CPU and the GPU are both processing units. At its most basic level though, the CPU is a master of serial complexity. It is optimized to have a small number of “threads” at any given time, with performance in these threads at an extremely high-level to take care of all the different programs, websites and other things you may have open on your machine. It is focused on low latency and is the central “brain”. The GPU on the other hand is optimized to perform a very large number of relatively basic steps in parallel. It is focused on high throughput. This is mostly vector shading and polygons - input a vertex to the graphics pipeline and then reassemble it into triangles. To avoid overstepping my own competence level, it is the difference between ability and rote memorization. GPUs in that sense are “teaching for the test” … General-Purpose GPU The sheer horsepower with which a GPU can do menial operational labor is astounding though! These are the trillionsof floating-point operations per second known as”teraflops” or "tflops". In graphics processing terms, the more tflops, the more polygons the GPU can draw all over the screen. But as the GPU moves beyond graphics to take on general-purpose computation, it has added significance. This is due to the nature of vector arithmetic; and it is the "arbitrary code" mentioned earlier. The applications of big data and artificial intelligence discussed above all rely on intense mathematical operations such as matrix multiplication, the Fourier transform, etc. That is the nature of deep-learning. The value-add in being able to construct computational frameworks though has all been at the software layer. These are frameworks such as Torch, Theano, & Caffe. But in order to use the frameworks, arbitrary data will need some form of standardization - hence an API - and the application interface layer in this case was CUDA. Many of these popular frameworks were based around code for Nvidia GPUs. This is how CUDA became the engine of deep learning. It brought niche frameworks over to the mainstream, and it is why Nvidia is leaning on it extremely hard. It is the same walled-garden type of approach that Intel (NASDAQ: INTC) takes on x86, Microsoft on Windows, Apple (NASDAQ: AAPL) with iOS, etc. It is a risky strategy though, and it is why I go so far as to say CUDA is the difference between a generational growth stock and a bubble. The Future is Decidedly ASIC ( and we will look at what GOOGL is building from the ground up) The problem is these applications are no longer niche. They are huge, and of enormous consequence to some of the largest technology companies in the world. The core numerical frameworks these companies are working with were built around CUDA, but most are open-source. ******************************************************************************************** To understand the advantages of GPU and TPU chips over CPU chips for DNN AI model training is discussed in the following ten and a half minute video. It’s all about parallel processing and matrix multiplication. For a discussion of the future of AI chips and NVIDIA’s role and the competition, see this ten minute video. Slainte! Frank “Sully”