The description of DeepSeek reminds me of my experience in networking in the late 80s - early 90s.

Back then a really big motivator for Asynchronous Transfer Mode (ATM) and fiber-to-the-home was the promise of video on demand, which was a huge market in comparison to the Internet of the day. Just about all the work in this area ignored the potential of advanced video coding algorithms, and assumed that broadcast TV-quality video would require about 50x more bandwidth than today’s SD Netflix videos, and 6x more than 4K.

What made video on the Internet possible wasn’t a faster Internet, although the 10-20x increase every decade certainly helped - it was smarter algorithms that used orders of magnitude less bandwidth. In the case of AI, GPUs keep getting faster, but it’s going to take a hell of a long time to achieve a 10x improvement in performance per cm^2 of silicon. Vastly improved training/inference algorithms may or may not be possible (DeepSeek seems to indicate the answer is “may”) but there’s no physical limit preventing them from being discovered, and the disruption when someone invents a new algorithm can be nearly immediate.


Keyboard Shortcuts

Key Action
o Source
e Edit
i Insight
r Random
s or / Search
www.joshbeckman.org/notes/844529234