All the tooling around them not keeping up anyway.
Actual “AGI” being distant and completely unrelated to contemporary models.
Inference costs plummeting.
The last one is critical.
Right this second, you can run Minimax H3 on a desktop for the tiny fraction of the compute/RAM OpenAI Sora took. And it’s better.
In a month, it will be ~8X faster.
You can run DeepseekV4 flash, dirt cheap, and get what Claude was less than a year ago. And it’s gonna spread to every host out there, to systems like Cerebas ASICs that don’t even need HBM.
So… Even if you’re an AI acolyte. And we go with that for the sake of argument…
We don’t actually need all that RAM for hosting generative models?
I’m very interested to see what happens to all these datacenters over the next two-three years.
They spent all this money on something that’s gonna be cheap as dirt to run, largely run locally, and that won’t need GPUs once bitnet takes off, sooo… they can’t make money off that.
What happens then?
What happens to all those Stargate RAM wafers, and excess datacenters?
You are 100% correct IMO for the AI stuff, just a question, what’s bitnet? Is it some supposed bitcoin-killer (because that’s not going to happen, IMO).
I also think we’re on the cusp of “having enough” personal computing power with the usual exceptions, take a bit, give a bit.
Our mobile phones are close to what’s needed for most people (8GB, 4k, gaming, …). Or so I think!
Bitnet is a catch-all title for ML models that use a specific mathematical trick.
If elements of a matrix are composed of only 1, 0, or -1, multiplying them is the same as as adding them.
That’s huge. LLM computation is basically all matrix multiplication, so if you replace that with simple addition, you reduce the computational requirements by orders of magnitude.
The catch is such models are hard to train effectively; its proven that it works, but research to get the technique usable and practical is still being done.
I mention it because, if it takes off, suddenly the massive matrix multiplier accelerators we have for LLMs aren’t as useful. Chips with simpler architectures could get the job done, at least for parts of models that are bitnet.
What happens to all those Stargate RAM wafers, and excess datacenters?
The bigger question is what happens to the world-economy when the largest players bottom out and the companies betting on them are massively downgraded because of their huge investments in a fantasy about eternal growth. Hope everyone is setting aside extra cash; it’s gonna make 2008 look like a walk in the park.
Traditionally during such busts the government starts outright confiscating bank accounts or otherwise enacts some fiscal policy that renders all savings (especially those in cash) completely worthless in a last bid to pass the buck onto the 99%.
You’re saying there’s precedent that indicates the gov’t will just “confiscate” peoples savings? Could you elaborate? I’m genuinely interested to know about this, because that’s… not good
Numerous individuals and companies were prosecuted related to Roosevelt’s Executive Order 6102. The prosecutions took place under the subsequent Executive Orders 6111,[15] 6260,[16] 6261[17] and the Gold Reserve Act of 1934.
There was a need to amend Executive Order 6102, as the one prosecution under the order was ruled invalid by Federal Judge John M. Woolsey on the grounds that the order was signed by the President, instead of the Secretary of the Treasury as required.[18] A New York attorney named Frederick Barber Campbell had a deposit at Chase National Bank of over 5,000 troy ounces (160 kg) of gold. When Campbell attempted to withdraw the gold, Chase refused and Campbell sued Chase. A federal prosecutor indicted Campbell the following day, September 27, 1933, for failing to surrender his gold.[19] Ultimately, the prosecution of Campbell failed but the authority of the federal government to seize gold was upheld, and Campbell’s gold was confiscated.
The case caused the Roosevelt administration to issue a new order under the signature of the Secretary of the Treasury, Henry Morgenthau Jr. Executive Orders 6260, and 6261 provided for the seizure of gold and the prosecution of gold hoarders. A few months later Congress passed the Gold Reserve Act of 1934, which gave legislative permanence to Roosevelt’s orders. A new set of Treasury regulations was issued providing civil penalties of confiscation of all gold and imposition of fines equal to double the value of the gold seized.
Back in 1933 the US was on the gold standard. If the government wanted to print the money they had to have the gold to back it up so that it would be convertible to gold on request. If the economy grows but the money supply doesn’t then the existing money becomes more valuable which is deflation.
Deflation sounds great - the money you already have can buy more! - but it can lead to a ton of economic problems. For example, if your money is becoming more valuable without you even doing anything, a temptation is to just do the equivalent of stuffing it under a mattress and to try to put off buying anything for as long as possible since the longer you can wait before buying anything, the more you can afford. That causes business to dry up because people are buying goods and services less frequently. So business revenues go down. Businesses with lost revenues have strong incentives to downsize by cutting their existing employment or reducing their wages. Those employees who are fired or with cut pay can’t spend as much which further reduces revenues. So that deflationary cycle keeps repeating and worsening the problem. In addition, consider loans and investment. Existing loans become more burdensome because they’re also worth more each day, so a lot of people will be defaulting because they didn’t expect the loan to be expensive. Those who lent the money can strike it rich if lucky or be ruined if unlucky so higher risk lending is disfavored in favor of low risk stuff like government bonds, similarly investments aren’t doing so great because so many companies are having worsening revenues so investing looks like setting money on fire and is less popular. No loans and no investment makes it extremely difficult to start or improve a business if you don’t already have money which means not many new startups are coming in to replace the folks downsizing or shutting down or come up with a new idea of doing things.
So a lot of government efforts were put at the time in trying to beat deflation. The one this guy fell afoul of was that Roosevelt had by executive order suspended the convertibility of the dollar to gold and demanded that people with amounts considered in excess to take that excess to the government and be compensated for it for a defined exchange to paper money. That would get the government enough gold to make enough new currency so that it could pay for the many ambitious projects and programs Roosevelt envisioned to get the massive number of unemployed people back to work and making money so that they would buy goods and services and break the deflationary cycle. This guy however did not exchange his gold and had 160 kg of gold in a deposit that he was trying to reclaim (then valued at ~$100000 which is what we would now consider millionaire tier money), so he ran afoul of that restriction against hoarding. But if he had converted the gold to dollars earlier they would not have confiscated it. There’s no particular reason for the government to go around confiscating our accounts because there’s no gold in them, even if it were the dollar is no longer tethered in any way to gold, and even if that were the case we aren’t experiencing a deflation cycle that the government wants to create more money to break itself out of.
Lol what? I just checked, on eBay I can find a lot of offers between 800 Euros and 1300 Euros for a used RTX 3090.
4-5k would get you a brand new RTX 5090.
Okay, not as bad as I thought, there’s a guy selling a 4090 for “only” 2k. There’s nobody selling a 3090 in the local hardware forum. I’m not going to buy off ebay because if someone on ebay sells a dud, I can’t go back to punch him in the face.
5090s start at about 5200, that’s true. To be fair, the “4-5k” for a used 3090 was a bit hyperbolic, I’m just mad that they’re so expensive that my last 2 cars cost less than a USED 4090.
Well, the current trajectory points to:
LLM capabilities topping out.
All the tooling around them not keeping up anyway.
Actual “AGI” being distant and completely unrelated to contemporary models.
Inference costs plummeting.
The last one is critical.
Right this second, you can run Minimax H3 on a desktop for the tiny fraction of the compute/RAM OpenAI Sora took. And it’s better.
In a month, it will be ~8X faster.
You can run DeepseekV4 flash, dirt cheap, and get what Claude was less than a year ago. And it’s gonna spread to every host out there, to systems like Cerebas ASICs that don’t even need HBM.
So… Even if you’re an AI acolyte. And we go with that for the sake of argument…
We don’t actually need all that RAM for hosting generative models?
I’m very interested to see what happens to all these datacenters over the next two-three years.
They spent all this money on something that’s gonna be cheap as dirt to run, largely run locally, and that won’t need GPUs once bitnet takes off, sooo… they can’t make money off that.
What happens then?
What happens to all those Stargate RAM wafers, and excess datacenters?
You are 100% correct IMO for the AI stuff, just a question, what’s bitnet? Is it some supposed bitcoin-killer (because that’s not going to happen, IMO).
I also think we’re on the cusp of “having enough” personal computing power with the usual exceptions, take a bit, give a bit.
Our mobile phones are close to what’s needed for most people (8GB, 4k, gaming, …). Or so I think!
Bitnet is a catch-all title for ML models that use a specific mathematical trick.
If elements of a matrix are composed of only 1, 0, or -1, multiplying them is the same as as adding them.
That’s huge. LLM computation is basically all matrix multiplication, so if you replace that with simple addition, you reduce the computational requirements by orders of magnitude.
The catch is such models are hard to train effectively; its proven that it works, but research to get the technique usable and practical is still being done.
I mention it because, if it takes off, suddenly the massive matrix multiplier accelerators we have for LLMs aren’t as useful. Chips with simpler architectures could get the job done, at least for parts of models that are bitnet.
The bigger question is what happens to the world-economy when the largest players bottom out and the companies betting on them are massively downgraded because of their huge investments in a fantasy about eternal growth. Hope everyone is setting aside extra cash; it’s gonna make 2008 look like a walk in the park.
Traditionally during such busts the government starts outright confiscating bank accounts or otherwise enacts some fiscal policy that renders all savings (especially those in cash) completely worthless in a last bid to pass the buck onto the 99%.
You’re saying there’s precedent that indicates the gov’t will just “confiscate” peoples savings? Could you elaborate? I’m genuinely interested to know about this, because that’s… not good
Per wikipedia:
Back in 1933 the US was on the gold standard. If the government wanted to print the money they had to have the gold to back it up so that it would be convertible to gold on request. If the economy grows but the money supply doesn’t then the existing money becomes more valuable which is deflation.
Deflation sounds great - the money you already have can buy more! - but it can lead to a ton of economic problems. For example, if your money is becoming more valuable without you even doing anything, a temptation is to just do the equivalent of stuffing it under a mattress and to try to put off buying anything for as long as possible since the longer you can wait before buying anything, the more you can afford. That causes business to dry up because people are buying goods and services less frequently. So business revenues go down. Businesses with lost revenues have strong incentives to downsize by cutting their existing employment or reducing their wages. Those employees who are fired or with cut pay can’t spend as much which further reduces revenues. So that deflationary cycle keeps repeating and worsening the problem. In addition, consider loans and investment. Existing loans become more burdensome because they’re also worth more each day, so a lot of people will be defaulting because they didn’t expect the loan to be expensive. Those who lent the money can strike it rich if lucky or be ruined if unlucky so higher risk lending is disfavored in favor of low risk stuff like government bonds, similarly investments aren’t doing so great because so many companies are having worsening revenues so investing looks like setting money on fire and is less popular. No loans and no investment makes it extremely difficult to start or improve a business if you don’t already have money which means not many new startups are coming in to replace the folks downsizing or shutting down or come up with a new idea of doing things.
So a lot of government efforts were put at the time in trying to beat deflation. The one this guy fell afoul of was that Roosevelt had by executive order suspended the convertibility of the dollar to gold and demanded that people with amounts considered in excess to take that excess to the government and be compensated for it for a defined exchange to paper money. That would get the government enough gold to make enough new currency so that it could pay for the many ambitious projects and programs Roosevelt envisioned to get the massive number of unemployed people back to work and making money so that they would buy goods and services and break the deflationary cycle. This guy however did not exchange his gold and had 160 kg of gold in a deposit that he was trying to reclaim (then valued at ~$100000 which is what we would now consider millionaire tier money), so he ran afoul of that restriction against hoarding. But if he had converted the gold to dollars earlier they would not have confiscated it. There’s no particular reason for the government to go around confiscating our accounts because there’s no gold in them, even if it were the dollar is no longer tethered in any way to gold, and even if that were the case we aren’t experiencing a deflation cycle that the government wants to create more money to break itself out of.
Thank you for this really detailed explanation!
They did it after the soviet union collapse for example.
Emory, if you’re out there somewhere ❤️
It’s more a failing socialist state thing, happened to my grandparents on my moms side, Capitalist states tend to go the inflation route.
Lower inference costs will lead to more demand for RAM
Will it though?
I posit there’s a saturation point where there’s “enough” LLM in use, and we are not far from that.
The whole pitch from Altman and such is scaling models up. But that’s not working.
Deepseek v4 Flash is insanely good and basically free. Especially the latest model which I think is only available from China right now.
Nah, 0731 weights were released. I’m running it locally right this second.
I don’t know about Deepseek API specifically, but there are tons of places to get it.
You are Mr. moneybags or you’re running 2 or 4bit quantized.
I have a 3090, 7800, and 128GB ram from pre-rampocalypse. It’s not nothing, but there are definitely crazier home labs.
Deepseek V4 is native 4 bit.
I am running this quant, but the quantization loss is measurably low: https://huggingface.co/Downtown-Case/DeepSeek-V4-Flash-0731-128GB-RAM-IK-GGUF
The native MXFP4 is only a bit bigger.
A used 3090 alone is like what, 4-5k now?
Your setup isn’t really achievable for most now.
The 3090 is from 2020. It’s under $1300 based on eBay prices I’m seeing (which is around what it launched at).
Crazy expensive, but not $5K.
The 128GB RAM is madly expensive now, though.
Lol what? I just checked, on eBay I can find a lot of offers between 800 Euros and 1300 Euros for a used RTX 3090. 4-5k would get you a brand new RTX 5090.
Okay, not as bad as I thought, there’s a guy selling a 4090 for “only” 2k. There’s nobody selling a 3090 in the local hardware forum. I’m not going to buy off ebay because if someone on ebay sells a dud, I can’t go back to punch him in the face.
5090s start at about 5200, that’s true. To be fair, the “4-5k” for a used 3090 was a bit hyperbolic, I’m just mad that they’re so expensive that my last 2 cars cost less than a USED 4090.