AWS Bedrock is other companies’ models running on separate dedicated AWS hardware, metered through AWS billing. AWS owns and operates all of the infrastructure and the client interface; the model provider basically hands over the model and weights to AWS and AWS Bedrock take it from there.
So, as an example, if you use Codex through Bedrock, that’s a totally separate instance of Codex from anything you would be interfacing with if you directly used OpenAI’s API; if you use Codex via Bedrock, OpenAI never sees your data or prompts because they stay sandboxed in an ephemeral Bedrock instance. For many large enterprise deployments this hard boundary is a big big deal.
Over the past year, Claude being available via Bedrock and ChatGPT/Codex not being available via Bedrock has been a huge competitive advantage for Anthropic in the enterprise space.
If you've used AI coding models in a large corporate setting, you'll know that a lot of big corporate deployments basically require using AWS Bedrock for two simple reasons:
1. Large companies tend to already have an existing relationship with AWS, which makes things way easier to go through vs. setting up a new vendor relationship
2. Large companies tend to have strong internal requirements about making sure that internal data stays under company control. With AWS Bedrock, you can be a lot more confident that what you're feeding into the models is not going to end up in someone's training set somewhere. For where I work, this requirement is a dealbreaker for going directly through OpenAI's API instead of going through AWS Bedrock.
To go a step further, the reason it's often impossible to add a new vendor if that you've signed a bunch of contracts with your customers saying you're not going to send their data to other vendors in all sorts of various flavors.
3. from my opportunity - For many (not all) LLMs, Bedrock gives you control over which country the data stays in. You have no control over that with the Claude API, for example. We do not work in the US and have strong requirements for the data to stay in our country, which Bedrock gives us control over.
> We do not work in the US and have strong requirements for the data to stay in our country, which Bedrock gives us control over.
It doesn't actually. The US can request data from whatever country US companies store it, and companies must comply.
So if you have strong requirements for data to stay in your country, using a US provider, whatever it is, is out of question no matter what the company's marketing claims (they are not maintaining these claims under oath for what it's worth: https://www.senat.fr/compte-rendu-commissions/20250609/ce_co... )
Ah yes, there is a gap between what our regulator wants and what the reality is. I have no qualms that they'll hover out the data if they want to, we know that since Snowden. But I have to comply with the regulator, not with reality.
4. AWS billing is already cross-charged to different departments per account. Copilot/Claude/Codex would need that setting up all over again, and is (probably) all coming out of a central bucket right now. Switching to Bedrock APIs is really easy, and solves a problem for people high enough up in the organisation that they can insist on it.
Curious to understand how AI will continue to grow if this is the trend. Assuming most valuable data is behind such firewalls. And whatever is public has been harvested, trained on top of whatever has been acquired illegally (this is a grey area).
Will it become a closed ecosystem without outside input?!
The pace of data creation is only increasing, and our capabilities of sharing and storing it is growing as well. Lots of this is out in the open, ready for anyone to crawl and scrape.
There probably is a point of “peak data” where the amount of new data will start decreasing, but that’s likely a 22nd or 24rd century problem.
Pace of data creation ignores the fact that the majority of the big gains in LLM “intelligence” has come from scraping and feeding in the huge amount of public data that already exists.
Unless we’re producing data on the order of an entire new internet every couple of years, then it’s hard to see how LLMs can achieve further huge leaps in capability compared to training on effectively 0% of the internet vs 100% of the internet.
That is without going into fact that many already use AI to type out and write stuff. I have a customer in Far East that routinely uses it even for simple emails, he is not so familiar with English.
The majority of the gains come from the size of the supercomputers used to train them on. That's still growing. The algorithms used, and how the data is annotated is also some secret sauce.
If anything, trend will go towards sharing data less. It will become more important to keep the knowhow and data to yourself so the companies will do that.
And individuals will loose motivation to share, because it wont be that pro-social activity anymore anyway.
What other people are saying, but also because Amazon does not want to fuck around in this space. They don't want the legal fight or the reputational damage that would come with it.
To take an easy example that has actually had lawsuits I can link to, you must be unfamiliar with the lawsuits against Amazon for misusing sellers' data in order to undercut them with their own products... https://www.reuters.com/sustainability/13-bln-uk-lawsuit-acc...
There's zero reason to "trust" Amazon about anything. (And yes, I know the retail and AWS sides of the company are different, but it's still the same company. The same rot is always there, just shuffled around.)
this is not related to AWS, but merely to amazon's retail business and their sellers know and sign up for the deal when they sell via amazon.
every single retail company does this, they allow suppliers to sell the product using retails's infrastructure, and then retailer turns around and create private label products using sales data (Costco's Kirkland Signature, Walmart's Great Value, are just some examples)
Yes, but Kirkland's signature comes from the same factory. If I'm the factory owner and Costco vis going to guarantee me sales albeit at a slightly lower margin, so long as I slap a different sticker on it, that's different than from Amazon finding out which of my products sells best and then gets someone else to rip it off so I don't get paid anything.
First of all, we don't know which factory kirkland's products are coming from. Even if they are coming from the same factory, who guarantees the same ingredients and quality control was used???
everything from amazon is coming from China, I dont understand why does a random person who resells stuff from Chinese factories via Amazon FBA feels entitled for exclusivity arrangement with Amazon?
Was such exclusivity encoded in some form of legally enforceable agreement ?
That’s not the case at all. Kirkland just ditched Huggies making their diapers. They just introduced a breaded chicken tender nug to compete with one on the shelf.
They absolutely go out and find who can make the product and the quality and price they want. It’s not always an identical product to the brand name on the same shelf. Sometimes it displaces the brand name.
Wow there must be an echo in here because I swear I said just that... And then pointed out that it's the same crap being recycled back and forth across the company. There is no real separation.
In contrast to Microsoft, OpenAI, and Anthropic, AWS has never done anything close to sneaking in unwanted training opt-outs after the fact.
They are the only ones I trust not to do that so far. And their terms are extremely clear on that, no fuzzy language. Exactly what we want to see. So we use Bedrock.
Laws and rules don’t hold anyone accountable. Anyone can say anything and then break that trust the next second.
Instead you trust your best friend because you have known them for 15 years and seen them in enough situations. It’s long term observation and predictability they ultimately gives trust.
AWS has been around 20 years and has never once shown a sign that that they would sell customer data. Could they still try? Sure, in the same way they my friend who hates seafood his entire life could suddenly flip 180 and love it. Yeah I guess it’s possible.
Actually yes, when it’s other huge corporations holding them accountable. It’s only when politicians who are much more cheaply bought get involved that creates problems. When the other side has a significant war chest to combat you with, suddenly behavior improves
You really don't understand what AWS offers if you think this is what is getting them workloads (including competitors and highly sensitive govt workloads).
Having worked with lots of companies, I can say that trust is there. But true test is competitors of Amazon. Does Walmart use them? Ebay? Although not in exact same business.
Contractual obligation, external third party audits, and above all, AWS’s reputation.
AWS isn’t going to risk their reputation, and thus huge chunks of their business, just so a few AI labs can get some extra training data. That’s an insane risk with zero upside for AWS. AWS knows full well they will make insane quantities of cash without breaking legal contracts with companies who pay them billions each year for infra.
they’re crap on a lot dimensions of how they treat customers but data privacy/security is one thats taken pretty seriously at AWS, perhaps owing to the massive reputational damage that would result if they played loose with it.
A medium-spicy take of mine is that a bytecode VM in a GPU kernel is not as bad of an idea as one might think, and in some cases it can actually be the most reasonable solution. Some fun examples:
1. As mentioned in the post above, the Dolphin emulator famously implements the entire Gamecube/Wii GPU pipeline in a single gigantic ubershader, and this is useful because it avoids shader compilation stalls [1].
2. Blender's Cycles renderer implements its shading graph eval system as a bytecode VM in a GPU kernel [2]. IIRC early versions of Vray GPU did something similar. There are better ways of course, but a VM gets you surprisingly far as a general approach.
3. Finally, a lot of ML frameworks (Tensorflow, PyTorch, etc) by default use the GPU relatively suboptimally (especially without kernel fusion and such). Tensor frameworks can extract a lot more perf out of GPUs using a VM-in-a-giant-kernel approach [3].
If you think abstractly about how a GPU SM actually works (using CUDA terminology here), all threads in a warp must execute in lockstep and the cost of execution divergence across threads in a warp is that you effectively run serially, losing the parallel advantage of the SM. This penalty gets magnified enormously if you are doing memory reads after wherever the execution divergence happens, since you now have multiple slow memory stalls in serial instead of one big memory read at once for all threads. If you're clever about implementing a bytecode VM, you can load as much state as you need upfront into shared memory, and then if your bytecode VM is just looping through executing a bunch of opcodes in a huge switch statement, then at least as far as the SM is concerned, there's no execution divergence! All threads look like they're doing the same thing at the same time; even if within the VM what is happening a lot is just no-ops, at the SM level you're not dealing with serialized memory stalls and serial scheduling and such.
Is it the _best_ most optimal approach imaginable? Almost certainly not! But can it be a _surprisingly good_ and possibly even reasonable approach for some problem domains and specific constraints? Yeah absolutely!
Imagine every bit of human knowledge as a discrete point within some large high dimensional space of knowledge. You can draw a big convex hull around every single point of human knowledge in a space. A LLM, being trained within this convex hull, can interpolate between any set of existing discrete points in this hull to arrive at a point which is new, but still inside of the hull. Then there are points completely outside of the hull; whether or not LLMs can reach these is IMO up for debate.
Reaching new points inside of the hull is still really useful! Many new discoveries and proofs are these new points inside of the hull; arguable _most_ useful new discoveries and proofs are these. They're things that we may not have found before, but you can arrive at by using what we already have as starting points. Many math proofs and Nobel Prize winning discoveries are these types of points. Many haven't been found yet simply because nobody has put the time or effort towards finding them; LLMs can potentially speed this up a lot.
Then there are the points completely outside of hull, which cannot be reached by extrapolation/interpolation from existing points and require genuine novel leaps. I think some candidate examples for these types of points are like, making the leap from Newtonian physics to general relativity. Demis Hassabis had a whole point about training an AI with a physics knowledge cutoff date before 1915, then showing it the orbit of Mercury and seeing if it can independently arrive at general relativity as an evaluation of whether or not something is AGI. I have my doubts that existing LLMs can make this type of leap. It’s also true that most _humans_ can’t make these leaps either; we call Einstein a genius because he alone made the leap to general relativity. But at least while most humans can’t make this type of leap, we have existence proofs that every once in a while one can; this remains to be seen with AI.
I like this construction, but I don’t think you take it far enough.
If you have a multi dimensional space, and you are trying to compute which points lie “inside” some boundary, there are large areas that will be bounded by some dimensions but not others. This is interesting because it means if you have a section bounded by dimensions A, B, and C but not D, you could still place a point in D, and doing so then changes your overall bounds.
I think this is how much of human knowledge has progressed (maybe all non-observational knowledge). We make observations that create points, and then we derive points within the created space, and that changes the derivable space, and we derive more points.
I don’t see why AI could do the same (other than technical limitations related to learning and memory).
I was a little muddy in my original post on distinguishing between what I think LLMs might be able to do and what AI broadly might be able to do. I'm skeptical LLMs can expand the hull or add dimensions to the space; but I also don't think the reasons for that skepticism necessarily apply to all AI system generally.
A lot of the space outside of the convex hull is just untried things. You can brute-force trying random things and checking the result and eventually learn something new. With a better heuristic, you can make better guesses and learn new things much more efficiently. There’s no reason to believe that kind of guess-and-check is outside of the reach of LLMs, or that most of our new discoveries are not found the same way.
I come back to something like this idea when I consider the distinction being made that LLMs can only combine and interpolate between points in their training material. I could write a brute-force program that just used an English dictionary to produce every possible one-billion-gazillion word permutation of the words within, with no respect for rules of language, and chances are there would be some provable, testable, novel insight somewhere in the results if you had the time to sift through and validate all of it. LLMs seem like a tool that can search that space more effectively than any we've had before.
If we managed to create very fast monkeys with typewriters and software that can review their output quickly enough that we end up with a result that's worth reading we'd still have people insisting that we've created intelligence. The monkeys however remain monkeys.
I think intelligence is an orthogonal, mostly philosophical question aside from whether these tools can produce novel, useful output vs purely recombinant output.
I think of most things you can get to by guess and checking as definitionally inside of the hull; most forms of guess and checking are you take some existing thing, randomize a bunch of its parameters, and see what you get. Whereas with something like relativity, there's not even a starting point that you can randomize and guess/check from the pre-existing knowledge space that will lead you to relativity. That's more like, adding a new dimension to the space entirely.
It's possible LLMs can handle this after all! But at least so far we only have existence proofs of humans doing this, not LLMs yet, and I don't think it's easy to be certain how far away LLMs are from doing this. I should distinguish between LLMS and AI more generally here; I'm skeptical LLMs can do this, I think some other kind of more complete AI almost certainly can.
I supposed you could just, I dunno, randomly combine words into every conceivable sentence possible and treat each new sentence as a theory to somehow test and brute force your way through the infinite possible theories you could come up with. But at that point you're closer to the whole infinite random monkeys producing Shakespeare thing than you are to any useful conclusion about intelligence.
I think your point about “you could randomly generate a sequence of words, which could in principle produce a text interpretable as expressing any particular expressible-as-a-sequence-of-words novel good idea” pretty much refutes the idea that guessing and checking can only result in things inside such a convex hull, unless said hull already contains everything.
Of course, there’s a significant role to play by the “checking” part.
Like, “take a random sequence of bits and interpret it as Unicode” is at one end of a scale, and “take a random sequence of words in a language” is just a tad away from it,
and the scale continues in that direction for quite a while.
This assumes that everything outside of the convex hull can already be described using existing language. If you need new language to describe what is outside of the convex hull, is this something an LLM can do?
I actually don't know the answer to that; my understanding is that LLMs by nature of what they are can't understand concepts that are independent of the existing language they are trained on, but I don't have enough in-depth nitty-gritty knowledge of like, core LLM implementation details and architecture and stuff to know if that understanding is correct or not.
I suppose it is conceivable that there are some useful ideas that cannot be described in terms of language we understand (e.g. if there are ideas that are alien to us and beyond what can be described using https://en.wikipedia.org/wiki/Natural_semantic_metalanguage#... ), but, if there is, I'm not sure those are ideas we can communicate to one-another?
By "If you need new language" do you mean like, coining new words?
I don't see what would prevent them from doing this? LLMs can process text that includes newly coined terms, and respond to that text in ways that use those newly coined words in accordance with the descriptions of the meanings given for those new words in the prompt. They can also make up new words+definitions when asked to do so. Now, whether they can, without being told to do so, recognize that it would be useful to coin a new word for something, and then start using it, I don't know of any instances of this, but based on the previous two things, I don't see a reason to expect this to be fundamentally beyond what they can do?
I don't know what it would mean for a concept to be "independent of the existing language they are trained on". If there are ideas that can't be expressed in terms of the semantic primes all ideas we can express can be expressed in terms of, then I guess such an idea would be independent of our language, but I think that's a much stricter condition than what you mean (and I'm not sure if there even are any good ideas that can't be indirectly expressed in terms of semantic primes -- I kind of suspect not, unless they are like, ideas that are too big to fit in a human mind anyway).
Of course, the outputs these models produce is causally downstream from the data they are trained on, and the distribution they produce over text is largely based on the distribution over text in the training data, but altered in a number of ways (for example, to make them implement the character of the "assistant" persona).
> You can brute-force trying random things and checking the result and eventually learn something new.
And most of the mathematicians seem to welcome this "brute forcing" by the LLMs. It connects pieces that people didn't realize could be connected. That opens up a lot of avenues for further exploration.
Now, if the LLMs could just do something like ingesting the Mochizuki stuff and give us a decent confirmation or disproof ...
It's also worth noting in that in very high dimension, the convex hull will contain massive volume. It could well be the case that humans established that convex hull millions of years ago, and all of our inventions and innovations sense have fallen inside it.
I found this thought provoking and just had to see how the new Gemini 3.5 Flash reasoned about this (I find it fun to go meta on modern AI like this), and I'm happy that I did! Also as an opportunity to trial this recent model.
At the time of the announcement IIRC the deal was only for Colossus 1. Is Anthropic also leasing Colossus 2 new?
At the time the consensus narrative was that SpaceX no longer needed Colossus 1 for Grok and that was why it could be leased to Anthropic while Colossus 2 would handle Grok training and inference. Does Anthropic also leasing Colossus 2 change this?
Right. This compute still being powered by an illegal amount of gas turbines in a residential neighborhood?
Claude is eating so much compute, the threat of that power being tuned down by lawsuit (rightfully) is worth the risk to Anthropic in the short-term. Instead of declaring "bubble", I'm just going to say that's so crazy.
Yes and both are getting sued? I wouldn't necessarily classify the NAACP as environmental activists, but they are concerned with the wellbeing of the people they represent.
"I suspect your definition of a lousy place to build infrastructure in might overlap with my definition of a relatively good place to live."
Which is quite close to the standard NIMBY attitude: "I want good grid, cheap electricity and other infrastructure, but not in my backyard. Either someone else's, or somewhere where no one lives at all."
Can't you see the fundamental problem with that attitude? We mostly live in densely populated regions. We do need roads, rail and power plants to sustain our way of living, you too.
Of course we need infrastructure. It’s annoying when a project that seems like a net good is held up due to environmental concerns, but this planet is very beautiful and I’d like to be able to continue to enjoy it.
In practice, everything is tradeoff. Your house, and mine, and everyone else's here stands in some place which was once a beautiful natural spot. But few people would support tearing their own homes down in order to restore that beauty.
A technical civilization of 8 billion people cannot exist without doing at least some damage to the environment, but most of the time, outright bans on further development are demanded instead of some reasonable mitigation.
Maybe you really deeply care about the environment. Most of the NIMBYs I met don't. When it comes to infrastructure, housing etc., the environmental concerns are quite often just a legal tool, and the real motivation of the people who wield them is more along the "I have mine, I don't care about yours, just sod off".
I wonder how many other people do this: when my wife and I do long roadtrips, we use Costcos as waypoints. Need to refuel? Costco gas is always cheaper than whatever other fill up station is nearby. Need to re-up on snacks (and maybe see what weird snacks the locals have that we don’t have back home)? Costco. Realize you forgot to pack enough socks halfway between Los Angeles and Salt Lake City? No prob, the St George Costco had the same exact socks I have at home because all of my socks are from Costco. Ran over a nail and need a tire swapped? Costco tire warranty babbbyyy. Bathroom? Costco bathrooms are very basic but always clean.
In big adventure RPG games there’s always some kind of shop in every new area that is the same inside everywhere that you can reliably go to for whatever gear you need, to heal, to save your game, whatever. Costco is that but in real life.
Huh. It's the exact opposite of what I'd look for in a roadtrip waypoint.
Gas line. Nightmare parking lot to get in and out of. Long walk to the store. Long walk to find anything. Slow checkout.
I guess narrowing it to "cheap gas, availability of familiar goods and bathrooms, with an on-site tire shop" helps make it make a _bit_ more sense, but the (lack of) speed would just be a deal-killer for me.
Something I really like about this post is that I think the high-level takeaway from this post is generally applicable far outside of the India AI industry. Sure, the specific application here is the AI industry and that's certainly the topic du jour, but I think this article does a great job of demonstrating how to think and see through the market distorting effects that subsidies produce, in order to figure out what the true cost structure and valuation in such an environment is.
Seems true but even at 2x that price people want some reliability. As someone that has used .gov.in services - it is unreliable. Yes, many gov services can be done online -great kudos (otherwise people will start personally attacking me for being unpatriotic). Portals like license, passport, Aadhar (identity) card sites frequently undergo so much down time.
At least for me Claude Code is still working on my Pro plan. I don't know if that's because the change simply hasn't propagated all the way through their systems yet (the change is now up on the main Claude pricing page and on their support pages, but not on the Claude Code landing page yet), or if it's because existing plans are grandfathered in, or what.
In general Anthropic seems to be pretty bad at clearly communicating what is going on. I have both Claude Pro for Claude Code and ChatGPT Plus for Codex, and lately I've been reaching for Codex first more and more often... at least for the hobby stuff I'm using Claude/Codex on, they seem pretty much equivalent in terms of practical capability/usefulness.
I'm really hopeful about John Ternus stepping into the CEO role. Pretty much everything he's done leading Apple's hardware engineering has been an enormous unqualified success, and for a company like Apple, having hardware lead the company seems like the right step.
So, as an example, if you use Codex through Bedrock, that’s a totally separate instance of Codex from anything you would be interfacing with if you directly used OpenAI’s API; if you use Codex via Bedrock, OpenAI never sees your data or prompts because they stay sandboxed in an ephemeral Bedrock instance. For many large enterprise deployments this hard boundary is a big big deal.
Over the past year, Claude being available via Bedrock and ChatGPT/Codex not being available via Bedrock has been a huge competitive advantage for Anthropic in the enterprise space.
reply