Good points. The easy experimentation factor is helpful for development, though I would gently encourage everyone to migrate to the 1st party APIs for pricing at scale.
OpenRouter is also a good place to find free LLM access with a catch: You should expect that any inputs and outputs are going into someone's training database. Clearly anyone who can pay should be using paid models with privacy protections, but the free models have been great for learning and experimenting. Especially for younger people learning API programming and LLMs who may not have access to a credit card or funds.
It’s interesting all the focus on opt-out from training. Sometimes I worry there is an intentional focus on that so people don’t think about the other ways the company might be profiting off our data. Like I pay for Anthropic and they don’t train on that but are they selling my “anonymized” usage data in some other way?
From what I recall, these companies don't offer any option to opt out of your session transcript data being used (and sold!) for "regular" adtech targeting purposes.
Anthropic explicitly state that they don't do this, even if you use the free plan and even if you don't opt-out of letting them use your data for training:
That answers for the "sold" part but not for the "used" part.
I.e. nothing about this statement prevents Anthropic from running ads within Claude, as long as they run the ad-placement auctions themselves, and so aren't leaking any of the data they're using to decide which placements are relevant to which users+sessions. (This is the same thing Google does for SERP ad auctions.)
But actually, and perhaps more interestingly, nothing about this statement prevents Anthropic from building a Google AdSense competitor either. Other sites (or mobile apps, etc) could plop in an Anthropic ad iframe; and it'd be Anthropic's knowledge of your interactions with Claude that would drive what ads would show up in that iframe. The embedding site doesn't know what ads the users are seeing, so that's still not "selling users' data to third parties", per se.
> You should expect that any inputs and outputs are going into someone's training database.
True enough, in theory; but what exactly are you imagining would be a useful-enough signal in the OpenRouter request+response stream, that any company would want their data as training material?
Even a single OpenRouter-API-key-identified subscriber's traffic, may consist of an mixture of traffic from multiple different sessions, under potentially multiple different end-users. (Where, if the subscriber is doing security correctly, then their OpenRouter key lives on a gateway rather than in a frontend app; and so the only IP address / UA / etc OpenRouter sees is that of the gateway itself.)
And the traffic stream may also invoke multiple models, and provide multiple different system prompts for those models; which, while marked in the traffic (i.e. conveyed as part of each request), makes the resulting data much less useful in aggregate, than if it were all training data for one model with one system prompt.
Plus, there are no RLHF signals in OpenRouter data. Even if OpenRouter wanted to build a general model-neutral framework for collecting RLHF-type data, it can't force subscriber apps to do the UI-level stuff necessary to collect it (i.e. the things ChatGPT/Claude do, with "thumbs-down" buttons, A/B tested responses, etc.) Analysis would have to rely on pure transcript-level user sentiment extraction.
> Privacy:
>
> Private I/O logging and the 1% data sharing discount are separate settings. You control each independently.
>
> Input & Output Logging stores prompts and completions for your use only and makes them visible in your logs. OpenRouter does not access this data. You can configure it in your observability settings.
>
> As always, more in the docs!
> Plus, there are no RLHF signals in OpenRouter data. Even if OpenRouter wanted to build a general model-neutral framework for collecting RLHF-type data, it can't force subscriber apps to do the UI-level stuff necessary to collect it (i.e. the things ChatGPT/Claude do, with "thumbs-down" buttons, A/B tested responses, etc.)
The majority of RLHF data doesn't need this. The majority is software development and/or tool calling where the agent gets a signal back as to if it succeeded (eg compilation errors, test errors). It's true that end-of-trajectory signals (eg, did this task do what you wanted) are even more useful but even partial signals are great for RL training.
> what exactly are you imagining would be a useful-enough signal in the OpenRouter request+response stream, that any company would want their data as training material?
Isn't this a treasure trove for any model distillation effort?
I've wondered this too - exactly how are our inputs and outputs useful as training data? So I asked Gemini. Apparently using negative sentiment in user or llm responses can serve as RLHF, and the human prompts can also serve as useful data for what problems the llms need to be able to solve. There's also that smaller models can train on and improve from data from larger models but that's less relevant when not switching models in context.
> Clearly anyone who can pay should be using paid models with privacy protections
Clearly, anyone who needs privacy should be using models with privacy protections. Some people build open source and the models will get the code anyway.
It's free, but not unlimited. Besides rate limits, new sign-ups get 1000 credits (requests), and once those are gone, they're gone for good. Only business accounts might get a couple of free refills.
It is unlimited under the free NVIDIA Developer Program. You're talking about a different sort of acct I think. The dev program acct is 40 rpm unlimited for personal use.
OpenRouter is also a good place to find free LLM access with a catch: You should expect that any inputs and outputs are going into someone's training database. Clearly anyone who can pay should be using paid models with privacy protections, but the free models have been great for learning and experimenting. Especially for younger people learning API programming and LLMs who may not have access to a credit card or funds.