It is nice to see more backends utilizing SQLite. The benchmarks and the Comparisions section also seem well done.
Just a nitpick - list the versions of the tested platforms.
Based on your benchmarks repo it looks like that the tests were done against PocketBase < v0.23 but note that PocketBase v0.23+ (especially with the Create API rule dry submit removal in v0.24+) has introduced significant changes and performance improvements - ~4x times in high concurrent scenarios in our own benchmarks[0] (if you want to retest it note that the CGO driver is no longer loaded by default and will have to be registered manually; see the example "db_cgo.go" in the PocketBase benchmarks repo or in the "Custom SQLite driver" docs[1]).
- It is nice to see more backends utilizing SQLite.
Hey thanks for chiming it. Huge fan of PocketBase, has been a major inspiration :applause:. For anyone driving by, certainly a more mature product.
- Based on your benchmarks repo it looks like that the tests were done against PocketBase < v0.23 but note that PocketBase v0.23+ (especially with the Create API rule dry submit removal in v0.24+) has introduced significant changes and performance improvements - ~4x times in high concurrent scenarios in our own benchmarks[0] (if you want to retest it note that the CGO driver is no longer loaded by default and will have to be registered manually; see the example "db_cgo.go" in the PocketBase benchmarks repo or in the "Custom SQLite driver" docs[1]).
You're right. I did run v0.22.21, which simply was current when I ran the benchmarks first. I absolutely will add the information, rerun, and thanks for the pointers. Glad to hear you got such a boost :clap:
I'm happy to report that v0.25.0 already w/o CGO driver (fighting it right now) and GOOS=linux CGO_ENABLED=0 GOAMD64=v4 improved about 35% from 61.7s per 100k inserts to about 40s :clap:
I still wanna get the mattn/go-sqlite3 driver to work and it's getting a bit late here for writing coherent text... I'll update the benchmarks ASAP
Back from the dead. I also now vaguely remember that there was some hiatus (probably between v0.22 and v0.23) where you took some time to rewrite the DB layer to enable custom DB drivers.
Anyway, w/o fog I managed to run with mattn/go-sqlite3. I'm not sure this is expected but it didn't seem to make much of a difference with my setup (For transparency, I do recently have some issues with repeatability likely due to btrfs). I'm certainly not seeing 4x but around 30-35%, which is still very impressive!
> I'm not sure this is expected but it didn't seem to make much of a difference with my setup
I expect it to be faster but it is also possible that maybe in your specific collection and execution scenario it somehow perform worst, or at least not the same as in my benchmarks, I'm not sure.
The ~4x mentioned speed up is based on the tests for Hetzner CAX41 with the CGO driver when creating 50k records with 500 concurrent clients:
FWIW, I did this locally (also happy to fork PB and check it in, certainly aids transparency). I've already updated the numbers in the benchmark doc but will continue to try squeeze more out of it.
As to 4x, my concurrency levels are significantly lower which could certainly explain it. Is PB juggling multiple connections increasing write lock congestion or are you serializing access, e.g. via a worker thread? Also happy to chat more (feel free to send me an email), I certainly want PB to have the best possible representation and maybe we can even speed things up on both sides
> As to 4x, my concurrency levels are significantly lower which could certainly explain it.
Yes I noticed that too so that it is very likely the reason.
In all cases my initial comment wasn't intended to "dispute", argue or anything like that, so please don't feel obligated to waste time updating the benchmarks. They are valid as they are!
My initial comment was more of a note/suggestion to simply list the used versions of the tested platforms (not just for PocketBase) because often they change over time and the shown results could be misleading if someone stumble on the article 2 years later for example.
> My initial comment was more of a note/suggestion to simply list the used versions of the tested platforms (not just for PocketBase) because often they change over time and the shown results could be misleading if someone stumble on it 2 years later for example.
FWIW, it never felt like a dispute and very much agree with your suggestion. I'm also just trying to do a decent enough job, both with the benchmarks and TrailBase itself. Either way, my offer to keep an open channel stands in case you want to share experiences or are in desperate need for a beverage :)
Keep in mind that PocketBase do a lot more than just executing a raw DB query. We perform data validation, normalization, serialization, enriching, auto fail-retry to handle additional SQLITE_BUSY errors, etc. All of this comes with some cost and will always have an effect when doing microbenchmarks like this.
The performance would also depend on what version of PocketBase did you try (before or after v0.10), whether you used CGO or the pure Go driver, etc.
There is definitely room for improvements (I haven't done any detailed profiling yet) but the current performance is "good enough" for the purposes the applications PocketBase is intended for (I've shared some numbers regarding a PocketBase app on production in https://github.com/pocketbase/pocketbase/discussions/4254).
Why? I think most modern bundlers and spa routers supports code splitting out of the box by just using dynamic imports, aka:
`const myComponent = await import("/path/to/component")`
I don't understand your concern with it and why this is a problem.
I haven't personally noticed the vite/esbuild memory consumption, but I also run build only once at the end before deployment or generating the prod artifacts.
It wouldn't be an issue if other work tools were taking absolutely nothing yes but as it is now, 16GB of RAM with a JS oriented stack isn't enough anymore.
And getting more RAM is just delaying the problem anyways since it takes more RAM every week as the product evolves.
It is really a cool project. We are using it as a default driver in PocketBase and
although it is not a "drop-in" replacement of the CGO alternative `mattn/go-sqlite3` (different dsn format, some differences in the error messages, etc.), with a small abstraction it works fine for most cases and greatly simplify cross compilation.
Performance wise I haven't done intensive benchmark tests yet, but from my local experiments last year it performed ~1.5-2x slower than the CGO version for some queries (it is especially noticeable with LIKE expressions on large string data), but as mentioned previously, for most use cases it is already good enough.
The existing admin interface could be used as a very rudimentary content management, but it was really intended for quick internal data explorations and edits. Additionally, currently there are no roles for admins so permission controls within the admin area are not possible.
You could always implement your own custom CMS SPA frontend that interacts with the PocketBase API (that is how I'm planning to use it with the next version of my other open source project - Presentator).
Currently any additional data transformation is left to the developers to extend via event hooks or custom client side handling.
I've worked with CRDT in the past (yjs), but it may not be very useful in PocketBase considering that the application was designed to run on a single server and db writes are practically queued (you can have only one sqlite writer at a time). I'll investigate it further and may consider it for future release.
I think if you integrate Yjs or a OT library first-party, you'll have a really exceptional framework - open source firebase alternative top-to-bottom, ready for "local-first" collaborate own-your-own-data apps. Sounds like my dream stack for small-scale projects or friend/family apps.
I guess there will be differences from use case to use case, but in general migrating from PocketBase to a more common horizontal supported stack is not different from rewriting your application.
Internally, each Collection creates a standard SQLite table that holds the collection records, so migrating the data structure shouldn't be too troublesome. The only thing that may prove difficult to migrate could be reimplementing the access rules and filters.
But in my opinion, when your application reaches that level of growth requiring multiple servers and services, your business use cases mostly likely will have changed several times already from your initial idea.
The documentation is not auto generated and is part of the static landing site (built with SvelteKit).
I didn't bother open source it because it is kinda messy and a little complicated to be edited by users not familiar with the codebase, but I'll try to find some time in the future and may publish it.
In the meantime, if you find typos or think that some of the wording could be improved, feel free to open an issue or discussion in the main repo and I'll fix them.
It is nice to see more backends utilizing SQLite. The benchmarks and the Comparisions section also seem well done.
Just a nitpick - list the versions of the tested platforms.
Based on your benchmarks repo it looks like that the tests were done against PocketBase < v0.23 but note that PocketBase v0.23+ (especially with the Create API rule dry submit removal in v0.24+) has introduced significant changes and performance improvements - ~4x times in high concurrent scenarios in our own benchmarks[0] (if you want to retest it note that the CGO driver is no longer loaded by default and will have to be registered manually; see the example "db_cgo.go" in the PocketBase benchmarks repo or in the "Custom SQLite driver" docs[1]).
[0]: https://github.com/pocketbase/benchmarks
[1]: https://pocketbase.io/docs/go-overview/#custom-sqlite-driver