However, I noted, that the base model is performing quite poorly on small text chunks (a few words) while the small version seems to be unaffected. Might this be some kind of side effect due to the way they deal with large contexts?
If you want to test, you can head over to SemanticFinder (https://do-me.github.io/SemanticFinder/), go to advanced settings, choose the Jina AI base model (at the very bottom) and run with "Find". You'll see that all other models perform just fine and find "food"-related chunks but the base version doesn't.
Just making up stuff here, but smaller models are great for serverless compute like functions, which would also benefit from lighter computation. Don't forget, some people are dealing with hundreds of millions of documents. Accelerating this by 4x may be worth a small performance hit.
That's awesome and so much more practical than scrolling through HN.
It would also be possible to integrate semantic search so people don't necessarily need to know the keywords. If you're interested, feel free to ping me or take a look at https://github.com/do-me/SemanticFinder. In case I could just create a pre-indexed version based on your data dump which would be quite convenient to use.
My open source blog mainly with (geospatial) niche tutorials.
It's a pretty personal thing as I mostly (but not only) derive the blog posts from challenges I encounter during work or my PhD research. In this way it documents my learnings and serves as a quite verbose personal wiki.
Awesome project! From the website I wasn't quite able to tell where exactly and how AI comes into play.
I think apart from fuzzy searching headings a semantic search functionality for the whole text would be a really nice default feature. I implemented frontend-only semantic search with transformers.js here: geo.rocks/semanticfinder. The implementation is very straightforward and it would be easy to integrate in the browser, maybe as a plugin or similar.
SemanticFinder now offers an easier way of pre-indexing files. Just like in all ML-flows the embedding calculation is the computation-intense part, so if you do this beforehand and load it in your browser, calculating cosine similarity is trivial (for a few thousand iterations).
It has the nice side-effect that this also allows for external computation of embeddings where you might have faster hardware than you're client device.
One could add so many features like adding full-text search by checking if the index contains certain keywords.
For smaller documents (but also entire books) you can even perform everything in your browser with JS without any DB or backend using transformers.js. It's surprisingly fast.
I think this could be really useful for visually impaired people. Just imagine, that instead of colors and letters you'd have unique shapes or to keep it simple something like a raster with holes. Every bus stop could have one of those raster plates while the people would have a sort of "key", designed to fit in "their bus".
I came up with a very similar approach to explain HyperLogLog a while ago and see some parallels.
I guess the key/lock principle is very similar to both, bloom filter and HyperLogLog and could yield other benefits as well, like counting distinct buses passing a stop and map them for city. In this way, one could easily create a heat map of the cities best- and worst-served areas.
That's a very cool idea. Thanks for the blog, it's well written. I love how you've applied the idea of a hashmap to a 2d foam board. I'm on the verge of having a breakthrough of improving my visualization but so far it hasn't arrived. If you think of anything feel free to let me know.
Pretty detailed guide! I'd just like to emphasize that nowadays you can literally create and deploy a personal homepage entirely for free.
For the individual touch, buy a custom domain for a few bucks, link it and even add a free custom mail (e.g. with Zoho's free tier).
It's remarkable how knowhow and budget requirements have dropped in recent years so that anyone can create something pretty good-looking in no time.
Personally, I use the combination of:
1) Hugo + GitHub + Netlify + Zoho for my personal blog and email (summed up here: https://geo.rocks/post/setup) and
2) Material for MkDocs + GitHub for projects.
- 𝟐𝟖.𝟓 𝐌𝐁 jina-embeddings-v2-small-en (https://huggingface.co/do-me/jina-embeddings-v2-small-en)
- 𝟏𝟎𝟗 𝐌𝐁 jina-embeddings-v2-base-en (https://huggingface.co/do-me/jina-embeddings-v2-base-en)
However, I noted, that the base model is performing quite poorly on small text chunks (a few words) while the small version seems to be unaffected. Might this be some kind of side effect due to the way they deal with large contexts?
If you want to test, you can head over to SemanticFinder (https://do-me.github.io/SemanticFinder/), go to advanced settings, choose the Jina AI base model (at the very bottom) and run with "Find". You'll see that all other models perform just fine and find "food"-related chunks but the base version doesn't.