I watched the presentation, but I'm getting the impression that you're still using a hierarchical XML-like structure for the document, which sounds like it still cannot merge or split nodes (without recreating the content after the merge or split point). Will issues like https://github.com/yjs/y-prosemirror/issues/205 be addressed by this? (Which not only breaks ProseMirror's position mapping, but also your own, and which I consider a serious enough bug that recommending y-prosemirror for anything feels unserious.)
Yes, we are still using XML to represent the document content, to represent the document as a flat list would require some research into something like Automerge's Block markers https://automerge.org/docs/reference/under-the-hood/rich-tex... which we don't have the funding to look into right now, but hope to work towards in the future.
We will have better support for ProseMirror's position mapping, but I'm not confident that we will make any progress on this specific issue, since it requires a much larger refactor than what we are contracted for (suggestions & versioning). For selection mapping specifically, we can of course make an effort here to fix it, but the underlying limitations of the split node operations would still be there.
I believe that node-splitting in y-prosemirror can be solved.
But I hope that you also can appreciate that mapping prosemirror to a CRDT structure is a very complex thing to do. Your schema implementation and node-splitting behaviors are extremely hard to map to existing conflict resolution libraries.
Other editors, like Quill, map better to CRDT structures.
This is, of course, not your problem. But if you - or any editor author for that matter - ever end up creating another editor library, I'd love to work with you on adaptability to existing conflict resolution libraries. There is a lot to gain from being compatible to the wider ecosystem.
After all, we shouldn't expect our users to set up a tailor made backend for each single collaborative component.
Encrypted editing apps like Proton Mail Docs wouldn't be possible with your solution.
I'll be the first to admit that ProseMirror's change representation is a bit messy, but I don't think that's the root of the issue here. The problem is about generalized tree-structured document representation (as opposed to the flat-string model of Quill). I believe such a representation has value, and is the appropriate choice for a system like ProseMirror or its successor. Joining and splitting blocks is not a weird quirk of ProseMirror—it is a basic, essential editing operation. But there doesn't appear to be a known appropriate way to map that structure to a CRDT in a way that can properly express such splitting and joining. And that's fundamentally limiting the approach taken in y-prosemirror.
This is not a fundamental issue. Node splitting can be represented in CRDTs. It's just really hard to map correctly to ProseMirror as the merge logic is complex and bound to a schema.
I don't blame you for ProseMirror for being how it is. I'm just offering my feedback for your next editor library.
But Kevin, you never really answered the question of the article. Unless I need a truly-masterless p2p topology, why would I do all this stuff, including throw away editor intent around things like block split, just to use Yjs? prosemirror-collab and prosemirror-collab-commit already seem to do all the things the Yjs docs claim to do (unbounded offline writes that reconcile automatically, optimistic updates, tolerant of all kinds of failures), and they work with 100% fidelity to the underlying model. AFAICT, the only thing that you need Yjs for, is true p2p editing.
This is a serious question, and the question of the article. I am here to learn what you mean, please explain.
Yjs is about making things easy. It is a good abstraction to make anything collaborative (not everyone can implement something like prosemirror-collab).
I'd take the slight performance overhead any day if I get guaranteed syncs. Network protocols are not as reliable as you think they are. Detecting random drops of messages is hard. At scale, you are going to appreciate the sync guarantees.
prosemirror-collab doesn't give you offline editing either. Because, guess what - if there is no central server you can't edit the same doc from multiple tabs.
I once had a customer that accidentally deleted part of their database containing Yjs docs. Few of his users noticed, because their docs synced through y-indexeddb.
And it's fun. You can Yjs on anything. There is a company that syncs Ydocs through QR codes.
As a generic collab library, it does a very good job. CRDTs really are a fun thing to use. A lot of people feel that way.
If you want to use something else, that's totally fine! Write an article about how great prosemirror-collab is.
But Kevin, prosemirror-collab does give me offline editing. I use it literally every day, entirely without issue. I write offline on different devices, and whenever I come online, it all syncs up. No issues.
It does not give me p2p topology. Is that what you mean?
Hi Alex, I'm the author of prosemirror-collab. I agree with your point that CRDTs are not the solution they often claim to be, and that CRDT editor integrations (at least the ones for Yjs and Automerge) are often shockingly sloppy.
But, seeing how I've had several people who read your article write me asking about this miraculous collab implementation, I want to push back on the framing that ProseMirror's algorithm is 'simple' or '40 lines of code'. The whole document and change model in ProseMirror was designed to make something like prosemirror-collab possible, so there is a lot of complexity there. And it has a bunch of caveats itself—the server needs to hold on to changes as long as there may be clients to need to catch up, and if you have a huge amount of concurrent edits, the quadratic complexity of the step mapping can become costly, for example. It was designed to support rich text, document schemas, and at least a little bit of keeping intentions in mind when merging (it handles the example in the first post of your series better, for example), but it's not a silver bullet, and I'd hate for people to read this and go from thinking 'CRDT will solve my problems' to 'oh I need to switch to ProseMirror to solve my problems'.
Ok Marijn I understand. I'm sorry I caused you an inconvenience. Of course, I know that implementing server-side prosemirror-collab is not entirely without problems (since we have done it) and take your point, which is correct. If I was to do this again I'd find a different way to say this than "40 lines of code."
With that said... I do not agree it is not "simple." Or at least, I think it is about as simple as it can possibly be.
This is mostly accurate for CodeMirror version 5. But this article is talking about version 6, which was intentionally built in a way that avoids the problem.
That would explain the weird experience of having your open software used in Apple products — no one from the company ever talks to you. I'm a bit baffled how they deal with upstream bugs with this mentality. Not my problem, of course.
Just weird.
There's the classic `tail -f` bug where they didnt upstream the fix, but on their own open source release have an #ifdef APPLE or something to that effect.
A bit late with this reply, but the answer is: Make patches internally, to work around the problematic areas. Often done to avoid the legal necessity to contribute it back upstream. For example, hook code at runtime to jump out to Apple specific changes that are now not part of the OSS codebase directly.
reply