At Manifest AI we have just released our open-source CUDA kernels to implement Symmetric Power Transformers, as described in our paper from back in August:
Since this is a variant of a linear attention, you get linear cost when training (as opposed to quadratic in regular attention), and constant when doing inference. This is especially attractive for longer contexts!
Have a look and play with it -- and of course contributions are very welcome! It's an early alpha!
I chose that word on purpose because of both its semantic and emotional power. Besides, I'm pretty happy that all the critics have been focused on either using the term "cancer" or the swearing at the end, so I guess my points are valid to almost everyone, and you just didn't like the way I expressed them. That sound ok.
I don't think that's Rails, or Ruby or whatever paritcular tech we're discussing. I think that's a common trait in human communities, and I'm worried about the one I feel mine, but I'm sure everyone can see a part of their community reflected on my post.
Postel's law (gently reformulated) is important when you have one-to-many communication. "be conservative in what you send, be liberal in what you accept".
Heh, I really liked the analogy with Postel's law, and I guess you're right. The main criticising I've received about the blog post has been about the swearing, so I guess I could skip that next time.
I didn't need to grow old to watch relatives and friends fighting cancer. I know how serious it is, and I really admire that you're working in that particular field. That said, I used the metaphor on purpose, because of its power, because I think it's a perfectly valid analogy.
I still think you lack perspective -- maybe standing "back from the fire" a bit. I would never, ever use that word to express anything about a lousy programming topic.
Agreed, but more importantly: I would never use that word unless I was trying to start a fight. And trying to use the word to stop a fight is just backwards.
(Like all rules for writing, this one has exceptions: For example, if it's in the service of a great pun, like:
I didn't mean to give a bad impression of Ruby. I think (and many people have pointed out) that the problems I outline are valid for many developer communities, if not all of them. In the end, those communities are human, and share a good amount of our defects.
You do the Ruby community that you seek to improve a disservice when you use a title that troll baitish. In fairness, you're not the only one I see doing this. There are many bloggers who use the same tactic. I have occasionally been guilty of this in the past though I cautiously avoid the practice now.
More specifically, have you attended any regional Ruby conferences? It is almost certainly a community unto itself: those developers who take the time away from their jobs and families to attempt to better themselves by learning from others. The (Ruby) conference-going community is demonstrably better than at least two out of your three arguments.
As for the Pareto principle, it seems unfair to paint the "Ruby community" with such a broad brush while then going on to say the same of the OSS community at large. It's a universal problem. It should not have been raised in the context of your post.
Finally, you provided no constructive advice in your blog post. How was this post supposed to be helpful?
I for one am tired of the negativity.
This post, by providing only destructive criticism, ironically only serves to exacerbate the public image that some paint Rubyists with.
I am currently organizing the Barcelona Ruby Conference, and have attended many other conferences in Europe. I know a lot of people put much effort in doing things right. But that's not the majority of the community, unfortunately. I also agree that the Pareto principle (and many other points raised in my post) are a universal problem, not just Ruby's.
I did provide constructive advice. If you read the post thoroughly, you can see a call to improve our learning manners, stay away from hot trends, and even a simple pattern you can use in your day to day to compare technologies, or to understand better some debates (like threads-processes-reactors).
Did it really not occur to you that an overly-dramatic lament, with f-bombs and references to cancer, might give a bad impression? There is really very little excuse for Rubyists to have so little awareness, especially if they hope to overcome the stigma they are so worried about...
I'm really proud that so far no one has argued about any point I make in the article, but rather about the "cancer" reference and the swearing at the end. I guess that means I happened to write a pretty damn good article.
I believe there are plenty of problems with the points of view expressed in your original post. Not the least of which is that the Rails community was built with carefully crafted factional mentality. People called it "opinionated" software and if you didn't like it, well, "F*ck you."
This is how Rails was marketed, by splitting people into camps: Us (cool, indie hackers) vs. Them (corporate stiffs). Fervent conflict was intentionally built into the Rails culture from the very start and complaining about it at this point seems slightly after the fact.
Which leads me to my next issue: the Pareto rule. This seems like a silly thing to be complaining about because it's also the reason that there are four revisions of the Rails book and countless other money-makers (books, confs, Peepcodes, etc).
I'm sure you can see why congratulating yourself may seem a little arrogant and perhaps give people the wrong impression of your "cancer-ridden" community.
Please, explain to me the difference between "following trends blindly" and "learning from older developers' experience"?
Because both of them look remarkably like cargo-culting appeals to authority.
Folks constantly rant about the reinvention of the wheel, but i think they're missing the point. Everyone talks about how wonderful it is that digital and web-based technologies allow for rapid prototyping and failing fast which in turn results in the ability to experiment.
And then they turn around and criticize people for experimenting accusing them of not learning from their elders or whatever. Learning is a process. You learn by doing. Even if you have to rediscover what other folks have discovered in the past.
Experimenting is good. I do it all the time. Just take a look at my repos on Github. Just a few of them are conceptually (and arguably) "new", while the others are just exercises to learn about new things. Experimenting about things, even about the basic ones, helps me a lot in my learning process. At a personal level.
But when you watch the bigger picture, with companies actually building software for clients that trust them, then choosing technology is a delicate, non-trivial matter. It's important, and I mean in actual dollars (or euros). There is where the expertise of older developers, people who have founded, sold or shut down profitable and unprofitable companies, who have worked in a gazilion of projects and made a lot of expensive mistakes, there is where that expertise comes to play. They can enlighten the sometimes childish, faction-like debates about technology that keep repeating themselves over and over. That can free us to think about new problems and new things, based on past experience. That's how science advances, and that's how software development advances too, in my opinion.
Having swearing as a personal signature is like having a tattoo. It doesn't make you any "cooler", and everyone does it so it's not much of a signature.
It's not that it's offensive, it just says you are unable to convey the strength of your argument without the vulgarities. You lose critical (logical) thinkers when you resort to crass language; primarily because it says your argument relies on emotion, not logic.
In a discipline such as computer science, appeals to emotion are inherently untrustworthy and distracting. A reader shouldn't have to shift through pandering rhetoric to find the kernel of a supportable argument.
https://manifestai.com/articles/symmetric-power-transformers...
Since this is a variant of a linear attention, you get linear cost when training (as opposed to quadratic in regular attention), and constant when doing inference. This is especially attractive for longer contexts!
Have a look and play with it -- and of course contributions are very welcome! It's an early alpha!