Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Rust vs. C++ Benchmark Game (debian.org)
51 points by tete on Oct 19, 2014 | hide | past | favorite | 48 comments


As pointed out in the reddit thread, this may be more a measure of the llvm backend used by rust to the gcc backend. A more accurate comparison would be against clang.


It doesn't matter much what the backends are, it's important to be able to demonstrate the effective implementation of the problems posed by the tasks being benchmarked. Especially for the new language. Of course, if your language is designed as to have the limited chances of being universaly fast, it's another story. But Rust is designed to be able to reach at least the speeds of C and also allow for a more modern approach. That's the real worth of the contributions to the "benchmark game:" demonstrating the capabilites of the language, even if it's just on some of all of possible tasks.

See also Rust vs Haskell: http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t...

And Rust vs Go: http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t...


It's not that easy to implement a language that's expressive with this kind of performance, even if you use llvm. Kudos to the Rust guys!


The Mono C# programs also use llvm ;-)

A more accurate comparison? A different comparison would be against clang.

The benchmarks game might well be described as "a good starting point" http://algs4.cs.princeton.edu/faq/ but let's not confuse that with some kind of definitive final analysis.

(Edit: Say what you disagree with, don't just down vote.)


I was under the impression that clang was still, generally, slightly slower than GCC for C++. Perhaps this has changed in recent years (as clang and llvm have been seeing tons of development effort and funding).


I've only started to investigate rust for server side programming ( websocket in particular) but saw many discussions about the limitations rust has regarding parallel programming. Yet, iirc servo did parallel rendering of html page, so that seems a contradiction. Anyone here knows about that ?

Specifically, i'm currently hesitating between golang and rust for coding a small Websocket chat server. I'm ready to use beta code because rust seems more promising to me in the long term than golang, but not if the language itself has severe fundamental limitations on server side programming.


Parallel rendering of an HTML page means an improvement of one renderer to 2 or more renderers. Writing a web server means possibly handling thousands to millions of concurrent requests, which isn't really doable with Rusts 1:1 threading model.

But the reality of concurrency and hardware is that all we have at the end of the day are threads anyway. So in Rust you could have a thread running an event-loop that serves your web sockets. People are building these sorts of abstractions for Rust but they are a work in progress as far as I know. MIO[0] is one such example.

Rust is a programming language for systems programming and "server side programming" is just a subset of systems programming.

0 - https://github.com/carllerche/mio


Thanks for your enlightening response. Wouldn't building event loop based server require non-blocking i/o libraries ? Does rust have that ?


To the best of my knowledge Rust's stdlib doesn't have non-blocking libraries yet. But since Rust is "close to the metal" you can easily use any of the non-blocking system calls in glibc. Mio (https://github.com/carllerche/mio#features) seems to be doing just that.


That's exactly what they require. mio is one such example, I am working on another implementation.


Depends on the kinds of parallelism you want. Rust has built-in support for parallelization, but tasks are mapped 1:1 to threads. Currently M:N threading is also supported, but that will be removed. Consequently, applications requiring lots of tasks (including highly-concurrent websocket servers) won't do well with the built-in mechanism.

Context: https://news.ycombinator.com/item?id=8459888


Can someone explain me why that is? It just seems like it is the dumbest thing in the world to me. The OS may be good at scheduling, but why not have a built in library providing you with an alternative? Native threads are usually really slow to start and stop, because they have high memory use and even modern OSs can only handle on the order of 10000 of them.

When I saw this decision my first reaction was "Well, that was a promising language. Shame about that." I can't be the only one...


This is actually not the case on Linux. Starting up a thread is really fast. One of Rust's contributors found that it was just as fast at spawning threads as Rust was at spawning green threads when both were pinned to one core.

Additionally, for a variety of reasons Rust's tasks were not smaller than OS threads, because they need to be bigger for a lot of things (calling into C, for example) and segmented stacks were found to be much too slow. Go gets around these problems by allocating a new stack when the current one runs out of space, but doing that properly requires being able to trace all the pointers into the stack to figure out where to transfer them--in other words, it requires a garbage collector :) So that solution wasn't really viable for Rust.

Finally, having to support both concurrency models required Rust to make significant sacrifices in its libraries, in both complexity and performance. Go was able to make the decision to focus entirely on green threading, but Rust did not take that approach, so it was stuck in a "worst of both worlds" situation.

It is possible that there is a better solution that avoids all these problems, but it will have to be found outside the main Rust tree.


A systems programming language without access to a systems feature isn't much of a systems language.

Also, it doesn't _preclude_ green threads. As others have mentioned, Rust is low-level enough that all of this is a library issue, not a language issue. You can write whatever concurrency primitives you need, and they'll be no less privileged as the ones the standard library gives you.


Well, yes, but anything that's merely a convenience and not in the standard library won't see much use, as I've understood programmer nature.


Rust's standard library will be much less privileged than in other languages, due to Cargo existing from the start. It's going to be much more slim than most, we've already pulled lots of things out into their own packages. Also, given that only part of the standard library will be stable at 1.0.


Sounds like Go is better for web servers because of better concurrency support for millions of users, while Rust is better suited for desktop apps that take advantage of all the PC's cores?


Actually, Rust is much closer to the system than Go is making it possible to implement lighter weight web services using Rust than Go. Rust features like not having a GC allows for much more predictable performance.

Go's concurrency features are nice if it's your kind of thing, but will get in your way when hitting that "last mile" of performance. For example, if you were to implement something like haproxy, you would probably want to use Rust over Go for performance.


If you want true lightweight threads, you need haskell or erlang.


Green threads (M:N) do seem to be available as a library though: http://doc.rust-lang.org/green/



Its being removed from the standard library. As I understand it will still be maintained as a separate library.


It's not being removed, it's being moved from the core of the language to an external library.



Although not directly related to your use-case, I wanted to mention that we received a pull request [1] implementing a test of the Iron framework. Expect to see this in Round 10 of our framework benchmarks project.

[1] https://github.com/TechEmpower/FrameworkBenchmarks/pull/1101


Rust really interest me. However I want them to get to 1.0 before I can evaluate whether or not to use Rust in my app. Currently I am using Go and it stable for my needs. Rust could make me switch once they get to 1.0 and people start writing more projects in Rust.

I want to add that stability in deciding to use a language in an application is very important. How good the standard library is and how much traction the language is gaining in projects and particularly open source projects is also important.


I agree. However sometimes it's good to do tiny side projects in a language. This allows you to learn, see if it's really nice, help the language and its ecosystem to develop and become stable.

For use outside of production it seems okay. There are two Sinatra/Flask/Express style web frameworks (Nickel and Iron), a browser engine (Servo), a game engine (Piston) actively developed. So if you have interest in any of those or interest in implementing something like that you might be able to make your project shine. Also really nice on your portfolio if you ever want to get a job where Rust is already in use.

Or one just has fun with it. :D


Agreed. I would do a small project just for fun.


Rust seems to do worse on the string-y tasks (fannkuch, k-nucleotides, regex-dna). A lot if you look at single-thread performance. Does it handle strings differently from C++?

(From what I understood, Rust gives you similar control over memory allocation as [a sane subset of] C++, so intuitively it should come out relatively close)


With respect to regex-dna, there is still substantial room for improvement in the Rust library. It's currently using a full NFA simulation (albeit specialized to native Rust code with the `regex!` macro), while the competing C++ library (RE2) has been tricked out by Russ Cox with a parallel DFA implementation. :-)



Incidentally -- reddit supports https by default now, so you don't need to use pay.reddit.com anymore.


Very impressive. What we need now is some sort of a good mainstream all that uses it to spread its popularity..

How is rust when it comes to deployment. C++ is a real pain to deploy for large projects. Compilation setup is also a massive headache in c++.


With Cargo, getting a nice repeatable build is much, much easier. There's still work to be done on this front, but I'm _incredibly_ excited by this RFC: https://github.com/rust-lang/rfcs/pull/403


This issue on the Rust repo is very relevant: https://github.com/rust-lang/rust/issues/18085


The server is down for me.


Same, but google had the time to cache it.

https://webcache.googleusercontent.com/search?q=cache%3Ahttp...


And up, presumably once this discussion thread moved off the front page ;-)


While I'm not a huge fan of benchmarks, this is quite impressive, given the age of Rust. C++ does better memory-wise in pretty much all benchmarks.


The main problem is the large (multi-megabytes) runtime that Rust currently is forced to load by default. This is disappearing.


Don't read too much into those memory use measurements - they are just enough to suggest programmers might have made a space/time trade-off, nothing more.


Considering that memory management is one of the headline features of Rust, memory use is certainly important factor to analyze.


That may well be, but the benchmarks game measurements are not a good basis for that analysis.


Actually LLVM vs GCC backend benchmark.


Is there a Rust tool chain based on GCC?


Not with full up to date language support, but there's a mature c++ toolchain on top of llvm


I wish there was a D comparison. It seems to have similar goals.


Make your wish reality - take the program source code and the measurement scripts and publish your own measurements.

http://benchmarksgame.alioth.debian.org/play.html#languagex

If you have a linux install the benchmarks game scripts should work fine, and you'd probably get help from the D community --

http://forum.dlang.org/thread/lv7h1r$mg3$2@digitalmars.com?p...




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: