Kaggle is pretty awesome and I hope to use it myself one day.

The article makes a good point that a lot of the most interesting data sets just aren't that big. This is worth pointing out because I've noticed that there has been some conflating of "Big Data" and "Data Science." These two buzzwords are related but one does not imply the other. Big data is just that, working with really large data sets (I think with the implication that the data are unstructured).

Data science can include Big data as well as little data. It also seems to cover both structured and unstructured data.

(Disclaimer: I'm the "accidental scientist" profiled in this article.)

Yes I've noticed this as well. People are often asking me for my opinion about Big Data, treating it as a synonym for Data Science. Since the journalist at Wired had recently written a couple of articles about Hadoop, I took this opportunity to explain to him how data size and data value are not necessarily directly related. I think he did a great job in the piece of getting this point across.

I think you'll find that Hadoop is often used in preparing the data into bite-size chunks for use in Kaggle contests.

Also, sometimes the contest winners do win by fetching more data related data than Kaggle provides. This happened in the Wikipedia participating-prediction contest, for example.

Thanks for Fastmail!

Aren't the prizes on offer extraordinarily low for the accomplishments desired?

For example to "Develop new models to accurately predict the market response to large trades." is valued at $10,000.

If a person were smart enough to "develop new models to accurately predict the market response to large trades", they would have to be very foolish to sell it for only $10K.

Assuming it was real and actually worked.

This is the part that really irks me.

It wouldn't bother me so much if I could test my model and get a score, without having to turn over the code in the event I won - of course without accepting the cash either.