Hacker Newsnew | past | comments | ask | show | jobs | submit | jrallison's commentslogin

"Just ignore the lidar in situations where lidar and vision conflict and you have the original decision."

Then why have lidar? If it's only purpose is to agree with vision or you just discard it, then why even have it?


I’m arguing against the idea that conflicting sensors is an issue. Clearly you can do no worse with sensors A and B than with just sensor A.

You might want to not have sensor B to lower the cost (as I suggested). Or you might want to have both sensors A and B because the provide some synergies (as gkop pointed out, better range of view, increased confidence, etc.). But it’s definitely not enough to just argue that the sensors may conflict.


Hypothetically, 1. Improved confidence when both signals agree 2. Improved field and range of view



It's totally legal everywhere I've visited (and I don't know a country where it's illegal).

Work visas are a thing because countries want to control the ability for foreigners to compete with locals for local jobs. Canada's definition of work (http://www.cic.gc.ca/english/resources/tools/temp/work/about...):

"What kind of activities are not considered to be “work”?

An activity which does not really ‘take away’ from opportunities for Canadians or permanent residents to gain employment or experience in the workplace is not “work” for the purposes of the definition."

Working for my own US company isn't competitive to local workers.

Every country I've visited has a similar definition, though I can't say for sure if every country does, so it's best to check before attempting to go.


Erm. From the source you just cited

'"Work" is defined in the Regulations as an activity for which wages are paid or commission is earned'

On another Canadian immigration page:

"Most visitors to Canada cannot work or study in Canada without a permit."

http://www.cic.gc.ca/english/visit/extend-stay.asp

It's problematic that most countries immigration websites don't explicitly point out that working for a company abroad or for yourself is not permitted on visitor visas, but if you write to their respective embassies then you will find this to be the case.


I have, quite often. :)

I've always gotten the answer from officials "if you aren't competing with local workers, you're fine". As always, depends on the country. Further down the source I just cited:

"Examples of activities for which a person would not normally be remunerated or which would not compete directly with Canadian citizens or Permanent Residents in the Canadian labour market and which would normally be part-time or incidental to the reason that the person is in Canada include, but are not limited to:

long distance (by telephone or internet) work done by a temporary resident whose employer is outside Canada and who is remunerated from outside Canada;"

It's the definition of "work", whether you're competing with locals, and whether you're being paid by a Canadian company.

Many countries don't have this clearly documented, as until very recently (last 10-15 years), the only real "work" was competitive with local citizens/residents. All work visa documentation is written with this implied definition. Many countries are starting to call this out as an exception (like Canada above, Thailand recently, etc, etc).

Until it's clear for a particular country, it's best to talk to an official.


Awesome that Canada has clarified that. The first I've seen.

Thailand's laws however explicitly don't. Work is defined very broadly. I've never seen it clarified beyond the report from Chiang Mai a couple of years back. Unfortunately the guy didn't have the authority to make the statement.


that means 100% of my work is done in Iceland. that's an awesome determination!


Did almost exactly this for all of 2014 (part of 2015). With a US citizenship, and visited 10 countries for a month+.

Working remotely for a foreign corporation generally isn't against the terms of a tourist visa (consult the specific country you're visiting). All officials I asked had no problem with it, from South America/Asia/Europe (Germany will actually let you stay longer, up to 2 years, if you're a remote worker/freelancer). You do need a work visa if you're planning to do work for a company that exists locally (competing with the locals, etc).

It was hugely income tax advantageous. There's a huge US tax credit if you aren't living in the US (your first $95-120k tax free), and I wasn't staying long enough in any country to meet their reporting requirements.


What types of places did you stay in?

I had forgotten the income tax advantage for expatriates. Makes it even more compelling.


Mostly Airbnb's, an occasional hostel/hotel if I didn't plan ahead.

My routine when arriving in a new country would be to quickly find a place to stay and a coworking space, then explore the city/country when I wasn't working.


Does this mean you did not have to file for local income tax in any of your host countries?


It's not guaranteed to be atomic in failure conditions. Also, the biggest difference between Redis and Geode would be the "distributed" part, which involves maintaining these guarantees across a cluster of machines (which Redis demonstratingly doesn't do).


That's pretty hand-wavey. All databases can be made to lose all guarantees under failure conditions.


Sometimes, just maybe, the priority isn't to build something from scratch (however simple), but rather press a few buttons and move onto something more important.


Don't know much about RAMCloud, but from the description:

"Many more issues remain, such as whether we can provide higher-level features such as secondary indexes and multiple-object transactions without sacrificing the latency or scalability of the system. We are currently exploring several of these issues."

Sounds it doesn't provide multi-key ACID transactions at the very least.


Yes, It looks like it doesn't support multi-key ACID transactions. As per:

https://ramcloud.atlassian.net/wiki/display/RAM/Deciding+Whe...

> If your application requires the higher-level data model features of a relational database, such as secondary indexes and transactions spanning multiple objects, then it may be difficult to get that application running on RAMCloud. The current RAMCloud data model is a key-value store with a few extension (such as conditional writes, table enumeration, and multi-read/multi-write operations); it does not yet support secondary indexes and multi-object transactions. Most of these operations can be implemented on top of the existing RAMCloud features, but it will take extra work and these operations will not run as fast as the built-in RAMCloud operations. We are currently working on adding higher-level features to RAMCloud.

Still. Aside from that, it is free, and open source and gives hundred times better latency?


> Still. Aside from that, it is free, and open source and gives hundred times better latency?

It doesn't have the same guarantees. If you need those guarantees, then there's no comparison and it doesn't make sense to compare latencies.


I'm not so sure. Have the latencies claimed by FoundationDB been measured on multi-key transactions?

And if you need these guaranties, it seems like "Most of these operations can be implemented on top of the existing RAMCloud features". I would expect it to be possible to implement that and stay well withing the latency budget, provided that a single-key durable write is 100 times faster.


All of the transactions in the test noted in the blog are multi-key transactions. (20 key updates)


When claiming the 14.4MHz number, is that per 20-key update? Or a single key update in the 20-key update? And what's the latency on the 20-key update?


The writes we are measuring are individual keys, modified atomically in transactions of 20 keys at a time. So this test is doing 14 million writes per second as 720 thousand transactions per second.


Multi-key transactions means coordination. Coordination is not cheap. You'll be paying a hefty cost somewhere.


Absolutely. Choosing the proper data model based on your access patterns is one of the best ways of keeping FDB performance high by reducing the likelihood of contention.


We've been using FoundationDB in production for about 10 months now. It's really been a game changer for us.

We continue to use it for more and more data access patterns which require strong consistency guarantees.

We currently store ~2 terabytes of data in a 12 node FDB cluster. It's rock solid and comes out of the box with great tooling.

Excited about this release! My only regret is I didn't find it sooner :)


Why do you need a 12 node cluster for 2 TB of data?


With a replication factor of 2 (for fault tolerance), it's ~4.5 TB.

FoundationDB requires SSD drives, which the best we can get efficiently in our data center is ~670 GB of usable space (3x480GB raided).

12x670 = 8040 GB

We try to keep extra space available for node failures (FDB will immediately start replicating addition data if it notices data with less than the configured number of replicas).

Our dataset currently grows at a decent pace, so we over provision a bit as well.


Why do you use RAID on your nodes if you have an RF==2?


A few reasons off the top of my head:

1) We're still interested in the nodes being as reliable as they can be. With RAID 5, we need two simultaneous disk failures to brick a node. With RAID 0 (to increase usable disk space), any of the 3 disks can brick the node.

Even with 12 nodes and RF of 2, an order of magnitude more node failures would be more likely to disrupt our service. Perhaps this makes more sense in a larger cluster with a higher RF factor?

2) We're using commodity hardware from a dedicated server provider, which uses RAID 1 or 5 configurations exclusively by default. We haven't reached the point where we felt that was enough of an issue or gain to investigate changing it.

3) Having more CPUs/disks in the cluster (12 nodes rather than cramming all data on 6) can be a good thing... as FDB scales fairly linearly.


I think you misinterpreted what I said. I explained it more clearly below. I suggested using the same 12 nodes but putting each one as a RAID 0, which would get you more reliability and more storage for the same cost. In your current config, two dead disks possibly bricks the system -- in the config I propose, you'd need four dead disks before anyone noticed.

What I'm suggesting is that you think of the cluster more holistically, since I assume your goal is a reliable cluster, not reliable nodes. As a nice bonus you get more "free" disk space.


That was my interpretation of your comment, but I'm still not sure I follow. In my understanding, by using RAID 0, any single disk failure will brick a node. Each node would then have 3 disks that are ticking time bombs (multiplying the failure rate by 3). How is that more reliable?

In RAID 5, I can have 1 disk failure on a node with no problem. 2 disk failures on the same node, and I only lose 1 node of my 12 node cluster (I.E. cluster is fine). I can also theoretically lose 12 (1 on each node) + 2*(RF-1) disks, and gracefully repair the situation with 0 interruption.

What's the benefit of RAID 0 other than increased usable disk space and perhaps write performance? It seems you're decreasing reliability significantly for those gains.


Perhaps to be able to recover without stressing other nodes? If a disk fails, your reads suddenly all go to 1 node in the replicaset. If then that same node also has to supply the data for the fresh harddrive, it might interfere with the read-performance and/or take a long time to restore full redundancy.

Also, in theory the raid 5 configuration would have faster reads.


But then you might as well just use an RF of 3. You get all the benefits you listed above, plus more storage (5.7TB vs 4.5TB), and less configuration hassle. And greater horizontal scalability.

And a RAID 5 will never be faster than a RAID 0 or a JBOD. :)


FDB has a substantial per-node license cost. It makes sense to beef up the hardware on individual nodes as much as possible first, then scale out to more nodes.

https://foundationdb.com/pricing


It would be unfortunate if their pricing model drove poor architectural decisions, but in this case that doesn't apply.

I suggested getting more storage by using the same number of nodes differently.


Ah, I mis-read your post, and was thinking you were suggesting more nodes rather than more disks. Sorry for that.

At any rate, I have quite a distaste for RAID-5, so I agree that it would be preferable to go with the higher RF instead.

Even more preferable to me would be to not use RAID-0, if FDB can just use multiple drive partitions as storage directly.


Yeah, JBOD is preferable to RAID if your workload supports it. Less overhead and less things to break.


It's not hypothetical. Twitter has significant revenue. When revenue is generated from eyeballs (ads), then yes, number of users and engagement is a valuable metric (yet not the only one).


The problem is Twitter has even more significant expenses. What worries the bond rating agencies is if this will ever change.


Of course. I wasn't challenging the bond rating, just the parent's notion that user engagement isn't a valuable metric for Twitter.


User engagement is not valuable to a bond holder since they see none of the upside. The only thing they care about is can twitter pay on time.


engagement -> ad views -> revenue -> bond payment


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: