I read you are going for an "Excel" type approach. I thought you were doing this to deal with non-local scope. I hope so because I think it is a good idea.
> are also pretty hard to represent and understand
Control flow is also really hard to represent but I don't think it is because of any aspect of the visual representation or core technology. You can remove coding from programming but you can't remove critical thinking and problem solving from programming: which is basically what programming is.
Control flow is critical thinking and problem solving. That is why when you "presented our prototype to a small number of non-programmers and sat back to watch the magic. To our horror, not a single one of them could figure out what the simple example program did or how it worked."
It is because there are inherent mental models that we use as tools to solve problems and think critically. People need to be given a visual representation of control flow in a way they are familiar with. I'm guessing you guys have done an amazing job (I've seen light table).
Frankly, I have a lot of experience in this space. I've spent ~5 to 6 years studying it and building out tools. I recognize a lot of the pains you are going through because I've been there. I would be more than happy to chat with you about it. I am totally stoked about Eve and would like to help.
> understanding code so it makes sense to default to visualisations that optimise for the latter.
I may be totally reading this wrong but you should not default visualisations to represent a given program. In my opinion, this is a mistake that all VPLish type things have made (as you said, it is a totally overloaded word). You can easily represent a given small part of a program in any domain you like: using the same underlying program. It could look like a mathematical equation to a mathematician, textual code to a programmer or process flow abstractions to someone in logistics.
This is one of the great strengths of hierarchical type programming languages: the same code can be represented using multiple visual abstractions tailored to the user.
This is why I totally agree with this statement "The environment should be optimised for the way that people use it, not the way that we think it should be used."
but think you should reconsider this "We should observe users and collect data to discover common actions and workflows." There are no common actions or assumptions you can make when going visual because there are so many different ways people approach solving a problem.
Programmers are only kind of aligned because we all go through this grinder that requires us to fight with compilers and figure out error messages and deal with syntax and so on. So, we all kinda mentally solve problems in the same way (and by kinda, it isn't like close but a lot closer than say an accountant and a painter solve their problems).
I hope this isn't too long winded but I really do have a lot of knowledge in this space.
> I thought you were doing this to deal with non-local scope.
Non-local data-flow (you can reference any table in a query) but only local control-flow (at the same level as excels 'if' statement) and no scoping (references are absolute and global - they don't change depending on where you are in the program).
> Control flow is critical thinking and problem solving.
Critical thinking and problem solving do not require programming with control flow and they are certainly not limited to programmers (you didn't claim the latter, but it is a common assertion on hn and it needs to die). Here are some of the people we've talked to about their programming needs, both for Eve and in past work:
* a radiologist with phds in both medicine and nuclear physics
* an ex- radar engineer who now manages and schedules multi-million dollar aerospace projects
* a group of traders at a prestigious prop trading company
These people are certainly capable of critical thinking, abstraction, problem solving etc. All of them build complex programs in Excel and other tools. None of them have got the hang of traditional programming. I've seen research that argues that supplying task-specific primitives and ensuring early success on the learning curve are both vital. I suspect that the distinction between traditional control-flow-heavy programming languages and static-data-flow end-user languages plays a big part in that learning curve.
That's not to say that non-local control-flow is evil or that it can't be taught, just that it seems to put a big hump in the learning curve and we can usually get away without it.
> It could look like a mathematical equation to a mathematician, textual code to a programmer or process flow abstractions to someone in logistics.
When I say visualisation I include all of the above, not just graphical representations. For example, our default table view is a grid of cells and our default rule view is a mixture of graphical meta-data and textual formulae.
> There are no common actions or assumptions you can make when going visual because there are so many different ways people approach solving a problem.
I think you are thinking much higher level than I am. There are lots of common actions that every programmer I know does every day. For example, I run my program in the browser, check the console for exceptions, put a breakpoint on the line where the exception happened and try to cause the exception to happen again so I can observe the local state, often having to click through the breakpoint multiple times to get the actual error case. In any sane environment (eg Common Lisp) that process would be optimised to: check the console for exceptions, click the 'open debugger here' button next to the exception.
> Non-local data-flow (you can reference any table in a query) but only local control-flow (at the same level as excels 'if' statement) and no scoping (references are absolute and global - they don't change depending on where you are in the program).
I'm quit surprised by this. In data-flow programming (e.g. FRP), non-local data-flow is a big no no, while non-local non-contiguous control-flow rules the day but is completely encapsulated (programmers only manipulate data flow via wiring, control flow is inferred).
> I suspect that the distinction between traditional control-flow-heavy programming languages and static-data-flow end-user languages plays a big part in that learning curve.
This is a guess, but I don't really see it. A cook can definitely follow a recipe, or even write one for others to read, which involves lots of control flow. We are totally steeped "in time" and our communications often make references to sequenced time-stepped actions.
Spreadsheets are great in accounting and were invented way before we had computers. Math and logic are also nice, and very specialized for the educated, but they came way after we developed our skills for natural language (which was the original OOP). It is quite difficult to say why programming is hard, and what makes it easy.
> Critical thinking and problem solving do not require programming with control flow
Ya bad choice of words. Control flow requires critical thinking and problem solving (but critical thinking and problem solving is not always about control flow).
Hmm. Though, I guess if you go way deep, every decision we make is based on two or more options.
> These people are certainly capable of critical thinking, abstraction, problem solving etc.
Ya. But the way they think critically, their internal mental model, might be totally different than how you are I think. So, we all may come to the exact same conclusion. Just through a very different thought process using different abstractions to represent that critical thinking.
All great points and I agree with them.
> Pretty much the only hierarchical example that comes to mind is Scratch.
I tend to lean towards Sean's post: https://news.ycombinator.com/item?id=8468838.
> non-local scope and control flow
I read you are going for an "Excel" type approach. I thought you were doing this to deal with non-local scope. I hope so because I think it is a good idea.
> are also pretty hard to represent and understand
Control flow is also really hard to represent but I don't think it is because of any aspect of the visual representation or core technology. You can remove coding from programming but you can't remove critical thinking and problem solving from programming: which is basically what programming is.
Control flow is critical thinking and problem solving. That is why when you "presented our prototype to a small number of non-programmers and sat back to watch the magic. To our horror, not a single one of them could figure out what the simple example program did or how it worked."
It is because there are inherent mental models that we use as tools to solve problems and think critically. People need to be given a visual representation of control flow in a way they are familiar with. I'm guessing you guys have done an amazing job (I've seen light table).
Frankly, I have a lot of experience in this space. I've spent ~5 to 6 years studying it and building out tools. I recognize a lot of the pains you are going through because I've been there. I would be more than happy to chat with you about it. I am totally stoked about Eve and would like to help.
> understanding code so it makes sense to default to visualisations that optimise for the latter.
I may be totally reading this wrong but you should not default visualisations to represent a given program. In my opinion, this is a mistake that all VPLish type things have made (as you said, it is a totally overloaded word). You can easily represent a given small part of a program in any domain you like: using the same underlying program. It could look like a mathematical equation to a mathematician, textual code to a programmer or process flow abstractions to someone in logistics.
This is one of the great strengths of hierarchical type programming languages: the same code can be represented using multiple visual abstractions tailored to the user.
This is why I totally agree with this statement "The environment should be optimised for the way that people use it, not the way that we think it should be used."
but think you should reconsider this "We should observe users and collect data to discover common actions and workflows." There are no common actions or assumptions you can make when going visual because there are so many different ways people approach solving a problem.
Programmers are only kind of aligned because we all go through this grinder that requires us to fight with compilers and figure out error messages and deal with syntax and so on. So, we all kinda mentally solve problems in the same way (and by kinda, it isn't like close but a lot closer than say an accountant and a painter solve their problems).
I hope this isn't too long winded but I really do have a lot of knowledge in this space.