To be honest most of our tests for UI components are fairly abstract which has pros and cons. The way we test UIs is by creating multiple prototypes that differ in various ways (hidden vs. not hidden scrollbar), giving the user time to familiarize themselves with the prototype, and then asking them to perform a series of tasks. For example after asking them to familiarize themselves with an abstract spreadsheet like software, we might ask them to find a specific data item and we check how they go about it. Or we might give them a form and ask them to fill it out and we see their usage. Or we might present to them a large set of changing data, ask them to mentally keep track of any numbers that change to be between 100 and 200, and after 2 minutes we ask them how many numbers did they see and measure that, then we change the UI (increase information density) and have them repeat it and see what the effect is.
After we give this to a bunch of our users, for our specific business, we split users up according to a bunch of criteria including people who performed the task the fastest in terms of time, performed it using the fewest actions, in terms of precision, sometimes eye tracking is important, and a host of things.
Now the next thing I will admit people will find distasteful but for our industry it matters a lot... we then design the software according to how the "optimal" users accomplished the task. Sometimes it's possible most users accomplished the task one way, but the optimal users accomplished it another way... in many cases we will then design our software for the optimal users even if they are a minority.
For this case we were predominantly interested in information density and the scrollbar was taking up about 10% of the width of a table (on 96 DPI the scrollbar is I think around 15px and the window as a whole is about 150px). The optimal users, and the vast majority of users overall tasked to find data in a table as well as the ones who did best in terms of keeping track of changing numbers, don't move the mouse over a scrollbar and use it directly. I don't have all the details off the top of my head although I could ask the UX researcher about it if there's a lot of interest, but I seem to recall that the small number of users who did drag the scrollbar are "clumsier" and slower.
Since I work in quantitative finance, it's better for our business to design our software to appeal to the so called optimal user and try to find ways to get the majority of our users to adopt more optimal usage patterns. That means if we can get our users to prefer using more precise ways of using our software through familiarity, training, or other means, we'd rather get them to do that instead of trying to design our software to accommodate usage patterns that they are currently comfortable with, but that we measure as being suboptimal.
None of this is perfect, we don't always get it right, but we do care about it and put quite a lot of effort into it and have seen a lot of substantive improvements in our users that translate to substantial earnings for our trading firm.
Fascinating! Could you tell me more about who these optimal users are? Are they optimal because they are the most valued users in their firms? (Make the best trades or some such?) are they optimal as their opinion on the software matters a lot? Or something else entirely that I am failing to imagine
After we give this to a bunch of our users, for our specific business, we split users up according to a bunch of criteria including people who performed the task the fastest in terms of time, performed it using the fewest actions, in terms of precision, sometimes eye tracking is important, and a host of things.
Now the next thing I will admit people will find distasteful but for our industry it matters a lot... we then design the software according to how the "optimal" users accomplished the task. Sometimes it's possible most users accomplished the task one way, but the optimal users accomplished it another way... in many cases we will then design our software for the optimal users even if they are a minority.
For this case we were predominantly interested in information density and the scrollbar was taking up about 10% of the width of a table (on 96 DPI the scrollbar is I think around 15px and the window as a whole is about 150px). The optimal users, and the vast majority of users overall tasked to find data in a table as well as the ones who did best in terms of keeping track of changing numbers, don't move the mouse over a scrollbar and use it directly. I don't have all the details off the top of my head although I could ask the UX researcher about it if there's a lot of interest, but I seem to recall that the small number of users who did drag the scrollbar are "clumsier" and slower.
Since I work in quantitative finance, it's better for our business to design our software to appeal to the so called optimal user and try to find ways to get the majority of our users to adopt more optimal usage patterns. That means if we can get our users to prefer using more precise ways of using our software through familiarity, training, or other means, we'd rather get them to do that instead of trying to design our software to accommodate usage patterns that they are currently comfortable with, but that we measure as being suboptimal.
None of this is perfect, we don't always get it right, but we do care about it and put quite a lot of effort into it and have seen a lot of substantive improvements in our users that translate to substantial earnings for our trading firm.