"We've done tons of user testing on this, and it turns out it doesn't work. Touch surfaces don't want to be vertical. It gives great demo but after a short period of time, you start to fatigue and after an extended period of time, your arm wants to fall off. it doesn't work, it's ergonomically terrible."<p>-Steve Jobs, 2010<p><a href="https://www.businessinsider.com/steve-jobs-touch-screen-mac-2010-10" rel="nofollow">https://www.businessinsider.com/steve-jobs-touch-screen-mac-...</a>
“Who wants a stylus? You have to get ’em and put ’em away, and you lose ’em. Yuck. Nobody wants a stylus. So let’s not use a stylus. We’re going to use the best pointing device in the world. We’re going to use a pointing device that we’re all born with—born with ten of them. We’re going to use our fingers.”<p>— Steve Jobs, 2007<p>(8 years before the introduction of the Apple Pencil)
When Steve Jobs said that, he was talking about a stylus as a main or even only input device. And he is still right about it. The Apple Pencil for the iPad never was a main input device but an alternative.
The Pencil isn’t a stylus. At least not primarily. It’s designed for freehand. This is probably why they insisted on it charging via Lightning by removing its end cap. They didn’t want people getting ideas.
I wonder what people will say when Apple releases the touch screen MacBook later this year then.
I had several PCs with touch screen and this absolutely true. Even intermittent use is not something I did, it’s just too inconvenient to ever become a habit, so the few times it’d be great, I don’t think about it being there because it’s not in my active list of affordances.
I have used a couple of laptops with touchscreens, and the experience was awful, even with the latest technology. If Apple gave us an iPhone or iPad-quality touchscreen on MacBooks, I am 100% sure the experience would be perfect.
Nobody forces you to use touchscreen exclusively?
If that's the case then the interface will remain a compromise that has to work for both point and touch, and ends up being suboptimal for both. Touch input necessitates bigger hit targets, lots of additional negative space around important buttons rather than groups, and a much lower information density on anything that has a click/touch handler like a list in order to avoid accidental presses. Apps need to be written with a touch UI in mind to work well.
Yeah, but this was also strategically in Apple's interest to sell the iPads with nerfed up iPad OS as a separate line up. I love Steve Jobs and all, but this did NOT age well. The millions of people using Surface and Surface Pro will absolutely disagree with this take.
I wouldn't want a touchscreen MBP even if it was free, anyone else feel similar?<p>I don't get the draw - we already optimize for keyboard commands to avoid living our fingers over to a touchpad. Why would I want to start clicking on my screen?<p>If you're using your computer for tasks (rather than entertainment) and you're not a visual designer, I don't get why Apple are apparently going to be putting them into the new MBP line later this year.
I feel like the point isn't "there should be a touch screen MacBook" but more "holy shit we simulated a working touch screen by looking at reflections coming off the glass, isn't that cool".
I don’t understand touchscreens on laptops that aren’t designed to fold flat. It’s got the feel of finger painting an unconstrained birthday balloon.
I had a laptop that folded to 180, without touch screen, and had a webcam hidden under F7 so it either looked straight up your nose or showed your huge fingers
You're essentially holding a large tablet upright, but all the weight is taken up up the base. Rather than finger-painting, try holding it on both sides like a tablet or gamepad and operating with thumbs.<p>Scrolling/controlling checkboxes and switches feels GREAT. Depends entirely what you're using it for.
Sometimes, if I’ve been using my iPad for awhile and switch over to my MBP, I might reach out and touch the screen out of habit. I can’t be the only one.
My father did within six months of owning an iPhone.
I had the opposite problem when work issued me a ThinkPad - I would accidentally brush my screen with my caveman knuckles once a day and somehow nuke a dozen lines of code.
My kids do this all the time. They also use the touchscreen in conventional laptop configuration (not folded-flat tablet mode) on their Chromebooks all the time. It's bizarre to me. I'm always trying to get them to use the keyboard, but they don't care. Example: enter password on the keyboard, then tap the log in button on the screen with their finger, rather than just pressing enter. ¯\_(ツ)_/¯
My 3 year-old never had access to tablets or phones but sees us using phones all the time. So when he gets curious and touches the laptop screen (something he does with our phones occasionally as well), he's <i>shocked</i> to see it doesn't react.
<i>I</i> do this on my iPad with Magic keyboard and I'm a die hard command line user otherwise.<p>I think the reason I started doing it on the iPad is that the keyboard focus is sometimes inconsistent, so clicking or tab-tab-tab-enter is slower and less reliable vs. just touching the screen. Definitely feel the gorilla arm though.
While I'm the same and totally agree with you, the few times I've been using touchscreen I find the habit sticks so hard that for days I keep touching my macbook screen, so there is definitely some subconscious desire for this (or I would have defaulted to using the trackpad even if my brain thought touch was available)
> I don't get why Apple are apparently going to be putting them into the new MBP line later this year.<p>Apple has apparently being going to put a touchscreen in a laptop every year since the iPad came out, and it's never materialized.
Macs are definitely not optimized for keyboard commands. If you feel the software you use is keyboard optimized, odds are it's not really Mac software.
Absolute base line example.<p>Copy text in terminal<p>Mac: command+c<p>Linux/windows: ctrl+shift+c (unless you want to cripple proper ctrl+c functionality in which case you can (maybe) activate it from a UI menu)<p>The command key on Mac is somewhat magical and engages in all sorts of productivity and finger efficiency related context switching so that you can do more with less physical movement.<p>I’m genuinely curious who you think does it better
macOS has been one of the best keyboard OSes for over a decade, maybe longer. Nearly everything is bindable without additional software or third party apps. This can be done on globally or app-specific. A lot of this comes from the deep script ability that used to be a priority but has fallen by the wayside in recent years.
Huh? Of all the wonky shit about my Mac, the flawless keyboard navigation is really none of that
I'm surprised you feel that way. I fight with my mac every day for one reason or another. At least it's not as bad as the days where some software used Cmd + letter and some used ctrl +letter, but for instance Cmd tab will switch to the wrong window when I go back and I have to use the mouse. Window switching in general is a lot harder if you only have the keyboard because the laptop is docked without a magic trackpad
strange. Some keyboard shortcuts in os x are kinda weird and not intuitive to linux or windows users, but they are there. It's totally possible to use mac without trackpad. even cmd+tab switcher has a lot of hidden (but googlable) things: while still holding cmd after initial cmd+tab, you can close apps with q, switch to other apps with tab and (cmd+)shift+tab or left/right arrows, show app windows with down, etc.
There's also a cmd+` for switching between one app's windows. I still find that distinction weird from usability perspective, but it's not too hard to adapt to it.
Really? I find that on MacOS apps are very inconsistent about whether popping open a menu shows me hints for selecting items in that menu. Those same apps are consistent about it on Linux.<p>And then there's the bonkers window manager which can't move focus directionally (e.g. Super + left) and so you have to fall back to Cmd + tab tab tab tab but even then there's no consistency about whether you're switching between app instances or windows instances within the same app...
“Flawless” is absolutely the opposite of how I’d describe the third class keyboard navigation in MacOS.<p>It’s actually more intuitive to use a magic keyboard on the iPad than on the desktop OS.
What do you find missing from macOS keyboard navigation?<p>I've been using macs since the 90s so I'm quite used to it, so I'd love to know what I've been missing out on.
I have a lot of complaints but I would say my three big gripes are:<p>- Window navigation within (rather than between) open programs. Mainly if one is on an external monitor, this is just a nightmare and I end up using expose and clicking the window instead.<p>- Window positioning (I installed 3rd party software called Rectangle for this last year so it’s kind of solved but if we’re talking about the vanilla experience this is a big one)<p>- Having to switch focus to the dock and navigate one by one through shortcuts to open them instead of the Super+Dock position shortcuts that Windows and KDE expose
They kinda added window positioning with Tahoe -- there are things I like more about it than Rectangle (resizing), but I found that it was janky enough I switched back to Rectangle.<p>I rarely use the Dock, it's somewhat eye candy I leave up, or add stacks for folders that I use, but typically for keyboard action I reach for spotlight (cmd+space). Now, spotlight occasionally shitting the bed, that's another issue...
Interesting, those are problems I don't have, I guess due to my work and workflow.<p>Command-` works for window switching as I expect, probably simply due to being used to it so I know exactly how It works.<p>Window positioning is an interesting one. I can't stand windows being positioned through tools, I stack them like you would with papers and shuffle through so the edge overlap is really important. Probably showing my age there!<p>And I never use the dock. Spotlight gets me everything I'd need from there.
Window navigation is just ctrl+direction
> Window navigation within open programs<p>Isn’t table cmd + ~
At least we have trackpads that are worth a crap.
Specific applications sure, but the base OS interface is unusable without a trackpad.
Cmd+Space to open spotlight, type in the first 3 or 4 letters of whatever you're trying to do (an application to open, or a system setting to change) and then Return gets me about where I need to go most of the time. Cmd+Tab and Cmd+` for window selection. I don't do much else on the OS itself so my bases are covered.
It just feels ancient and weird now that I can tap every screen I own, except my Mac. I don't want to replace the Mac's keyboard & mouse with a touchscreen, I would simply like it to support touch.<p>(This also made me realize the impending obsolescence of the Studio Monitor XDR: <i>no touch support</i>.)
But the Touch Bar was such a resounding success …
I very much would want a touchscreen for my use cases.
Why? Better than a track pad.<p>Use a Surface Pro some time. If you are just casually browsing or reading a website. I find it much nicer to just tap on a link or swipe to scroll.
Sometimes I feel the urge to do some art, and the bigger surface might allow that. Perhaps in lieu, make a bigger trackpad.
I have 2-3 old touchscreen laptops lying around. The touchscreen is useless to me. Worse than useless. If I ever use it, it’s accidentally, and I end up annoyed.
I wouldn't want a touch screen to become the primary input device, but I think it would be useful on occasion. Not entirely unlike how we still have touchpads even though we try to use keyboard commands.
It would make sense if the screen folded over. In a laptop form factor a touch screen is just annoying because it keeps pushing the screen back.
Yes, I feel like it'll be a degrade in quality if they do this with any of their current line up. If they want to make a Macbook Ultra or whatever with it, that's fine -- I would have no interest in it.
The benefit of a touchscreen MBP is that Apple will be forced to make their screens more protective.
as long as it works well, would rather have it than not (but don't want to pay extra, so yeah... leave it out)
I agree. I've never wanted a touch screen on my laptop. My screen gets smudged enough already.
The ergonomic aspects are horrible. I believe there's actual research on this from the 70s/80s/90s.
All my Windows laptops have touch screens and I love it. What is the problem with having another input method available? You only use it when it’s appropriate.
[dead]
I don't even imagine how I would be tapping on the screen of my MacBook, not because of its form factor or design, but because of the macOS system itself. It's a different story with the iPad where you can do it endlessly...
This is cool. Simple prototype. Is it dependent on lighting ... what if you are outside or backlit or glare etc...?
Touch screens are not pleasant for laptops. I prefer not to have them.
It's actually quite pleasant user experience for scrolling. Some interactions are better with a pointer, others are better with touch.<p>You can try it on an iPad with Magic Keyboard attached, it's very good to be able to do precision through the trackpad and then casually move large things on the screen with your fingers.
Honestly I just hate having fingerprints on a screen. And I use pageup/pagedown mostly which to me is better than scrolling.<p>Trackpad is nice for a device you can lay flat on a table or keep on one hand while sitting on the sofa, not too much when the device has a keyboard permanently attached to it and it cannot fold. I know I have a thinkpad like that and I never use the touchscreen.
Agree for iPad. But for a laptop trackpads ftw!
You don't have to use it.
As long as there's a way to maintain the current display density, that would be just fine.<p>However, like on Windows, I suspect macOS would increase the tap target size on lots of the touchable elements. Even if I don't use the touchscreen, I would still have to pay the touch target real estate tax in my applications.
That's a fair ask. My dream would be a simple toggle in something like control center for macOS that can flip between "touch mode" and "desktop mode" with most of the under the hood stuff being the same and just UI changes for the task. No doubt this would create new hurdles for software devs but again I'm dreaming here. Windows 10 actually had this with "tablet mode" in the notification center but I think they already soured people on the touch Windows thing by this point. I think Apple could reasonable do it better if they had the will but they'd much rather you buy and iPad for touch and a mac for desktop and everyone who doesn't want an extra device for certain use cases is left out in the rain.
Note on Windows you can disable the touch device and it goes back to the old density. Don't know if Macs support it.
you will accidentally touch the screen more often that you think.
People use their laptops under various lighting conditions. I can imagine it would be difficult (or likely impossible) to bring this PoC to a solid production level technology. It looks like a fun project though.
What a super neat application of computer vision. Cool writeup. Thanks for sharing the code and making it open source too!
Ignoring whether touchscreen laptops are actually a good idea, I "OOF"ed out loud at this line.<p>> <i>Filter for skin colors and binary threshold</i><p>Skin has an extremely broad range of colors that are also lighting dependent. I'd have gone with background subtraction.
I was laughing so much. Thank you. Unexpected tech!
This is amazing. They should start to install upward looking cameras to implement this officially.
Love it! I appreciate the ethos of doing more with existing hardware. Adding an actual touchscreen would add real COGs to a macbook, and many potential failure points. Using the existing camera hardware + software seems to produce a "good enough" result for most people for casual use. I'm sure with some time and eng, Apple could make the "hack" shippable. But it doesn't earn product managers the big big bonuses, so it'll never happen.
Don't love touchscreens that much.<p>But I did love my Toshiba Satellite. It was like writing on paper!<p>Down with capacitive screens and long live Active Digitizers!
I always say, people who want a touchscreen on their Laptop never used a really good trackpad.
I never missed a touchscreen on my MacBook but when I do something on someone else’s Windows Laptop I often prefer to touch the screen because the trackpad is just terrible.
Using an external webcam is that not more than $1? cool project though; reminds me of how you could use a Wii remote to create a interactive whiteboard.
It feels real because of the dirty touchscreen.
As other people mentioned this is obviously not something I would want in my notebook... but I can still appreciate the cool tech!<p>I can also definitely see this kind of thing being used in things budget outdoor displays, specially if the UI is made to accommodate the lack of accuracy, and the camera is positioned on the side (since these displays are usually vertical).
Difficult to capture reflections across a large screen while also dealing with outdoor lighting, glare, and moisture. The touchscreen part isn't usually what makes outdoor signage expensive compared to IP65, temperature control, and a secure housing, all of which would still need to apply here.<p>This looks like a neat option for retrofitting, and I suspect it'd work for some non-screen glass applications too. A combined IR/visible light solution would be interesting too, since I suspect those are complimentary (IR touch has issues with radiant light, while this wouldn't; this would have issues with low/no light, while IR wouldn't).
Is there a coating you can apply to the glass to help with smudge marks?
Oleophobic coating is standard on phones and tablets, which is part of why they don’t pick up fingerprints as easily.<p>Some brands offer coating you can DIY yourself (eg ProofTech OLEOPEL) but these seem mostly designed for phone screens. I don’t know whether they’d be as effective on laptop screens
I think I could do this for less than 15 cents: four small peices of double sided tape, and the tiny mirror, and two hair pins... but the software? Priceless.
The reason we buy macbooks is because they aren't touchscreens.
I wonder how well this would work with my bright blue fingernails that are about .5" longer than my finger.<p>I then wonder how much recalibration I would have to do when one of them broke and I was poking directly at the screen.
You could still use the keyboard and track pad
You chose to suffer not just with touchscreens. That said, it would probably work just fine.
Brilliant!
[dead]
[dead]
[dead]
Checking this profile of a random hacker in 2018, of course they are now working on AI.
Neeeeveeeer!!! please let macbooks be as they are .. why would I ever choose to put fingers on that beautful screen ... I don't get it!
> why would I ever choose to put fingers on that beautful screen ... I don't get it!<p>I'm not sure, but bare in mind that the iPad is almost as large of a market as the Mac at this point, and the iPhone has long surpassed Mac revenue. Touching your computer is a very popular sentiment among the grimy-handed public.
Still an amazing hack today and I love it. However, I heard Apple are developing a touch screen MacBook this year, and I simply don't get why they're doing that. I don't know what's worse, the ergonomics or the fingerprints.
i used to have a bad touch screen laptop like 7 years ago and back then it made the mobile development more pleasant (both native and web based)
So you mean they're developing the iPad, an insanely popular device, and you're not sure why they would make such a device?