This post might seem at first that this should live some where else than in Android UI Design Patterns blog. I will not be talking about Android smartphone user interfaces (well at least not that much) and I will not be talking about design patterns for Android phone apps. I will, however, be talking about Android and user interface. This post is about my thoughts and personal views about smart TV industry and especially GoogleTV and what I think needs to happen before the next living room revolution truly gets started.
User Interface Problem
By far the largest problem standing between us and mass smart TV adoption is the user interface. All current TVs are just few short evolutionary steps from the TVs of 1960s. The lack of progress is actually very surprising. TV has been big business for a long time. For decades basically every home has had one.
Thinking back for the last 70 years where are the revolutionary innovations? While of course the technology that is in TVs has advanced tremendously true revolutionary ideas are missing. 3D TV? Digital TV? HD TV? No, that is innovation and progress but not revolution. People really don't care about 3D!
 |
| Poll on Slashdot, http://slashdot.org/poll/2323/when-it-comes-to-3d-tv |
In my opinion the only true revolution has been the remote control. A remote lets us stay on the couch instead of getting up when we want to watch something else. It was a massive step forward in TV user interface. Remote isn't a new idea though. The first wireless remote was built 1955 and it hadn't changed much after that.
 |
| 1956 - Zenith Space Command, photo by Jim Rees (CC) |
Adding More Pixels
All TV manufacturers seem to be hellbent to add more pixels and more functions to their TVs. This whole situation has great parallels to smartphone industry. It took an outside forces to step in and shake up the industry before the current revolution started. What Apple and Google did in smartphones must be done in TV industry before we start moving on.
Interestingly parallels of Smartphones and TVs don't end there. Take a look at this recent quote from Samsung AV product lead Chris Moseley:
"TVs are ultimately about picture quality. Ultimately. How smart they are...great, but let's face it that's a secondary consideration. The ultimate is about picture quality and there is no way that anyone, new or old, can come along this year or next year and beat us on picture quality." (quote source)
That is exactly what Nokia, RIM and Palm said about iPhone and later about Android before they were all in trouble for not realising what users really want. And it wasn't more Gs or MPs in their phones.
The Remote
People say that in TV content is king. But if people can't access the content without a fight. Take a look at the remotes you have laying around. What percentage of the buttons you use?
This is one of the three remotes I have to use whenever I want to watch a movie. I've never used more than 4 to 5 buttons of this remote.
This is one of the three remotes I have to use whenever I want to watch a movie. I've never used more than 4 to 5 buttons of this remote.
 |
| See the problem? |
YouTube Remote
OK. I've now established what I think abut the current state of things but how about some ideas how things should be? Here goes. I don't know how many of you have tried Android YouTube remote app and YouTube lean back but if you haven't try it out now.
Step one:
Open http://www.youtube.com/leanback in any non-mobile browser. A laptop or desktop works fine. Make sure you're logged into your YouTube / Google account.
Step two:
Install YouTube remote app to your Android phone from the Android Market, launch it and give it permission to login to your Google account.
Done!
.png)
.png)
You now have a very easy to use remote to your laptop / desktop for watching anything on YouTube. The Android app interface is intuitive and easy to use. If you want to search for a video you can use the familiar interface you have on your phone.
Let's think how this would transfer to a GoogleTV. The TV already knows your Google account so that step can be skipped. The TV is capable of launching a YouTube app when start the remote app on your phone. The whole step 1 can therefore be skipped altogether.
As for step 2, once you have the app installed only thing that have to done is to launch the app. That's it. Seamless, practically zero configuration remote for watching videos on a TV (well on a computer in this case). Also noteworthy is that the two devices don't need to be in the same network so the user doesn't have to worry about that either.
So which device is actually extending which?
So which device is actually extending which?
A Remote I Want to See, Wild Speculation
We clearly have the technology to do better things that we do now. Someone just needs to step up do it right. Shaking up old industries is difficult form inside and small new players like Boxee are struggling to gain significant traction.
Google is doing great work with its GoogleTV platform. GoogleTV has a lot going on. They keep improving the platform by trying to keep up with Android releases, adding better app support and better APIs. This isn't enough though. There's only so much Google can do alone (at least for now). The main interface of TV (the remote) is still build by OEMs. Google either needs to step in and make the industry shakeup happen or someone else will.
That someone else will most likely be Apple. Think what you want about Apple but indisputable fact is that Apple did revolutionize both smartphone and tablet industry.
That someone else will most likely be Apple. Think what you want about Apple but indisputable fact is that Apple did revolutionize both smartphone and tablet industry.
At the CES 2012 there was panic in the air. While Apple's entrance to the TV business was only rumor (and still is) every major manufacturer were competing against this yet to be announced device. But the traditional manufacturers are so stuck into their way of thinking that it seems to be impossible to innovate. What we saw at CES was frankly saddening. Every manufacturer took the same approach. They took their existing hardware and tried to jam every conceivable feature into their TV and remote. That doesn't work.
So, let me introduce the device that will disrupt this industry and take it into next level. This is what Apple TV's remote control will look like:
The remote will run all iOS apps. In fact the remote will run iOS. Apple will introduce set of new APIs that let the apps use connected TV to extend app functionality. You will be able, for example, to move a game, video or song you're playing to your TV seamlessly. Most of the functionality will stay on your remote as that is the familiar and powerful tool you've already used to. You can, of course, use any of your iDevices in the same way. Only configuration that will be needed is you to login to your iTunes account on your TV or accompanying set-top box. The TV will become a simple extension of your phone or tablet instead of being a separate entity.
While this all is just pure speculation and guessing (I have no insider information) I believe that we need to change the way we think about TV. TV must become a part of the ecosystem we're using instead of us inventing another ecosystem.
There's much more
In this post I willfully ignored voice and motion control and some other related technologies. While I believe that they can easily be viable control methods and might play big role I don't think any of them are the big shift and revolution. They will simply be technologies that will allow us to build more natural UIs.
TL;DR or Conclusion
The shift we need must be more than new technology. We must start thinking TV as extension of our other devices and not to try make our other devices (or remotes) extension of TV. A TV is a large display. Let's treat it as one. TVs will be part of the smartphone ecosystems we already have. Tools for devs will allow them very easily move display part of their apps to any TV any time. Our TVs will become a natural extension of the best technology we already have.
I visited a friend this weekend who has a new Panasonic big screen TV and a new Panasonic Bluray player. Despite both being new and from the same company, they are inconsistent with each other. When using them for DLNA they support different codecs so he has to try both devices to play content and almost always one works while the other doesn't, but there is no pattern.
ReplyDeleteThe TV has builtin Youtube support. In order to search you have to type stuff. Rather than providing a wireless keyboard or bluetooth, you have to use the numbers on the remote with T9 text entry (each digit has 3 letters you can cycle between). It was painful typing just one word let alone what we were actually searching for! (We stopped using T9 in 2007)
I'm always convinced that the employees actually making these devices don't get paid enough to actually own them, so they have no idea just how bad the interfaces are.
Contrast this with the original research done by Toyota before designing Lexus. They sent employees travelling all over the US and made them stay in more expensive areas, hotels, restaurants etc so their employees would better understand their prospective customers.
a very refreshing, excellent and intelligent read.
ReplyDeleteWhy can't we get out-of-the-air television channels on our smartphones? Why can't a smartphone manufacturer add a TV decoder into the device? Does it require a different antenna? Too expensive?
ReplyDeleteSome thought-provoking ideas here; I am sure the folks at Google and Apple would very much like to move in this direction if they haven't already.
ReplyDeletePersonally, however, I find I use my iphone and tablet only when I have to -- when I'm on the move and can't use a keyboard. When I'm at home, the mouse and keyboard are my preferred way of interacting with the TV. I say TV, but really my 140" projector screen is just a computer monitor. Most of my entertainment comes from Netflix, Youtube and Hulu, as well as the occasional DVD. I could use any device (PS3, Xbox, PC or mac) as the front-end; in practice I still use Windows 7 because desktop PC's have massive internal HDD capacity and Mac Pro's are expensive!
Rather than use an iOS-like device to navigate content, I find the trusty old mouse is good for 99% of tasks. The best improvement in user experience would be for the PC applications and web portals I use to enable *less keyboard* and *more mouse* usage. I'm not wedded to the mouse; any reasonably accurate tracking device would do (perhaps Kinect?)
In a few years time touch screen devices as companion devices will be common place. To extend the TV experience IOS/Android apps are appearing in large amounts. What won't happen is that touch screen devices will replace remote controls with physical buttons. The reason for this is that users tend to focus on one screen at a time and if you're focused on the TV then a touch screen device does not give the haptic feedback needed. I've worked on projects that use touch screen devices as remotes (working on one at the moment) so know this pitfall from first hand experience. Future physical remotes do have to be much simpler, with the logical minium buttons being four without dedicated volume buttons. Voice control and gesture control I've worked with, again both are side shows to a remote with buttons on it. We won't see physical buttons disappear from the TV experience as there's nothing that gets close to their ease of use.
ReplyDeleteThe future of TV is much more about getting the interface right. I've written a something on that here....
http://ux.stewdean.com/2012/01/tv-ux-breaking-down-the-divide/
You *can* use touch screen devices for remotes, even without looking at them - if you use gestures (previous, next, volume up/down would be a breeze to implement). But I'm not going to buy a phone or a tablet, just to go with the TV.That locks you up to a provider even more.
ReplyDeleteOh, and universal remote control apps have existed for years for some Nokia phones. It's pretty entertaining to do tricks on people's TVs with an N900 and watch them wonder what's going on :)
I think your approach is flawed from the beginning. If you think about it, the average person don't want multitouch and nifty controllers for watching TV: you are concentrated on a screen, and need a SIMPLE device that can be used without looking at it, accurately. a flat surface wont work, it needs to have buttons that can be recognized by touch and shape, without having to take my eyes from the screen.
ReplyDeleteIt also needs to be strong so I can drop it, sit on it or toss it without damaging it. That's why the remote still rules.
So the next thing will not only be a matter of technology and user interface, but simplicity and being able to manage it without looking at it.
Read more on usability from Nielsen, and remember that not everything can be solved with an android device, an expensive screen with no shape response.
Thanks for your comments all! A lot of interesting ideas and point of views. Thank you for taking the time to write what you think.
ReplyDeleteJavo, I try to ignore your condescending tone and answer to your point as it is very common and makes a lot of sense.
You might very well be right but I think you missed what I was trying to say a bit. In my opinion we need to start thinking about the TV differently. Your comment is, of course, very accurate if you want to stick with the old mindset. But the world is changing and the way we consume entertainment is also. There is a growing number of people who don't watch TV the same way we've used to do. TVs are being replaced by our more advanced devices. Tablets and phones (and computers) are more and more common replacement for TV as a whole.
I don't believe that the one way Tube is going to survive for long. Content through Internet is already gaining momentum even though large part of that is illegally consumed or just very niche content (take a look at eSports for example). When the content becomes mainly non-linear and even the linear content changes to a totally different format we need better ways to access them. My thoughts about using mobile phones as remotes wasn't based on me wanting push Android or iOS because I happen to like them. The thought is derived from the fact that we are already using them and we know how to access everything very easily. I believe that what is going to happen is that the main focus is going to be on our phone / tablet and only occasionally when we have photos, videos etc we want to see on a larger screen or share with people we move them to the big screen. And once we start to think about the TV as an extension instead of a separate device the flat control surface no longer matters.
Juhani, I apologize if I sounded condescending, it was not my intention. My opinion is based on the majority of television viewers, and these are the moms, workers and average Joe around the world, not only in the US.
ReplyDeleteThat put aside I agree that a lot of content will be consumed online, will be able to carry with you and to move from device to device.
Good article. I enjoyed it very much.
ReplyDeleteI feel that every TV OEM has taken very deep breath and waiting for Apple to finish its R&D for them. They are afraid of innovation. I also feel that we shouldn't forget Microsoft. They are the rulers of home entertainment at the time, plus they invested a lot of cash in kinect, and that's where I see a potential, also.
I am disappointed, but not surprised with this article. Like everyone in software design today, there is a lot of tech babble and a complete lack of solutions.
ReplyDeleteWhere is the haptic feedback of real buttons for channel surfing in the dark? Many adults watch in darkened rooms. Even in the light, a flat touch screen can be inadvertently touched in the wrong place resulting in unintended input. Do you expect the user to take their attention away from the TV to check that their thumb is hovering over the right piece of flat glass? I know that gets really old really fast. I purchased a Chameleon remote online almost a decade ago. I used it for two weeks. I did not realise that it was a flat piece of plastic with no discernible buttons. Volume and channel up/down buttons are critical. Mute and power buttons are close to critical (imagine all the accidental input power button presses that will piss the user off if you depend on a touch screen power button).
Voice control is not a second thought feature. I would never expect a remote to have extensive voice control, but there are specific actions that are as annoying as can be on current remotes. These problems can be solved by voice control.
"Channel 81"
"Channel 12"
"Channel NBC/ABC/FOX/ITV/BBC2"
Looking at the remote numeric keypad and then pushing the buttons is so annoying I just do not do it. I do not know anyone who does it past single number channels.
Add internet TV and you need voice control further.
"YouTube kittens"
"Vimeo short movies"
"FilmRiot"
"Current Walking Dead"
"Current Big Bang Theory"
"Suggest New Content"
If you hope to do this on a tiny touch screen with pages of buttons, program lists and previews, then you do not understand customers at all.
Now, lets look at the simple things that should have been added, but moronic programmers need to have their idiotic faces ground into these ideas before they realise that NORMAL people want them.
Why can't the remote become a unified interface for controlling the TV? When did programmers forget that computers can be programmed by users to achieve tasks wihtout direct user input at all times? I would like the remote to run the TV through a complex sleep cycle.
No input or movement for 2 hours = dim the TV brightness 50%
No input or movement for 3 hours = dim to 0%, & reduce the audio to 75%.
Wake on remote movement or input.
Wake at 6am.
Wake at Reno 911 & raise volume to 150% for 2 minutes.
Buffer favourite internet shows at 3am.
The remote should be able to do this "IN REMOTE". If every Android remote has the same interface/programmable interface, then users would become accustomed to the Android remote way of operation across all brands of TV. The alternative is to let the TV brands create a disparate set of control features and processes. Confusing the consumer further, and making Apples unified/simplified/dumbed down control system appealing to the end consumer.
Apple's siri has a real opportunity to become the new way (or maybe the standard) to remotely control our TVs
ReplyDeleteEvery network enabled AV device (TV/Receiver/Blueray-player) I've used that was manufactured in the last 3 years has had an Android and iPhone app that you can download for free to control the device. They typically provide easy text entry, gestures and some sort of media guide. The future you speak of is already here for the most part. There's definitely room for improvement on the interfaces and there's no 3rd party support which is frustrating but expected.
ReplyDeleteTo add to the discussion, I think it important to consider why today's remote control needs revolutionizing at all? Granted, we often use just a few of the 20+ buttons on the remote, but that in itself doesn't call for radical change.
ReplyDeleteI believe that the motivation for changing the remote will ultimately be a reflection of the changing face of TV content. Let me elaborate.
Today's we have a relatively small set of linear (aka scheduled) TV content grouped around channels. The remote solves the problem of how to navigate this data. Up/down to browse channels, left/right to shift time periods and enter to select.
However TV content in the future will be an enormous set of mostly non-linear video. Why make such a claim? Because I believe that tomorrow's content will be streamed to our TVs via the internet - we are already seeing this today (Netflix, Hulu anyone?). Basically watching TV will become watching online video, i.e. what you want, when you want - and no, I'm not talking cat videos here, but propper long-form albeit niche content.
What does this have to do with the remote control? Well, today's remote will be poorly suited to navigating this vast sea of non-linear content. Even today, switching through the 50+ channels available to me is a poor experience with the remote.
We need to revolutionize the remote because it just doesn't scale well.
The 'channel hopping' we do today will make less sense in the future. Search will be one way we find something to watch, but personal recommendations and curated experiences will be more the norm. And in this world, discoving content will be more like our usage of the internet, for which the smartphone and tablet is better suited than a TV remote.
There is quite a big difference between TV and Streaming content. I wish this distinction would not have been omitted when talking about controling the access to the content.
ReplyDeleteTV: Almost fully passive consumption.
Streaming: All the choices all the time.
I do agree on tv remotes being ridiculous. Receiver, Blueray-Player, Screen... each with a 28 Button or more remote. To assist the knowledgable television customer in resolving the following complex tasks:
Turn on TV. Switch through the interesting channels. Turn volume up/down. Pause for biobreak. Turn off TV.
But hey, maybe that's an important part? Maybe it would feel too stupid to just have a little thing with 5 buttons and have that represent the intellectual requirement of ones pastime.