Chromecast: Understanding its Potential beyond Video Streaming

Google launched the $35 Chromecast into the US market in July 2013 and it became available in the UK in March 2014.  It’s primarily marketed as a device for streaming video to your TV, and, as such, is usually compared with the Roku streamer, Apple TV and the new Amazon Fire TV.

Once you look under the covers, you discover that such comparisons are misleading: unlike the other devices, the Chromecast is actually much more than just a streaming device for TV.  This became clear when Google launched their Software Development Kit (SDK), allowing developers to build their own custom applications for the Chromecast.  Having had my hands on one since almost the day they became available in the UK, I’ve come to think that it’s one of the most exciting and interesting devices of recent times, and I believe it has huge potential within many market sectors, in particular healthcare.  From the reviews and articles that have been written about the Chromecast, I’ve also concluded that it’s a very poorly-understood piece of technology: I think it’s partly Google’s fault – they deliberately focus attention on its role as a video streamer to the extent that few people have thought to understand how it really works and what it’s really capable of doing.

In this article I’ll explain what’s actually inside the Chromecast, how it works and how applications run on it: you’ll hopefully start to understand why I’m so excited about the Chromecast.  Needless to say, I’ve already extended EWD.js to allow anyone to very rapidly develop Chromecast applications, but more about that in a future article!

The Chromecast is basically a device that runs a stripped-down version of ChromeOS (some say it’s actually more a stripped-down Android OS) and a Chrome browser.  Other than that it has:

  • an HDMI output that can deliver the output from Chrome to an HDMI-interfaced TV, monitor or projector;
  • a wireless network interface.

Screen Shot 2014-04-05 at 10.38.11

 

 

 

 

 

 

 

 

So a Chromecast device can run any page of HTML plus JavaScript and CSS that you could run in any Chrome browser.  The difference is that the Chromecast has no direct means of controlling Chrome’s user interface (UI).  Instead, pages are loaded into its Chrome browser indirectly via what’s known as a Sender application, and the UI of the page that’s loaded into the Chrome browser is also controlled via the Sender application.  A Sender application is simply another application that runs on a device that does have a means of controlling its UI.  Google’s Chromecast SDK supports three classes of Sender application that run in:

  • Android devices
  • iOS devices
  • Chrome browsers (eg on a Windows, Mac OS X, Linux or ChromeBook device)

In this article I’m going to focus on the latter, but the principles are similar for Android and iOS devices.  The difference is that the Chrome browser approach allows everything to be done in HTML and JavaScript which, of course, lends itself to using EWD.js.

So, a typical environment for the Chromecast is shown below:

Screen Shot 2014-04-05 at 10.59.03

 

 

 

 

 

 

 

 

As you can see, the one key additional thing you need to do is install the free Chromecast Extension for Chrome on the machine that you’ll be using for your Sender application(s).  The first step, then is to load the Sender application into Chrome by specifying its URL.  The one requirement is that the Sender application HTML page must load the Chromecast APIs from Google’s web site:

 

Screen Shot 2014-04-05 at 11.03.12

 

 

 

 

 

 

 

 

 

The JavaScript that is specific to the Sender Application you load must specify a Chromecast Receiver Application Id.  A Receiver application is a pre-built “container” page that will run in the Chromecast.  Google provides several pre-built generic ones that are used primarily for video streaming.  However, a registered Chromecast Developer can create a Custom Receiver application.  When such an application is registered and made public using the Google Cast SDK Developer Console, it is allocated a unique ID and any Chromecast device can be instructed to load it by a Sender Application.  I’ve created one for use with EWD.js: its Id is CCBCB93C.

The Chromecast SDK Sender APIs include the function chrome.cast.requestSession().  When this is invoked, the Chromecast Extension opens a pop-down window within Chrome showing all Chromecast devices that are available on the wireless network:

Screen Shot 2014-04-05 at 11.16.18

 

 

 

 

 

 

When you click on the Chromecast device you want to use, a message is sent to the Chromecast device to load the Receiver application with the specified Id (in this case CCBCB93C).  A Receiver application is nothing more than a page of HTML, and the Registered Id simply maps to the URL at which that page resides.  In the case of the EWD.js Chromecast Receiver application, that URL is https://ec2.mgateway.com:8080/ewd/chromecastrx/index.html.  You can, of course, try loading it into any browser, but it won’t do much unless it’s loaded into the Chrome browser in a Chromecast device (actually it is possible to get any Chrome browser to emulate a Chromecast device – itself something that has interesting possibilities).

As you’ll see if you inspect the source of the EWD.js Receiver application page, typically, a Chromecast Receiver application is simply:

  • a page into which the Chromecast Receiver JavaScript APIs are loaded
  • a “container” page that includes an iframe

Screen Shot 2014-04-05 at 11.29.46

 

 

 

 

 

 

 

 

 

 

The piece of magic that now happens is that the Sender and Receiver applications establish a private WebSocket connection across the wireless network:

Screen Shot 2014-04-05 at 11.31.53

 

 

 

 

 

 

 

 

 

The idea, then, is that the Sender application can now act as a proxy UI for the application that runs in the Chromecast, with instructions signalled via WebSocket messages.  Normally the first message that is sent from the Sender application instructs the Receiver application to set the src attribute of the iframe, causing it to load the actual application that you want to run in the Chromecast.  When you use a Chromecast to watch YouTube videos or Netflix films, this is basically what has happened: the YouTube or Netflix-specific HTML video-streaming application is loaded into the generic Receiver application’s iframe.  In our case, however, the WebSocket message from the Sender application instructs the Receiver application to load an EWD.js application into the iframe:

Screen Shot 2014-04-05 at 11.40.14

 

 

 

 

 

 

 

 

 

 

The really important thing to understand about applications that run within the Chromecast is that they run completely within the Chromecast and can communicate over the Internet.  So, if you’re using a Chromecast for streaming a Netflix film, it’s streaming it directly to the Chromecast: the Sender application’s role is simply one of acting as a UI controller for the application within the Chromecast, eg telling it to start, pause or stop the streaming video.  In other words, the streaming does not take place via the Sender application.

So, if you’re running an EWD.js application on a Chromecast, what you end up with is the following environment:

Screen Shot 2014-04-05 at 11.47.02

 

 

 

 

 

 

 

 

 

 

 

 

Both the Sender and Chromecast applications are standard EWD.js applications, communicating via WebSocket messages with their corresponding remote Node.js-based back-ends.  Meanwhile, the Sender and Receiver applications can communicate with each other via WebSocket messages across the local wireless network (and note that this is two-way communication: the Receiver application can send messages back to the Sender application).  Meanwhile, within the Chromecast, the Receiver application “container” can communicate in a two-way fashion with the application running within its iframe using postmessage messaging.

Of course, the Sender and Chromecast applications could be hosted by the same EWD.js/Node.js back-end, and this could be connected to an application such as VistA:

Screen Shot 2014-04-05 at 11.54.04

 

 

 

 

 

 

 

 

 

 

 

 

Potential Applications

At this point you might be thinking: “All very interesting, but so what?”  Well, let’s think about what this is providing us with: the $35 Chromecast turns any available TV or HDMI-interfaced monitor into a device that can run a fully interactive application without that TV or monitor needing to be physically directly connected to a computer.  The application that is now running in the Chromecast and being displayed on the TV or monitor can be controlled remotely from another Sender device: a PC, tablet or phone.

So, for example, any TV that’s mounted on a wall in a clinic or hospital can be used as an interactive display for an EWD.js application.  A patient could be shown information during a consultation.  Information about the hospital could be displayed on TVs or monitors mounted around the hospital or clinic without the need for them being wired to dedicated computers, and/or replacing expensive proprietary displays.  The content and its styling and format is completely under the control of the hospital or clinic and easily defined and modified: it’s just standard HTML, JavaScript and CSS!

Furthermore, the WebSocket-based communication of EWD.js applications means that real-time information can be sent to the application running in the Chromecast: so life-signals could be sent in real-time to the Chromecast application, updating a graph or chart in real-time without any polling.

Think further “out of the box” and consider the Sender application’s role as a proxy UI controller for the Chromecast application.  Instead of using a keyboard and mouse, what about the inexpensive LeapMotion controller?  Now the Chromecast application can be controlled by hand gestures, once again all developed using JavaScript via the LeapJS APIs.  In fact, if you have a Chromecast and a LeapMotion controller, you can try this idea out for yourself now.  Simply install the Chromecast Extension on your Chrome browser on your PC or Mac and launch the following URL to load a demonstration Sender application:

http://ec2.mgateway.com/ewd/chromecast/index.html

Login with a username of ‘rob‘ and a password of ‘secret‘.

Load the Chromecast application by clicking the Green Power On button.  Once loaded into the Chromecast, you’ll see a new set of buttons in the Sender application, one of which turns on the LeapMotion controller interface.  Move one hand side to side and up and down and watch the pointer on both the Sender and Chromecast applications (Note: I’m trying to get LeapMotion to fix a bug that currently stops the controller interface working properly if you stop and then restart the LeapMotion interface).  To stop the Chromecast application, just click the Red Power Off button on the Sender application’s UI and the Receiver application automatically unloads itself from the Chromecast.

It’s a pretty simple demo, but consider some potential applications of these techniques:

  • turning any TV or monitor into a display for bringing up patient imaging, eg X-Rays, moving them around using swipe gestures (yes, in true Minority Report style!)
  • assessing a patient’s dexterity or hand tremors and displaying the results in real-time on a TV or monitor for the patient to also see (thanks to Steve Owen for this idea!)

All done using a $35 Chromecast and a $90 LeapMotion controller!

Developing your own EWD.js Chromecast Applications

Hopefully I’ve now whetted your appetite and you’re now interested in developing your own applications for the Chromecast!  If so, in my next post I’ll introduce the new Chromecast extensions to EWD.js that makes it a piece of cake: you’ll be able to build sophisticated Chromecast applications in just a matter of a few hours.  You won’t even need to be a registered Chromecast developer: just use my registered EWD.js Receiver application and you can do the rest using standard EWD.js coding.  And if you’re a VistA user, start thinking how its power and functionality can be further enhanced by using the Chromecast.

All the power of EWD.js, coming to a TV near you!

 

About these ads

10 comments

  1. Rob,

    This is very, very interesting !

    Thanks for sharing such a detailed article.

    Luis

  2. Harry A. (Andy) Bruce · · Reply

    Ditto…well done Rob. Once again you lead the charge with integration/configuration assistance of new capabilities.

  3. markbarton · · Reply

    Sounds interesting.

    What prevents someone from controlling the chromecast device if they are on the same wireless network?

    1. Very good question, Mark. At present it’s one of the limitations of the Chromecast – anyone with access to the wireless network to which the Chromecast is connected can cast to it without any limitations. I’ve tried to get the message to Google (perhaps someone from Google will read this!) that there’s a whole world of potential applications for the Chromecast that would benefit from an improved (probably optional) security layer – perhaps something as simple as a password or PIN – to prevent such unauthorised casting.

      At present, Google’s suggested solution is to ensure that the Chromecast is connected to a private, secured wireless network which, of course, may or may not be practical in the scenarios I’m suggesting for its use.

      I think we need to lobby Google for such a feature – I’m sure it’s something that ought to be do-able as a simple software upgrade.

      Rob

  4. […] Nice use of the Chromecast plus Node.js and EWD.js for interactive applications – robtweed.wordpress.com/2014/04/05/chr… […]

  5. In the same vein check out AutoCast : http://youtu.be/pDCiiX48mq8
    The idea is to be able to be able to build any chromecast app right from your Android device :-)

  6. habeebq · · Reply

    Very nicely written and informative :)
    Just some clarification needed in the beginning of the article, that it is actually the receiver that runs on the Chromecast and the Sender is merely controlling the receiver via media/custom channels.
    I think the Chromecast needs more web developers to exploit rather then Android/iOS developers.
    Thanks!

  7. […] Thanks Ilya Grigorik for pointing me to Chromecast: Understanding its Potential beyond Video Streaming | The EWD Files. […]

  8. I’m hoping to try out some of the healthcare-related ideas for the Chromecast that I’ve listed in the main article at the forthcoming NHS Hack Day here in the UK: http://nhshackday.com/

    I’ll report back here on the outcome.

    Google UK should send someone to the Hack Day to help out maybe? :-)

  9. […] in community developments: building Leap Motion-enabled Chromecast apps in JavaScript, designing a biomedical model app, and the future of real-time industrial site management. To […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 51 other followers

%d bloggers like this: