Google I/O 2009  Native Code for Compute Intensive Web Apps

Google I/O 2009 Native Code for Compute Intensive Web Apps

November 9, 2019 7 By Peter Engel


Brad Chen: Good morning. Thanks very much for joining
this session of Google IO. My name is Brad Chen, and I’m the Engineering Manager
of the Native Client project. And I’m really looking forward to telling you
a little bit about that today. I also wanted to mention
right away that we have other events
planned for you today. There’s a Fireside Chat
at 1:00 and office hours
for the Native Client project at 3:00. And a bunch of the team
members are here today. Can you guys say hello? We’re looking forward
to answering any questions you have
about how to use Native Client, how to get comfortable with
the system, that sort of thing. So…let’s get on
with it then. Let’s talk a little bit about
Native Client and native code. Now, why would anybody want native code in a web browser
anyways? I mean, it’s been such a big
source of problems in the past. Well, there’s a number
of really basic things that are motivating us. Firstly, performance. The JavaScript implementations that are available in browsers
like Chrome nowadays are really quite
incredibly good. But in spite of that, there’s still about two
orders of magnitude difference between what you can do
with JavaScript and what you can do
with native code. And the hardware architecture
trends actually suggest that that gap is going to
continue to broaden. Things like vector instructions
and threading are just not useful features
in today’s browser for JavaScript-based systems. And so that performance, approximately
two orders of magnitude, just isn’t available. So performance is really
the main thing driving us, and you’ll see some other
evidence of that as I tell you
more about the system. Apart from performance,
I kind of think that people should
be able to use any programming language
they want. She shouldn’t be obliged to only
use one particular language. And so native code
and native compilation makes a lot of additional
languages available. And not only are the languages
available that you like then, but also all the legacy code
that exists in those languages. And we’re doing a lot
to try and make it easy to import
and use legacy code. So…with these benefits, you might wonder,
well, gee whiz, you know, ActiveX and NPAPI,
you know, Netscape and Microsoft
introduced those ten years ago, so what’s the big deal? Well, a couple of things
you never got with these previous systems
are portability and safety. An ActiveX plugin
only runs on Windows, and NPAPI plugins typically
do use the native OS interfaces, so you can’t move them from operating system
to operating system. Moreover,
because they have access to those native operating
system interfaces, they have full access
to the file system and full access to the network. They can do anything you want, which is, you know,
great on the one hand, unless you happen to be
the recipient of such a module that wants to send junk mail or infect your computer
with viruses. And so, these are things
that we’ve made very specific design decisions
and done a lot of work to try and make
much more difficult. Our goal, in terms of
portability and safety, is to deliver a system that gives about the same level
of portability and safety as JavaScript,
and eventually, you know, we’d like to even do better
than JavaScript. So that’s sort of the background
on why native code. Now, what do we mean
by performance? ‘Cause, I mean, you can write
pretty fast JavaScript nowadays. Well, what we mean is,
literally, we wanna be within
single digit percentages of the best you can do
with desktop native code. And in particular, it means supporting
features like threads. Modern CPUs are multicore and also will tend to be
multithreaded, and so we’ve put a lot
of effort into providing a subset of POSIX threads
in Native Client… also sometimes known as
“enough rope to hang yourself”. But we’re also providing
abstractions on top of the threads
to make them easier to use. So for example,
Intel’s Thread Building Blocks builds cleanly
on top of Native Client, and that’s a kind of
higher level interface. Another important
performance feature which I already mentioned
briefly is vector instructions. And we’ll be showing you
an example today of an application
that uses SSE instructions to accelerate photo processing. And, last but not least,
for the people who really need the highest level of performance
available in their applications, we make it relatively
straightforward to use hand-coded assembler for the small number
of people who are actually interested
in doing that. But for the really high end
performance-oriented things, the computational kernels, animation applications,
that sort of thing, all that performance
is available and substantially as available as in a desktop
programming environment. So what does this mean
for the Web? What are the new things that we
think Native Client will enable? Well, as an example,
by giving the CPU performance– making it available
in web applications, we think we can enable safer
multimedia codecs. You’ll be able to have
a multimedia codec that’s embedded
in a safe environment instead of running
as unconstrained native code. We’ll show you some examples, and we have other examples
in our open source project of real-time audio and video
synthesis. This is animation, right, so substantially more ambitious
than just displaying a video. We’re actually
creating the video and then showing it to you. Another thing
that’s really important in a lot of modern user
interfaces and games is physics simulations. You actually need to have
Newtonian physics with object colliding
into each other and responding
in realistic ways. And again, in our Quake demo, which is available
in our open source release, you can see an example of that. You know, just briefly
going through the list, audio and video analysis
and recognition, things like face recognition
we think should be possible in browser-based applications
with, you know, with native code. Multimedia editors. Eventually high-throughput
cryptography and more flexible
than just using SSL. You’ll be able to encode
and decode the content as you determine. And application-specific
data compression, which has a potential
to give you much more dense representations
of domain-specific data than just a generic compression
algorithm like gzip. Together with this,
with O3D, that is the 3-dimmensional
graphics interfaces for the Web that Google
has recently introduced, we think we’ll be able
to enable things like high quality games, games that start to approach
the quality of things that you’re accustomed
to seeing on consoles, as well as CAD applications
and a whole variety of things. So these are the kind
of applications that we want to be able
to enable with Native Client. Now…so that’s enough
for background. What I want to do right now– and what we’re gonna spend
most of the time on today– is going through some examples of actual Native Client
applications that we’ve built and show you some of
the nuts and bolts of how things are put together. Before showing you that,
I’m gonna go through a series of architecture
diagrams so you can sort of see
how the pieces– what are the pieces
and how they work together before we do the examples. So suppose, for example, you wanted to write
a little game– Global Cooling we’re gonna
call it in this case– and it’s implemented
as a web-based application. It happens to use
a Native Client module, so being a web-based
application, there’s gonna be
some HTML someplace. Now, there’s two models that
we’re gonna talk about today for using
Native Client modules. One is as a plugin, and that’s what we’re gonna
talk about in this example. The other
is using web workers, an emerging HTML standard that you probably
heard about yesterday. So in this case,
if you’re using a plugin, you’ll commonly use
an object tag. And there’ll be
a source argument and a number of other
arguments, but there’ll be a source
argument to the object tag that says where is the content
that’s gonna be activated by the plugin? When Native Client
sees the object tag and the mime type
associated with it, when the browser sees that, it actives the Native Client
infrastructure and, in particular,
a helper module that knows how to handle
the native code and enable it for
the web-based application. The browser, when it sees
the source argument, is going to download
the untrusted native code from a website
someplace on the internet. Now, in the case
of climate modeling, we probably will need
some physical simulation. So that’s what most of… the scenario’s
gonna be based on. But suppose,
just for a moment, suppose that somebody
was trying to trick you and get you to download and run
something dangerous, okay? So before Native Client
is gonna enable this module, it’s going to pass it through
a security inspection, a static analysis
of the actual machine code, to make sure
that there are no sharp edges, nothing dangerous. And if it sees a module
that breaks the rules, that does some of the things that are explicitly
not allowed in Native Client, that module will be rejected,
and we will refuse to run it. And that’s exactly
what will happen for random x86 binaries
that you might choose. Because there’s
a set of instructions, a set of constructs
in the machine code that we simply don’t allow because they lead
to safety problems. But if the Native Client module
in question has been built properly
and follows all of the rules that we’ve identified
for the system, then it will pass through
the security scan. And the next step
is to load it into a container. That’s where
it will be contained and allowed to run safely. We actually think
of this container as kind of like
being in jail, ’cause there’s still
a set of things that we’re not gonna allow
this native code to do. It’s still gonna be
somewhat constrained. But now that it’s activated
on your desktop and established and running, we can connect it so it can communicate
freely with the browser. And at that point,
it’s gonna be able to actually interact with the browser,
and the browser, of course, will be able
to interact with it– graphics, you know, events,
all that sort of thing. So we’ve got our Native
Client-enabled web application. So that’s a high level model
of what’s going on. Just to be explicit
about a couple of details– a couple of nuances
of the system– we run the Native Client
modules in their own address space, and that’s actually mostly
for exception isolation. We don’t need that isolation
for memory protection. We’ve actually implemented that
in our sandbox. But exception handling
is more challenging. The operating systems don’t
know how to handle exceptions except by killing the task. So by putting the Native Client
process module in its own process, we prevent the operating system
from terminating the browser. Another thing is
the communication interface that’s indicated
with the arrows there allows for a simple message
passed in communication. In addition to that, something that we’ve done
that’s really very powerful is we provide
a shared memory– you can establish
shared memory regions between the Native Client
module and the browser. And that’s essential
for doing things like graphics that require really high
communications throughput. The shared memory regions
also allow you to implement things
like spinlocks in the case that you need really high frequency
communication. So although you can actually
do pretty well in terms of latency with the message passing
interface too. So that’s the architecture
at a really high level, and keep this in mind as we go
through the three examples. I want to tell you
a little bit about security of Native Client, ’cause it’s a legitimate
concern for anybody who’s ever used an ActiveX
or an NPAPI control. And it’s really what we spent
most of our time on so far in the project. Our goal,
like I mentioned earlier, is to make native code at least as safe as JavaScript
and HTML and the other things
that people, you know, basically take for granted
in web content. So some examples
of steps that we’ve taken to make Native Client safer– firstly, multiple internal
security reviews. We code review all our code. That’s a standard
Google practice. And anything that gets
presented publicly from Google goes through a design review
and an implementation review. The implementation review
is a really fun one. That’s one–we hire
all these scary black hat guys, and they spend time– they get paid for finding
bugs in our code. So we managed to survive that. But that wasn’t enough,
actually. We felt compelled to do more. So we’ve open sourced
the system. The system has been open sourced
since December 8th, and we’ve encouraged
public critical review, in part
by submitting papers to prestigious
technical conferences. We recently had a publication
in IEEE Security and Privacy, which is a really good
security conference. And more than that,
we held a security contest. We actually challenged
the community to find bugs in our system, and we’re gonna be paying
cold, hard cash to the people who have found
the most interesting exploits. That contest actually ran
for about two months, ending on the 5th of May. And just some kind of background on what we put ourselves
through– over 400 teams signed up
to compete in this contest, and over 600 individuals. Of those people, there were only
22 issues submitted that were actually valid. Valid in the sense that,
you know, we agreed they were
a vulnerability of any sort. And I provided
a little high-level profile of the issues that we saw. Most of them were related
to browser integration, issues with, you know,
dealing with NPAPI or, you know,
memory allocation and management in that context. There are a couple of issues
with the inner sandbox too. That’s that validator that does the static
analysis of the program. And to just give you
the next little detail, I wanted to share with you a couple of the more
interesting issues that we saw in the contest. You can see
the entire list of issues if you go to the Native Client
code site. But just kind of for
the purpose of the talk here, one interesting bug– the Native Client system
relies on being able to disassemble the program
accurately and understand
all possible control flow. And so this first bug
was an example where we had
a control flow analysis issue. Now, the Native Client system has disallowed
the x86 prefix byte for 16-bit address computation. Addr 16,
it’s commonly called. And that’s important, ’cause that lets you do
kinds of address calculations that let you jump
anywhere in the code, so defeating our intention
of being able to reliably
disassemble the program. Well, but we missed
a case of that. It turns out that you can
use a different prefix, the data16 prefix, to do
16-bit address calculations for a certain set of jump
instructions that we neglected. And so it happened to be
two-byte jump instructions. So the solution
to address this problem and prevent this arbitrary
control-flow transfer is to disallow
the data16 instruction for a bunch of additional
two-byte instructions that we hadn’t considered
before. And furthermore,
we took an additional step of un-mapping the entire
first 64 kilobytes of the address base
of the Native Client module so that any 16-bit address would never be a valid address
for a control-flow transfer. So one other quick example– I know this
is a little bit gritty, but just want to, you know,
make the point of the kinds of
steps we’re taking to secure the system. There was a stack smashing
attack vulnerability found dealing with the eflags
direction flag. Now, on the x86 architecture, you probably know
there’s some repeat instructions that can be used to copy
very long strings. And with those
mov instructions, there’s a flag
that determines whether it copies forwards
or backwards. Well, it turns out
there was a place where we didn’t check
those flags before invoking system calls
of the native operating system. And furthermore,
there was a window system call that used a repeat movs
instruction without checking
the direction flag first. So the combination
of those two problems meant that you could
actually use the Windows stack smashing
prevention code to do a stack smashing attack, which is kind of ironic. Fortunately, again, the
solution was not too difficult. It was just a matter
of clearing the flags carefully before every time we invoke
anybody’s system calls. So that gives you hopefully
a little bit of intuition about the kind of tricky stuff
that is involved if you’re going to attempt
to defeat the system. The good news, and the thing that we’re
actually very pleased about, is although there were
a few, you know, 22 issues submitted
to the contest, there were no real fundamental
architectural issues. They were all basically
implementation defects, and the kind of defects
that we had no trouble fixing in a matter of days. So that kind of reinforced
our belief that it’s possible
to implement a system that achieves a substantial
level of safety. Okay. So before we dive into
the details of the examples, I wanted to tell you
a little bit about where the project is today and where we’re going
in the future. Today, Native Client
is delivered as a NPAPI plugin. This design is really mostly
intended for research release, and it means that anybody who
has any NPAPI-capable browser including Opera, Safari,
Firefox, Chrome, any of those, can use our plugin
and explore creating and even trying to find
security defects in the system. The system, as you might have
noticed from my discussion, does rely quite heavily
on x86 machine code. It’s the sort of
fundamental part of how we make things
secure. And the graphics support
we provide today is pretty much limited
to raster graphics which means, you know,
software graphics, not hardware accelerated
BitBlt operations. And our code development tree, for those of you
who have sent us change lists, is hosted internally. It turns out that these
are basically details for the research release, and going forward,
we’re thinking much more about a developer-oriented system. So some of the implications
of that is rather than designing
the system as an NPAPI plugin, we want it to be built
directly into the browser. The web worker’s demo
that we’re gonna be showing is a first example of that. But we’re also going to be
preparing revisions to the current version
of NPAPI, things that enhance
the portability and performance of NPAPI
as it’s currently designed, so that this will be
a better interface for hosting safe plugins,
safer plugins and web browsers. We also believe very strongly
that the Web should be portable, and it is not really appropriate
to lock all future web content that uses this kind of
acceleration into a single instruction set. So we’re putting a lot of
effort currently into 64-bit x86 support
and also ARM support. Now, in terms of delivering
better graphics, we’ve got an effort– a collaboration
with the O3D team to deliver 3D graphics and make it available
to native code. A lot of developers have asked
about an open GL interface and the possibility of providing
an open GL interface in Native Client. That’s definitely a request that we’d like to be able
to respond to at some point. And then finally, the other
detail I wanted to mention is we’re moving into
a public SVN repository right alongside
the Chrome repository, which will make it much easier
for external developers to make contributions
directly into the system. So…the demos
that we’re gonna show you today actually preview
some functionality that is not in our research
release yet but is a part of what
we’re working on to provide
in our developer release. And the first example
is native web workers. Now, you’ve heard
some discussion in other sessions here
at Google IO about web workers, a way of creating threads
in JavaScript. Well, native web workers
are just like web workers except instead of being
written in JavaScript, they’re written
in native code. But otherwise, they have
the same execution model and the same interfaces
as web workers. In particular, web worker
supports no shared data. There’s no way to share data
between web workers and the browser,
and as a consequence of that, web workers can’t actually
do direct DOM access. Web workers support
a postMessage API as well as XMLHTTPRequests, to make HTTP requests
of the hosting website and openDatabase to provide
access to stable storage. And you can see lots of details about the web worker
specification at the URL
that we provided up there. Our goals in Native Client
for native web workers are firstly
to support web workers and other languages
besides JavaScript, for example, C, C++ or Ruby. Also we still want to maintain the simplicity
of the web worker model, which was very deliberately
designed to prevent the kinds
of race conditions and other kinds of issues that can make
threaded programming sometimes very difficult and provide an interface
that’s basically appropriate for low frequency applications. Things where like
you make a request and then, you know,
a second or so later, you get the information back
and can use it. So as an example of that, David Sehr, Tech Lead
for the Native Client project is gonna walk you through
what we’ve done with web workers
for Native Client. David Sehr: Thanks, Brad. As Brad said, I’m gonna
be talking a little bit about some early work we’re doing on
supporting native web workers. As you probably heard,
Matt Papakipos talked yesterday about web workers and about
the sort of programming paradigm that it supports. I’m gonna show you
how we think one possible direction
that embedding Native Client into this sort of
programming model would be used, so… So here’s a kind of
classic example of an application that
you might like to thread. It’s a compute-intensive
application. In this case,
I’m going to be showing you computing a Mandelbrot picture. And each one of those boxes is eventually gonna be populated
with some colored bits. Mandelbrot is a compute-
intensive application. Basically you’re computing
a bunch of floating point values for each one of the points
in the region. And we’ve taken this
as a little demo and panelized it by splitting each one of those
squares into four tiles. Each tile will be handled
by a separate web worker. On these modern multicore
processors, you can have four web workers, each having access
to one core, and you’ll be able to… update Adobe, it appears,
at the moment. Anyway…[laughs].
So… So here’s the JavaScript code that does a typical web worker
sort of…application, the one that I was
talking about before. There’s a doubly nested loop. That’s actually
just walking over the… each of the tiles. That’s here. And each one of the tiles
is computed by an instance
of a web worker, and that’s created
by this just new worker. And in the HTML 5 spec, the worker URL is
a JavaScript, uh, script. And then communication
is back from the web worker by assigning this
onmessage property to be a function
that takes an event. And that event will,
in this case, be the result
of the computation. Basically, it’s a big
string containing the pixel RGB values. And…then there’s some
GU here for putting the RGB
up on the screen. The host JavaScript, the JavaScript, actually,
that runs in the browser, is able to talk to the worker by sending–or using
these postMessage APIs. And so in this case,
each one of the tiles is started off. There’s an onmessage
bound for it that will in-cue the tile and eventually
put it up on the screen when the last tile
is received. And then the tile computation
is actually started by being passed a string
that says “which tile are you” and “how big is the tile?” So the code, then, in the…
the worker JavaScript has a sort of similar flavor. There’s an onmessage
that says “when I get a message
from the host JavaScript, what am I gonna do?” In this case, I’m going to do
an awful lot of… floating point stuff and then some computing RGB
values from the floating point. And eventually,
I’m going to put together… a big string that contains
all the RGB pixel values. And I’m going to do
a postMessage to pass that screen back
to the host JavaScript and the browser. So…pretty straightforward,
I think. So…our approach to this… for native web workers
will actually use the same API. The initial work we’re doing, just basically the text
using a different URL– in our case,
looking for a .nexe or native executable. That’s our name for
Native Client executables. If you see new worker
with the worker URL being mandel.nexe
instead of mandel.js, well, then you will start a Native Client
execution environment and…run the code that was
contained in that executable. Now, you’ll be seeing this again in Nicholas’s demo
in just a second. We have defined
in Native Client… there’s an RPC interface which we call the simple RPC
or SRPC interface. And SRPC methods are always
of this sort of prototype where you basically
have a list input arguments and a list of return values,
and so… what we have done for this
is we bind one Native Client RPC method to be postMessage. And so that means an RPC to postMessage from
the host JavaScript is called– it will invoke this function
called Mandel worker up here. And Mandel worker
will extract its parameters. In this case, we’re passing
strings as methods. The web worker interface
allows you to pass slightly richer
data types, but I’m not talking about
that in this particular example. So you can extract
the string value by saying the first
input argument, and it’s a string,
so you extract that. Then it does some computing. This Mandel computes each one
of the point values and produces a string. And then when it’s done
producing the string, it will call
a postMessage API, which is a postMessage
back to the browser saying here’s the result
of my string. So other than the fact
that this is C, it looks an awful lot like
the JavaScript sort of approach. We defined an RPC handler
that does some computes, and it eventually
does a postMessage back to the host JavaScript. So…the two are very similar. So…and… forgive me the… posting things to the screen is actually slower
than doing most of the computes in these things. So the performance delta
is not obvious, but if the timing
is actually done, you’ll see that the native
one is quite a bit faster than the JavaScript. But you can see the two things have functionally
the same behavior, the same host JavaScript code
is used for both the JavaScript
web worker and the Native Client
web worker. And we’ll be developing
some more applications to try and showcase
the power of native executables in the setting as the web worker support
becomes more and more mature. It should be said that–
Brad said we’re going live soon with our SV entry. The demo code
is in the repository. It’ll be going out. The web worker support will be appearing
in a Chrome release that hasn’t yet gone out. So sometime over
the next couple of weeks, we hope to get that out
so you’ll be able to try this. And with that,
I’ll turn it back to Brad. Chen: Thanks, David. Okay. So, uh…native web workers is probably gonna be
the simplest interface, simplest way to use
native code and get native code performance
in your web applications. For people
who want more interactivity, who want to write the kind
of high frequency applications like animation or audio
or video decoding, we’re planning revisions
to NPAPI. We currently have NPA support
in our current system, but to address some of
the portability and performance issues
that we’ve found with that API. Now, for those of you who are
familiar with browser plugins, you probably recognize
that use today is quite limited. There’s a lot of well-known
security issues, most of which
hopefully we’ve convinced that we can do something about– we’ve convinced you we can
do something about. Pop-up boxes asking people
questions that they really shouldn’t
be asked in the first place… you know, overall,
web portability and the ubiquity of content
just kind of falls apart. We are determined
to create a better future for native plugins by addressing some of
the well-known misfeatures of NPAPI and ActiveX. And compared to web workers, we’re going to actually
address some of the limitations
of that system. We will support shared data. We will provide the kinds
of things you need to implement high frequency
applications. And we’ll also be providing
mechanisms through NPAPI to provide
synchronous DOM access. So as a first example of how these more direct
interfaces work, I want to show you
a video player, video decoder that we’ve been working on. Now, another thing
I want to point out, and the main point
of this example, is just how simple it is to take an existing
Linux application and make it work
under Native Client. This particular video decoder
is something that had been balancing around
inside Google for a while. The original test program
took an H.264 decoded video– encoded video– and wrote it as raw frames
into a file on disk. It only took 20 lines
of source code changes to take that original
test program and turn it into a program that’s a simple Native Client
video player. About 210 additional lines
were required to add audio
and frame-rate control. But for now,
I just want to focus on the video modifications. So I’m actually
gonna walk you through the 20 lines of source changes
that it took to make this thing run
in Native Client. Firstly, in order to use the multimedia subsystem
that Native Client provides, and this subsystem, by the way,
leverages, you know, the communications
substrate NPAPI and the shared memory system
that I mentioned earlier. You make this call
to nacl_multimedia_init then four lines
of error checking. And then after initializing
the multimedia system, the next highlighted line
to nacl_video_init initializes
the video subsystem of the multimedia system. Four more lines of error
checking. We’re gonna null out a pointer so that the application
doesn’t try and write all of this video to a file
on disk. And that’s the…
initialization code. The next change we had to make is when the frame
becomes available, to actually
display it on the screen. So the original
internal representation of frames in the system
was in YV12. We use this call, which was
already a part of the decoder, to create an RGB representation. and then a nacl_video_update
takes the RGB frame and causes it to be
blitted onto the screen. the screen real estate
that was established with the earlier
initialization calls. And that’s it, actually. Those lines are all you need to actually get the video
displaying on the screen. Now, there’s a couple of
includes that had to be fixed and there was a shutdown
that I didn’t bother including. But that’s pretty much
all it took. So the next thing I wanted
to actually show you is a demonstration
of the decoder. I was actually
going to build it for you first just so you can see
that, you know, that building
a Native Client module is actually not that different
from building anything else. Surprise, surprise. The path to the compiler
is a little bit messy. But after that,
this is pretty much just a normal old
invocation of GCC. It’s got a bunch of
source files from the decoder itself, and then starting around here, we’re pulling in some
additional source files that implement the audio part
of the system. And now it’s built. So the next thing I wanna do
is actually invoke it. Now, I’m going to,
for the purposes of the demo, invoke it the way– typing the whole
command line out, we need to– and I’m not gonna be running
this in the browser, you’ll notice. I’m gonna use the debug
environment that we use ourselves and that we provide as a part
of our open source release. So…let’s see, staging… There’s a program called
cell loader which is a container for running
these Native Client modules outside the browser. I’m gonna pass it
the debug flag, and that’s because since
we’re not running in a browser, I need to get the video file
off the file system. And so the debug flag is going to actually
take down the firewall that prevents
file system access and let us access
the file system directly. If you run without
the debug flag, then you wouldn’t be able
to open any files or see any of the local
file system. Then I’ll next specify
the Native Client module which is called
G.264release.nexe. That’s what we built
up here someplace. And lastly,
the name of the video. And so we get to enjoy a 47-second ad
for Google Chrome. So a couple of things
I wanted to point out while we’re watching
the squirrel run around. This just like
the office, by the way. This is a portable binary. It will run on Windows,
Mac, and Linux, and the Native Client modules
will run in NPAPI browsers Opera, Safari, Firefox,
and Chrome. It’s entirely portable. We’re getting pretty close
to the same performance as the original decoder. And… in addition,
in Native Client, we’ve gone through
a lot of effort to maintain the kind of facilities
you need to actually do
real-time applications in a reasonable way, including access
to the time stamp counter and performance
for other system calls that allow you to manage
time effectively. So that’s our video decoder, and hopefully you all saw it ran pretty much
like a normal video decoder. That’s the whole point, right? The next demo
we want to show you is actually an example of the same kind of interface
being used but running
in a web application. So we’re gonna have
most of the user interface implemented in JavaScript, and the fast part,
the image manipulation, is gonna be implemented
in Native Code. And I’d like to introduce
Nicholas Fullagar now. He wrote this application, and he’s another one of
the lead developers on the Native Client team. Nicholas Fullagar:
Thanks, Brad. Hello. So today I’m gonna demonstrate
Native Client Darkroom. And…let’s see. And first we’ll do a little
walk-through in Chrome and launch the application. And note, it’s just like
visiting any other web page. If you have the Native Client
plugin installed, there’s nothing else you need. So we’re gonna open
the photo, and it’s a nice little picture. And…so why… why Native Client? Why would you need
Native Client for this? Let’s say, for example, this photo is something
that you’ve uploaded somewhere, and you want your friends to be able
to critique it for you… and they want to make
suggestions about, well,
maybe it looks better if it’s in black and white. Or maybe it would look better if you add little bit more
contrast to the image. These are all
nondestructive edits, so we still have
the original image available to us. This is nice, because if you have it
sitting on the server, you don’t want to start
creating multiple copies
for every edit. You really just want
to send around the values of the sliders
that people have suggested. So…these sliders
are implemented in JavaScript and HTML, and they control most of
the GUI for this application. This area here
is in Native Client. And it’s running
in the native C code. And you can see
it updates pretty quickly. We can zoom in.
We can zoom out. We’ve got bitmap scaling
done in Native Client. We’ve got some
geometry operations. You can rotate the image. And these are all occurring
fairly quickly. And this is modeled after
some of the popular photo editing software that you may have
experience with. So the other important
thing to keep in mind is that this same web page will work on Linux, Mac,
and Windows as long as they have the Native Client
plugin installed. So… next, we’ll look at the… source code
for a little bit. Here’s the photo HTML code
and… Oh. What we’re gonna
look for here is… what happens
when you update the angle. So… here we have
a little bit of HTML that sets up the slider
that the user can adjust to change the angle. And we’ve added
an update function, a JavaScript function that gets called every time
the slider gets changed. That calls
this JavaScript function which passes in
the value of the slider and… what we’ve done is we have
a Native Client object which has simple
RPC calls bound to it. And one of these calls
is Updateangle, and we pass along
the value of the slider. This now jumps into C code. And as David showed earlier, we have these SRPC methods
that we’ve set up, and… we’ve got the name
of the JavaScript function, of the JavaScript binding, and it passes one argument,
which is an integer, and it calls this C function
in our C code. So jumping to that C function, we’ve got the parameters
coming in, and we know
the first parameter is going to be an integer,
so we extract it here. And we convert that
slighter value to a float and then we pass it to
the darkroom class and call it setangle function. Now, it should be noted this SRPC is happening
asynchronously on a listening thread. So we’re gonna… quickly just pass it over
to the darkroom class and in its
own application thread, when we’re rendering the photo, we’ll take into account
the new angle. Now we’re gonna make
a bit of a jump, and we’re gonna go straight
into the low-level code that does the pixel processing
when that image rotation occurs. This is, again, in C code. And we do the rotation
as a product of three shears. We do two horizontal shears
and a vertical shear, and that makes some of the
filtering a little bit easier because we just filter
in one dimension instead of having to filter
in two dimensions. So this is–if you’ve written
low-level pixel blitting code, this should look
pretty straightforward. There’s quite a bit
of setup code here for calculating
what the filter value should be. And if we go further down, we get to the actual
pixel processing loop that takes the image
and does the shear. So for each pixel, we’re gonna read the source, and if we go outside of range, we’re gonna read
the border color. And then we’re gonna apply
some SSC intrinsic code, vector instructions, to apply the filter that’s
necessary to shear the pixels. Then we combine the result
and store it in the destination. So this is what’s driving
the low-level… image rotation
that we looked at earlier, when I make this slider move
and we see the image rotate. So this…example
will be online soon. Keep your eye on, uh… Google Code and, uh… thanks for listening. Thanks. [applause] Chen: Okay, so…with that… we’re pretty much
at the end here. Our project, like I mentioned,
has been open source since December, and we would really
be delighted if people had ideas for how
to move the system forward or had an application
that they thought would benefit from the speed of native code
in the browser. I wanted to mention at 1:00,
which is in about 15 minutes, there’s a Fireside Chat where some of the product
management staff will be taking questions
about Native Client and other client technologies. And we’ll be holding
office hours, the Native Client team,
at 3:00, and we’d be happy to tell you
more about these examples, other examples
in our open source release. Or if you’ve got
a piece of code or an idea for a system that you think
might benefit from this, we’d be there to talk
with you about it. So right now we’ve got plenty
of time for questions, and we’ll be happy to hear about questions or ideas
about Native Client. You guys–
can you guys join me? man: In the mic, or–
Chen: Please, yeah. It’s being filmed. man: You mentioned explicitly
encryption, and you said eventually. Could you elaborate? Chen: Uh… well, I guess only a little bit, since we don’t have
any specific plans. But there’s a high level notion
of, you know, today SSL takes an entire
web page and encrypts all of it. And there’s situations
where it might be beneficial to actually
only encrypt part of it. But doing that over SSL
is messy. If, on the other hand, you had an encryption tunneled
through an HTTP request, it would give some more
flexibility. man: My specific problem is encryption and signing using X.509 certificates
under JavaScript control. And right now,
I’m forced into using IE with ActiveX controls. And I would love an alternative,
if that’s on the horizon. Chen: Very interesting. Well, the one thing that I can
promise you about Native Client is if it’s pure computation,
then we can deliver this almost as fast as what you get
with the native CPU. So this sounds like actually
a pretty good application. man: Thanks. Chen: Other questions. Other questions
about Native Client. man: Hi. What’s the lifetime
of the web worker itself? Like how many pages
does it remain? If I move forward, how long will the web worker,
the native version operate? Chen: That’s a good question, and David might know the details
of this better than me. The lifetime of a web worker. Sehr: The lifetime
of the web worker is actually, once the object
has been created, as long as there’s
a reference to it, it’s basically
a reference-counted object in the JavaScript
execution environment just like anything else. So as long as there’s a handle
still held to the worker, that process is still alive. There’s some startup
and shutdown capabilities as well, so I can send
an explicit message to shut down the web worker,
but– man: If a navigator
the page forward, and if I come back… Sehr: We’re not currently
working on persistent web workers
that Matt talked about. So in that case,
you’d be sort of… you’d restart the web worker,
yes. Chen: But there is this proposal for a thing called
persistent web workers which would give
a worker thread a lifetime that’s independent
of the lifetime of any particular page. And as that kind of progresses in terms of
the JavaScript version and in terms of standardization, you should expect the native
web workers to follow. man: Thanks.
Chen: Any other questions? No? Okay, well, enjoy
the rest of the Google IO and thanks very much
for listening to our talk about Native Client. [applause]