Node.js Now with Dan Shaw
Articles,  Blog

Node.js Now with Dan Shaw


[APPLAUSE] DAN SHAW: How’s everybody doing? So before we kick off into my
talk and my janky graphics, is this on? Can you hear me? AUDIENCE: Yeah. DAN SHAW: Great. I’m going to kind of lay out
where we’re going this evening. So we have a very interesting
kind of eclectic mix for you, and that very much
represents the world of Node. We’re in the enterprise,
in embedded devices, in mobile, all over
the place, so I’m going to share some of my
experience, some of what I’ve been helping grow
in the Node community. And then after me, we’ll
have Trevor Norris, Node core contributor, and we’re
going to go really deep into some core stuff that
is sort of very low level. And then we’re going
to come up from there, and Eric Toth of Paypal
is going to share some of the experience of
the transformation that’s taken place at Paypal and
really the pervasive Node that is driving Paypal now. So it’s really
exciting, and we’re going to close out with
Guillermo Rauch, the creator of Socket.io and WebSockets and
connecting it back to Kaazing. This meetup is
really special to me because it connects back
to the very earliest parts of my career that
were with WebSockets. I, unfortunately,
made the choice to go, not with Java, but over
to this Node thing, and it was WebSockets
and Socket.io that really kickstarted
that part of my career. So we’re going to bring it home
with Guillermo and WebSockets and Socket.io. Socket.io got to 1.0
before Node got to 1.0, so that’s really exciting and an
integral part of Node’s growth. But that’s the evening. Let me dive into Node.js
and what’s going on. So I’m DShaw. I am the co-founder of a
company called NodeSource. We do Node for the enterprise. We help large companies
run Node effectively. Before that, I
co-founded a consortium, a group of individuals that
came together early on in Node’s history to provide professional
services for the Node firm. And I’ve worked at a bunch of
startups with Node, at Storify, Spreecast, Voxer, ClassDojo. And if you enjoy hearing me um
and ah and all that stuff you, can listen in your eardrums to
me on my podcast called NodeUp, and I run a couple
events around bots and Arduino stuff, nodebots.sf. Nebrius is here. I don’t know if
Miles is also here. Great group of people,
where we come together, and it’s really kind of the
antithesis of this event. This event is special
person dropping knowledge on other people. The Node bots, I compare it to
getting together and basically building LEGOs with friends. Close as I can get to that. It’s just hacking,
sharing, and hanging out. So NodeBots Day and NodeBots.sf,
it’s very special, kind of very different,
meetup that I created. So let’s dive in and see why
Node has been so important and why Node is relevant
and why Node will continue to be relevant for,
let’s say, 10 years. I’m not going to say that
Node is going to be relevant forever, but there’s a
incredibly positive story for Node in the next 10 years. And the foundation
anchor of that is because Node is a
platform for running JavaScript on the server. It has increasingly been a part
of how we build applications. With the dawn of
HTML5, we really now have the tools to build
what we want with JavaScript on the web, and that is driving
ever more greater importance of JavaScript and the need for
dynamic and effective backends. On top of that, we’re
building services that need to be responsive. I’m using– I’m stealing the
responsive word from responsive design to illustrate
a pattern that we’ve seen in the backend, where
perhaps you have a API endpoint that returns
a lot of data. Awesome for your web app. Really horrible for
your mobile application. Mobile application,
you need something that’s tailored for that. It’s been an area where
Node has really excelled. You can craft and deliver
content that is precisely targeted to the
needs of the device, and Node is an incredible
lightweight engine to drive that. The Internet of Things
has been another factor in increasing the
relevance of Node, the democratization
of programming, really, with JavaScript. And it has been augmented by
the democratization of hardware with Arduinos and
Raspberry Pi’s, so connecting that to our
services and our infrastructure has been really compelling. And beyond that, there’s
new opportunities inside devices, inside of
switches, inside of the devices that are powering the internet
that is ever more increasingly important. So we have this
beautiful language, beautiful with warts
language, JavaScript. But the fantastic
thing about JavaScript, that it’s incredibly
approachable and everywhere, and it powers all
of these platforms. When I went through
the transition from being a Java,
JavaScript developer, middle tier developer, doing
the backend services in Java and the frontend
services in JavaScript, I was blown away, once I moved
to JavaScript on the server side, how fast I was moving. It was absolutely incredible. So this rapid cycle
time, and what I’d like to call the
time from ideation to realization, really,
really becomes much smaller when you’re working
with a dynamic language, if you want to get
language geeky. But JavaScript is a
fantastic expression of that. And then if you’re looking at
JavaScript and its relevance, this is being reinforced
and becoming a competency that business leaders are
looking for and hiring around. And the fact that, now, we
have frontend pervasive, tooling pervasive,
backend pervasive– it’s growing and snowballing into
a really relevant career path. So as we explore Node,
let’s go and look at what it took to get
here because it was not all roses for the
entire history. Back in 2009, we had the
original presentation of Ryan Doll’s hacks. 2010, when I started adopting
Node and then left my job and then basically came out
San Francisco and abandoned everything to come
work on Node– I’m not allowed to curse. Things were really broken. The ecosystem was
really fragmented. We didn’t have a
package manager. We had Kiwi, which is kind of
janky, from TJ Holowaychuk, and Isaac came along with NPM. Thankfully, we kind
of came together and decided to
have one mechanism for delivering packages. Your promise lovers, we
got rid of promises then. So we explored a
lot of patterns, broke a lot of
things, and really brought together a
fantastic platform. Fast forward a year, and
2011 with the .6 release, we introduce Windows. And we don’t have
laptops out, but there’s a sea of silver
in here, I’m sure, if you pull your laptops
out, and the systems that are not Macs are
probably Linux machines. But if you go outside
of the West Coast, you go outside the Bay
Area, Windows still dominates the world,
and if it weren’t for Microsoft’s
investment in Node, I honestly don’t think
the platform would be where it is today. So I am really grateful
for that evolution in Node, even though I spend very little
time in the Windows ecosystem for Node and its success. If we didn’t have that cross
platform ability, if you look at other platforms that
have chosen sort of eschew and ignore Windows, they
haven’t evolved as much, so this is a pivotal
point in Node’s growth. 2012 is all about the real time. Socket.io is booming. Everyone is working with
Node, building great things. 2012 spent a lot
of time with Steven out at [INAUDIBLE] not Node, but
great architecture, fantastic. Real time has been
pivotal to Node’s growth. The low latency IO is just
a fantastic fit for that. Then in 2013, we
introduce the enterprise. Walmart comes along, and
they have their Black Friday event running all
their mobile, I think, large portion
of their mobile traffic through Node and having it
be a rock solid Black Friday. That really encouraged many
others to come along behind it, and so you had Walmart
and Paypal really paving the road for large scale Node. And 2014, it’s really
become pervasive, and one of the
default choices where new development, new
platforms, are growing and the productivity that
is being done in Node is letting these
organizations choose that. And all of the foundation of
why we’re building JavaScript applications has driven
the growth of Node. So as were looking at
building applications, let’s see what has led to Node’s
success in these environments. If you look at large systems
that have evolved over years, and you look at– I’m going
to pick on Java a little bit. –you look at Java apps, it is
typically one big monolithic application. And I know dyed-in-the-wool Java
developers will say that you can go POJOs. You can go simple minimalist
Java services– awesome. Engineering good practices
are, in a way, universal, and you can apply them
in most every language. But typically, what you
see in Java applications, Ruby applications,
PHP applications, are large monolithic
applications, and this tends to
slow everybody down. And over time,
having all that code, having to go through
this bottleneck, these applications were
incredibly behind schedule, and in fact, six months to
two years behind schedule with some of these
other platforms. The reality is that
shipping code wins, and the time to
execution, the time to realizing that
engineering effort, is greatly reduced in building
out an application that– I’ll step away from Node. We’ll latch onto the
principles of engineering. So a smaller thing, you can
iterate on it faster, period. Right? Node tends to encourage that. So one of the biggest
impact in the last year or so areas of Node’s growth
has been the frontend backend. It’s a really odd slice of
the application architecture, but these rich client teams
that have been building out, these fantastic dynamic
JavaScript experiences, needed something more. The monoliths that were
driving the delivery of these applications
were just too slow. You need, as a
frontend engineer, to rapidly iterate on
the customer experience to respond to their needs. So the introduction
of a frontend backend– it’s a backend
for your frontend services. So templating and CSS
and all that stuff is moved out of the monolith,
the big application, into this lightweight
dynamic Node layer, and it’s not incredibly
sexy code, there. It’s very utilitarian, but
it’s extraordinarily functional and absolutely allows these
user interface engineers, these frontend teams, to
operate in a whole new way. And from my experience
as a developer that did backend services
and frontend, the ability to also
think in JavaScript is insanely liberating,
so let’s look at the impact of the
frontend backend. These engineers that are
crafting the user experience are the front line of
what everyone experiences, so it’s extremely important
that area of the code base be dynamic. Your backend services that talk
to a database or data schema probably doesn’t evolve
weekly, but your frontend may evolve moment to moment. There’s a baseball game. There’s an earthquake in Napa. You need to address
what’s changing and deliver an experience
that is appropriate. So Node is empowering
these frontend teams by allowing us to really
put in front of them the tools that they
need to deliver the experience
that they want to. So we’re able to
pull away from some of these weird architectures
that are delivering HTML, HTML5, whatever, HTML,
CSS, and the JavaScript. If you’ve gone to Java Land or
C++ Land and had a C++ or Java abstraction for CSS that you
have to– it’s a nightmare scenario. You can’t take your expertise
in all these technologies and apply it. It’s so incredibly
separate there. So we go in, and we reduce
that crazy abstraction, get rid of that, allow
frontend engineers to operate within
their specialization and be able to focus
on what they’re doing. They don’t have to go
through a middleman, essentially, to
produce the experience. You can rely on the
knowledge and your expertise that you’ve built
up over your career. So the biggest thing for me
in going through my transition from having to do backend
services in Node– in something other than Node,
and delivering a frontend experience is not having
the context switch. I would go build UI components,
and then on a separate day, have to go and sort of go
into Java mode database, ORMs in Java, Hibernate,
all that stuff. It was really, really
mentally challenging to go back and forth. Enjoyed programming
in both, but not having that context
switch– a lot of times, we’re crafting API
elements that we’re delivering new
experiences to the user. You can rapidly
iterate on the backend, rapidly iterate on the frontend,
and really pull together a fantastic experience without
that cognitive load that’s involved in going back and
forth between platforms. So what are the ways
that we can really embrace Node, incorporate
Node into our ecosystem? How do we accomplish
this frontend backend? One of the biggest
things that I’ve seen in the success
of a frontend backend is the use of an API, having
an effective abstraction layer for your data. I’ve thrown out the concept
that databases must die. I got a lot of
backlash for that, but essentially,
what I mean there is that you should
be talking to an API. You should not be
exposing– and again, this is not Node specific. This is engineering in general. You should be exposing,
to other tiers in your architecture,
an API, and then if it’s your responsibility
in the various tiers of things to talk to a
database, that’s fine. There, you have
database connections. But if you’re doing
template rendering and rendering JavaScript
files and CSS rendering, you should probably
focus on just that. Separate out those concerns. Focus on that. Create a lightweight
API layer that talks to your data
services, aggregates them, and really focus the areas. You’ll design better, smaller,
more effective applications that way. So that ends up basically
looking like proxies all the way down, so
the smaller apps– it can feel a little bit weird. But there are– once
you break things out into these API
layers that are talking, you’re essentially
handing off the data through these various tiers. So if you’re looking at
an application that’s going through all
these hops, that is a very effective
pattern in Node. And if you need to
take a data point away for, oh, my god,
latency bad, network bad in an internal
network with Node, you’re looking at about six
milliseconds added latency going through Node
because Node is very, very efficient at doing streaming
I/O. So that allows you to have very fluent interfaces for
communicating with their data. To do this, we’re using
a dynamic language, rolling this out into any scale. Do yourself a favor–
don’t set aside testing. Test early, test often,
and to make that effective, automate it. Automate your testing. Have that baked
into your ecosystem. Automate the process
of shipping code. I’ve always been a fan
of continuous integration and continuous delivery. Not so much for
the fanciness of, oh, we’re shipping code
every xyz times a day. The reason why I love
continuous integration and continuous delivery is
it forces you to do this. If your code is going
out all the time, and you don’t have
effective tests in place, someone’s going to be
unhappy, so they’re great patterns that allow
you to work effectively. So Node is really
growing and becoming an essential part of
how we build things from servers to build tools
throughout our ecosystem because of JavaScript, and
I’m grateful for the ability to write JavaScript everywhere. Fantastic. Thank you very much. [APPLAUSE] DAN SHAW: So I’m going to
bring out Trevor Norris. And I guess as Trevor’s
getting set up– we’ll have a larger
Q&A panel later today, but if you want to
ask any questions, happy to take one as Trevor
recompiles his kernel. Linux user. Yes? AUDIENCE: [INAUDIBLE]? DAN SHAW: Right. AUDIENCE: [INAUDIBLE]? DAN SHAW: Sorry, I need
to repeat the question. So the question is
there’s 10 years out. What is out there? I think that would
be driven by what web technology’s exposed to us. The web is the
driver for JavaScript and one of the reasons why
JavaScript got to this level. If we begin to see
pluralities in the web– I’m at Google, so Dart. I don’t think it’s
going to Dart, but if the browsers can
introduce a language that begins to become as
pervasive as JavaScript, I would look to that. On NodeUp, we have
a show scheduled for next week on Node versus Go. Again, I don’t think– Go
is a fantastic language. It really does a beautiful
job at being a better C, but it was designed to
be a systems language. Node was designed to be
an applications language. Node’s extremely,
extremely effective at being an
application language; OK at doing systemsy things. At NodeSource, we
actually end up recommending that
a lot of companies do not use Node for
all their things, their systems tools that
are really good and really effective at doing what they
do, and build your applications. Use the JavaScript
and that Velocity to build your applications. But it is not a
universal panacea. It’s a fantastic tool, but
in every large installation of code that I’ve seen,
there’s an incredible plurality of languages that
continue to exist. And even though JavaScript has
taken over a lot of the new growth in many of
these instances, the presence of Java and
presence of C++ at your larger code bases are still there. So what comes in
after JavaScript, in my mind, whatever the web
drives in and very possibly could still be JavaScript. Yes? AUDIENCE: [INAUDIBLE]? DAN SHAW: So the
specifics of what are the effective tools for REST
API’s with JavaScript, right? So the contract patterns
there– I think, actually, Eric Toth can
probably speak better to this. I tend to be a little bit
lower level than that, but I believe they have
been exploring RUM. The API contracts, what
are the expected patterns? That is an area that the Java
world has really excelled at, and that the JavaScript world
has done less of a good job at addressing and tackling as
a fundamental part of what’s going on. So schemas and– what is the,
I know you guys are using, for your REST API’s? Swagger? Swagger seems to be
increasingly popular. I do think there’s a– I
always get a nice cringe when I mentioned WADLs,
you know, WSDL, WADLs. WSDL was really
influential in introducing us to service oriented
architectures. WADL is XML to describe
JSON, and that sucked. So it’s not a great pattern,
and if you ask any JavaScript developer about XML,
they’re going to back away. But I do think there’s a lot
of room for improvement there, and if you look at API
definitions, Discovery, and a lot of the things that
came out of the SOAP era, the patterns for
that have not really concretized in JavaScript,
even though the use of it is probably even greater. So we’ll be back. We’ll have a panel and
probably some discussion. Hand it over to Trevor Norris. I’m going to steal my water. TREVOR NORRIS: Thank you. DAN SHAW: Thank you.

9 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *