The Node Module Diaries: Large App Architecture from the Trenches

Jonathan Lipps

Director of Engineering, Sauce Labs

The Node.js community has evolved a number of application design patterns for JavaScript applications, including application-internal package management patterns. The all-around awesomeness of NPM has lent itself to a hyper-modular approach for Node code not seen in many other language communities. This is great, especially when writing open-source tools designed to be used by others’ applications. Meanwhile, the “microservices” or “distributed systems” architectural patterns have begun their ascendancy. It makes sense to isolate components so they can be managed independently of one another. One could be forgiven for thinking that “many modules” and “microservices” are similar answers to similar questions, but in fact they are completely orthogonal, as we found out the hard way.

Recently, the Appium project, a large open-source application, decided to rewrite its codebase using ES2015, the newest version of JS. Not only that, we wanted to do a complete architectural overhaul, and in the process avail ourselves of all of the amazing new practices we’d heard lauded during conferences over the previous years. We went for it whole hog, and six months later we had our new application—all 45 modules’ worth. It was beautiful! And it solved many of our problems. But alas, it came with new ones we had not anticipated in our headlong rush towards the hottest new ideas.

In this talk, I’ll tell the story of what worked and what didn’t. What were the goals of our rewrite and how did our choices deliver on those goals? Specifically, how did ES2015 and the “many modules” approach work for us? What could have worked better and where are we going from here? What tools did we use or invent to cover over any drawbacks of our approach?

A degree of caution is important for any rewrite, especially when you are tempted to choose new languages or patterns because of their tantalizing buzzwordiness. On the other hand, there are many things you cannot know before you jump in, complete a project, look back, and ask how it went. What we learned is that the accepted Node wisdom, at least as we internalized it, doesn’t apply to every kind of application. We needed to try things out and allow reality to humble us in order to eventually find the right patterns for our specific use case. Please, allow the false starts and dead ends we encountered to help you when you come to a rearchitecture of your own!

Jonathan’s talk is now available on the Chariot Solutions site.

Tags: , , ,

Location: Salon E
April 12th, 2016
1:30 PM - 2:30 PM