What Do Software Engineers Do?

“What does it take to build a site like Artsy?” I’ve been asked this question a lot during my five years leading our engineering teams and my answer tends to be straightforward: a seven-figure dollar value and a 20-plus engineer headcount. When pressed for more, I prefer to address the deeper question, which resonates far beyond Artsy: “What exactly do all of those engineers do?” In this post I will try to answer that question.

Software is about the practice and discipline of constructing. You build a layer and then climb up and build the next, and while it’s possible to remodel the foundation, it’s hard once you’ve built it.

A common misconception is that engineers just make websites. They certainly do, but the creation of a website such as Artsy.net is more metaphorically similar to building a real-life skyscraper. Software is about the practice and discipline of constructing. You build a layer and then climb up and build the next, and while it’s possible to remodel the foundation, it’s hard once you’ve built it. A skyscraper is equal parts art and science, form and function. Software reflects these intersections: code fits together beautifully and simply, algorithms and information are architected in, and it ultimately serves a purpose to the user. 

To illustrate this metaphor, consider the famous Flatiron Building in Manhattan. It’s a triangular 22-story steel-framed landmarked building and is considered to be a groundbreaking skyscraper. Upon completion in 1902, it was one of the tallest buildings in the city at 20 floors high and was called the Fuller building. It was both a functional office space and a true work of art, hiding all its complexities behind the scenes and leaving only the polished beauty to inspire you.

Before the Fuller building, the physical site in front of Madison Square Park hosted billboards made of electric lights. Similarly, Artsy.net began as Exhibytes—a splash page with some basic wiring. A single engineer wrote the HTML (HyperText Markup Language) and CSS (Cascading Style Sheets) code that a browser knew how to display. HTML and CSS are so-called “markup” languages that include ways to tell a browser where to show the Artsy logo or what typeface to use.

A software engineer had to figure out how to design and develop a custom search engine for artworks and artists. This was similar to the construction of the narrow part of the Flatiron building: There were fewer windows to worry about on this side, but their shape was quite particular. At the same time another engineer worked on the artwork discovery experience. They chose a more powerful programming language to write code in (Ruby), a web framework that retrieved data and produced web pages (Rails), and a database (MongoDB), to store information about artworks and artists. They wrote tens of thousands of lines of code within multiple systems that sent information back and forth, and that all added up to an effortless experience of discovering art online.

Knowing what languages or tools to choose was part of the job of early software engineers at Artsy. Experienced ones brought their knowledge to the making of Artsy.net and devised a workflow in which a feature could be added quickly. For example, they chose Git, a system that allows many software engineers to add code to a project without stepping on each other's changes. Similarly, site supervisors brought their experience from previous projects and designed the flow of concrete trucks arriving to the intersection of 23rd Street and 5th Avenue during the Flatiron building construction that ensured a constant supply of fresh cement.

Each part of the codebase grew in complexity with more users and data. In 2015 Artsy engineers rewrote many systems, ranging from the Content Management System (CMS) that our partners used to upload artworks to the mobile app that hundreds of thousands of people enjoyed every day. It had to be done “in-place,” without ever taking the website down, similarly to how the original cast-iron birdcage elevators were replaced in the Flatiron, one elevator or one application at a time. While the Flatiron building elevators became noticeably safer and faster, it was a compromise in cost and time—the hydraulic power system in the Flatiron and Artsy’s data aggregation systems were yet to be updated, amounting to a significant amount of so called “technical debt”. Software engineers made similar compromises by deciding what code to write and what work to postpone every day, based on product priorities, available time and key business indicators.  

Why does a small technology company need 20 engineers? Why will it need more? A fast-growing business causes the number of applications and moving parts to increase rapidly. Complexity and sophistication is constantly on the rise. Meanwhile, old systems always need to be maintained, evolved or replaced.

Next time you see a website or an app, think about it as a construction project. It may be a small house or the tallest skyscraper in the world, and a team of one or more software engineers is behind it. I hope this metaphor was helpful to you!