Developer Experience (DevEx) is at the heart of how software teams work and thrive. It encompasses everything from the tools developers use, to the processes and workflows they follow, to the culture of the teams they work within. And at its core, DevEx isnât just about productivity. Itâs about empowerment, satisfaction, and the ability to do meaningful work without unnecessary friction.
In this asynchronous interview series, we explore DevEx through the eyes of the people who live it every day. Iâll be speaking with software professionals across roles and industries to uncover how they define DevEx, the challenges theyâve faced in creating good developer experiences for themselves and other developers, and the creative ways theyâve sought to improve the experience of developing software in todayâs modern professional landscape.
Our second interview is with Platform Engineer Annabelle Thomas Taylor. We hope you enjoy the conversation!
Kristen: Thank you for agreeing to have this conversation about Developer Experience with me, Annabelle! I reached out to you for this interview really specifically because a few years ago, I got to watch you successfully transition from a software development role into a DevOps role, which I still think is just so impressive. I think it would be interesting to examine the differences between DevEx in those two different domains of shipping software.
Before we really get into it, though, can you tell the audience a little about yourself?
Annabelle: Itâs lovely to chat with you, Kristen! I like to think of my professional experience so far as a meandering walk through the traditional development stack. I began as a front-end developer primarily focused on building data visualizations, and gradually began to wonder how my data got from Postgres to the browser. So, I took a full-stack role to find out. There I got a taste for backend programming, but also a load of questions about how and where we were actually deploying our applications. That opened the door to CI/CD, cloud infrastructure, container orchestration, and the rest is history. A classic âif you give a mouse a cookieâ scenario (if we were giving mice Docker containers, I suppose).
Outside of work, I love playing the bassoon in my community orchestra, watching Jeopardy!, and hanging out with my toddler, Arthur. Heâs really been on a board-book kick lately, so most evenings youâll find us reading Brown Bear, Brown Bear, What Do You See? for the umpteenth time. Itâs always a party.
Kristen: Okay, I love that you have this âHow does this work under the hood?â mindset, as reflected in your career story. I imagine this is part of what makes you an excellent engineer. A former colleague taught me the (totally positive) term âcurious nerdâ, and you, my friend, are a curious nerd! Meant as the highest compliment.
(Also, Arthur sounds delightful! Iâll be eager to know if your love of learning rubs off on him đ)
So, I think when a lot of people think about DevEx, they think about the factors that cause friction and frustration during the process of developing and shipping software. The authors of the original DX Framework have proposed a set of DevEx metrics centered around feedback loops, cognitive load, and flow state:
Image Source: DevEx: What Actually Drives Productivity, 2023
As the chart indicates, we can look at these three through the additional dimensions of human attitudes and opinions and system and process behaviors. Now, this chart isnât exhaustive â these are just some examples - but I think they illustrate the kinds of things that can affect a software practitionerâs experience of building and shipping software.
In your experience, which of these DevEx factors are heavily shared between the software engineer/developer role and the DevOps engineer role? What factors do you feel are fairly distinct?
Annabelle: The flow state metrics really stand out to me as a shared DevEx factor, particularly âthe perceived disruptiveness of being on-call.â In my experience as a DevOps Engineer embedded on product teams, thereâs something of an open question as to who all should join the teamâs on-call rotation. Often the question is framed as âare the alerts due to application code, or due to infrastructure?â but it isnât always so cut and dry. As a toy example, letâs say an endpoint starts eating up memory, but there isnât an autoscaling policy to handle that load or a health check to terminate the instance. Is that a âsoftware problemâ or a âdevops problem?â Honestly, itâs kind of both.
The metrics around feedback loops resonate more with me when I wear my Software Hat, whereas those around cognitive load ring true for DevOps Annabelle. When I was working in a particular codebase every day, I had a much larger personal stake in, say, how long it took to run integration tests than I do now. I think itâs easy to fall into an âout of sight, out of mindâ trap, and I try to stay cognizant of that.
Now, regarding cognitive load: once I took on a DevOps role I found myself relying on patterns across multiple applications to work through problems. Production issues donât always happen in a vacuum. If one microservice is having trouble connecting to a central data store, itâs likely that other microservices are too (even if they arenât monitored and observable, but thatâs another can of worms). I try to lean on what Iâve learned from prior incidents when assisting in new ones, especially if I havenât worked in that particular codebase.
I suppose a rough distinction that Iâve seen is âsoftware friction occurs in depth, devops friction occurs in breadth.â And no one is having a good time when an alert goes off in the middle of the night.
Kristen: Amen to that, sister đ
You mentioned that you âtry to lean on what (youâve) learned from prior incidents when assisting in new ones.â For me, one of the hardest parts of being a software engineer was having to remediate production failures; chasing down bugs that have brought down parts of the system (or the entire system!) can be high-stress, high-stakes, and involve lots of stakeholders peering over your shoulder, watching for progress, frequently asking for updates.
For me, this situation induces anxiety, and for me and many others, anxiety impedes learning (lots of research around this in multiple domains). And so something Iâve found in my career is that some of the systems I have to use to debug and track down production-system issues are ones that Iâm really only interacting with when thereâs a production issue.
Iâm curious: Do you engage in any kind of focused learning in order to prepare yourself for the big, high-stress fires that sometimes occur and that â as a DevOps engineer â youâre often pulled in to help address?
Annabelle: I actually feel a bit sheepish in answering. In short, I donât. I used to attempt what was very unfocused learning to prepare for incident response, but that just looked like frantically jumping from intro doc to 101 video to blog post about a smattering of technologies, then panicking when it wasnât all immediately clicking. I felt like I needed to be prepared for anything, but that isnât realistic. There will always be something in an incident response that surprises you.
I think I actually got a lot better at incident response and management when I let go of that super-preparedness. Because Iâm never going to be fully ready, Iâm as ready as Iâm going to be. If anything, I focused on learning who all on my team was the âgo-to personâ for databases or Kubernetes or IAM permissions or what have you. When I donât know the answer, I focus on staying calm and coordinating the people who might have the answer. When I keep a clear head, Iâm able to pick up clues as to what might be worthwhile to brush up on after the incident is resolved.
Kristen: Not sure if you have dreams of being an engineering manager, Annabelle, but this response positively smacks of âleadership materialâ đ
You mentioned in an earlier response that âsoftware friction occurs in depth, devops friction occurs in breadth.â Can you explain a little more about this for me and our readers?
Annabelle: âSoftware friction occurs in depth, devops friction occurs in breadthâ â thatâs probably a wild overgeneralization. But Iâm thinking of how, when I was solely focused on one or two domains as a software engineer, I was best served by deeply understanding the full stack of that particular problem space. If I could cobble together a script that made my particular tests run faster, that was a huge benefit. However, as a devops engineer I find myself tripping over project-specific idiosyncrasies.
For example, when I was making the transition from software to devops, I was a big fan of AWS CDK (and to a degree, I am still a fan!). CDK is an Infrastructure as Code (IaC) tool very similar to Terraform, but it is written with traditional software languages like TypeScript and Python. I wrote all of my teamsâ infrastructure with AWS CDK, and for those projects, it was great! The app was written in Typescript, the infrastructure was written in Typescript, and the developers were closely involved in the infrastructure maintenance. However, when the larger DevOps team tried to implement organization-wide updates across our IaC (like updating mandatory tags), we had to chase down infrastructure written in multiple tools, maintained in multiple repos, and deployed in multiple ways. Grappling with all of these configurations is tricky, even if each configuration serves that particular software team well.
Like I said, though, pretty broad generalization. This isnât to insinuate that software engineers are inherently selfish in their coding practices, or that devops engineers are mandatory fun-haters!
Kristen: Okay, that makes so much sense - thanks for clarifying that! Should we make a shirt that says "DevOps Engineers are NOT mandatory fun-haters"!? I feel like this could be a big seller at KubeConâŠ
Annabelle: The shirts would practically sell themselves!
Kristen: What kind of work do you do as a DevOps engineer to remove friction around CI/CD processes for the teams you support?
Annabelle: A good CI/CD pipeline strikes a balance between thoroughness and speed. In other words, you want to vet what youâre sending to production, and you want to be able to validate how it was vetted later on. But an overly long and complicated process can cause understandable frustration!
I work with my teams to determine which processes can run in parallel and which need to be sequential. For example, in many cases all tests can run simultaneously; unit tests wonât conflict with integration tests or snapshot tests. But if you also have end-to-end testing, or multiple suites that make changes to a test database, you donât want to risk race conditions by running those all at once.
When Iâm scaffolding new pipelines, I like to start by focusing on the run conditions. Each job will simply start with an echo command saying âIâm the database migrationsâ or âDocker build goes hereâ. That way, I can make sure the DAG (short for Directed Acyclic Graph) logic is working exactly as I expect. Once thatâs set, I can tackle actually adding the migrations or the build in bite-sized chunks.
As far as decreasing build times, Iâve become a big fan of utilizing multi-stage Docker builds. Depending on how you structure your Dockerfile, you can rely on cached layers for less-frequently changed files. You can also maintain separate images for, say, testing versus productionârip out the tests, dev dependencies, and whatnot from your production image and youâve got a smaller image thatâll ship faster.
Full disclosure, creating a multi-stage image doesnât always save me tons of time. But starting that as a practice did help move Docker images from âmagic boxes of computer stuffâ to âwhat exactly do I need to run this program?â So still a worthwhile exercise, in my opinion.
Kristen: Iâve come to really appreciate and (full disclosureâŠ) just even understand the value of layer caching in the build process. Depot just released a remote caching service, in fact, that utilizes a build cache to implement incremental builds and accelerated tests. Itâs wild how much faster these builds are running using this strategy.
Alright, Annabelle - one final question for the road: What developer tool most helps you get shit done? If you had complete control over what dev tooling you use (and maybe you do!), which dev tool would be non-negotiable?
Annabelle: This is tough â there are a lot of tools I love! But many of them fit into the category of âstrong opinion, loosely held.â I love Obsidian for note-taking and Typescript for development, but I can be happy with any link-friendly notes app or typed language. So, if we throw ânon-negotiableâ in there, Iâd have to go with my Kubernetes tool of choice: K9s. It strikes the perfect balance between friendly UI and bare-bones CLI, so I can quickly navigate my clusters or drill into a particular resource. It was one of the first things I set up on my laptop when I started my new job.
If I may, I want to give an honorable mention to the actual first thing I installed: my favorite VSCode theme! It surprised me how quickly having a familiar environment helped me feel empowered to navigate a new codebase. I think itâs the digital equivalent of setting up photos and knick-knacks on your desk to make it feel homier. There might not be a quantifiable impact of installing one theme over another, but having a developer experience that feels uniquely yours (no matter how minimalist or maximalist) can make all the difference.