1

How to push for changes when not directly in a leadership position?

Profile picture
Senior DevOps Engineer at Taro Communitya year ago

Hi Taro,

I'm cross posting this from the premium slack because it was raised that the answers might help the broader community.

I work for a small company - the engineering org is approximately 60-70 people all told. The company is about a decade old, but has grown more recently, and I joined the small SRE/Developer Tooling team within the last year. Historically, the company has operated at a relatively slow pace, and followed practices that are, politely, out of date. Just to give an example of the kind approach the company takes:

  • We operate out of a single AWS Region, with no DR or failover capabilities
  • infrastructure was provisioned ad-hoc and manually, with effectively no Infrastructure as code
  • Developers would typically bypass deployment pipelines to manually update files or run commands, even for production systems
  • QA is primarily manually performed for our SaaS application. There is some automation, but this is something that QA runs and checks the output, instead of automatically tracking the output in some way.

In my role, I've been pushing for change where possible, trying to evangelize the better ways of working, such as Infrastructure as Code, logs sent to a centralized location like Splunk, and deploying to other AWS regions to assist in both regional lag and general DR/failover concerns.

Thankfully, there's definitely some purchase there by leadership, at least on a high level, as they're generally receptive to these changes and recognize that they cannot continue with the same old practices. However, this mentality doesn't appear to be flowing through to the rest of the engineering organisation. My team and I are repeatedly asked to revert changes we've made, often because developers are merely used to the way things used to be, or because PMs/teams want to stick to a schedule or speed that was only possible via shortcuts (such as manually provisioned infrastructure). All of this has happened despite repeated public comments by some in leadership against those requests specifically.

What can I do to push for these kinds of changes, when I'm not in any kind of official management or leadership position? I have no official power beyond a general remit by my manager to uphold certain standards for my team.

200
2

Discussion

(2 comments)
  • 12
    Profile picture
    Engineer @ Robinhood
    a year ago

    At my previous startup, I was in the same boat. I was constantly trying to learn and apply best practices because the codebase I was working on wasn't great and the engineers took a slower, more relaxed approach to development. Ultimately, close to zero attempts to change the culture happened and I ended up leaving for greener pastures. While my scenario might not exactly be the same as yours, hopefully bringing in some of that perspective will help paint a clearer path forward.

    How I (as a tenured senior engineer) would approach a world where things aren't great is to identify the problem as 1 simple truth. When problems are identified as a single, simple truth, it's much easier to understand them and to align on them (less is more). If you provide a list, you will overhelm others. If you jump directly to executing this list of things, you're perceived as engineering for the sake for engineering (which reduces your social capital). From your description of the response to your changes, it's clear to me that you started with the list and not a simple truth. Develop the 1-liner statement first and get alignment on that statement. When asked to clarify, state both your perceived observations and the impact together. Bucket things into higher-level problem groups: again, simple truths are much easier to be understood compared to "here's a list of everything to fix". In your situation, you can try something like "Our legacy tooling and practices is slowing development down. I have observed x, y, z themes of scenarios in which this is visible".

    Once you've aligned on the simple truth, then identify what is the single, most impactful effort you can do to mitigate the problem. This will narrow down the surface area of your changes, lowering the changes you need to make and how many new things other engineers need to learn. You also want to treat each effort as a risk. Every effort you attempt can fail by making it appear you made the status quo worse. If you reach a point where you've made multiple changes and ended with more unsuccessful changes than successful changes (your value is negative from making things worse more than better), you'll have no social capital and will likely get PIP'd/fired soon. Comitting to one thing also establishes focus: you won't need mental overhead to juggle competing priorities.

    Once you're identified what effort to focus on, you need to minimize the blast radius of developers impacted by your efforts.

    • The first step here is to act as an owner. Do whatever it takes to make sure you don't break builds and you don't break/delete existing tooling and processes: whatever you do, make sure the boat doesn't rock. It's incredibly frustrating whenever workflows change while working on a timeline (the tighter timelines are, the more frustrating it'll be). Every single issue that is potentially related your efforts, you need intercept and resolve them as fast as you can. This will establish that you're proactive as an owner and that you're not a boat rocker.
    • The next step is to start with 1 person. Ideally you want to start with who has the most trust in the company, but anyone who's willing to test your efforts will do. (Relative to more than 1 person) it's much easier to be attentive to one person and to get a commitment from management for volunteers. Your goal with this one person is to sell to them that your efforts will (without a doubt) benefit their workflow. Do as many walkthroughs as needed to teach them, write all the documentation is needed for them to reference, and address every question/comment/concern/feedback they have. If they are clearly sold on the idea of your changes and have mastered them, then you can move onto onboarding more people.

    Once that individual is clearly sold on your changes, look to onboard their team. Work with this individual to help sell the team on the benefits of your changes and help onboard the team to them. As a member of the team, their team will trust what they say (and that includes their vouch for you). After some reasonable amount of time (up to your discretion), have them fill a simple survey with a single yes/no question: are these changes an improvement to your current workflow? If less than 67% of respondents say "yes", iterate on the feedback of people who said "no" until you have at least that 67% ratio.

    After that, it should be a relative breeze: onboard more and more teams onto your chages and keep surveying them to make sure you maintain at least a 67% ratio for people who are happy with your changes. Once the majority of folks who benefit from the change on onboarded, work with management to put a date to delete the old stuff and to move everyone onto the new changes. At this stage, there will be little to no questions: there's clear metrics around the benefits your changes and there's people who will vouch for you. The worst scenario for people now is that they are bregrudgingly willing to migrate.

    Finally, you're done! Now if people want you to keep going, pick the next most impactful change and repeat the process starting at working with 1 person.

    To compress this down to make it easier to extract learnings: you are probably in the right for taking on efforts to improve tooling and infrastructure, but your gaps are around understanding the people around you and your standing with them. Given their responses to your effort, you tried to directly replace existing tools/processes without direct and thorough consulation with ICs to build clarity, empathy, and trust in your efforts. For me, your current approach would only work if you were a tenured staff+ FAANG engineer in a company with 0 ex-FAANGs (then it's inefficient for everyone/anyone to fight your efforts compared to unceremoniously complying). The key lesson is that social capital impacts your ability to execute: if your name or resume doesn't immediately demand respect, then you need to build up social capital. The larger the blast radius, the more social capital you need. To avoid efforts with larger social capital needs (like company-scoped infra migrations), break down the effort into an incremental set of small steps. The process will be slow, but that reflects on how we (as humans) are generally slow to change.

  • 4
    Profile picture
    Tech Lead @ Robinhood, Meta, Course Hero
    a year ago

    Jonathan's response is excellent. I have some additional thoughts, also coming from a company where I tried to make things better and didn't execute that well (I made a lot of the mistakes Jonathan mentioned):

    • Don't knock bad code too much - Your current company's tech is bad, but that doesn't change the fact that it was good enough to build a business that gives you a paycheck. At the end of the day, the point of code is to deliver economic value. If it does that, it's not reasonable to hate it too much. I hate working with bad code and messy tech as well, but after I came to realize this, it made me more at peace with the spaghetti. It also put my headspace in a better situation to improve it (i.e. not just create a massive list of everything that's wrong and overwhelm people).
    • Pick your battles - You simply cannot fix everything, and if you try to (and are vocal about it), your team will get annoyed with you very fast. Find the most important and damaging issue of your tech debt and laser focus on that.
    • Be patient - As Jonathan mentioned, you need lots of social capital to do any sort of major refactor/migration/revamp. The tricky part is that relationships take time to build, especially if you want to get them to a state where they're willing to stick their neck out for you and get behind a giant engineering improvement effort that isn't roadmap feature work. At Instagram, I revamped the entire oncall system without too much resistance, but that is because I had been cultivating trust and building a reputation as an empathetic high-performer for 2 years.
    • Make impact clear - You should always strive to connect every action you do to a business need. In other words, you have to craft the narrative (and have proof) behind how your current tech debt is harming users. For my Instagram Ads oncall revamp, the problem was obvious: Lack of ownership led to issues being ignored/beach-balled around and users (some of which were paying advertisers) would suffer. You need to find that same connection for your efforts. I am almost allergic to the word "modern" now, because I have seen so many engineers throw it as a buzzword to lazily try to convince people to do massive overhauls, which is just wasting time (it's engineering for the sake of engineering). Don't fix what ain't broke.

    Here's a link to the oncall revamp case study I mentioned before: [Case Study] Revamping Oncall For 20 Instagram Engineers - Senior to Staff Project

    I recommend this thread as well: "How to convince my engineering org to participate in large-scale migration?"