So, even before I complete one year at trivago, I had the opportunity to lead a technical project within our team. The title of the role was “Technical Driver”, and I learned/enjoyed a lot while doing it.
Brace yourselves for a very long read!
TLDR; This article is about what I’ve learned on driving a tech team to complete a project smoothly.
Who is a “Technical Driver”?
The role of a technical driver usually comes when you’re working in an engineering team, where the majority are engineers and you’ll also have a Product Owner and maybe several Project Managers. Usually, when there is a new feature request raised towards the team, while the PO and PMs deal with the business level functionalities, engineers are responsible for implementing that requirement.
Even though the PO/PM can manage the user stories (or tasks), they might have less understanding of technical implementations. If there need to be technical discussions, research, documentation
So, as the name suggests, a Technical Driver will drive the team from the technical perspective of the project.
The best way of having a Technical Driver (hereinafter mentioned as TD) is to appoint one engineer from the engineering team to take up the lead/driver role and let him drive the technical team. In this case, it was me. 😉
Responsibilities of a Technical Driver
1. Be the translator between the PO/PM and the engineering team
TD should understand the business requirement properly and explain that to the engineering team in their language. And also, TD should keep on updating the PO with the progress of the technical tasks and complications. For instance, if the team needs to do technical research before going forward with a particular technology/framework, it’s TDs responsibility to make sure they’re done properly and the PO is updated.
Again, this does NOT mean that the engineers don’t talk with the PO/PM. Engineers anyway talk with them to get things clarified regarding the business logic, and PO/PM will talk to engineers if they have any questions regarding the implementations.
Depending on how technical the requirement is, depth and the context of these discussion may vary. For instance, I was the TD for a fully technical initiative (aka project) where I had to make sure we meet the business KPIs (Key Performance Indicators) for the project. So, I had to summarize all the implementation details, and progress and update the PO highlighting those KPIs.
2. Arrange, facilitate and moderate technical discussions
Apart from the usual team meeting (i.e. scrum ceremonies) and business level meetings which will usually be initiated by the PO, TD also should initiate many meetings regarding technical aspects of the project. Since these are many technical discussions, there is a high chance of it going out of topic/scope or even time. So, it’s the responsibility of the TD to moderate the meeting in order to make sure the team stays within the topic and the time. When you feel like the team is diverging from the scope of the meeting, it’s your responsibility to barge in and
However, if the planned topic is interesting to the TD or (s)he has more experience in that topic, we always can assign someone else as the moderator for that meeting.
3. Make sure everything is documented
There is only one rule about documentation.
Document everything!
Starting from the architecture, findings of research tasks, discarded approaches (and why), results of tasks, we should document everything! Now I’m NOT going to explain why documentation is so important, but if you think it’s any less important, you shouldn’t even be reading this 😛
Make sure you have up-to-date diagrams and descriptions of the pipelines at every step. And this doesn’t mean that the Technical Driver should do everything. We can always delegate tasks to other team members.
4. Take (overruling) decisions when needed
Sometimes you will find the team having back-to-back discussions and end up not having any conclusion regarding a certain topic. And also when there are conflicts of ideas within the team and the discussions are dragging too much, you will feel helpless as you don’t have anything to do. Because you should be driving the project, not ruling it.
But… Sometimes, you need to make decisions that may sound harsh, but it’s for the greater good. For instance, when team members are arguing whether to try a new technology or not, at some point you have to barge in and end it. You (should) have the power to (politely) overrule others’ opinions and say “okay, let’s try this, it may helpful” or “there is no point of trying because it’s out of the scope“. And if you do it often, then the team won’t like it, and if you don’t do at all, the team won’t proceed. So, and you should be conscious of when and how to do it.
And most importantly, it comes with the trust inside your team. If everyone knows that there is nothing personal, and you do it only to make the project run, there is nothing to worry. I was so lucky to be able to work with that kind of a team!
5. Drive the team
The most important responsibility of the Technical Driver is to drive the team to the end goal while creating a friendly environment for the team members to work freely and comfortably. So, the role of the TD is mostly a facilitator role, even though (s)he can also do engineering work from the work backlog.
Technical Driver should try his/her best to make all the meetings well-organized and fruitful. And also, keep the team motivated and encouraged. One thing I used to do it give a small update about the project progress and also praise team members for their task completions during the daily standup meeting. And members said it helped them a lot to have a general understanding of where we’re headed.
Lessons learned
1. Keep in touch with the PO (always)
Product Owner is the one with a clear understanding of the business requirement. It doesn’t matter how cool your output is if it doesn’t fulfill the business requirement(s). In my case, the business level requirement was to improve the consistency of our pipelines (along with some more KPIs). So, I kept on working with our PO to make sure we achieve this while encouraging the engineering team.
I had frequent meetings with our PO and discussed about the progress, blocks and the scope refinements. It also helped us to set clear expectations with the external stakeholders with respect to the features and deadlines.
2. Delegate responsibilities (when needed)
I used to think that I have to do everything by myself because I’m responsible for the delivery of the project in technical terms. However, I got proved wrong in many cases and I ended up being super stressed. My teammates kept on telling me I should distribute the managerial tasks, even creating tasks on Jira. After I stepped out of that obsessive mindset, everything was smoother than I ever expected. I could sleep way better 😛
Here are some things I delegated.
- Moderating the meetings and discussions
- Writing technical user stories/tasks
- Drawing architecture diagrams
- Writing documentation
- Drive a defined portion of the project
This was super helpful, especially when I had to take vacation. I was out of office for two weeks and I asked two team members to drive two shipments while I’m away.
Seems like I delegated all my responsibilities? Well, now I have to put my trust on my team members to complete the assigned tasks and make sure they do so. Then I could spend more time on engineering tasks since I only had to keep track of the progress of those assigned tasks.
It doesn’t matter who does it, as long as it’s done (right)!
3. Be the “devil’s advocate” (as needed)
Sometimes when you see the team is going out of focus, it’s your responsibility to bring them back to the path. When you work in an engineering team and need to research on a topic, there can be hundreds of options to explore.
For instance, when we were considering options for streaming data in our RDS (Relational Database Service) changes, we came across many ways of doing it. We could use
One of the most important questions you can ask is, how much value will it add to the final KPIs? If it only provides a good-to-have feature after refactoring the entire pipeline and takes a couple of months, you have to decide whether it makes sense to follow that path. You have to provide a valid set of reasons why you’re going to try or drop that idea. If you don’t have valid reasons, you should find them! That’s where you can (ask the team
There was an incident where we were going back and forth with several approaches related to storing our master data. Some suggested to use Amazon Athena over Amazon S3, but some kept on the idea of using Amazon RDS. The option of Athena came several times, but we skipped it due to many reason, especially because it required a lot of changes in the existing pipelines. But it came to a point that team was (hypothetically) split into two groups and arguing about it. After many discussion with many people, I decided to overrule the arguments. My solution was to make a PoC with Athena and see if that caters our solution. After finishing that PoC after one or two weeks, we realized it clearly doesn’t cater all our requirements, yet we couldn’t see it for sure without trying. But, the time and energy was never a waste because we learned many things about Athena over S3 and we will be able to use if we come across a valid use-case.
However, you can’t do these things if there is no trust and the openness between your team members. But fortunately, my team was superb in that aspect! ( I already said that 😛 )
4. Be involved (as much as possible)
If you become so busy with the managerial tasks (because you don’t delegate), it’s very hard to contribute to the team in engineering tasks. This might cause you to loose the sense of what the engineering team is doing. The best way of handling these kinds of scenarios is to take up some engineering tasks during the sprint. Then you’ll have more understanding of the technical details of the implementations, and it will be much easier to perform all the other responsibilities.
You’re just the driver of the project, not a boss!
Once, there was a task to create a PoC to stream out RDS changes, and we chose Debezium library for that. And I picked up the task and created a PoC and it was promising. Then we selected that approach, and I was able to contribute more during code/design reviews for that component because I had the understanding of how it works.
5. Don’t be afraid to try
When it comes to implementing a new project, there are many unknowns. Usually people are restricted from trying new things either due to the pressure from the management (reflected by the Project Managers and team leads), and the fear of failing.
If you are facing the first reason, maybe it’s the time to change the company (because you can’t really change the management).
If you are having the latter concern, we should create the environment of considering failing is a part of the learning process. It never comes from a single person nor within one day. The management MUST create that environment. Instead of firing up hundreds of emails around when something goes wrong to put the blame of employees, they should nurture the idea of trying and learning.
We did so many PoCs for many technical requirements and sometimes ended up using the current solution at the end. But, we consider it as gaining knowledge and that’s why documenting the findings is very important.
6. Moderate the meetings (always)
When people go into meetings to talk about something or take a decision, there is a huge probability where we end up talking about random stuff or go too much into details, and we feel like the meeting was utterly useless.
Here are some tips to make it more productive.
- ALWAYS have an agenda and share it beforehand!
- ALWAYS request participants to prepare for the meeting.
- NEVER invite people who are not required.
- NEVER have a meeting more than one hour at a stretch.
- ALWAYS have a moderator assigned for the meeting.
- FREQUENTLY remind the remaining time and points to discuss.
- ALWAYS summarize the decisions made before wrapping up.
- ALWAYS document the outcome of the meeting/discussion.
7. Praise the team members for the achievements
When the team achieves something special, mention it during a standup meeting or a retrospective meeting. As a practice, we either openly praise the colleague, or even use Kudos Cards. This will definitely boost the confidence and the mood of the team!
8. Get/give feedback (always)
The best of learning is to get feedback. If you don’t already have a culture for giving/receiving or even asking feedback, you have to try different ways to get them. However, in our team we have a habit to get/ask and give feedback frequently.
I ask for feedback openly from the entire team during meetings, or ask in person. It could be very simple questions such as “how did you handled these type of situations when you were a technical driver?” or “do you think I could have handled that situation differently?” etc. I could ask these questions directly because of the openness we have within the team. Because I know, their feedback is not personal and they know that I don’t take them personally. I also give feedback to team members either openly or in person depending on the type of feedback.
If you don’t have it, it’s high time to think of creating that.
It’s been quite a ride for me since this is the first time I’m playing any kind of a technical lead role in my careers. It’s been a lot stressful in the beginning, but the team helped me throughout and finally we completed the project with promising KPIs values.
How?
We spent around three months finishing our project, where one of the major KPIs was to reduce the down time for our data streaming components, while making sure we have proper data recovery procedures. It’s very challenging because we didn’t know how to do it. So there were a lot of uncertainties and research involved.
The flow
These were the general steps of the projects.
- PO comes up with an idea for an initiative (3 months projects) after discussing with the team.
- Pitches the idea to the company and get feedback.
- Refine it till the idea is approved.
- The team selects a Technical Driver with the help of the Responsibility Lead.
- PO explains the idea to the TD in details.
- Have the kick-off meeting
- Have frequent feature discussions
- Have milestone meetings and evaluate the progress
- Release the project and send for C-Testing
- Celebrate when/if accepted.
Types of meetings
When conducting a project, it’s filled with meetings and discussions. Here are the meetings we had apart from the usual scrum ceremonies.
The kick-off meeting
When a new requirement is given, there should be a kick-off meeting: a technical discussion held to decide the architecture and the technologies to use. This usually is a workshop where all the engineers are present, and sometimes even the PO in case the team needs more information. If the scope is too big, you may have this is several parts, but we should not overwhelm the participants with too many meetings.
Outcome:
- Everyone has a general understanding of the requirement.
- The technology stack should be decided.
- The higher level architecture diagram.
- Identifying user stories (or individual features)
- Whether we need to do any technical research.
- Any open questions.
After this meeting, PO and TD sit together and refine the high-level user stories and make sure they align with the initial business requirement(s).
Feature discussions
When the higher level user stories are decided, the
This shows how agile a team can be, as the initially suggested architecture could even be completely redesigned. However, TD should make sure that the team won’t reiterate the discarded solution unless it’s really required. For this, every decision should be well documented along with the reasons and references.
When I was the Technical Driver, we started with 8 shipments, but decided only to focus of 4 due to the scope being too wide and we wanted to have more focus.
Outcome:
- A d
eeper understanding of the requirement and the user story (feature) - If the user stories should be changed or split into smaller parts
- Possible break down of the tasks (not mandatory)
- Any technical research to be done.
These user stories and research tasks will be refined and estimated during the usual refinement meetings.
Ad-hoc technical discussions
In case the team needs to decide on something or even to review a design, any team member can arrange a meeting. Usually, the engineer will talk to the Technical Driver and discuss the matter beforehand, and then they agree whether to have a separate discussion with the team or not.
Usually, these meetings are about taking critical decisions which affect the end product of the project. TD should make sure the team has enough information to come into the conclusions. This means, (s)he should go through all the required documentation and the outcomes of the research tasks and be ready to present them to the team. And we should make sure these meetings don’t take a lot of time, otherwise we’ll spend our entire time in meetings.
Outcome:
- The intended goal is accomplished
- If there are significant decisions made, document them with the reasoning.
Milestone meetings
Whenever we do a big project, it’s best to divide it into smaller parts and then it’s so much easier to keep track of the progress. Therefore, it’s better to have frequent self-evaluations within the team to check where the team and the project reside. Given that a project lasts for three months, we can do this once a month or after 1.5 months.
During this meeting, we can not only review our work and progress but also do a small retrospect of the process to see what things can be improved.
Outcome:
- Review the progress of the project.
- Discuss the blocks/problems faced and make sure we know how to tackle them in the future.
- Give feedback to the team.
Release and celebrations
When/if the project is completed, it’s the time to release the project/product and celebrate! Technical Driver will help the PO to write the press-release of the project (could be a blog post or an email etc)We should always give recognition to the team members and their contributions. If there are any other ways of recognizing them in addition to mentioning their names or directly praising them, we should do that too.
This usually happens when the feature/project is accepted, and could be even after a few weeks or months after finishing the project. Yes, there is a chance that we wouldn’t finish the project or even not accepted after all the effort we put. But still, it’s not considered as a failure as long as we learn from it.
Outcome:
- The project is completed and (everything set to be) delivered
- Party!!!
Summary
It was a fantastic experience to drive a team to finish a technical project within three months. I realized that technical skills are very less important when it comes to handling many aspects of an engineering team. It needs a lot of other skills to understand people and enable smooth process to reach the goal. It would have been so hard, but my teammates were wonderful and super helpful.
So we ended the project with:
- Our own Amazon RDS streaming solution using binary logs.
- Now our number of missed records is 0!
- Merge two RDS databases into one instance to increase performance and remove data redundancy.
- Read this article for more information.
- Proper alarming mechanism for AWS components
- Automatically detect data lags of sinks and notify us
- Many scripts to run in case of a data inconsistency.
- A series of run-books explaining different disaster scenarios (related to data) and how to handle them.
We planned a lot more to do, but we reduced the scope during the project itself to make sure we focus on the most important things.
It’s been an amazing experience for sure! Maybe this might work for you too!
You reached the end of the article. You’re a true hero! Here is a potato!
