[Effective Engineer] Execution (Part 2)
Oct 29, 2022A continuation of [Effective Engineer] Mindset (Part 1), this article of the series focuses on execution techniques that help enforce those mindsets.
[Chapter 4] Accelerate iteration speed
- The faster you can iterate, the more that you can learn what works and what doesn’t.
- Because learning compounds, the sooner you iterate fast, the faster your learning rate is.
Continuous Deployment
- Help engineers make small, incremental changes rather than large features.
- When making small changes, it helps trace down bugs and enforce fine-grained A/B testing.
- Gated large feature behind a configuration flag.
Investing in time-saving tools.
- Faster compilation cycle shorten the validation loops when writing code.
- Using REPL (read-eval-print loop) to verify expressions in an interactive programming environment.
- Invest in those tools early on with a potential upfront cost rather than pay the tax every iteration.
Shorten debugging and validation loop
- Make a minimal, reproducible test case that triggers the bugs with only necessary configs.
- Test and debug with a tight feedback loop.
Master programming environment
- Get proficient with text editors and IDE (code reformatting, navigating, version control, etc).
- Be expert in at least one high-level language with powerful built-in abstractions (Python over Java, C/C++).
- Get familiar with Linux shell commands.
- Keyboard over mouse.
- Run unit tests that only associated with your changes.
Unblock non-engineering bottlenecks
- Dependency upon other teams: overcommunicate to align with priorities.
- How to obtain approval from senior leadership: use informal coffee chat/office hour to build relationship over time.
- Review process from PM/legal: plan ahead.
- Some of these are out of engineer’s control, so some companies set up the role of TPMs to help cross-team communications.
[Chapter 5] Measure what you want to improve
Engineers should relentlessly measure and quantify metrics to improve product quality.
Use metrics to drive progress
- Focus on right things. Metrics confirm that your product changes actually achieve your objectives.
- Help guard against future regressions.
- Drive forward progress. Make sure the thing that you are optimizing is going at the right direction.
- Measure the effectiveness, compare the leverage against other activities, and prioritize accordingly. See prioritization section.
Pick the right metrics
- Maximize impact: profit per
x
in economics, etc. - Actionable: use
rate/percentage
overgross/total
. - Responsive yet robust: Updates quickly enough to give feedback over a period of time. Example:
xxx
over the last week is more responsive than over the last month, while more rebust than over the last hour. - In terms of team management, engineers can also pick some metrics over the other:
Productivity per week
overHours per week
: Long working hours decreases productivity, resulting in “undertime”.Outstanding bug fixed
overTotal bug fixed
: Prioritize impactful bugs over trivial bugs.
Instrument the system
- Metric is top-level strategy; instrumentation is low-level implementation detail.
- More instrumentations provides better visibility of the system.
- Create dashboard to visualize instrumentations over time.
Internalize common numbers
- Keep the table at hand: Latency numbers that every engineer should know.
- These back-of-the-envelop calculations can quickly estimate performance metrics of your design.
- Help spot anomalies in data measurements.
- Common useful numbers can benchmark your current design and scope for improvements.
Prioritize data integrity
- Log the accurate, unbiased data. Wrong data is worse than no data.
- Usually metrics-related code is less robust than feature code.
- Be skeptical about data integrity: Data can get dropped in network traffic, bugs in data collection pipelines or dashboard, etc.
[Chapter 6] Validate ideas early and often
- Optimize for feedback ASAP helps get right things done.
- Resonate with a more recent idea: build in public.
Find low-effort ways to validate the work
- Identify the scariest, the riskiest, the most uncertain part of the project, and validate it first.
- Build a MVP/prototype: the version of a product that allows a team to collect maximum amount of validated learning about customers with the least effort.
- The MVP can be creative. Dropbox “fakes” a placeholder button for Google account login and monitors its click-through rate, to validate whether the feature can gain tractions.
Continuously validate product changes via A/B testing
- A random subset of users see a change, while everyone else doesn’t.
- Using variable control, A/B testing validates that any significant change in metrics can be solely attributed to the product change.
- A/B testing turns the black box of of user behavior into understandable, actionable knowledge.
Beware the one-person team
- Leverage design doc, code reviews, or informal coffee chat to keep other engineers aware of your progress.
- Keep code changes small and incremental.
- Design API interface/protobuf first to surface early issues.
- Solicit buy-in with conversations for large features.
Build feedback loop for any decision
- Formulate a hypothesis, design an experiment to test it, define metrics, run the experiments, learn from the results.
[Chapter 7] Improve project estimation skills
Use accurate estimates to drive project planning
- Keep the deadline constant and deliver what is possible.
- Decompose the project into granular tasks.
- Set a hard deadline for minimal delivery and an internal deadline for stretch goals.
- While one engineer can finish one task,
n
engineers can’t delivern
tasks with the same efficiency because of communication overhead, new hire ramp-up, etc. - Once you have an estimate, validate it against historical data.
- Use timeboxing to restrict with open-ended tasks.
Budget for the unknown
- There are always uncertain problems that we fail to account for when estimating. Several minor yet unexpected tasks compound and wreak havoc with the schedule.
- Separate effective work time from total calendar time, as many non-engineering duties may slip in. When setting schedules, build in buffer time for the unexpected interruptions.
1 engineering day = 2 calendar days
is a ballpark benchmark.
Define specific goals and measurable milestones
- A well-defined goal helps scope the project: separate must-haves from nice-to-haves.
- It also builds clarity and alignment across key stakeholders.
- A concrete milestone keeps the team honest and tracks current progress. Rather than
almost done
, a measurable milestone looks likefeatures X, Y, Z by a completion date
.
Reduce risks early
- A common risk is system integration, where functional subsystems are integrated together but fails as a whole.
- Build an end-to-end scaffolding (skeleton) and test it early on.
Don’t sprint in the middle of marathon
Working overtime doesn’t produce linear impact
- Productivity decreases with additional hours worked
- When you realize you can’t make the deadline, you are more behind schedule than you think, because you likely underestimated the entire projects, not just the previous months.
- Working extra hours can burn out team members and hurt team morale.
Working overtime can be used as a compromised option, if you:
- Communicate with the team on why working overtime is necessary.
- Revise the plan and schedule to adapt to the new reality.
- By any means, it’s not an excuse for not practicing project estimation skills.
Continue reading the next section: [Effective Engineer] Long-term Value (Part 3)