Every Sunday I sit down and ask myself four questions: What went well? What would I do differently? What did I actually learn? What am I carrying into next week? Here is this week honest answer to each.
What Went Well: Automating the Unglamorous Stuff
I finally automated a data ingestion job that had been running manually for months. A Python script wrapped in an Airflow DAG, pulling from an SFTP server, transforming data, and landing it in BigQuery. It saved 2 hours of manual work per day — that is 500+ hours a year back to the team. The boring automations often have the highest ROI.
What I Would Do Differently: Ask Sooner
I spent four hours debugging a dbt model with a cryptic error. After four hours I sent a Slack message. My teammate replied in five minutes with the fix. Four hours. Five minutes. Same result. Next week: if I am stuck for more than 30 minutes on something a teammate might know, I am asking. Full stop.
What I Learned: Airflow Dynamic Task Mapping
I finally made peace with Airflow dynamic task mapping — a feature in Airflow 2.3+ that lets the number of tasks be determined at runtime using .expand() instead of being hardcoded at DAG write time.
@dag(start_date=datetime(2026, 1, 1), schedule="@daily")
def dynamic_pipeline():
@task
def get_files() -> list:
return ["file_a.csv", "file_b.csv", "file_c.csv"]
@task
def process_file(filename: str):
print(f"Processing {filename}")
process_file.expand(filename=get_files())
dynamic_pipeline()Some tools look harder than they are. The barrier is often just willingness to read the documentation carefully instead of skimming.
What I Am Carrying Into Next Week: Collaborate First
The best moment of the week was a 15-minute call with a teammate who had seen a similar problem before. In 15 minutes we covered what would have taken me 2 hours alone. My intention for next week: collaborate first, solo-debug second.
The Habit Behind the Reflection
If you do not do a weekly reflection yet, try it — just 10 minutes on a Sunday. Four questions: What went well? What would I do differently? What did I learn? What is my intention for next week? Growth as an engineer is not just about learning new tools. It is about getting better at learning itself.
What is one thing you learned this week? Drop a comment below.
— Pushpjeet Cholkar, Data Engineer