Tech Lead

Tech Lead

Responsibilities:- Develop modular and reusable applications using Python.- Implement configuration-driven design, robust exception handling, logging, debugging, and data processing.- Utilize BigQuery SQL for complex joins, CTEs, window functions, MERGE operations, and performance tuning.- Develop production DAGs and manage task orchestration and dependencies using Airflow / Cloud Composer.- Handle dynamic/runtime configuration, retry strategies, failure handling, backfills, and reruns in production.- Manage GCP services including BigQuery, Cloud Storage, IAM, service accounts, and Pub/Sub integrations.- Conduct testing and delivery processes using pytest, unit and integration testing, and Git workflows.- Implement CI/CD pipelines and validate deployments across environments.- Develop JSON-driven pipeline architecture and reusable hooks during onboarding.- Ensure data reliability and financial processing through idempotent reruns, deduplication strategies, and audit checks.

Required Skills:- Proficiency in Python.- Experience with BigQuery SQL.- Familiarity with Airflow / Cloud Composer.- Knowledge of GCP services and data pipeline operations.- Strong testing and delivery skills, including CI/CD practices.- Understanding of data reliability and financial processing principles.