Dan Kruse
London, UK · Remote-first
I design, remediate, and scale cloud data platforms for growing teams. Expect calm delivery, candid advice, and hands-on help that gets you shipping faster.
- Consultant data engineer specialising in end-to-end platform delivery
- Trusted to fix failing migrations and accelerate greenfield builds
- Partner to exec, product, and engineering teams alike
Download a printer-friendly PDF or scroll to explore experience and skills.
👤 Profile
Experienced Data Engineer Consultant with a proven track record of designing and optimising scalable data platforms. I specialise in end to end delivery across infrastructure, ingestion, transformation and presentation.
I’ve worked on key projects, including rescuing stalled cloud migrations and leading greenfield data platform development, ensuring robust, scalable, and cost-effective solutions.
🛠️ Skills
- Cloud Platforms: GCP, AWS, Azure, Snowflake, Databricks
- Programming: Python, SQL, JavaScript, TypeScript
- Orchestration: Airflow, Step Functions
- Transformation: dbt
- Compute: ECS, EKS, GKE, EC2, Lambda, Function Apps
- Data Lake: S3, Glue Data Catalog, Apache Iceberg
- Databases: PostgreSQL, MySQL, MongoDB, SQL Server
- Data Warehouses: BigQuery, Redshift, Snowflake
- APIs: API Gateway
- Infrastructure as Code: Terraform, Pulumi
- Data Visualisation: Tableau, QuickSight, GoodData, Power BI
🎓 Education
BSc (Hons) Civil Engineering
Anglia Ruskin University, 2006 - 2009
📜 Certifications
- Building Batch Data Pipelines on GCP, Coursera (Sept 2021)
- Modernizing Data Lakes and Warehouses with GCP, Coursera (Aug 2021)
- Advanced React, Bostype Inc. (Jul 2021)
- React for Beginners, Bostype Inc. (Jan 2021)
- Master Gatsby, Bostype Inc. (Nov 2020)
- Google Cloud Big Data and ML Fundamentals, Coursera (Nov 2018)
- Relational Database Support for Data Warehouses, Coursera (Jul 2017)
- Data Warehouse Concepts and Integration, Coursera (Jan 2017)
- Database Management Essentials, Coursera (Nov 2016)
- Web Development Bootcamp, Lighthouse Labs Vancouver (Sept 2015)
💼 Experience
Consultant Data Engineer - Hartree Partners
June 2025 - present
- Reviewed the current platform and authored recommendations, including tooling options and comparisons.
- Implemented an orchestrator to solve dependency management across transformation, ingestion, and external data sources.
- Built a transformation project mesh with projects per domain team for clarity and maintainability.
- Advised on data engineering best practice and patterns for incremental and full‑refresh workflows, including idempotency.
Technical Skills: AWS, dbt, Snowflake, Airflow
Consultant Data Engineer - Rubik Tech
March 2025 - Oct 2025
- Delivered an API layer on top of the data warehouse, enabling the product frontend and external customers to access governed data.
- Architected the platform end to end with ingestion connectors for APIs, flat files, and relational databases.
- Implemented secure cloud foundations including VPC, IAM, logging, secrets management, CICD, and environment isolation.
- Built data models, tests, and deployment flows to keep transformations reliable and auditable.
- Set up orchestration for scheduled and event driven runs with alerts and retries.
Technical Skills: AWS, dbt, API Gateway, Lambda, Redshift, Step Functions
Consultant Data Engineer - Dr. Martens
Jan 2024 - June 2025
- Performed in-depth discovery to understand the existing setup and challenges.
- Optimised the technology stack for scalability and efficiency.
- Streamlined and consolidated data pipelines to simplify operations.
- Ensured client alignment and clear communication throughout the project.
Technical Skills: Snowflake, Azure, Airflow, Terraform
Consultant Data Engineer - Patchwork Health
Dec 2022 - Mar 2025
- Documented the existing infrastructure and identified improvement opportunities.
- Collaborated with stakeholders to gather technical and business requirements.
- Conducted vendor and tool evaluation to support data science goals.
- Redesigned and implemented a robust data model and platform architecture.
Technical Skills: AWS, dbt, Redshift, Airflow, Docker
Senior Data Engineer - HealthHero
Jan 2022 - Oct 2022
- Implemented IaC and automated deployment pipelines for scalable infrastructure.
- Designed and deployed containerised pipelines to integrate datasets.
- Developed a data lake and warehouse for efficient large-scale data management.
- Optimised resource usage and costs with orchestration and autoscaling.
Technical Skills: AWS, Airflow, dbt, Docker, SQL
Senior Data Engineer - MADE.COM
Jan 2021 - Jan 2022
- Oversaw engineering code for quality and maintainability.
- Designed and implemented scalable engineering solutions.
- Collaborated with stakeholders to align engineering efforts with business goals.
Technical Skills: GCP, Docker, Kubernetes, BigQuery, Python, SQL
Data Engineer - MADE.COM
Jun 2018 - Jan 2021
- Built and maintained robust ETL pipelines for data ingestion and transformation.
- Optimised data models to enhance query performance and reduce processing times.
- Managed workflows and task automation using orchestration tools to improve efficiency.
- Implemented automated testing and deployment processes to ensure reliability and consistency.
- Developed custom tools and scripts to address unique business requirements.
Technical Skills: GCP, Docker, Kubernetes, BigQuery, Python, SQL
Data Developer - King's Health Partners
Apr 2016 - Jun 2018
- Developed interactive data visualisations for research and publications.
- Extracted and transformed data for clinical research and service design.
- Created SQL triggers, functions, and stored procedures for data processing.
Technical Skills: SQL, Tableau, R