Senior Data Engineer Consultant – Data Quality and Modern Cloud Platform

Function : Data & Digital Consulting

Location : South Africa Hybrid / client-site as required

Reports to: Data Engineering or Consulting Practice Lead

About Keyrus

Keyrus is an internationally recognised specialist in Data and Digital and a trusted partner to organisations across industries. We deliver practical business solutions using reputable, modern and scalable technologies. Our purpose is to help clients improve performance through transformation enabled by data.

We are passionate about innovation, teamwork and collective success, driven by exceptional individuals who combine technical depth with sound consulting judgement.

The role

The Senior Data Engineer Consultant designs, builds and operationalises secure, scalable and governed modern data platforms for Keyrus clients. The role converts business and data requirements into reliable ingestion, transformation, data-quality, storage and serving solutions that support operational reporting, analytics, migration and responsible artificial intelligence use cases.

The consultant works with data architects, business analysts, data-quality specialists, governance teams and client stakeholders. The role provides technical leadership, contributes to solution architecture and pre-sales, mentors other engineers, and ensures that solutions are tested, observable, documented, auditable and transferable to client teams.

Key outcomes

• Production-grade cloud data platforms and pipelines that are secure, scalable, reliable and cost-conscious.
• Trusted, curated and reporting-ready data products with measurable quality and clear ownership.
• Repeatable data profiling, cleansing, remediation, reconciliation and monitoring capabilities.
• Traceable metadata, lineage, transformation logic and evidence supporting governance and formal acceptance.
• AI-ready data foundations and responsible, approved use of AI-assisted engineering tools.
• Operational runbooks, knowledge transfer and sustainable transition into business-as-usual support.

Responsibilities

1. Data discovery and solution design

• Assess client systems, interfaces, datasets, integration patterns, business processes and reporting dependencies.
• Produce current-state data-flow, integration, dependency and lineage documentation.
• Translate business requirements into technical designs, source-to-target mappings, data contracts, acceptance criteria and delivery backlogs.
• Contribute to target-state data lake, lakehouse, warehouse and integration architecture.
• Identify technical risks, assumptions, dependencies and constraints and maintain appropriate delivery evidence.
• Explain complex architectures, trade-offs and recommendations clearly to technical and non-technical stakeholders.

2. Data platform engineering

• Design and implement reusable ingestion, transformation, orchestration and serving frameworks.
• Integrate relational databases, APIs, files, SaaS platforms and event or streaming sources where appropriate.
• Implement batch, incremental, change-data-capture and synchronisation patterns with reconciliation and recoverability.
• Build curated, business-ready data products using suitable dimensional, lakehouse or domain-oriented modelling patterns.
• Apply modular design, peer review, automated testing, version control, CI/CD and controlled environment promotion.
• Optimise SQL, Spark processing, pipeline execution and cloud storage for performance, reliability and cost.

3. Data quality and remediation

• Profile complex datasets and establish baselines for completeness, validity, consistency, uniqueness, timeliness and referential integrity.
• Identify and quantify duplicates, anomalies, gaps, conflicting records and cross-system misalignment.
• Implement validation, standardisation, enrichment, entity matching, deduplication and survivorship rules.
• Build repeatable cleansing and remediation pipelines with version-controlled rules and auditable before-and-after evidence.
• Develop exception handling, reconciliation, control totals, remediation workflows and escalation mechanisms.
• Implement data-quality scorecards, monitoring, alerts and preventative controls, including correction at source where practical.
• Work with data owners and stewards to validate rules, resolve exceptions and obtain business acceptance.

4. Metadata, governance, security and privacy

• Implement technical metadata, data catalogue, classification and end-to-end lineage capabilities.
• Maintain source-to-target mappings, transformation specifications, data standards and traceable change records.
• Apply role-based access, least privilege, secrets management, encryption, masking or tokenisation as appropriate.
• Implement logging, monitoring, alerting and evidence retention for data access, pipeline execution and data changes.
• Handle personal, banking, customer and other sensitive data in accordance with POPIA, contractual commitments and client-approved security, privacy and residency controls.
• Align technical controls with the client's established governance programme, decision rights and stewardship model.

5. Operational and migration enablement

• Implement pipeline observability, retry, recovery, backup and operational support procedures.
• Define and monitor data service levels, operational controls, ownership and escalation paths.
• Prepare deployment guides, support procedures, operational runbooks and maintainable technical documentation.
• Support migration readiness through profiling, reconciliation, exception management, acceptance gates and sign-off evidence.
• Conduct structured knowledge transfer, training and transition into client business-as-usual teams.

6. Responsible AI and continuous learning

• Design governed data foundations that can support advanced analytics and AI use cases, including curated training or retrieval datasets where approved and justified.
• Evaluate appropriate uses of AI and machine learning for profiling, anomaly detection, classification, entity resolution, metadata generation and engineering productivity.
• Use only Keyrus- and client-approved AI tools, models and processing environments.
• Ensure AI-assisted code, documentation, mappings and analysis are tested, reviewed and subject to human accountability before use.
• Never submit client data, credentials, proprietary code or confidential information to unapproved public AI services, and never permit client data to be used for model training without written authorisation.
• Understand and communicate AI risks including hallucination, bias, privacy, intellectual property, explainability, data residency and vendor lock-in.
• Maintain current knowledge of cloud data platforms, data-engineering practices, AI-assisted development and responsible-AI controls, sharing learning through demonstrations, standards, reusable assets and mentoring.

7. Consulting, leadership and commercial contribution

• Lead or supervise data-engineering workstreams from discovery through production handover and post-implementation support.
• Plan and estimate work, manage priorities, monitor delivery quality and escalate risks early.
• Define evidence-based acceptance criteria and support formal client review and sign-off.
• Mentor engineers, conduct design and code reviews, and promote consistent engineering standards.
• Contribute to proposals, solution demonstrations, client bids and technical pre-sales activities.
• Translate technical capabilities into clear business benefits, costs, risks and implementation choices.
• Build trusted stakeholder relationships and identify legitimate improvement opportunities through high-quality delivery.

Role requirements

Education and professional standing

A relevant degree or diploma in computer