Transparent Insights logo

Senior AWS Data Engineer

Transparent Insights
3 days ago
Full-time
On-site
Cape Town, South Africa
Data Science & Analytics

Introduction

About Transparent Insights

Transparent Insights was founded to bring transparency and trust to a world driven by data. We are a technology-agnostic, global data consultancy with offices in the Netherlands, Switzerland and South Africa, blending Dutch efficiency, Swiss quality, and South African resilience.

What sets us apart is our hands-on “Decision-makers on the Floor” approach, where our leaders are actively involved in client engagements, alongside an agile, flexible approach to both solution delivery and consultant placement.

We use accelerators to deliver value faster to our clients, and we invest deeply in our people through rigorous vetting and continuous certified upskilling—building lasting partnerships with both our clients and our consultants.

Role Overview

The AWS Data Engineer will be responsible for designing, building, and maintaining scalable data platforms, data pipelines, and data warehouse solutions in cloud and hybrid environments.

The role supports Group Finance and accounting functions, enabling reliable, governed data for financial reporting, analytics, and operational systems. Experience working with financial data, accounting processes, or IFRS17 reporting environments would be advantageous.

The data engineer will leverage modern cloud technologies and big data processing frameworks to ensure trusted, well-structured data for analytics and reporting use cases

 

 

Duties & Responsibilities

Data Platform Development

  • Design and develop data pipelines for ingestion, transformation, and delivery
  • Build and maintain data lakes, data warehouses, and analytical datasets
  • Develop scalable data solutions that support financial reporting, analytics, and operational use cases

Data Engineering & Delivery

  • Translate business and finance requirements into technical data solutions
  • Develop solutions from design through deployment and support
  • Perform data profiling, testing, and peer reviews
  • Ensure data pipelines are reliable, maintainable, and scalable

Collaboration & Delivery

  • Work with cross-functional teams, including engineering, analytics, and finance stakeholders
  • Deliver solutions through Agile development cycles
  • Contribute to engineering standards and best practices

Governance & Compliance

  • Ensure solutions meet data governance, security, and regulatory standards
  • Maintain documentation and support operational compliance requirements

Desired Experience & Qualification

Cloud & Data Platform

  • Strong experience with Amazon Web Services data services and infrastructure
  • Experience building cloud-based data lakes and data warehouse solutions
  • Hands-on experience with Databricks

Data Engineering & Processing

  • Strong Python development skills
  • Experience working with Apache Spark using PySpark
  • Experience building scalable data ingestion and transformation pipelines

Data Warehousing & Modelling

  • Experience designing and implementing data warehouse solutions
  • Strong understanding of dimensional modelling (Kimball methodology)
  • Experience working with relational and analytical data models
  • Broader data modelling knowledge (e.g., 3NF, dimensional models) expected from experienced data engineers
  • Exposure to Data Vault is beneficial but not required

Engineering Practices

  • Experience working in Agile and DataOps environments
  • Knowledge of CI/CD pipelines and version control (e.g., GitHub)
  • Understanding of data governance, data quality, and security practices, particularly in regulated environments

Domain Knowledge (Advantageous)

  • Exposure to financial services data environments
  • Experience working with accounting or finance data
  • Knowledge of IFRS17 reporting frameworks or financial regulatory reporting

Qualifications

  • Bachelor’s Degree in Computer Science, Information Systems, or related field (Requirement)
  • Experience designing and developing cloud-based data pipelines
  • Experience building data warehouse and analytical data solutions
  • Experience working with AWS-based data platforms
  • AWS certifications (e.g., Data Engineer or Solutions Architect) advantageous

Interested?