Senior Data Engineer: AI Data Lakehouse Architect

Senior Data EngineerLocation: Abu Dhabi, UAE
- Remote GlobalType: ContractRole SummaryWe are seeking a Senior Data Engineer to architect and build the foundational data infrastructure fora large-scale AI transformation programme. You will move beyond standard ETL tasks to design anddeploy enterprise-grade Data Lakehouses that serve as the "Single Source of Truth" for AI agents andadvanced analytics platforms. You will lead the technical execution of data and platform foundation initiatives. Your mandate is tomap rigid enterprise systems (e.g., SAP S/4HANA, Ariba, etc.) and collaborate with teams tounderstand complex unstructured data (technical drawings, regulatory text) while building highperformance pipelines and systems that power Intelligent Supply Chain forecasting and generative AItools. You will operate within a structured "Sprint Zero" environment, ensuring data lineage, governance,and security meet strict enterprise standards.

Key Responsibilities
1. Data Lakehouse Architecture (Supply Chain)
* ERP Integration: Architect and deploy ingestion pipelines to extract high-volumetransactional data from SAP S/4HANA, Ariba, and PLM systems, ensuring near real-timeavailability for forecasting models.
* External Feed Integration: Build connectors for external market intelligence feeds (e.g., S&PGlobal, Orbis, EcoVadis) to enrich internal procurement data with macroeconomic andgeopolitical signals.
* Unified Data Model: Design and implement a standardized procurement data model andtaxonomy across multiple entities, harmonizing fragmented datasets into a cohesive layer foranalytics.2. Unstructured Data Pipelines
* Complex Ingestion: Engineer pipelines to ingest and process unstructured technical data,including PDF tender documents, CAD metadata, and historical CONOPS, transforming theminto vector-ready formats for RAG (Retrieval-Augmented Generation) applications.
* Vector Database Management: Manage and optimize Vector Databases (e.g., Weaviate) tostore embeddings of archival proposals and engineering snippets, ensuring high-speedretrieval for AI drafting assistants.
* Digital Thread Implementation: Establish data lineage and traceability protocols that linkrequirements to physical components, supporting Model-Based Systems Engineering (MBSE)Digital Thread initiatives.3. Governance & Security
* Enterprise-Grade Security: Implement Role-Based Access Control (RBAC), audit logging,and data redaction policies to ensure compliance with export controls and strict on-premisesecurity requirements.
* Quality Control: Deploy automated data quality frameworks to validate BOM (Bill ofMaterials) completeness and cost data accuracy before it reaches AI models.
* Infrastructure Optimization: Optimize pipelines for on-premise GPU clusters and air-gappedenvironments, ensuring efficiency within existing infrastructure limits. Technical Requirements
* Core Stack: Expert proficiency in Python, SQL, and modern data engineering frameworks(Apache Spark, Kafka, Airflow).
* Enterprise ERP: Strong experience extracting data from complex ERP environments,specifically SAP S/4HANA and SAP Ariba. Familiarity with SAP BTP is a plus.
* Database Technologies: Deep understanding of Data Lakehouse architectures(Databricks/Delta Lake), Relational Databases (PostgreSQL), and Vector Databases(Weaviate/Milvus).
* Data Pipeline Development: Experience building pipelines for RAG solutions, conversationalagents, and classical ML models using tools such as dbt, Dagster, or Prefect.
* DevOps/DataOps: Proficiency with containerization (Docker, Kubernetes) and CI/CDpipelines for deploying data workflows in secure environments. Professional Qualifications
*

Experience:
5+ years of experience in Data Engineering, with at least 2 years focused onbuilding pipelines for Machine Learning or Generative AI applications in an enterprise setting.
* Domain Knowledge: Experience in Supply Chain, Manufacturing, or Defence sectors ishighly desirable. Ability to understand Bill of Materials (BOM) structures and procurementlifecycles.
* Problem Solving: Ability to navigate the "Governance Collision" between agile data work andrigid systems engineering requirements, ensuring data deliverables meet formal Stage Gatereviews.
* Collaboration: Proven ability to work alongside Data Scientists and Backend Engineers todefine data schemas that support predictive modelling and AI agents. Why This Role? You will be the architect of the data foundation behind advanced AI capabilities. Your work will directlyenable AI agents to support procurement workflows, forecasting, analytics, and engineering decisionmaking. If you are ready to build the robust infrastructure that turns raw data into strategic advantage,this role is for you. GCS is acting as an Employment Business in relation to this vacancy.