Data Engineering Lead

1 week ago

Dubai, Dubai, United Arab Emirates Dicetek LLC Full time
Roles and responsibilities
  • Overall 5-7 years of experience in data engineering and transformation on Cloud
  • 3+ Years of Very Strong Experience in Azure Data Engineering, Databricks
  • Expertise in supporting/developing lakehouse workloads at enterprise level
  • Experience in pyspark is required – developing and deploying the workloads to run on the Spark distributed computing
  • Candidate must possess at least a Graduate or bachelor's degree in Computer Science/Information Technology, Engineering (Computer/Telecommunication) or equivalent.
  • Cloud deployment: Preferably Microsoft azure
  • Experience in implementing the platform and application monitoring using Cloud native tools

  • Team Leadership and Management:

    • Team Supervision: Lead and mentor a team of data engineers to ensure high performance, professional growth, and alignment with organizational goals.
    • Task Allocation: Assign tasks and manage workloads effectively across the team, ensuring timely delivery of projects while maintaining quality.
    • Collaboration: Work closely with cross-functional teams (e.g., data scientists, analysts, business stakeholders) to ensure the data infrastructure supports their needs and objectives.
    • Training and Development: Provide guidance, training, and upskilling opportunities to junior and mid-level data engineers.

  • Data Architecture Design:

    • Pipeline Development: Design and implement scalable, reliable, and efficient data pipelines to process and transform large datasets from various sources.
    • Architecture Strategy: Lead the design and architecture of data systems and solutions, ensuring data is easily accessible, clean, and properly formatted for analytics.
    • Data Modeling: Work on data models, including data warehouses, lakes, and other data storage solutions, and design the schema that supports efficient querying and reporting.

  • Data Integration and ETL:

    • ETL Processes: Lead the development of Extract, Transform, Load (ETL) processes to integrate data from multiple sources, ensuring data quality, consistency, and reliability.
    • Data Quality Assurance: Ensure that the data pipelines and processes meet high standards of data accuracy, consistency, and timeliness. Establish data validation processes and error handling mechanisms.

  • Data Warehousing and Storage:

    • Data Storage Solutions: Lead efforts in selecting and implementing data storage solutions, such as data lakes, data warehouses, and cloud storage platforms (e.g., Amazon Redshift, Google BigQuery, Snowflake).
    • Scalability and Performance: Design and optimize data storage systems for performance, scalability, and cost-efficiency, taking into account future growth and business requirements.
    • Database Management: Oversee the management of relational and non-relational databases, ensuring optimal performance and security.
Desired candidate profile

  • Technical Expertise:

    • Expertise in data engineering concepts, such as ETL, data warehousing, data lakes, and data pipelines.
    • Proficiency in programming languages such as Python, Java, Scala, SQL, and frameworks like Apache Spark or Apache Flink for big data processing.
    • Strong experience with cloud platforms (AWS, Google Cloud, Azure) and associated services like S3, Redshift, BigQuery, EMR, and Databricks.
    • Familiarity with containerization tools (e.g., Docker, Kubernetes) for orchestrating and deploying data pipelines.

  • Database and Storage Knowledge:

    • Deep knowledge of relational and NoSQL databases, including MySQL, PostgreSQL, MongoDB, Cassandra, and HBase.
    • Familiarity with data storage solutions, including Data Lakes (e.g., AWS S3, Google Cloud Storage), Data Warehouses (e.g., Snowflake, BigQuery), and distributed file systems like HDFS.

  • Leadership and Management:

    • Experience managing and leading a team of data engineers, with strong organizational, mentoring, and communication skills.
    • Ability to manage multiple projects simultaneously, ensuring deadlines are met without compromising quality.
    • Conflict resolution, team-building, and performance management skills.

  • Data Modeling and Architecture:

    • Expertise in data modeling techniques and building scalable data architectures.
    • Ability to design efficient data models that support analytics, business intelligence (BI), and machine learning (ML) use cases.

  • Problem Solving and Optimization:

    • Strong analytical and problem-solving skills to optimize data pipelines and ensure efficient data processing and storage.
    • Experience in debugging and troubleshooting complex data issues.

  • Soft Skills:

    • Excellent communication skills to interact with both technical and non-technical stakeholders.
    • Strategic thinking to understand business needs and align technical solutions with those needs.
    • Ability to work in a fast-paced, collaborative environment and manage competing priorities.

  • Agile Methodologies:

    • Familiarity with agile methodologies (e.g., Scrum, Kanban) for managing projects and ensuring iterative delivery of data solutions.

  • Azure Data Engineer Lead

    1 week ago

    Dubai, Dubai, United Arab Emirates Capgemini Engineering Full time

    Candidates interested in the position of Azure Data Engineer Lead should have extensive knowledge of various Azure services and their functionalities. They will understand cloud computing principles, including infrastructure as a service (IaaS), platform as a service (PaaS), and software as a service (SaaS). Familiarity with cloud deployment models like...

  • Data Engineer

    1 week ago

    Dubai, Dubai, United Arab Emirates Data Labs - LANDMARK GROUP Full time

    As a Junior Data Engineer, you will play a key role in supporting our data engineering initiatives that drive automation, data quality, and accessibility across the organization. Working closely with senior team members and business stakeholders, you will contribute to the creation of scalable, cloud-based data solutions on Microsoft...

  • Data Engineer

    23 hours ago

    Dubai, Dubai, United Arab Emirates Data Labs - LANDMARK GROUP Full time

    As a Junior Data Engineer, you will play a key role in supporting our data engineering initiatives that drive automation, data quality, and accessibility across the organization. Working closely with senior team members and business stakeholders, you will contribute to the creation of scalable, cloud-based data solutions on Microsoft Azure. Responsibilities:...

  • Azure Data Engineer

    3 weeks ago

    Dubai, Dubai, United Arab Emirates Capgemini Engineering Full time

    Get AI-powered advice on this job and more exclusive features.Direct message the job poster from Capgemini EngineeringTalent Acquisition Lead - Strategic and Leadership Hiring at Capgemini Engineering (World leader in engineering and R&D services)Capgemini Engineering, a global leader in engineering services, brings together engineering, science and...

  • Azure Data Engineer

    2 weeks ago

    Dubai, Dubai, United Arab Emirates Capgemini Engineering Full time

    Get AI-powered advice on this job and more exclusive features. Direct message the job poster from Capgemini Engineering Talent Acquisition Lead - Strategic and Leadership Hiring at Capgemini Engineering (World leader in engineering and R&D services) Capgemini Engineering , a global leader in engineering services, brings together engineering, science and...

  • Azure Data Engineer

    2 days ago

    Dubai, Dubai, United Arab Emirates Capgemini Engineering Full time

    Get AI-powered advice on this job and more exclusive features. Direct message the job poster from Capgemini Engineering Talent Acquisition Lead - Strategic and Leadership Hiring at Capgemini Engineering (World leader in engineering and R&D services) Capgemini Engineering , a global leader in engineering services, brings together engineering, science and...

  • Data Engineer Lead

    5 days ago

    Dubai, Dubai, United Arab Emirates G42 Full time

    G42 OverviewG42 is a leading technology company that combines cutting-edge research and development with innovative engineering practices to deliver transformative solutions. We're seeking a talented Lead Data Engineer to join our team of experts in data engineering, machine learning, and artificial intelligence.About the RoleWe're looking for a seasoned...

  • Data Center Mechanical Operations Lead

    1 week ago

    Dubai, Dubai, United Arab Emirates Edgnex Data Centers by DAMAC Full time

    Data Center Mechanical Design ManagementEdgnex Data Centers by DAMAC is seeking a highly experienced Mechanical Design Manager to lead the successful design of our data center projects. The ideal candidate will have a strong background in mechanical engineering, with extensive knowledge of designing mechanical systems for data centers.The Mechanical Design...

  • Lead Data Engineer

    3 weeks ago

    Dubai, Dubai, United Arab Emirates Robert Walters Full time

    Direct message the job poster from Robert Walters Consultant - Technology & Transformation at Robert Walters Our client is a leading AI-driven organization seeking an experienced Data & AI Engineering professional to elevate their tech team. What you'll do: Lead, mentor, and scale a high-performing team of engineers. Develop and execute the data and AI...

  • Lead Data Engineer

    4 days ago

    Dubai, Dubai, United Arab Emirates Robert Walters Full time

    Direct message the job poster from Robert Walters Consultant - Technology & Transformation at Robert Walters Our client is a leading AI-driven organization seeking an experienced Data & AI Engineering professional to elevate their tech team. What you'll do: Lead, mentor, and scale a high-performing team of engineers. Develop and execute the data and AI...

  • Lead Data Engineer

    1 week ago

    Dubai, Dubai, United Arab Emirates Xpoint Full time

    Join Xpoint, a trailblazer in the fast-growing online sports betting and gaming industry. We solve complex regulatory challenges by providing cutting-edge geolocation solutions to ensure operators comply with strict regulations. As a key player in the industry, Xpoint is gaining market share by offering superior technology and delivering exceptional customer...

  • Data Engineering Lead

    3 weeks ago

    Dubai, Dubai, United Arab Emirates Dicetek LLC Full time

    Roles and Responsibilities Overall 5-7 years of experience in data engineering and transformation on Cloud 3+ Years of Very Strong Experience in Azure Data Engineering, Databricks Expertise in supporting/developing lakehouse workloads at enterprise level Experience in pyspark is required – developing and deploying the workloads to run on the Spark...

  • Lead Data Engineer

    3 weeks ago

    Dubai, Dubai, United Arab Emirates Xpoint Full time

    Join Xpoint, a trailblazer in the fast-growing online sports betting and gaming industry. We solve complex regulatory challenges by providing cutting-edge geolocation solutions to ensure operators comply with strict regulations. As a key player in the industry, Xpoint is gaining market share by offering superior technology and delivering exceptional customer...

  • Data Engineering Lead

    5 days ago

    Dubai, Dubai, United Arab Emirates Wunderman Thompson Full time

    About the RoleWe are seeking a highly experienced Senior Data Engineer to lead our team in developing a cutting-edge data integration for a key agency client. As a Senior Data Engineer, you will work closely with our Data Architect and collaborate with stakeholders to design and implement scalable data management systems.Key ResponsibilitiesDesign data...

  • Data Engineering Lead

    4 weeks ago

    Dubai, Dubai, United Arab Emirates Al-Futtaim Full time

    ResponsibilitiesGuide and develop a high-performing Data Engineering team, fostering a culture of innovation, efficiency, and collaboration.Apply your deep expertise in Azure Data Platform and other Data Engineering tools to refine and expand our data capabilities.Lead the end-to-end process of designing, documenting, developing, implementing, and testing...

  • Data Engineering Lead

    1 week ago

    Dubai, Dubai, United Arab Emirates Al-Futtaim Full time

    Guide and develop a high-performing Data Engineering team, fostering a culture of innovation, efficiency, and collaboration.Apply your deep expertise in Azure Data Platform and other Data Engineering tools to refine and expand our data capabilitiesLead the end-to-end process of designing, documenting, developing, implementing, and testing innovative...

  • Data Engineering Lead

    2 weeks ago

    Dubai, Dubai, United Arab Emirates Al-Futtaim Full time

    Responsibilities Guide and develop a high-performing Data Engineering team, fostering a culture of innovation, efficiency, and collaboration. Apply your deep expertise in Azure Data Platform and other Data Engineering tools to refine and expand our data capabilities. Lead the end-to-end process of designing, documenting, developing, implementing, and...

  • Senior Data Engineering Lead

    1 week ago

    Dubai, Dubai, United Arab Emirates Lumina Tech Group Full time

    Job OverviewThis role involves leading architectural decisions for cloud-based data infrastructure on AWS, mentoring and guiding data engineering teams, designing and implementing scalable data pipelines for GenAI applications, and driving technical excellence and quality standards across deliverables.Lead architectural decisions for cloud-based data...

  • Data Engineering Team Lead

    1 week ago

    Dubai, Dubai, United Arab Emirates M2 Exchange Full time

    Job SummaryThe Head of Data Engineering role at M2 Exchange involves leading the development and implementation of our data architecture and infrastructure. This position requires expertise in data engineering, technical leadership, and strategic planning.About M2 ExchangeM2 Exchange is a cutting-edge company that relies on innovative technologies to drive...

  • Data Engineering Lead

    1 week ago

    Dubai, Dubai, United Arab Emirates Black Pearl Full time

    Black Pearl is seeking a highly skilled Senior Data Engineer to lead the design, deployment, and management of our SAP HANA FSDP platform in Dubai. The ideal candidate will ensure optimal performance, scalability, and reliability.Your Key Responsibilities:SAP HANA FSDP Architecture: Develop and implement best practices for architecture, configuration, and...