Data Engineering Lead

2 days ago

Dubai, Dubai, United Arab Emirates Dicetek LLC Full time
Roles and responsibilities
  • Overall 5-7 years of experience in data engineering and transformation on Cloud
  • 3+ Years of Very Strong Experience in Azure Data Engineering, Databricks
  • Expertise in supporting/developing lakehouse workloads at enterprise level
  • Experience in pyspark is required – developing and deploying the workloads to run on the Spark distributed computing
  • Candidate must possess at least a Graduate or bachelor's degree in Computer Science/Information Technology, Engineering (Computer/Telecommunication) or equivalent.
  • Cloud deployment: Preferably Microsoft azure
  • Experience in implementing the platform and application monitoring using Cloud native tools

  • Team Leadership and Management:

    • Team Supervision: Lead and mentor a team of data engineers to ensure high performance, professional growth, and alignment with organizational goals.
    • Task Allocation: Assign tasks and manage workloads effectively across the team, ensuring timely delivery of projects while maintaining quality.
    • Collaboration: Work closely with cross-functional teams (e.g., data scientists, analysts, business stakeholders) to ensure the data infrastructure supports their needs and objectives.
    • Training and Development: Provide guidance, training, and upskilling opportunities to junior and mid-level data engineers.

  • Data Architecture Design:

    • Pipeline Development: Design and implement scalable, reliable, and efficient data pipelines to process and transform large datasets from various sources.
    • Architecture Strategy: Lead the design and architecture of data systems and solutions, ensuring data is easily accessible, clean, and properly formatted for analytics.
    • Data Modeling: Work on data models, including data warehouses, lakes, and other data storage solutions, and design the schema that supports efficient querying and reporting.

  • Data Integration and ETL:

    • ETL Processes: Lead the development of Extract, Transform, Load (ETL) processes to integrate data from multiple sources, ensuring data quality, consistency, and reliability.
    • Data Quality Assurance: Ensure that the data pipelines and processes meet high standards of data accuracy, consistency, and timeliness. Establish data validation processes and error handling mechanisms.

  • Data Warehousing and Storage:

    • Data Storage Solutions: Lead efforts in selecting and implementing data storage solutions, such as data lakes, data warehouses, and cloud storage platforms (e.g., Amazon Redshift, Google BigQuery, Snowflake).
    • Scalability and Performance: Design and optimize data storage systems for performance, scalability, and cost-efficiency, taking into account future growth and business requirements.
    • Database Management: Oversee the management of relational and non-relational databases, ensuring optimal performance and security.
Desired candidate profile

  • Technical Expertise:

    • Expertise in data engineering concepts, such as ETL, data warehousing, data lakes, and data pipelines.
    • Proficiency in programming languages such as Python, Java, Scala, SQL, and frameworks like Apache Spark or Apache Flink for big data processing.
    • Strong experience with cloud platforms (AWS, Google Cloud, Azure) and associated services like S3, Redshift, BigQuery, EMR, and Databricks.
    • Familiarity with containerization tools (e.g., Docker, Kubernetes) for orchestrating and deploying data pipelines.

  • Database and Storage Knowledge:

    • Deep knowledge of relational and NoSQL databases, including MySQL, PostgreSQL, MongoDB, Cassandra, and HBase.
    • Familiarity with data storage solutions, including Data Lakes (e.g., AWS S3, Google Cloud Storage), Data Warehouses (e.g., Snowflake, BigQuery), and distributed file systems like HDFS.

  • Leadership and Management:

    • Experience managing and leading a team of data engineers, with strong organizational, mentoring, and communication skills.
    • Ability to manage multiple projects simultaneously, ensuring deadlines are met without compromising quality.
    • Conflict resolution, team-building, and performance management skills.

  • Data Modeling and Architecture:

    • Expertise in data modeling techniques and building scalable data architectures.
    • Ability to design efficient data models that support analytics, business intelligence (BI), and machine learning (ML) use cases.

  • Problem Solving and Optimization:

    • Strong analytical and problem-solving skills to optimize data pipelines and ensure efficient data processing and storage.
    • Experience in debugging and troubleshooting complex data issues.

  • Soft Skills:

    • Excellent communication skills to interact with both technical and non-technical stakeholders.
    • Strategic thinking to understand business needs and align technical solutions with those needs.
    • Ability to work in a fast-paced, collaborative environment and manage competing priorities.

  • Agile Methodologies:

    • Familiarity with agile methodologies (e.g., Scrum, Kanban) for managing projects and ensuring iterative delivery of data solutions.

  • Data Engineering Lead

    10 hours ago

    Dubai, Dubai, United Arab Emirates Robert Walters Full time

    Company Overview:Robert Walters is a leading organization in the field of Artificial Intelligence, seeking an experienced professional to lead their data engineering efforts.Job Description:We are looking for a skilled Data Engineering Lead to join our team. As a key member of our organization, you will be responsible for overseeing the entire engineering...

  • Data Engineering Expert

    7 days ago

    Dubai, Dubai, United Arab Emirates Capgemini Engineering Full time

    We are seeking an experienced Data Engineer to join our team at Capgemini Engineering. As a key member of our engineering services group, you will be responsible for designing and implementing cloud-based data solutions using Azure services. Your expertise in cloud computing concepts, Azure services, and data management will enable us to deliver high-quality...

  • Lead Data Engineer

    2 days ago

    Dubai, Dubai, United Arab Emirates Robert Walters Full time

    Direct message the job poster from Robert WaltersConsultant - Technology & Transformation at Robert WaltersOur client is a leading AI-driven organization seeking an experienced Data & AI Engineering professional to elevate their tech team.What you'll do:Lead, mentor, and scale a high-performing team of engineers.Develop and execute the data and AI...

  • Data Engineering Lead

    4 weeks ago

    Dubai, Dubai, United Arab Emirates Dicetek LLC Full time

    Roles and responsibilitiesOverall 5-7 years ofexperience in data engineering and transformation onCloud3+ Years of Very Strong Experience inAzure Data Engineering, DatabricksExpertise insupporting/developing lakehouse workloads at enterpriselevelExperience in pyspark is required– developing and deploying the workloads to run on theSpark distributed...

  • Data Engineering Lead

    1 week ago

    Dubai, Dubai, United Arab Emirates Dicetek LLC Full time

    Roles and responsibilitiesOverall 5-7 years ofexperience in data engineering and transformation onCloud3+ Years of Very Strong Experience inAzure Data Engineering, DatabricksExpertise insupporting/developing lakehouse workloads at enterpriselevelExperience in pyspark is required– developing and deploying the workloads to run on theSpark distributed...

  • Edgnex Data Center Mechanical Engineering Director

    6 days ago

    Dubai, Dubai, United Arab Emirates Edgnex Data Centers by DAMAC Full time

    We are seeking a highly experienced Edgnex Data Center Mechanical Engineering Director to lead our data center project teams. As a director, you will be responsible for overseeing the mechanical design and implementation of our data center projects.This role requires a strong background in mechanical engineering, with experience in designing and delivering...

  • Data Engineering Lead

    1 week ago

    Dubai, Dubai, United Arab Emirates Al-Futtaim Full time

    Guide and develop ahigh-performing Data Engineering team, fostering a culture ofinnovation, efficiency, andcollaboration.Apply your deep expertise inAzure Data Platform and other Data Engineering tools to refine andexpand our data capabilitiesLead theend-to-end process of designing, documenting, developing,implementing, and testing innovative prototypes that...

  • Data Engineering Lead

    4 days ago

    Dubai, Dubai, United Arab Emirates Al-Futtaim Full time

    Guide and develop ahigh-performing Data Engineering team, fostering a culture ofinnovation, efficiency, andcollaboration. Apply your deep expertise inAzure Data Platform and other Data Engineering tools to refine andexpand our data capabilities Lead theend-to-end process of designing, documenting, developing,implementing, and testing innovative prototypes...

  • Data Engineering Lead

    4 weeks ago

    Dubai, Dubai, United Arab Emirates Al-Futtaim Full time

    Guide and develop ahigh-performing Data Engineering team, fostering a culture ofinnovation, efficiency, andcollaboration.Apply your deep expertise inAzure Data Platform and other Data Engineering tools to refine andexpand our data capabilitiesLead theend-to-end process of designing, documenting, developing,implementing, and testing innovative prototypes that...

  • Data Engineering Lead

    3 weeks ago

    Dubai, Dubai, United Arab Emirates Al-Futtaim Full time

    Responsibilities Guide and develop a high-performing Data Engineering team, fostering a culture of innovation, efficiency, and collaboration. Apply your deep expertise in Azure Data Platform and other Data Engineering tools to refine and expand our data capabilities. Lead the end-to-end process of designing, documenting, developing, implementing, and testing...

  • Data Engineering Lead

    2 days ago

    Dubai, Dubai, United Arab Emirates Al-Futtaim Full time

    Guide and develop a high-performing Data Engineering team, fostering a culture of innovation, efficiency, and collaboration.Apply your deep expertise in Azure Data Platform and other Data Engineering tools to refine and expand our data capabilitiesLead the end-to-end process of designing, documenting, developing, implementing, and testing innovative...

  • Mechanical Design Lead

    6 days ago

    Dubai, Dubai, United Arab Emirates Edgnex Data Centers by DAMAC Full time

    The Edgnex Data Centers by DAMAC team is seeking a highly skilled Mechanical Design Lead to oversee the design and implementation of our data center projects. As a key member of our team, you will be responsible for developing and managing the mechanical design process from concept to completion.This role requires a strong background in mechanical...

  • Lead Data

    11 hours ago

    Dubai, Dubai, United Arab Emirates Robert Walters Full time

    Lead Data & AI Engineer (7TW60E-DE71975D) Dubai, United Arab EmiratesOur client is a leading AI-driven organization seeking an experienced Data & AI Engineering professional to elevate their tech team and lead the engineering strategy.What you'll do:As a Lead Data & AI Engineer, you will lead a high-performing team of engineers, fostering an environment that...

  • Data Engineering Team Lead

    1 day ago

    Dubai, Dubai, United Arab Emirates Robert Walters Full time

    Job OverviewWe are seeking an experienced Data & AI Engineering professional to elevate our tech team. As a key member of our organization, you will be responsible for leading, mentoring, and scaling a high-performing team of engineers.You will develop and execute the data and AI engineering strategy, collaborate with internal stakeholders to define...

  • Lead Data Engineer

    2 days ago

    Dubai, Dubai, United Arab Emirates G42 Full time

    QualificationsSkills and attributes for success Bachelor's degree in Computer Science, Engineering, Mathematics, or a related field; or equivalent work experience.3+ years of experience in a Data Engineering role.Proficiency in SQL and programming languages like Python, Java, and Scala.Hands-on experience with big data technologies like Hadoop, Spark and...

  • Data Engineering Lead

    7 days ago

    Dubai, Dubai, United Arab Emirates RemoteWorker US Full time

    **About the Job:**We are looking for a highly skilled Data Engineer to join our team at RemoteWorker US. As a key member of our Data Engineering team, you will design and develop tools, pipelines, and systems that enable us to operate reliably at scale.About the Company:At RemoteWorker US, we strive to provide music creators with the best possible tools to...

  • Development Strategy Lead

    1 week ago

    Dubai, Dubai, United Arab Emirates LEAD Development Full time

    The Senior Development Manager at LEAD Development oversees the entire development and design process, ensuring the successful execution of projects within given timeframes. This involves collaborating with finance and sales teams to integrate commercial requirements into the design and development process, encompassing the Masterplan and its components....

  • Lead Data Ops Engineer

    6 days ago

    Dubai, Dubai, United Arab Emirates Emirates Global Aluminium (EGA) Full time

    The Lead Data Ops Engineer will play a critical role in building, scaling, automating, and maintaining the company's Big Data infrastructure and Machine Learning operations (MLOps). This individual will work in close collaboration with our Data Architect, Data Science, Data Engineering, and IT teams to ensure the development, deployment and scale of robust,...

  • Data Center Structural Engineering Lead

    2 days ago

    Dubai, Dubai, United Arab Emirates BSO Full time

    Job OverviewWe are seeking a Senior Structural Engineer to lead the design and analysis of advanced AI data center facilities. In this role, you will be responsible for ensuring the structural integrity, safety, and cost-effectiveness of large-scale, mission-critical infrastructure.You will collaborate closely with architects, mechanical engineers,...

  • Lead Data Engineer

    1 week ago

    Dubai, Dubai, United Arab Emirates G42 Full time

    QualificationsSkillsand attributes for successBachelor's degree inComputer Science, Engineering, Mathematics, or a related field; orequivalent work experience.3+ years ofexperience in a Data Engineeringrole.Proficiency in SQL and programminglanguages like Python, Java, andScala.Hands-on experience with big datatechnologies like Hadoop, Spark...