Data Engineering Lead

5 days ago

Dubai, Dubai, United Arab Emirates Dicetek LLC Full time
Roles and Responsibilities
  • Overall 5-7 years of experience in data engineering and transformation on Cloud
  • 3+ Years of Very Strong Experience in Azure Data Engineering, Databricks
  • Expertise in supporting/developing lakehouse workloads at enterprise level
  • Experience in pyspark is required – developing and deploying the workloads to run on the Spark distributed computing
  • Candidate must possess at least a Graduate or bachelor's degree in Computer Science/Information Technology, Engineering (Computer/Telecommunication) or equivalent.
  • Cloud deployment: Preferably Microsoft azure
  • Experience in implementing the platform and application monitoring using Cloud native tools

  • Team Leadership and Management :

    • Team Supervision : Lead and mentor a team of data engineers to ensure high performance, professional growth, and alignment with organizational goals.
    • Task Allocation : Assign tasks and manage workloads effectively across the team, ensuring timely delivery of projects while maintaining quality.
    • Collaboration : Work closely with cross-functional teams (e.g., data scientists, analysts, business stakeholders) to ensure the data infrastructure supports their needs and objectives.
    • Training and Development : Provide guidance, training, and upskilling opportunities to junior and mid-level data engineers.

  • Data Architecture Design :

    • Pipeline Development : Design and implement scalable, reliable, and efficient data pipelines to process and transform large datasets from various sources.
    • Architecture Strategy : Lead the design and architecture of data systems and solutions, ensuring data is easily accessible, clean, and properly formatted for analytics.
    • Data Modeling : Work on data models, including data warehouses, lakes, and other data storage solutions, and design the schema that supports efficient querying and reporting.

  • Data Integration and ETL :

    • ETL Processes : Lead the development of Extract, Transform, Load (ETL) processes to integrate data from multiple sources, ensuring data quality, consistency, and reliability.
    • Data Quality Assurance : Ensure that the data pipelines and processes meet high standards of data accuracy, consistency, and timeliness. Establish data validation processes and error handling mechanisms.

  • Data Warehousing and Storage :

    • Data Storage Solutions : Lead efforts in selecting and implementing data storage solutions, such as data lakes, data warehouses, and cloud storage platforms (e.g., Amazon Redshift , Google BigQuery , Snowflake ).
    • Scalability and Performance : Design and optimize data storage systems for performance, scalability, and cost-efficiency, taking into account future growth and business requirements.
    • Database Management : Oversee the management of relational and non-relational databases, ensuring optimal performance and security.
Desired Candidate Profile

  • Technical Expertise :

    • Expertise in data engineering concepts, such as ETL, data warehousing, data lakes, and data pipelines.
    • Proficiency in programming languages such as Python , Java , Scala , SQL , and frameworks like Apache Spark or Apache Flink for big data processing.
    • Strong experience with cloud platforms (AWS , Google Cloud , Azure ) and associated services like S3 , Redshift , BigQuery , EMR , and Databricks .
    • Familiarity with containerization tools (e.g., Docker , Kubernetes ) for orchestrating and deploying data pipelines.

  • Database and Storage Knowledge :

    • Deep knowledge of relational and NoSQL databases, including MySQL , PostgreSQL , MongoDB , Cassandra , and HBase .
    • Familiarity with data storage solutions, including Data Lakes (e.g., AWS S3 , Google Cloud Storage ), Data Warehouses (e.g., Snowflake , BigQuery ), and distributed file systems like HDFS .

  • Leadership and Management :

    • Experience managing and leading a team of data engineers, with strong organizational, mentoring, and communication skills.
    • Ability to manage multiple projects simultaneously, ensuring deadlines are met without compromising quality.
    • Conflict resolution, team-building, and performance management skills.

  • Data Modeling and Architecture :

    • Expertise in data modeling techniques and building scalable data architectures.
    • Ability to design efficient data models that support analytics, business intelligence (BI), and machine learning (ML) use cases.

  • Problem Solving and Optimization :

    • Strong analytical and problem-solving skills to optimize data pipelines and ensure efficient data processing and storage.
    • Experience in debugging and troubleshooting complex data issues.

  • Soft Skills :

    • Excellent communication skills to interact with both technical and non-technical stakeholders.
    • Strategic thinking to understand business needs and align technical solutions with those needs.
    • Ability to work in a fast-paced, collaborative environment and manage competing priorities.

  • Agile Methodologies :

    • Familiarity with agile methodologies (e.g., Scrum , Kanban ) for managing projects and ensuring iterative delivery of data solutions.
#J-18808-Ljbffr

  • Data Engineer

    4 days ago

    Dubai, Dubai, United Arab Emirates Data Labs - LANDMARK GROUP Full time

    As a Junior Data Engineer, you will play a key role in supporting our data engineering initiatives that drive automation, data quality, and accessibility across the organization. Working closely with senior team members and business stakeholders, you will contribute to the creation of scalable, cloud-based data solutions on Microsoft...

  • Data Engineer

    2 weeks ago

    Dubai, Dubai, United Arab Emirates Data Labs - LANDMARK GROUP Full time

    As a Junior Data Engineer, you will play a key role in supporting our data engineering initiatives that drive automation, data quality, and accessibility across the organization. Working closely with senior team members and business stakeholders, you will contribute to the creation of scalable, cloud-based data solutions on Microsoft...

  • Azure Data Engineer

    5 days ago

    Dubai, Dubai, United Arab Emirates Capgemini Engineering Full time

    Get AI-powered advice on this job and more exclusive features.Direct message the job poster from Capgemini EngineeringTalent Acquisition Lead - Strategic and Leadership Hiring at Capgemini Engineering (World leader in engineering and R&D services)Capgemini Engineering, a global leader in engineering services, brings together engineering, science and...

  • Azure Data Engineer

    2 weeks ago

    Dubai, Dubai, United Arab Emirates Capgemini Engineering Full time

    Get AI-powered advice on this job and more exclusive features. Direct message the job poster from Capgemini Engineering Talent Acquisition Lead - Strategic and Leadership Hiring at Capgemini Engineering (World leader in engineering and R&D services) Capgemini Engineering , a global leader in engineering services, brings together engineering, science and...

  • Lead Data Engineer

    2 weeks ago

    Dubai, Dubai, United Arab Emirates Robert Walters Full time

    Direct message the job poster from Robert WaltersConsultant - Technology & Transformation at Robert WaltersOur client is a leading AI-driven organization seeking an experienced Data & AI Engineering professional to elevate their tech team.What you'll do:Lead, mentor, and scale a high-performing team of engineers.Develop and execute the data and AI...

  • Lead Data Engineer

    10 hours ago

    Dubai, Dubai, United Arab Emirates Robert Walters Full time

    Direct message the job poster from Robert Walters Consultant - Technology & Transformation at Robert Walters Our client is a leading AI-driven organization seeking an experienced Data & AI Engineering professional to elevate their tech team. What you'll do: Lead, mentor, and scale a high-performing team of engineers. Develop and execute the data and AI...

  • Data Engineering Lead

    4 hours ago

    Dubai, Dubai, United Arab Emirates Dicetek LLC Full time

    Roles and responsibilities Overall 5-7 years of experience in data engineering and transformation on Cloud3+ Years of Very Strong Experience in Azure Data Engineering, DatabricksExpertise in supporting/developing lakehouse workloads at enterprise levelExperience in pyspark is required – developing and deploying the workloads to run on the Spark...

  • Lead Data Engineer

    2 days ago

    Dubai, Dubai, United Arab Emirates Xpoint Full time

    Join Xpoint, a trailblazer in the fast-growing online sports betting and gaming industry. We solve complex regulatory challenges by providing cutting-edge geolocation solutions to ensure operators comply with strict regulations. As a key player in the industry, Xpoint is gaining market share by offering superior technology and delivering exceptional customer...

  • Senior Data Engineer Lead

    2 days ago

    Dubai, Dubai, United Arab Emirates M2 Exchange Full time

    M2 Exchange is seeking an experienced professional to fill the role of Senior Data Engineer Lead. This key position will be responsible for overseeing the development and implementation of our data engineering projects.Responsibilities:Data Engineering Projects:Oversee the development and implementation of data engineering projects, ensuring they are...

  • Data Engineering Lead

    3 weeks ago

    Dubai, Dubai, United Arab Emirates Dicetek LLC Full time

    Roles and responsibilitiesOverall 5-7 years ofexperience in data engineering and transformation onCloud3+ Years of Very Strong Experience inAzure Data Engineering, DatabricksExpertise insupporting/developing lakehouse workloads at enterpriselevelExperience in pyspark is required– developing and deploying the workloads to run on theSpark distributed...

  • Senior Azure Data Engineer

    5 days ago

    Dubai, Dubai, United Arab Emirates Capgemini Engineering Full time

    **About the Role:**We are seeking an experienced Senior Azure Data Engineer to join our team at Capgemini Engineering. As a key member of our engineering department, you will be responsible for designing and implementing data solutions using various Azure services.**Responsibilities:**Develop end-to-end applications using multiple programming languages...

  • Data Engineering Lead

    3 weeks ago

    Dubai, Dubai, United Arab Emirates Al-Futtaim Full time

    Guide and develop ahigh-performing Data Engineering team, fostering a culture ofinnovation, efficiency, andcollaboration.Apply your deep expertise inAzure Data Platform and other Data Engineering tools to refine andexpand our data capabilitiesLead theend-to-end process of designing, documenting, developing,implementing, and testing innovative prototypes that...

  • Data Engineering Lead

    1 week ago

    Dubai, Dubai, United Arab Emirates Al-Futtaim Full time

    ResponsibilitiesGuide and develop a high-performing Data Engineering team, fostering a culture of innovation, efficiency, and collaboration.Apply your deep expertise in Azure Data Platform and other Data Engineering tools to refine and expand our data capabilities.Lead the end-to-end process of designing, documenting, developing, implementing, and testing...

  • Data Engineering Lead

    4 hours ago

    Dubai, Dubai, United Arab Emirates Al-Futtaim Full time

    Guide and develop a high-performing Data Engineering team, fostering a culture of innovation, efficiency, and collaboration.Apply your deep expertise in Azure Data Platform and other Data Engineering tools to refine and expand our data capabilitiesLead the end-to-end process of designing, documenting, developing, implementing, and testing innovative...

  • Lead Data

    2 weeks ago

    Dubai, Dubai, United Arab Emirates Robert Walters Full time

    Lead Data & AI Engineer (7TW60E-DE71975D) Dubai, United Arab EmiratesOur client is a leading AI-driven organization seeking an experienced Data & AI Engineering professional to elevate their tech team and lead the engineering strategy.What you'll do:As a Lead Data & AI Engineer, you will lead a high-performing team of engineers, fostering an environment that...

  • Data Engineering Lead

    2 days ago

    Dubai, Dubai, United Arab Emirates VML South Africa Full time

    At VML South Africa, we are seeking a seasoned Data Engineering Lead to spearhead the development of a cutting-edge data integration for a prominent agency client. As part of our team, you will collaborate closely with cross-functional stakeholders to design and implement scalable data management systems that meet business objectives.We are looking for an...

  • Data Engineer Lead

    2 days ago

    Dubai, Dubai, United Arab Emirates TPConnects Full time

    TPConnects is an IATA-certified, global travel content aggregation and distribution company seeking a highly skilled Senior Data Engineer to play a critical role in advancing best practices for data engineering within our organization. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining robust data architecture,...

  • Financial Data Engineering Lead

    2 days ago

    Dubai, Dubai, United Arab Emirates BHFT Full time

    BHFT is a leading global algo trading company driven by cutting-edge technology. We're looking for a talented Financial Data Engineering Lead to join our team.About the RoleThis is an exciting opportunity to work on designing, developing, and maintaining complex systems that handle historical market data from multiple sources.Main Responsibilities:You will...

  • Lead Data Systems Engineer

    3 days ago

    Dubai, Dubai, United Arab Emirates Xpoint Full time

    Xpoint is an innovative technology company that empowers the online sports betting and gaming industry with cutting-edge geolocation solutions. We prioritize collaboration, accountability, and continuous improvement, shaping our culture and success.Job DescriptionWe seek a seasoned Lead Data Systems Engineer to join our tech team, focusing on designing,...

  • Data Protection Specialist

    1 week ago

    Dubai, Dubai, United Arab Emirates Alpha Data Full time

    Company OverviewAlpha Data is a leading provider of data protection solutions. Our team is dedicated to delivering exceptional service to our clients, ensuring their data is secure and recoverable at all times.