Lead Data Engineer

3 weeks ago


Abu Dhabi, Abu Dhabi, United Arab Emirates Dicetek LLC Full time
Roles and responsibilities
  • Overall 5-7 years ofexperience in data engineering and transformation onCloud
  • 3+ Years of Very Strong Experience inAzure Data Engineering, Databricks
  • Expertise insupporting/developing lakehouse workloads at enterpriselevel
  • Experience in pyspark is required– developing and deploying the workloads to run on theSpark distributed computing
  • Candidate mustpossess at least a Graduate or bachelor's degree inComputer Science/Information Technology, Engineering(Computer/Telecommunication) orequivalent.
  • Cloud deployment: PreferablyMicrosoft azure
  • Experience in implementing theplatform and application monitoring using Cloud nativetools
  • 1. DataArchitecture &Design

  • Strongunderstanding of database architectures, including relational,NoSQL, and data warehousing.
  • Ability to designand implement data pipelines, data lakes, and datawarehouses.
  • Knowledge of ETL (Extract,Transform, Load) processes.
  • 2.Programming &Scripting

  • Languages:Expertise in Python, Java, Scala, andSQL.
  • Familiarity with datamanipulation libraries like Pandas, Dask, and NumPy(for Python).
  • Ability to write efficientqueries and scripts for data processing andanalysis.
  • 3. Big DataTechnologies

  • Proficiencyin big data frameworks like Hadoop,Spark, andFlink.
  • Experiencewith distributed computing systems and handling largedatasets.
  • 4. CloudPlatforms

  • Knowledge ofcloud platforms such as AWS,Google Cloud Platform (GCP), andMicrosoftAzure.
  • Experience withcloud-based data solutions like AmazonRedshift, GoogleBigQuery, or Azure DataLake.
  • 5.DataModeling

  • Expertise indata modeling techniques, such as dimensionalmodeling, ER modeling,and star/snowflakeschemas.
  • Ability to designschema structures that support business intelligence and analyticalneeds.
  • 6. Data PipelineManagement

  • Experiencein creating and maintaining data pipelines for large-scale dataprocessing.
  • Familiarity with tools likeApache Airflow,Luigi, ordbt for orchestratingworkflows.
  • 7. DataWarehousing & SQLOptimization

  • Expertisein designing and optimizing data warehousingsolutions.
  • Strong skills in SQLtuning, optimization, and performance improvementfor handling large datasets.
  • 8.Data Governance &Security

  • Understandingof data governance practices, such as data privacy, metadatamanagement, and ensuring data quality.
  • Abilityto enforce data security standards andpractices for data protection.
  • 9.Team Leadership & ProjectManagement

  • Ability tolead a team of data engineers, collaborate with cross-functionalteams, and manage data engineeringprojects.
  • Familiarity with project managementframeworks like Agile andScrum.
  • 10.Automation &CI/CD

  • Experience inautomating data workflows and deploymentpipelines.
  • Knowledge ofCI/CD tools (e.g., Jenkins, GitLab)for version control andautomation.
  • 11. MachineLearning Integration(Optional)

  • While notstrictly necessary for all data engineering roles, experienceintegrating data pipelines with machine learning models orsupporting the deployment of models can bevaluable.
Desired candidate profile

1.Data Architecture &Design

  • Strongunderstanding of database architectures, including relational,NoSQL, and data warehousing.
  • Ability to designand implement data pipelines, data lakes, and datawarehouses.
  • Knowledge of ETL (Extract,Transform, Load) processes.

2.Programming &Scripting

  • Languages:Expertise in Python, Java, Scala, andSQL.
  • Familiarity with datamanipulation libraries like Pandas, Dask, and NumPy(for Python).
  • Ability to write efficientqueries and scripts for data processing andanalysis.

3. Big DataTechnologies

  • Proficiencyin big data frameworks like Hadoop,Spark, andFlink.
  • Experiencewith distributed computing systems and handling largedatasets.

4. CloudPlatforms

  • Knowledge ofcloud platforms such as AWS,Google Cloud Platform (GCP), andMicrosoftAzure.
  • Experience withcloud-based data solutions like AmazonRedshift, GoogleBigQuery, or Azure DataLake.

5.DataModeling

  • Expertise indata modeling techniques, such as dimensionalmodeling, ER modeling,and star/snowflakeschemas.
  • Ability to designschema structures that support business intelligence and analyticalneeds.

6. Data PipelineManagement

  • Experiencein creating and maintaining data pipelines for large-scale dataprocessing.
  • Familiarity with tools likeApache Airflow,Luigi, ordbt for orchestratingworkflows.

7. DataWarehousing & SQLOptimization

  • Expertisein designing and optimizing data warehousingsolutions.
  • Strong skills in SQLtuning, optimization, and performance improvementfor handling large datasets.

8.Data Governance &Security

  • Understandingof data governance practices, such as data privacy, metadatamanagement, and ensuring data quality.
  • Abilityto enforce data security standards andpractices for data protection.

9.Team Leadership & ProjectManagement

  • Ability tolead a team of data engineers, collaborate with cross-functionalteams, and manage data engineeringprojects.
  • Familiarity with project managementframeworks like Agile andScrum.

10.Automation &CI/CD

  • Experience inautomating data workflows and deploymentpipelines.
  • Knowledge ofCI/CD tools (e.g., Jenkins, GitLab)for version control andautomation.

11. MachineLearning Integration(Optional)

  • While notstrictly necessary for all data engineering roles, experienceintegrating data pipelines with machine learning models orsupporting the deployment of models can bevaluable.
Key Skills
Engineering,EngineeringMathematics,Data Reporting
Employment Type: Part-time
Department / FunctionalArea: Data Engineering
Experience: Not Mentioned years
Gender: Male
Vacancy: 1
Joining Date: Wed, 19 Feb2025

  • Abu Dhabi, Abu Dhabi, United Arab Emirates Alpha Data Full time

    Job OverviewWe are seeking a skilled Data Protection Engineer to join our team at Alpha Data. In this role, you will be responsible for administering and managing Veeam Backup as a service platform, including vCloud Connect, Kasten, and VBR.Key ResponsibilitiesDevelop customized plans to maximize value from Veeam products and services.Identify pain points...

  • Lead Data Engineer

    3 weeks ago


    Abu Dhabi, Abu Dhabi, United Arab Emirates Dicetek LLC Full time

    Roles and ResponsibilitiesOverall 5-7 years of experience in data engineering and transformation on Cloud3+ Years of Very Strong Experience in Azure Data Engineering, DatabricksExpertise in supporting/developing lakehouse workloads at enterprise levelExperience in pyspark is required – developing and deploying the workloads to run on the Spark distributed...


  • Abu Dhabi, Abu Dhabi, United Arab Emirates Whiteblue Cloud Services Full time

    Job DescriptionWe are seeking an experienced Data Engineer Lead to join our team at Whiteblue Cloud Services. The ideal candidate will have a strong background in designing, developing, and maintaining data pipelines in Azure Data Factory (ADF) for ETL from on-premise and cloud-based sources. Additionally, they should have expertise in designing, developing,...

  • Lead Data Engineer

    1 week ago


    Abu Dhabi, Abu Dhabi, United Arab Emirates Dicetek LLC Full time

    Roles and responsibilitiesOverall 5-7 years ofexperience in data engineering and transformation onCloud3+ Years of Very Strong Experience inAzure Data Engineering, DatabricksExpertise insupporting/developing lakehouse workloads at enterpriselevelExperience in pyspark is required– developing and deploying the workloads to run on theSpark distributed...

  • Lead Data Engineer

    1 day ago


    Abu Dhabi, Abu Dhabi, United Arab Emirates Dicetek LLC Full time

    Roles and responsibilities Overall 5-7 years ofexperience in data engineering and transformation onCloud 3+ Years of Very Strong Experience inAzure Data Engineering, Databricks Expertise insupporting/developing lakehouse workloads at enterpriselevel Experience in pyspark is required– developing and deploying the workloads to run on theSpark distributed...


  • Abu Dhabi, Abu Dhabi, United Arab Emirates Segula Technologies Full time

    Job Overview We are seeking a skilled and experienced Geospatial Data Engineer Lead to oversee all Geographic Information System (GIS) activities for a high-profile Oil & Gas pipeline construction project in Abu Dhabi. This role requires technical expertise in GIS technologies, leadership skills, and experience in large-scale infrastructure projects. Key...


  • Abu Dhabi, Abu Dhabi, United Arab Emirates Caliberly Full time

    Calmberly is seeking a talented Healthcare Data Engineering Lead to spearhead the development of our healthcare data infrastructure on Azure Cloud and Databricks.The ideal candidate will have expertise in designing data architectures optimized for healthcare workloads and a strong understanding of ELT processes, data modeling, and data pipeline...


  • Abu Dhabi, Abu Dhabi, United Arab Emirates The Chemical Engineer Full time

    Job DescriptionWe are seeking a highly experienced PMMC Senior Process Safety Engineer to join our team at Wood, a global leader in consulting and engineering. This is an exciting opportunity for a professional with a proven track record in implementing and managing process safety systems.The successful candidate will be responsible for executing assigned...


  • Abu Dhabi, Abu Dhabi, United Arab Emirates The Chemical Engineer Full time

    JOB DESCRIPTION Remarkable people, trusted by clients to design and advance the world. The Role This is an exciting opportunity for an experienced PMC Senior Process Safety Engineer with a proven track record in implementation and management of process safety systems, mitigating risks and ensuring compliance with safety standards in oil and gas...


  • Abu Dhabi, Abu Dhabi, United Arab Emirates Gulf Data International Full time

    How to apply: Data & Integration SpecialistGDI takes pride in creating opportunities for personal growth and achievement. To apply for a job opening, please email your latest CV in MS Word or PDF format to careers@gdi.ae, with the job title applied for in the subject line.Job DescriptionWe are actively seeking a Data & Integration Specialist with expertise...


  • Abu Dhabi, Abu Dhabi, United Arab Emirates Gulf Data International Full time

    How to apply: Data & Integration Specialist GDI takes pride in creating opportunities for personal growth and achievement. To apply for a job opening, please email your latest CV in MS Word or PDF format to , with the job title applied for in the subject line. Job Description We are actively seeking a Data & Integration Specialist with expertise in...


  • Abu Dhabi, Abu Dhabi, United Arab Emirates Datatime Full time

    The Senior Data Center Engineer will lead the technical design and engineering aspects of data center projects at Datatime. Key responsibilities include ensuring data center functionality, scalability, and efficiency, while adhering to industry standards.Key TasksTranslate operational needs into technical specifications with clients and vendors.Cordinate...

  • Lead Data Scientist

    2 weeks ago


    Abu Dhabi, Abu Dhabi, United Arab Emirates 10Pearls Full time

    Lead Data Scientist Company Overview 10Pearls is an end-to-end digital technology services partner helping businesses utilize technology as a competitive advantage. We help our customers digitalize their existing business, build innovative new products, and augment their existing teams with high-performance team members. Our broad expertise in product...


  • Abu Dhabi, Abu Dhabi, United Arab Emirates Datatime Full time

    Datatime seeks a highly skilled Lead Engineering Designer to specialize in data centers. As a key member of the team, this professional will oversee the technical design and engineering aspects of data center projects.Main ResponsibilitiesWork closely with clients, contractors, and vendors to define operational needs.Collaborate with architects, IT teams,...


  • Abu Dhabi, Abu Dhabi, United Arab Emirates 10Pearls Full time

    Data Science OpportunitiesWe are looking for a highly skilled Senior Data Scientist to join our team and lead advanced analytics initiatives in the oil and gas industry.About the Job:Leverage data-driven insights to optimize exploration, production, and operational efficiency.Build predictive models, deploy machine learning algorithms, and lead a team to...


  • Abu Dhabi, Abu Dhabi, United Arab Emirates The Chemical Engineer Full time

    ResponsibilitiesThe PMC Senior Process Safety Engineer will be responsible for:Executing assigned scopes of workProviding discipline expertise to the businessParticipating in project HAZOP, HAZID, SIL studies, and other safety workshops as requiredCoordinating with other disciplines during all stages of design development, construction, pre-commissioning,...


  • Abu Dhabi, Abu Dhabi, United Arab Emirates 10Pearls Full time

    Job OverviewWe are seeking a highly skilled Senior Data Scientist to lead advanced analytics initiatives in the oil and gas industry. This role involves leveraging data-driven insights to optimize exploration, production, and operational efficiency.About the Role:Design and implement end-to-end data science projects to solve complex business challenges.Built...


  • Abu Dhabi, Abu Dhabi, United Arab Emirates Sense7ai Data Solution Pvt Ltd Full time

    This is a remoteposition.ProjectPartners has a passion for delivering projects. Not justsuccessfully, but on-time and to budget We started thisjourney on a mission to empower people to deliver projectssuccessfully. Being project professionals ourselves we know whatworks.Our insider knowledge means we know exactly whatgreat project delivery needs. We went on...


  • Abu Dhabi, Abu Dhabi, United Arab Emirates Sense7ai Data Solution Pvt Ltd Full time

    This is a remoteposition. ProjectPartners has a passion for delivering projects. Not justsuccessfully, but on-time and to budget We started thisjourney on a mission to empower people to deliver projectssuccessfully. Being project professionals ourselves we know whatworks.Our insider knowledge means we know exactly whatgreat project delivery needs. We went on...

  • Data Engineer

    1 week ago


    Abu Dhabi, Abu Dhabi, United Arab Emirates Presight Full time

    Overview Role: Data Engineer Location: Abu Dhabi About Presight Presight is an ADX-listed public company with Abu Dhabi based G42 as its majority shareholder and is the region's leading big data analytics company powered by GenAI. It combines big data, analytics, and AI expertise to serve every sector, of every scale, to create business and positive...