Lead Data Engineer

2 days ago

Abu Dhabi, Abu Dhabi, United Arab Emirates Dicetek LLC Full time
Roles and responsibilities
  • Overall 5-7 years of experience in data engineering and transformation on Cloud
  • 3+ Years of Very Strong Experience in Azure Data Engineering, Databricks
  • Expertise in supporting/developing lakehouse workloads at enterprise level
  • Experience in pyspark is required – developing and deploying the workloads to run on the Spark distributed computing
  • Candidate must possess at least a Graduate or bachelor's degree in Computer Science/Information Technology, Engineering (Computer/Telecommunication) or equivalent.
  • Cloud deployment: Preferably Microsoft azure
  • Experience in implementing the platform and application monitoring using Cloud native tools

  • 1. Data Architecture & Design

  • Strong understanding of database architectures, including relational, NoSQL, and data warehousing.
  • Ability to design and implement data pipelines, data lakes, and data warehouses.
  • Knowledge of ETL (Extract, Transform, Load) processes.

  • 2. Programming & Scripting

  • Languages: Expertise in Python, Java, Scala, and SQL.
  • Familiarity with data manipulation libraries like Pandas, Dask, and NumPy (for Python).
  • Ability to write efficient queries and scripts for data processing and analysis.

  • 3. Big Data Technologies

  • Proficiency in big data frameworks like Hadoop, Spark, and Flink.
  • Experience with distributed computing systems and handling large datasets.

  • 4. Cloud Platforms

  • Knowledge of cloud platforms such as AWS, Google Cloud Platform (GCP), and Microsoft Azure.
  • Experience with cloud-based data solutions like Amazon Redshift, Google BigQuery, or Azure Data Lake.

  • 5. Data Modeling

  • Expertise in data modeling techniques, such as dimensional modeling, ER modeling, and star/snowflake schemas.
  • Ability to design schema structures that support business intelligence and analytical needs.

  • 6. Data Pipeline Management

  • Experience in creating and maintaining data pipelines for large-scale data processing.
  • Familiarity with tools like Apache Airflow, Luigi, or dbt for orchestrating workflows.

  • 7. Data Warehousing & SQL Optimization

  • Expertise in designing and optimizing data warehousing solutions.
  • Strong skills in SQL tuning, optimization, and performance improvement for handling large datasets.

  • 8. Data Governance & Security

  • Understanding of data governance practices, such as data privacy, metadata management, and ensuring data quality.
  • Ability to enforce data security standards and practices for data protection.

  • 9. Team Leadership & Project Management

  • Ability to lead a team of data engineers, collaborate with cross-functional teams, and manage data engineering projects.
  • Familiarity with project management frameworks like Agile and Scrum.

  • 10. Automation & CI/CD

  • Experience in automating data workflows and deployment pipelines.
  • Knowledge of CI/CD tools (e.g., Jenkins, GitLab) for version control and automation.

  • 11. Machine Learning Integration (Optional)

  • While not strictly necessary for all data engineering roles, experience integrating data pipelines with machine learning models or supporting the deployment of models can be valuable.
Desired candidate profile

1. Data Architecture & Design

  • Strong understanding of database architectures, including relational, NoSQL, and data warehousing.
  • Ability to design and implement data pipelines, data lakes, and data warehouses.
  • Knowledge of ETL (Extract, Transform, Load) processes.

2. Programming & Scripting

  • Languages: Expertise in Python, Java, Scala, and SQL.
  • Familiarity with data manipulation libraries like Pandas, Dask, and NumPy (for Python).
  • Ability to write efficient queries and scripts for data processing and analysis.

3. Big Data Technologies

  • Proficiency in big data frameworks like Hadoop, Spark, and Flink.
  • Experience with distributed computing systems and handling large datasets.

4. Cloud Platforms

  • Knowledge of cloud platforms such as AWS, Google Cloud Platform (GCP), and Microsoft Azure.
  • Experience with cloud-based data solutions like Amazon Redshift, Google BigQuery, or Azure Data Lake.

5. Data Modeling

  • Expertise in data modeling techniques, such as dimensional modeling, ER modeling, and star/snowflake schemas.
  • Ability to design schema structures that support business intelligence and analytical needs.

6. Data Pipeline Management

  • Experience in creating and maintaining data pipelines for large-scale data processing.
  • Familiarity with tools like Apache Airflow, Luigi, or dbt for orchestrating workflows.

7. Data Warehousing & SQL Optimization

  • Expertise in designing and optimizing data warehousing solutions.
  • Strong skills in SQL tuning, optimization, and performance improvement for handling large datasets.

8. Data Governance & Security

  • Understanding of data governance practices, such as data privacy, metadata management, and ensuring data quality.
  • Ability to enforce data security standards and practices for data protection.

9. Team Leadership & Project Management

  • Ability to lead a team of data engineers, collaborate with cross-functional teams, and manage data engineering projects.
  • Familiarity with project management frameworks like Agile and Scrum.

10. Automation & CI/CD

  • Experience in automating data workflows and deployment pipelines.
  • Knowledge of CI/CD tools (e.g., Jenkins, GitLab) for version control and automation.

11. Machine Learning Integration (Optional)

  • While not strictly necessary for all data engineering roles, experience integrating data pipelines with machine learning models or supporting the deployment of models can be valuable.

  • Data Protection Engineer

    3 days ago

    Abu Dhabi, Abu Dhabi, United Arab Emirates Alpha Data Full time

    Job OverviewWe are seeking a skilled Data Protection Engineer to join our team at Alpha Data. In this role, you will be responsible for administering and managing Veeam Backup as a service platform, including vCloud Connect, Kasten, and VBR.Key ResponsibilitiesDevelop customized plans to maximize value from Veeam products and services.Identify pain points...

  • Lead Data Engineer

    3 weeks ago

    Abu Dhabi, Abu Dhabi, United Arab Emirates Dicetek LLC Full time

    Roles and ResponsibilitiesOverall 5-7 years of experience in data engineering and transformation on Cloud3+ Years of Very Strong Experience in Azure Data Engineering, DatabricksExpertise in supporting/developing lakehouse workloads at enterprise levelExperience in pyspark is required – developing and deploying the workloads to run on the Spark distributed...

  • Data-Centric Solutions Engineer

    2 days ago

    Abu Dhabi, Abu Dhabi, United Arab Emirates Big Data Technology Solutions Full time

    Job RequirementsWe are seeking a highly skilled Data-Centric Solutions Engineer to join our team at Big Data Technology Solutions. As an expert in data-centric solutions, you will be responsible for designing and implementing scalable solutions that meet the needs of our clients.Main Responsibilities:Design and deploy HPC systems using Dell PowerEdge...

  • Data Engineering Lead

    21 hours ago

    Abu Dhabi, Abu Dhabi, United Arab Emirates G42 Full time

    About the RoleThis is an exciting opportunity for a motivated and experienced Data Engineer to join our team at G42. As a Senior Data Engineer, you will play a key role in developing and maintaining our data infrastructure, ensuring that it is reliable, scalable, and secure.You will be responsible for designing and implementing data pipelines, ensuring data...

  • Data Engineering Lead

    2 days ago

    Abu Dhabi, Abu Dhabi, United Arab Emirates G42 Full time

    G42 is the UAE's national-scale enabler in AI Research and Development. Partnering with Microsoft's AI SaaS, we offer domain-specific Agentic AI Orchestrator platforms utilizing reasoning agents for precise and cost-effective services.Our Technology StackHadoopSparkFlinkWe are looking for a Senior Data Engineer to develop and maintain data systems that...

  • Data Warehouse Engineering Lead

    5 days ago

    Abu Dhabi, Abu Dhabi, United Arab Emirates Whiteblue Cloud Services Full time

    Job DescriptionWe are seeking an experienced Data Engineer Lead to join our team at Whiteblue Cloud Services. The ideal candidate will have a strong background in designing, developing, and maintaining data pipelines in Azure Data Factory (ADF) for ETL from on-premise and cloud-based sources. Additionally, they should have expertise in designing, developing,...

  • Lead Data Engineer

    4 weeks ago

    Abu Dhabi, Abu Dhabi, United Arab Emirates Dicetek LLC Full time

    Roles and responsibilitiesOverall 5-7 years ofexperience in data engineering and transformation onCloud3+ Years of Very Strong Experience inAzure Data Engineering, DatabricksExpertise insupporting/developing lakehouse workloads at enterpriselevelExperience in pyspark is required– developing and deploying the workloads to run on theSpark distributed...

  • Lead Data Engineer

    2 weeks ago

    Abu Dhabi, Abu Dhabi, United Arab Emirates Dicetek LLC Full time

    Roles and responsibilitiesOverall 5-7 years ofexperience in data engineering and transformation onCloud3+ Years of Very Strong Experience inAzure Data Engineering, DatabricksExpertise insupporting/developing lakehouse workloads at enterpriselevelExperience in pyspark is required– developing and deploying the workloads to run on theSpark distributed...

  • Lead Data Engineer

    4 days ago

    Abu Dhabi, Abu Dhabi, United Arab Emirates Dicetek LLC Full time

    Roles and responsibilities Overall 5-7 years ofexperience in data engineering and transformation onCloud 3+ Years of Very Strong Experience inAzure Data Engineering, Databricks Expertise insupporting/developing lakehouse workloads at enterpriselevel Experience in pyspark is required– developing and deploying the workloads to run on theSpark distributed...

  • Geospatial Data Engineer Lead

    7 days ago

    Abu Dhabi, Abu Dhabi, United Arab Emirates Segula Technologies Full time

    Job Overview We are seeking a skilled and experienced Geospatial Data Engineer Lead to oversee all Geographic Information System (GIS) activities for a high-profile Oil & Gas pipeline construction project in Abu Dhabi. This role requires technical expertise in GIS technologies, leadership skills, and experience in large-scale infrastructure projects. Key...

  • Healthcare Data Engineering Lead

    3 days ago

    Abu Dhabi, Abu Dhabi, United Arab Emirates Caliberly Full time

    Calmberly is seeking a talented Healthcare Data Engineering Lead to spearhead the development of our healthcare data infrastructure on Azure Cloud and Databricks.The ideal candidate will have expertise in designing data architectures optimized for healthcare workloads and a strong understanding of ELT processes, data modeling, and data pipeline...

  • PMC Senior Process Safety Engineer

    7 days ago

    Abu Dhabi, Abu Dhabi, United Arab Emirates The Chemical Engineer Full time

    JOB DESCRIPTION Remarkable people, trusted by clients to design and advance the world. The Role This is an exciting opportunity for an experienced PMC Senior Process Safety Engineer with a proven track record in implementation and management of process safety systems, mitigating risks and ensuring compliance with safety standards in oil and gas...

  • IT Services Delivery Lead

    2 days ago

    Abu Dhabi, Abu Dhabi, United Arab Emirates Alpha Data Full time

    Job OverviewAlpha Data is seeking a seasoned Veeam Backup Engineer to lead our efforts in delivering high-quality IT services to clients. As a key member of our team, you will be responsible for the administration and management of Veeam Backup as a service platform (vCloud Connect), Kasten, and VBR.Responsibilities and DeliverablesPlatform Administration:...

  • Data & Integration Specialist

    3 weeks ago

    Abu Dhabi, Abu Dhabi, United Arab Emirates Gulf Data International Full time

    How to apply: Data & Integration SpecialistGDI takes pride in creating opportunities for personal growth and achievement. To apply for a job opening, please email your latest CV in MS Word or PDF format to careers@gdi.ae, with the job title applied for in the subject line.Job DescriptionWe are actively seeking a Data & Integration Specialist with expertise...

  • Oracle Data Architect and Integrator

    2 days ago

    Abu Dhabi, Abu Dhabi, United Arab Emirates Gulf Data International Full time

    About the PositionGulf Data International is looking for a talented Data & Integration Specialist to join our team. As a key member of our team, you will be responsible for creating and implementing data models that support business operations.In this role, you will work closely with business stakeholders, data analysts, and IT teams to understand data...

  • Data & Integration Specialist

    4 days ago

    Abu Dhabi, Abu Dhabi, United Arab Emirates Gulf Data International Full time

    How to apply: Data & Integration Specialist GDI takes pride in creating opportunities for personal growth and achievement. To apply for a job opening, please email your latest CV in MS Word or PDF format to , with the job title applied for in the subject line. Job Description We are actively seeking a Data & Integration Specialist with expertise in...

  • Engineering Project Lead for Data Centers

    7 days ago

    Abu Dhabi, Abu Dhabi, United Arab Emirates Datatime Full time

    The Senior Data Center Engineer will lead the technical design and engineering aspects of data center projects at Datatime. Key responsibilities include ensuring data center functionality, scalability, and efficiency, while adhering to industry standards.Key TasksTranslate operational needs into technical specifications with clients and vendors.Cordinate...

  • Help Desk Engineer

    7 hours ago

    Abu Dhabi, Abu Dhabi, United Arab Emirates Alpha Data Full time

    Job SummaryWe are seeking a highly experienced Help Desk Engineer to join our team at Alpha Data.In this role, you will be responsible for providing technical support to our clients, troubleshooting complex issues, and collaborating with senior engineers to resolve escalated problems.The ideal candidate will have a strong background in IT, excellent...

  • Data Protection Specialist

    2 days ago

    Abu Dhabi, Abu Dhabi, United Arab Emirates Alpha Data Full time

    About the PositionWe are looking for an experienced Veeam Backup Engineer to join our team at Alpha Data. As a key member of our IT services delivery team, you will be responsible for the administration and management of Veeam Backup as a service platform (vCloud Connect), Kasten, and VBR.Main ResponsibilitiesService Platform Administration: Ensure the...

  • Lead Data Scientist

    3 weeks ago

    Abu Dhabi, Abu Dhabi, United Arab Emirates 10Pearls Full time

    Lead Data Scientist Company Overview 10Pearls is an end-to-end digital technology services partner helping businesses utilize technology as a competitive advantage. We help our customers digitalize their existing business, build innovative new products, and augment their existing teams with high-performance team members. Our broad expertise in product...