Data Engineer

5 days ago


Dubai, Dubai, United Arab Emirates ValueLabs Full time

Direct message the job poster from ValueLabs

Talent Acquisition Executive at ValueLabs

Experience - 5+ yrs

Location - Dubai (On-site)

Notice Period - Immediate - 30 days

Roles and Responsibilities:

  • Data Pipeline Development: Design, develop, and maintain highly scalable and optimized ETL pipelines using PySpark on the Cloudera Data Platform, ensuring data integrity and accuracy.
  • Data Ingestion: Implement and manage data ingestion processes from a variety of sources (e.g., relational databases, APIs, file systems) to the data lake or data warehouse on CDP.
  • Data Transformation and Processing: Use PySpark to process, cleanse, and transform large datasets into meaningful formats that support analytical needs and business requirements.
  • Performance Optimization: Conduct performance tuning of PySpark code and Cloudera components, optimizing resource utilization and reducing runtime of ETL processes.
  • Data Quality and Validation: Implement data quality checks, monitoring, and validation routines to ensure data accuracy and reliability throughout the pipeline.
  • Automation and Orchestration: Automate data workflows using tools like Apache Oozie, Airflow, or similar orchestration tools within the Cloudera ecosystem.
  • Monitoring and Maintenance: Monitor pipeline performance, troubleshoot issues, and perform routine maintenance on the Cloudera Data Platform and associated data processes.
  • Collaboration: Work closely with other data engineers, analysts, product managers, and other stakeholders to understand data requirements and support various data-driven initiatives.
  • Documentation: Maintain thorough documentation of data engineering processes, code, and pipeline configurations.

Qualifications

Education and Experience

  • Bachelor's or Master's degree in Computer Science, Data Engineering, Information Systems, or a related field.
  • 3+ years of experience as a Data Engineer, with a strong focus on PySpark and the Cloudera Data Platform.

Technical Skills

  • PySpark: Advanced proficiency in PySpark, including working with RDDs, DataFrames, and optimization techniques.
  • Cloudera Data Platform: Strong experience with Cloudera Data Platform (CDP) components, including Cloudera Manager, Hive, Impala, HDFS, and HBase.
  • Data Warehousing: Knowledge of data warehousing concepts, ETL best practices, and experience with SQL-based tools (e.g., Hive, Impala).
  • Big Data Technologies: Familiarity with Hadoop, Kafka, and other distributed computing tools.
  • Orchestration and Scheduling: Experience with Apache Oozie, Airflow, or similar orchestration frameworks.
  • Scripting and Automation: Strong scripting skills in Linux.

Soft Skills

  • Strong analytical and problem-solving skills.
  • Excellent verbal and written communication abilities.
  • Ability to work independently and collaboratively in a team environment.
  • Attention to detail and commitment to data quality.
Seniority level

Mid-Senior level

Employment type

Full-time

Job function

Information Technology

Industries

IT Services and IT Consulting

#J-18808-Ljbffr
  • Data Engineer

    1 day ago


    Dubai, Dubai, United Arab Emirates Data Labs - LANDMARK GROUP Full time

    As a Junior Data Engineer, you will play a key role in supporting our data engineering initiatives that drive automation, data quality, and accessibility across the organization. Working closely with senior team members and business stakeholders, you will contribute to the creation of scalable, cloud-based data solutions on Microsoft...


  • Dubai, Dubai, United Arab Emirates The Chemical Engineer Full time

    About the RoleWe are seeking an experienced Senior Chemical Engineer to join our team as a Process Engineering Lead. In this role, you will be responsible for designing and preparing process engineering requirements for projects, reviewing and approving key process documentation from EPC Contractors, and supporting tender or proposal processes.Key...


  • Dubai, Dubai, United Arab Emirates The Chemical Engineer Full time

    JOB DESCRIPTION Wood is currently recruiting for Lead Process Engineer to strengthen our team in Dubai with occasional trips to Basra, South of Iraq RESPONSIBILITIES Overview & ResponsibilitiesThe Lead Process Engineer is engaged in the design, preparation of specification, data sheets and analysis of Process engineering requirements on a project.Review and...

  • Data Scientist

    1 day ago


    Dubai, Dubai, United Arab Emirates Data Labs - LANDMARK GROUP Full time

    We are seeking a skilled Data Engineer to join our team at Data Labs - LANDMARK GROUP. As a key player in our data engineering initiatives, you will drive automation, data quality, and accessibility across the organization.You will work closely with senior team members and business stakeholders to create scalable, cloud-based data solutions on Microsoft...


  • Dubai, Dubai, United Arab Emirates Data Labs - LANDMARK GROUP Full time

    At Data Labs - LANDMARK GROUP, we are looking for an experienced Data Engineer to join our team. As a Junior Data Engineer, you will play a vital role in supporting our data engineering initiatives that drive automation, data quality, and accessibility across the organization.About the Role:Cloud-Based Solutions: Work closely with senior team members and...

  • Big Data Developer

    1 day ago


    Dubai, Dubai, United Arab Emirates Data Labs - LANDMARK GROUP Full time

    We are seeking a talented Data Engineer to join our team at Data Labs - LANDMARK GROUP. As a key player in our data engineering initiatives, you will drive automation, data quality, and accessibility across the organization.About the Job:Cloud-Based Solutions: Create scalable, cloud-based data solutions on Microsoft Azure.Data Ingestion and Integration:...


  • Dubai, Dubai, United Arab Emirates Edgnex Data Centers by DAMAC Full time

    Data Center – AutoCAD Draughtsman (Mechanical)Undertake the preparation of Mechanical Data Center Design documentation in accordance with DAMAC Design Standards and deliver in accordance with the approved design program.Job role functions:1. Work with the Mechanical Data Center Design Team to implement high quality Mechanical Data Center Design solutions...

  • Data Engineer

    3 days ago


    Dubai, Dubai, United Arab Emirates GSSTech Group Full time

    Job Title: Data Engineer (PySpark)________________________________________About the RoleWe are seeking a highly skilled Data Engineer with deep expertise in PySpark and the Cloudera Data Platform (CDP) to join our data engineering team. As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines that ensure...

  • Data Engineer

    3 weeks ago


    Dubai, Dubai, United Arab Emirates LanceSoft UAE Full time

    We have a new opportunity for a Data Engineer with our client.Title: Data EngineerDuration: 6 MonthsLocation: DubaiQualifications:A Bachelor's degree in Data Engineering, Computer Science, Information Technology, or a related discipline is required.A Master's degree is preferred but not mandatory.Certifications in ETL tools, data engineering platforms, or...

  • Data Engineer

    3 weeks ago


    Dubai, Dubai, United Arab Emirates VML South Africa Full time

    Position Name: Data EngineerPosition Location: Dubai & Abu Dhabi, UAE THE ROLE We are seeking a skilled Data Engineer to join our team. The successful candidate will work closely with the Senior Data Engineer and be responsible for designing and managing the data structure and ETL processes of our retail bank client. They will handle data from various...


  • Dubai, Dubai, United Arab Emirates Data Labs - LANDMARK GROUP Full time

    Data Labs - LANDMARK GROUP is a leading organization in the industry, and we are seeking a highly skilled Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for driving automation, data quality, and accessibility across the organization.You will work closely with senior team members and business stakeholders...

  • Data Engineer

    10 hours ago


    Dubai, Dubai, United Arab Emirates Teliolabs Communication Private Limited Full time

    Location: DubaiWho Can Apply: Candidates who are currently in DubaiJob Type: ContractExperience: Minimum 8+ yearsJob Summary:We are looking for an experienced Data Engineer to design, develop, and optimize data pipelines, ETL processes, and data integration solutions. The ideal candidate should have expertise in AWS cloud services, data engineering best...

  • Data Engineer

    4 days ago


    Dubai, Dubai, United Arab Emirates VML South Africa Full time

    At VML, we are a beacon of innovation and growth in an ever-evolving world. Our heritage is built upon a century of combined expertise, where creativity meets technology, and diverse perspectives ignite inspiration. With the merger of VMLY&R and Wunderman Thompson, we have forged a new path as a growth partner that is part creative agency, part consultancy,...

  • Data Engineer

    6 days ago


    Dubai, Dubai, United Arab Emirates Overture Rede Full time

    Job Description : Data EngineerLocation Dubai Job Summary We are looking for an experienced Data Engineer. The suitable candidate should have demonstrated experience in designing and implementing ETL/ ELT solutions onpremises modern platforms to support Enterprise data warehouse Data Mart and advanced analytics capabilities. Success in this role comes...

  • Data Engineer

    6 days ago


    Dubai, Dubai, United Arab Emirates ClickJobs Full time

    Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers, doers and disruptors, who solve real problems and meet real customer needs. We are seeking Data Engineers...

  • Data Engineer

    1 day ago


    Dubai, Dubai, United Arab Emirates Property Finder Group Full time

    Founded by Michael Lahyani in 2005 as a magazine (Al Bab World), Property Finder today is a single technology platform and brand across multiple countries in the MENA region. We offer the most advanced tools and best-in-class user experience for homeseekers, real estate brokers, and developers. Property Finder's most recent valuation secures our status among...

  • Data Engineer

    1 day ago


    Dubai, Dubai, United Arab Emirates Artefact Full time

    Artefact is a new generation of data service providers specialising in data consulting and data-driven digital marketing. It is dedicated to transforming data into business impact across the entire value chain of organisations. We are proud to say we're enjoying skyrocketing growth.The backbone of our consulting missions, today our Data consulting team has...

  • Data Engineer

    4 days ago


    Dubai, Dubai, United Arab Emirates Property Finder Full time

    Founded by Michael Lahyani in 2005 as a magazine (Al Bab World), Property Finder today is a single technology platform and brand across multiple countries in the MENA region. We offer the most advanced tools and best-in-class user experience for homeseekers, real estate brokers, and developers. Property Finder's most recent valuation secures our status among...

  • Data Engineer

    3 weeks ago


    Dubai, Dubai, United Arab Emirates Property Finder Group Full time

    Founded by Michael Lahyani in 2005 as a magazine (Al Bab World), Property Finder today is a single technology platform and brand across multiple countries in the MENA region. We offer the most advanced tools and best-in-class user experience for homeseekers, real estate brokers, and developers. Property Finder's most recent valuation secures our status among...

  • Data Engineer

    4 weeks ago


    Dubai, Dubai, United Arab Emirates Property Finder Group Full time

    Founded by Michael Lahyani in 2005 as a magazine (Al Bab World), Property Finder today is a single technology platform and brand across multiple countries in the MENA region. We offer the most advanced tools and best-in-class user experience for homeseekers, real estate brokers, and developers. Property Finder's most recent valuation secures our status among...