Data Engineer
5 days ago
Direct message the job poster from ValueLabs
Talent Acquisition Executive at ValueLabsExperience - 5+ yrs
Location - Dubai (On-site)
Notice Period - Immediate - 30 days
Roles and Responsibilities:
- Data Pipeline Development: Design, develop, and maintain highly scalable and optimized ETL pipelines using PySpark on the Cloudera Data Platform, ensuring data integrity and accuracy.
- Data Ingestion: Implement and manage data ingestion processes from a variety of sources (e.g., relational databases, APIs, file systems) to the data lake or data warehouse on CDP.
- Data Transformation and Processing: Use PySpark to process, cleanse, and transform large datasets into meaningful formats that support analytical needs and business requirements.
- Performance Optimization: Conduct performance tuning of PySpark code and Cloudera components, optimizing resource utilization and reducing runtime of ETL processes.
- Data Quality and Validation: Implement data quality checks, monitoring, and validation routines to ensure data accuracy and reliability throughout the pipeline.
- Automation and Orchestration: Automate data workflows using tools like Apache Oozie, Airflow, or similar orchestration tools within the Cloudera ecosystem.
- Monitoring and Maintenance: Monitor pipeline performance, troubleshoot issues, and perform routine maintenance on the Cloudera Data Platform and associated data processes.
- Collaboration: Work closely with other data engineers, analysts, product managers, and other stakeholders to understand data requirements and support various data-driven initiatives.
- Documentation: Maintain thorough documentation of data engineering processes, code, and pipeline configurations.
Qualifications
Education and Experience
- Bachelor's or Master's degree in Computer Science, Data Engineering, Information Systems, or a related field.
- 3+ years of experience as a Data Engineer, with a strong focus on PySpark and the Cloudera Data Platform.
Technical Skills
- PySpark: Advanced proficiency in PySpark, including working with RDDs, DataFrames, and optimization techniques.
- Cloudera Data Platform: Strong experience with Cloudera Data Platform (CDP) components, including Cloudera Manager, Hive, Impala, HDFS, and HBase.
- Data Warehousing: Knowledge of data warehousing concepts, ETL best practices, and experience with SQL-based tools (e.g., Hive, Impala).
- Big Data Technologies: Familiarity with Hadoop, Kafka, and other distributed computing tools.
- Orchestration and Scheduling: Experience with Apache Oozie, Airflow, or similar orchestration frameworks.
- Scripting and Automation: Strong scripting skills in Linux.
Soft Skills
- Strong analytical and problem-solving skills.
- Excellent verbal and written communication abilities.
- Ability to work independently and collaboratively in a team environment.
- Attention to detail and commitment to data quality.
Mid-Senior level
Employment typeFull-time
Job functionInformation Technology
IndustriesIT Services and IT Consulting
#J-18808-Ljbffr-
Data Engineer
1 day ago
Dubai, Dubai, United Arab Emirates Data Labs - LANDMARK GROUP Full timeAs a Junior Data Engineer, you will play a key role in supporting our data engineering initiatives that drive automation, data quality, and accessibility across the organization. Working closely with senior team members and business stakeholders, you will contribute to the creation of scalable, cloud-based data solutions on Microsoft...
-
Engineering Process Lead
1 day ago
Dubai, Dubai, United Arab Emirates The Chemical Engineer Full timeAbout the RoleWe are seeking an experienced Senior Chemical Engineer to join our team as a Process Engineering Lead. In this role, you will be responsible for designing and preparing process engineering requirements for projects, reviewing and approving key process documentation from EPC Contractors, and supporting tender or proposal processes.Key...
-
Lead Process Engineer
1 day ago
Dubai, Dubai, United Arab Emirates The Chemical Engineer Full timeJOB DESCRIPTION Wood is currently recruiting for Lead Process Engineer to strengthen our team in Dubai with occasional trips to Basra, South of Iraq RESPONSIBILITIES Overview & ResponsibilitiesThe Lead Process Engineer is engaged in the design, preparation of specification, data sheets and analysis of Process engineering requirements on a project.Review and...
-
Data Scientist
1 day ago
Dubai, Dubai, United Arab Emirates Data Labs - LANDMARK GROUP Full timeWe are seeking a skilled Data Engineer to join our team at Data Labs - LANDMARK GROUP. As a key player in our data engineering initiatives, you will drive automation, data quality, and accessibility across the organization.You will work closely with senior team members and business stakeholders to create scalable, cloud-based data solutions on Microsoft...
-
Cloud Data Architect
1 day ago
Dubai, Dubai, United Arab Emirates Data Labs - LANDMARK GROUP Full timeAt Data Labs - LANDMARK GROUP, we are looking for an experienced Data Engineer to join our team. As a Junior Data Engineer, you will play a vital role in supporting our data engineering initiatives that drive automation, data quality, and accessibility across the organization.About the Role:Cloud-Based Solutions: Work closely with senior team members and...
-
Big Data Developer
1 day ago
Dubai, Dubai, United Arab Emirates Data Labs - LANDMARK GROUP Full timeWe are seeking a talented Data Engineer to join our team at Data Labs - LANDMARK GROUP. As a key player in our data engineering initiatives, you will drive automation, data quality, and accessibility across the organization.About the Job:Cloud-Based Solutions: Create scalable, cloud-based data solutions on Microsoft Azure.Data Ingestion and Integration:...
-
Data Center – AutoCAD Draughtsman
7 hours ago
Dubai, Dubai, United Arab Emirates Edgnex Data Centers by DAMAC Full timeData Center – AutoCAD Draughtsman (Mechanical)Undertake the preparation of Mechanical Data Center Design documentation in accordance with DAMAC Design Standards and deliver in accordance with the approved design program.Job role functions:1. Work with the Mechanical Data Center Design Team to implement high quality Mechanical Data Center Design solutions...
-
Data Engineer
3 days ago
Dubai, Dubai, United Arab Emirates GSSTech Group Full timeJob Title: Data Engineer (PySpark)________________________________________About the RoleWe are seeking a highly skilled Data Engineer with deep expertise in PySpark and the Cloudera Data Platform (CDP) to join our data engineering team. As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines that ensure...
-
Data Engineer
3 weeks ago
Dubai, Dubai, United Arab Emirates LanceSoft UAE Full timeWe have a new opportunity for a Data Engineer with our client.Title: Data EngineerDuration: 6 MonthsLocation: DubaiQualifications:A Bachelor's degree in Data Engineering, Computer Science, Information Technology, or a related discipline is required.A Master's degree is preferred but not mandatory.Certifications in ETL tools, data engineering platforms, or...
-
Data Engineer
3 weeks ago
Dubai, Dubai, United Arab Emirates VML South Africa Full timePosition Name: Data EngineerPosition Location: Dubai & Abu Dhabi, UAE THE ROLE We are seeking a skilled Data Engineer to join our team. The successful candidate will work closely with the Senior Data Engineer and be responsible for designing and managing the data structure and ETL processes of our retail bank client. They will handle data from various...
-
Junior Engineering Specialist
1 day ago
Dubai, Dubai, United Arab Emirates Data Labs - LANDMARK GROUP Full timeData Labs - LANDMARK GROUP is a leading organization in the industry, and we are seeking a highly skilled Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for driving automation, data quality, and accessibility across the organization.You will work closely with senior team members and business stakeholders...
-
Data Engineer
10 hours ago
Dubai, Dubai, United Arab Emirates Teliolabs Communication Private Limited Full timeLocation: DubaiWho Can Apply: Candidates who are currently in DubaiJob Type: ContractExperience: Minimum 8+ yearsJob Summary:We are looking for an experienced Data Engineer to design, develop, and optimize data pipelines, ETL processes, and data integration solutions. The ideal candidate should have expertise in AWS cloud services, data engineering best...
-
Data Engineer
4 days ago
Dubai, Dubai, United Arab Emirates VML South Africa Full timeAt VML, we are a beacon of innovation and growth in an ever-evolving world. Our heritage is built upon a century of combined expertise, where creativity meets technology, and diverse perspectives ignite inspiration. With the merger of VMLY&R and Wunderman Thompson, we have forged a new path as a growth partner that is part creative agency, part consultancy,...
-
Data Engineer
6 days ago
Dubai, Dubai, United Arab Emirates Overture Rede Full timeJob Description : Data EngineerLocation Dubai Job Summary We are looking for an experienced Data Engineer. The suitable candidate should have demonstrated experience in designing and implementing ETL/ ELT solutions onpremises modern platforms to support Enterprise data warehouse Data Mart and advanced analytics capabilities. Success in this role comes...
-
Data Engineer
6 days ago
Dubai, Dubai, United Arab Emirates ClickJobs Full timeDo you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers, doers and disruptors, who solve real problems and meet real customer needs. We are seeking Data Engineers...
-
Data Engineer
1 day ago
Dubai, Dubai, United Arab Emirates Property Finder Group Full timeFounded by Michael Lahyani in 2005 as a magazine (Al Bab World), Property Finder today is a single technology platform and brand across multiple countries in the MENA region. We offer the most advanced tools and best-in-class user experience for homeseekers, real estate brokers, and developers. Property Finder's most recent valuation secures our status among...
-
Data Engineer
1 day ago
Dubai, Dubai, United Arab Emirates Artefact Full timeArtefact is a new generation of data service providers specialising in data consulting and data-driven digital marketing. It is dedicated to transforming data into business impact across the entire value chain of organisations. We are proud to say we're enjoying skyrocketing growth.The backbone of our consulting missions, today our Data consulting team has...
-
Data Engineer
4 days ago
Dubai, Dubai, United Arab Emirates Property Finder Full timeFounded by Michael Lahyani in 2005 as a magazine (Al Bab World), Property Finder today is a single technology platform and brand across multiple countries in the MENA region. We offer the most advanced tools and best-in-class user experience for homeseekers, real estate brokers, and developers. Property Finder's most recent valuation secures our status among...
-
Data Engineer
3 weeks ago
Dubai, Dubai, United Arab Emirates Property Finder Group Full timeFounded by Michael Lahyani in 2005 as a magazine (Al Bab World), Property Finder today is a single technology platform and brand across multiple countries in the MENA region. We offer the most advanced tools and best-in-class user experience for homeseekers, real estate brokers, and developers. Property Finder's most recent valuation secures our status among...
-
Data Engineer
4 weeks ago
Dubai, Dubai, United Arab Emirates Property Finder Group Full timeFounded by Michael Lahyani in 2005 as a magazine (Al Bab World), Property Finder today is a single technology platform and brand across multiple countries in the MENA region. We offer the most advanced tools and best-in-class user experience for homeseekers, real estate brokers, and developers. Property Finder's most recent valuation secures our status among...