Data Engineer

4 days ago


Dubai, Dubai, United Arab Emirates Property Finder Group Full time
Roles and responsibilities

1. Design and Implement Innovative Data Solutions: Develop and maintain advanced ETL pipelines using SQL, Python, and Generative AI, transforming traditional data processes into highly efficient and automated solutions.
2. Orchestrate Complex Data Workflows: Utilize tools such as Dagster and Airflow for sophisticated pipeline orchestration, ensuring seamless integration and automation of data processes.
3. Leverage Generative AI for Data Solutions: Create and implement smart data solutions using Generative AI techniques like Retrieval-Augmented Generation (RAG). This includes building solutions that retrieve and integrate external data sources with LLMs to provide accurate and contextually enriched responses.
4. Employ Prompt Engineering: Develop and refine prompt engineering techniques to effectively communicate with large language models (LLMs), enhancing the accuracy and relevance of generated responses in various applications.
5. Utilise Embeddings and Vector Databases: Apply embedding language models to convert data into numerical representations, storing them in vector databases. Perform relevancy searches using these embeddings to match user queries with the most relevant data.
6. Incorporate Semantic Search Techniques: Implement semantic search to enhance the accuracy and relevance of search results, ensuring that data retrieval processes are highly optimised and contextually aware.
7. Collaborate Across Teams: Work closely with cross-functional teams, including data science and business analytics, to understand and deliver on unique and evolving data requirements.
8. Ensure High-Quality Data Flow: Leverage stream, batch, and Change Data Capture (CDC) processes to ensure a consistent and reliable flow of high-quality data across all systems.
9. Enable Business User Empowerment: Use data transformation tools like DBT to prepare and curate datasets, empowering business users to perform self-service analytics.
10. Maintain Data Quality and Consistency: Implement rigorous standards to ensure data quality and consistency across all data stores, continuously innovating to improve data reliability.
11. Monitor and Enhance Pipeline Performance: Regularly monitor data pipelines to identify and resolve performance and reliability issues, using innovative approaches to keep systems running optimally.

Desired candidate profile

1. 7+ years of experience as a data engineer.
2. Proficiency in SQL and Python.
3. Experience with modern cloud data warehousing and data lake solutions such as Snowflake, BigQuery, Redshift, and Azure Synapse.
4. Expertise in ETL/ELT processes, and experience building and managing batch and streaming data processing pipelines.
5. Strong ability to investigate and troubleshoot data issues, providing both short-term fixes and long-term solutions.
6. Experience with Generative AI, including Retrieval-Augmented Generation (RAG), prompt engineering, and embedding techniques for creating and managing vector databases.
7. Knowledge of AWS services, including DMS, Glue, Bedrock, SageMaker, and Athena.
8. Familiarity with dbt or other data transformation tools.

Other Desired Experience:

1. Familiarity with AWS Bedrock Agents and experience in fine-tuning models for specific use cases, enhancing the performance of AI-driven applications.
2. Proficiency in implementing semantic search to enhance the accuracy and relevance of data retrieval.
3. Experience with LangChain techniques and platforms for building applications that require complex, multi-step reasoning, such as conversational AI, document retrieval, content generation, and automated decision-making processes.
4. Experience with AWS services and concepts, including AWS OpenSearch, EC2, ECS, EKS, VPC, IAM, and others.
5. Proficiency with orchestration tools like Dagster, Airflow, AWS Step Functions, and similar platforms.
6. Experience with pub-sub, queuing, and streaming frameworks such as AWS Kinesis, Kafka, SQS, and SNS.
7. Familiarity with CI/CD pipelines and automation, ensuring efficient and reliable deployment processes.

#J-18808-Ljbffr
  • Data Engineer

    6 days ago


    Dubai, Dubai, United Arab Emirates Data Labs - LANDMARK GROUP Full time

    As a Junior Data Engineer, you will play a key role in supporting our data engineering initiatives that drive automation, data quality, and accessibility across the organization. Working closely with senior team members and business stakeholders, you will contribute to the creation of scalable, cloud-based data solutions on Microsoft...


  • Dubai, Dubai, United Arab Emirates The Chemical Engineer Full time

    About the RoleWe are seeking an experienced Senior Chemical Engineer to join our team as a Process Engineering Lead. In this role, you will be responsible for designing and preparing process engineering requirements for projects, reviewing and approving key process documentation from EPC Contractors, and supporting tender or proposal processes.Key...


  • Dubai, Dubai, United Arab Emirates The Chemical Engineer Full time

    JOB DESCRIPTION Wood is currently recruiting for Lead Process Engineer to strengthen our team in Dubai with occasional trips to Basra, South of Iraq RESPONSIBILITIES Overview & ResponsibilitiesThe Lead Process Engineer is engaged in the design, preparation of specification, data sheets and analysis of Process engineering requirements on a project.Review and...


  • Dubai, Dubai, United Arab Emirates Edgnex Data Centers by DAMAC Full time

    Mechanical Engineering Draughtsman - Data CentersEdgnex Data Centers by DAMAC is seeking a highly skilled Mechanical Engineering Draughtsman to join our team. As a key member of our Mechanical Design Team, you will be responsible for designing and preparing mechanical design documentation for data centers.About the Role:Work closely with our Mechanical...

  • Data Scientist

    6 days ago


    Dubai, Dubai, United Arab Emirates Data Labs - LANDMARK GROUP Full time

    We are seeking a skilled Data Engineer to join our team at Data Labs - LANDMARK GROUP. As a key player in our data engineering initiatives, you will drive automation, data quality, and accessibility across the organization.You will work closely with senior team members and business stakeholders to create scalable, cloud-based data solutions on Microsoft...


  • Dubai, Dubai, United Arab Emirates Data Labs - LANDMARK GROUP Full time

    At Data Labs - LANDMARK GROUP, we are looking for an experienced Data Engineer to join our team. As a Junior Data Engineer, you will play a vital role in supporting our data engineering initiatives that drive automation, data quality, and accessibility across the organization.About the Role:Cloud-Based Solutions: Work closely with senior team members and...

  • Big Data Developer

    6 days ago


    Dubai, Dubai, United Arab Emirates Data Labs - LANDMARK GROUP Full time

    We are seeking a talented Data Engineer to join our team at Data Labs - LANDMARK GROUP. As a key player in our data engineering initiatives, you will drive automation, data quality, and accessibility across the organization.About the Job:Cloud-Based Solutions: Create scalable, cloud-based data solutions on Microsoft Azure.Data Ingestion and Integration:...


  • Dubai, Dubai, United Arab Emirates Edgnex Data Centers by DAMAC Full time

    Data Center – AutoCAD Draughtsman (Mechanical)Undertake the preparation of Mechanical Data Center Design documentation in accordance with DAMAC Design Standards and deliver in accordance with the approved design program.Job role functions:1. Work with the Mechanical Data Center Design Team to implement high quality Mechanical Data Center Design solutions...


  • Dubai, Dubai, United Arab Emirates Edgnex Data Centers by DAMAC Full time

    Data Center – AutoCAD Draughtsman (Mechanical)Undertake the preparation of Mechanical Data Center Design documentation in accordance with DAMAC Design Standards and deliver in accordance with the approved design program.Job role functions:Work with the Mechanical Data Center Design Team to implement high quality Mechanical Data Center Design solutions...

  • Data Engineer

    7 days ago


    Dubai, Dubai, United Arab Emirates GSSTech Group Full time

    Job Title: Data Engineer (PySpark)________________________________________About the RoleWe are seeking a highly skilled Data Engineer with deep expertise in PySpark and the Cloudera Data Platform (CDP) to join our data engineering team. As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines that ensure...

  • Data Engineer

    3 days ago


    Dubai, Dubai, United Arab Emirates GSSTech Group Full time

    Job Title: Data Engineer (PySpark)________________________________________About the RoleWe are seeking a highly skilled Data Engineer with deep expertise in PySpark and the Cloudera Data Platform (CDP) to join our data engineering team. As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines that ensure...

  • Data Engineer

    4 weeks ago


    Dubai, Dubai, United Arab Emirates LanceSoft UAE Full time

    We have a new opportunity for a Data Engineer with our client.Title: Data EngineerDuration: 6 MonthsLocation: DubaiQualifications:A Bachelor's degree in Data Engineering, Computer Science, Information Technology, or a related discipline is required.A Master's degree is preferred but not mandatory.Certifications in ETL tools, data engineering platforms, or...

  • Data Engineer

    4 weeks ago


    Dubai, Dubai, United Arab Emirates VML South Africa Full time

    Position Name: Data EngineerPosition Location: Dubai & Abu Dhabi, UAE THE ROLE We are seeking a skilled Data Engineer to join our team. The successful candidate will work closely with the Senior Data Engineer and be responsible for designing and managing the data structure and ETL processes of our retail bank client. They will handle data from various...


  • Dubai, Dubai, United Arab Emirates Data Labs - LANDMARK GROUP Full time

    Data Labs - LANDMARK GROUP is a leading organization in the industry, and we are seeking a highly skilled Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for driving automation, data quality, and accessibility across the organization.You will work closely with senior team members and business stakeholders...

  • Data Engineer

    5 days ago


    Dubai, Dubai, United Arab Emirates Teliolabs Communication Private Limited Full time

    Location: DubaiWho Can Apply: Candidates who are currently in DubaiJob Type: ContractExperience: Minimum 8+ yearsJob Summary:We are looking for an experienced Data Engineer to design, develop, and optimize data pipelines, ETL processes, and data integration solutions. The ideal candidate should have expertise in AWS cloud services, data engineering best...

  • Data Engineer

    3 days ago


    Dubai, Dubai, United Arab Emirates the ENTERTAINER Full time

    Get AI-powered advice on this job and more exclusive features.About Us The ENTERTAINER is a leading digital company dedicated to adding value for consumers by bringing them the best incentive offers globally. We are a 100% digital, data-driven tech company providing first-rate offers across renowned dining, leisure, entertainment, and hotel brands worldwide....

  • Data Engineer

    1 week ago


    Dubai, Dubai, United Arab Emirates VML South Africa Full time

    At VML, we are a beacon of innovation and growth in an ever-evolving world. Our heritage is built upon a century of combined expertise, where creativity meets technology, and diverse perspectives ignite inspiration. With the merger of VMLY&R and Wunderman Thompson, we have forged a new path as a growth partner that is part creative agency, part consultancy,...

  • Data Engineer

    3 days ago


    Dubai, Dubai, United Arab Emirates Overture Rede Full time

    Job Description: Data EngineerLocation: Dubai Job Summary: We are looking for an experienced Data Engineer. The suitable candidate should have demonstrated experience in designing and implementing ETL/ELT solutions on-premises modern platforms to support Enterprise data warehouse, Data Mart, and advanced analytics capabilities. Success in this role...

  • Data Engineer

    6 hours ago


    Dubai, Dubai, United Arab Emirates ClickJobs Full time

    Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers, doers and disruptors, who solve real problems and meet real customer needs. We are seeking Data Engineers...

  • Data Engineer

    1 week ago


    Dubai, Dubai, United Arab Emirates ValueLabs Full time

    Direct message the job poster from ValueLabsTalent Acquisition Executive at ValueLabsExperience - 5+ yrsLocation - Dubai (On-site)Notice Period - Immediate - 30 daysRoles and Responsibilities:Data Pipeline Development: Design, develop, and maintain highly scalable and optimized ETL pipelines using PySpark on the Cloudera Data Platform, ensuring data...