Data Engineer

Found in: beBee S AE - 1 month ago

Dubai, United Arab Emirates Agility Full time
:

As a Data Engineer, you are responsible for the development, performance, quality, and scaling of our data pipelines, with a special focus on distributed processing and data quality.

You will be working with stakeholders throughout the company to ensure we have high quality data to power our Data Science initiatives.

You will be collaborating closely with Data Scientists to ingest and efficiently process required data and facilitate the deployment of data science models into production.

Your Responsibilities: Build solutions and tools to enable, organize and process large amounts of data. Design and implement data ingestion and transformation in cloud-based environment and in on-premise platform. Create and maintain optimal data pipeline architecture for both batch and real-time processing. Developing scalable, flexible, and high-performance distributed data pipelines with real time stream processing. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability. Work closely with data scientists to define the best way to ingest and aggregate required data and process it efficiently, and how to get their data science models to run in production. Collaborate with business stakeholders to ingest new sources, cleaning and preparing the data and writing the code that connect it to the data lake or to specific application(s). Evaluate new technologies and build prototypes for continuous improvements in Data Engineering Requirements:

Your Qualifications: MSc degree (or equivalent) in Computer Science, Informatics, Information Systems or another quantitative field 4+ years\' experience in data engineering / architecting in big data environment both on-premise and in the cloud, with hands-on work covering the followings: Your Proficiencies: Building and optimizing complex data pipelines and architectures. Experience architecting, designing, and implementing cloud solutions on AWS environment is a must. Orchestration and workflow management tools: Airflow, AWS Step Function, MLflow, Kubeflow, etc. Developing and managing stream-processing systems: Hadoop/Spark-Streaming, Kafka/Flink, AWS Kinesis\xe2\x80\xa6 Object-oriented/object function scripting languages: Python, Java, Scala, etc. Deployment and provisioning automation tools e.g., Docker, Kubernetes, Terraform Knowledge of search engines such as Elastic search Technical aspects of Data Governance including glossary, metadata, data quality, master data management & data lifecycle management Security, authentication and authorization DevSecOps / MLOps and CI / CD frameworks

Agility

  • Data Engineers

    Found in: Talent AE C2 - 2 weeks ago

    Dubai, United Arab Emirates Ruaa Data Management Services Full time

    Your challengesGather and address technical and design requirements.Architect our data platform & develop real-time apps ingesting large volumes of data and processing messages real-time.Handle all our data pipeline and contribute towards our data strategy and its execution.Architecting, building, testing & managing our data platform infrastructure from data...

  • Data Scientists

    Found in: Talent AE C2 - 2 weeks ago

    Dubai, United Arab Emirates Ruaa Data Management Services Full time

    Your challenges as a data scientist You will be designing, developing, testing and documenting the data collection framework. The data collection consists of (complex) data pipelines from (IoT) sensors and low/high level control components to our Data Science platform.You will build a monitoring solution of data pipeline which enables data quality...

  • Site Engineer Jobs In Dubai UAE 2023 | Alpha Data

    Found in: Whatjobs AE C2 - 2 weeks ago

    dubai, United Arab Emirates Alpha Data Full time

    Site Engineer Jobs In Dubai UAE 2023 | Alpha DataFor More Top Organization Jobs Click HereLooking for a Site Engineer in Dubai, United Arab Emirates (UAE)Job Qualifications and Experience:The suitable candidate must have a Bachelor’s Degree in Civil Engineering or a related fieldMinimum 5+ years of proven experience as a site engineer, preferably within...

  • Energy, Refining and Chemicals Insurance Risk Engineer

    Found in: Talent AE 2A C2 - 3 weeks ago

    Dubai, United Arab Emirates The Chemical Engineer Full time

    £65 to £110 K + housing allowanceInsurance riskCompanyOur client is a leading player in the International high hazard Insurance sector where it is involvedIn cover associated with a variety of sectors including oil and gas, refining, petrochemicals, LNG, metals, and mining, maritime, mass- transport and so on. The organisation’s Gulf hub is enjoying...

  • Energy, Refining and Chemicals Insurance Risk Engineer

    Found in: Whatjobs AE C2 - 2 weeks ago

    dubai, United Arab Emirates The Chemical Engineer Full time

    £65 to £110 K + housing allowanceInsurance riskCompanyOur client is a leading player in the International high hazard Insurance sector where it is involvedIn cover associated with a variety of sectors including oil and gas, refining, petrochemicals, LNG, metals, and mining, maritime, mass- transport and so on. The organisation’s Gulf hub is enjoying...

  • Data Engineer

    Found in: beBee S AE - 4 weeks ago

    Dubai, United Arab Emirates Connecting Research to Development Full time

    Company Description Smart Development is part of the CRD Group and is a forward-thinking think tank based in the vibrant city of Dubai. Our aim is to leverage the power of data and innovative technologies to solve complex challenges and drive sustainable growth. We believe in fostering a culture of continuous learning, collaboration, and impact.Role...

  • Data Engineer

    Found in: Talent AE 2A C2 - 7 days ago

    Dubai, United Arab Emirates SimplyApply Full time

    Data EngineerDo you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers, doers and disruptors, who solve real problems and meet real customer needs. We are seeking...

  • Data Engineer

    Found in: Whatjobs AE C2 - 1 week ago

    dubai, United Arab Emirates SimplyApply Full time

    Data EngineerDo you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers, doers and disruptors, who solve real problems and meet real customer needs. We are seeking...

  • Data Engineer

    Found in: Whatjobs AE C2 - 1 week ago

    dubai, United Arab Emirates SimplyApply Full time

    Data EngineerDo you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers, doers and disruptors, who solve real problems and meet real customer needs. We are seeking...

  • Data Engineer

    2 weeks ago

    Dubai, United Arab Emirates SimplyApply Full time

    Data EngineerDo you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers, doers and disruptors, who solve real problems and meet real customer needs. We are...

  • Data Engineer

    Found in: Whatjobs AE C2 - 3 weeks ago

    dubai, United Arab Emirates Tyde Digital Full time

    Tyde Digital Data Engineer Tyde Digital, a reputable digital transformation consultancy, is seeking a highly analytical and detail-oriented Data Engineer to join our fast-growing team. In this role, you will be responsible for developing, implementing, and optimizing data pipelines with multiple endpoints, while also serving as a data analyst to extract...

  • Data Engineer

    Found in: Talent AE C2 - 2 weeks ago

    Dubai, United Arab Emirates Tyde Digital Full time

    Tyde Digital, a reputable digital transformation consultancy, is seeking a highly analytical and detail-oriented Data Engineer to join our fast growing team.In this role, you will be responsible for developing, implementing, and optimizing data pipelines with multiple endpoints, while also serving as a data analyst to extract actionable insights from our...

  • DevOps Engineer – Data Engineering

    Found in: Whatjobs AE C2 - 2 weeks ago

    dubai, United Arab Emirates Marc Ellis Full time

    Job Duration: 12 months extendableJob DescriptionWe’re on the lookout for a skilled DevOps Engineer with a strong background in data engineering. In this role, you’ll play a pivotal part in enhancing the data engineering processes by building and maintaining CI/CD pipelines, automating data workflows, and optimizing our data...

  • Data Engineer

    3 days ago

    Dubai, United Arab Emirates ClickJobs.io Full time

    Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers, doers and disruptors, who solve real problems and meet real customer needs. We are seeking Data...

  • Data Engineer

    Found in: Talent AE A C2 - 2 days ago

    Dubai, United Arab Emirates ClickJobs.io Full time

    Do you love building andpioneering in the technology space? Do you enjoy solving complexbusiness problems in a fast-paced, collaborative, inclusive, anditerative delivery environment? At Capital One, you'll bepart of a big group of makers, breakers, doers and disruptors, whosolve real problems and meet real customer needs. We are seekingData Engineers...

  • DevOps Engineer – Data Engineering

    Found in: Whatjobs AE C2 - 2 weeks ago

    dubai, United Arab Emirates Tech Junction Ltd Full time

    Job Duration: 12 months extendableJob Description We’re on the lookout for a skilled DevOps Engineer with a strong background in data engineering. In this role, you’ll play a pivotal part in enhancing the data engineering processes by building and maintaining CI/CD pipelines, automating data workflows, and optimizing our data infrastructure....

  • Data Engineer

    Found in: Talent AE A C2 - 2 weeks ago

    Dubai, United Arab Emirates SimplyApply Full time

    DataEngineerDo you love building and pioneeringin the technology space? Do you enjoy solving complex businessproblems in a fast-paced, collaborative, inclusive, and iterativedelivery environment? At Capital One, you'll be part of abig group of makers, breakers, doers and disruptors, who solve realproblems and meet real customer needs. We are seeking...

  • Data Engineering Manager

    Found in: Whatjobs AE C2 - 2 weeks ago

    dubai, United Arab Emirates Michael Page International (UAE) Full time

    We have been made aware of a global phishing campaign where employees from companies are being impersonated. We are confident that no PageGroup system has been breached. Find out how to protect yourself Must have extensive experience within the Banking or Financial Services sector.Opportunity to lead and develop a skilled data engineering teamAbout Our...

  • Data Engineer

    Found in: Whatjobs AE C2 - 2 weeks ago

    dubai, United Arab Emirates Dautom Full time

    Client Introduction: In this role, you will have the opportunity to work closely with one of our esteemed clients. This client is a global leader in the Real Estate industry known for its commitment to quality and innovation. They have chosen Dautom as their trusted partner for their upcoming projects. Job Title: Data Engineer Job Purpose: As a Data...

  • Data Engineering Manager

    Found in: Talent AE 2A C2 - 3 weeks ago

    Dubai, United Arab Emirates Michael Page International (UAE) Full time

    We have been made aware of a global phishing campaign where employees from companies are being impersonated. We are confident that no PageGroup system has been breached. Find out how to protect yourself Must have extensive experience within the Banking or Financial Services sector.Opportunity to lead and develop a skilled data engineering teamAbout Our...