Senior Data Engineer

4 weeks ago


Abu Dhabi, United Arab Emirates Core42 Full time
Overview

The opportunity

Core42 is the UAE's national-scale enabler for cloud and generative AI, combining G42 Group's expertise across multiple technology disciplines into a single platform for public sector and large enterprise transformations. Building on our capabilities as sovereign cloud and HPC specialist, we bring generative AI, cybersecurity, professional and managed services expertise to enable national-scale program deployments across industries.

Responsibilities

Your key responsibilities

Data Pipeline Architecture and Development

  • Design, construct, install, test, and maintain highly scalable data pipelines with a focus on machine learning models and analytics.

Data Integration

  • Work closely with data scientists, ML engineers, and stakeholders to ensure that data is accessible, consistent, and reliable for ongoing projects.

API and Data Services

  • Develop and maintain APIs for data access and manipulation, and integrate with external data services as needed.

Data Storage

  • Manage and optimize data storage solutions, including relational databases, Search Engines like Elasticsearch and NoSQL databases, to support the requirements of machine learning models.
  • Understand data engines and structure to effectively design solutions for transactional, analytics, and search purposes.

Data Quality and Governance

  • Implement processes to monitor data quality and ensure production data is always accurate and available for key stakeholders.

Collaboration and Support

  • Collaborate with ML engineers to assist in data-related technical issues and provide architectural guidance and solutions.

Security and Compliance

  • Ensure compliance with data security and privacy policies.

Documentation

  • Maintain clear and up-to-date documentation including data dictionaries, metadata, and architectural diagrams.

Qualifications

Skills and attributes to success

  • Bachelor's degree in Computer Science, Engineering, Mathematics, or a related field; or equivalent work experience.
  • 5+ years of experience in a Data Engineering role.
  • Proficiency in SQL and programming languages like Python, Java, and Scala.
  • Hands-on experience with big data technologies like Hadoop, Spark and Flink.
  • Familiarity with machine learning frameworks such as TensorFlow, PyTorch, or similar.
  • Strong understanding of data warehousing concepts, ETL processes, and data modeling.
  • Experience with API development and integration with data services.
  • Experience with cloud platforms like AWS, GCP.
  • Knowledge in DevOps, CI/CD methods, and containerization technologies like Docker or Kubernetes.
  • Experience with real-time data processing.

Technical stack

  • Programming Languages: Python, Java, Scala, SQL, Bash
  • Big Data Technologies: Hadoop, Spark, Flink
  • Databases: MySQL, PostgreSQL, MongoDB, Cassandra, HBase, Redis
  • Cloud Platforms: Azure
  • API Development: RESTful APIs, GraphQL, OpenAPI
  • Data Services: Kafka, RabbitMQ
  • Containers: Docker, Kubernetes

What we look for

If you are a performance-driven, inquisitive mind with the agility to adapt to ambiguity, you will fit right in. You should be eager to explore opportunities to build meaningful collaborations with stakeholders and aspire to create unique customer-centric solutions. Bias for action and a passion to conquer new frontiers in the AI space is at the heart of the Core42 community.

What working at Core42 offers

Culture: An open, diverse and inclusive environment with a global vision that encourages personal growth and focuses on ground-breaking, industry-first innovations.

Career: Outstanding learning, development & growth opportunities via structured training programs and innovative, high-tech projects.

Work-Life: A hybrid work policy to strike the perfect balance between office and home.

Rewards: A competitive remuneration package with a host of perks including healthcare, education support, leave benefits and more.

If you can confidently demonstrate that you meet the criteria above, please contact us as soon as possible.

  • Abu Dhabi, United Arab Emirates ALPHA DATA PROCESSING SERVICES LLC Full time

    Create Data-warehouses / Data-marts• In-depth understanding of healthcare data (Preferred)• Informatica DEI (Data Integration) and DQ (Data Quality) tool (Preferred)• Create Data Architecture optimized for analytics and collaborative research• Create Data Dictionary, ER models and other technical documentation• Train and support data analysts and...


  • abu dhabi, United Arab Emirates ALPHA DATA PROCESSING SERVICES LLC Full time

    Create Data-warehouses / Data-marts• In-depth understanding of healthcare data (Preferred)• Informatica DEI (Data Integration) and DQ (Data Quality) tool (Preferred)• Create Data Architecture optimized for analytics and collaborative research• Create Data Dictionary, ER models and other technical documentation• Train and support data analysts and...


  • abu dhabi, United Arab Emirates ALPHA DATA PROCESSING SERVICES LLC Full time

    Create Data-warehouses / Data-marts• In-depth understanding of healthcare data (Preferred)• Informatica DEI (Data Integration) and DQ (Data Quality) tool (Preferred)• Create Data Architecture optimized for analytics and collaborative research• Create Data Dictionary, ER models and other technical documentation• Train and support data analysts and...


  • Abu Dhabi, United Arab Emirates Talent Pal Full time

    Abu Dhabi United Arab Emirates BigData Development BCM Industry15/04/2024 Req. VR103447Project description We are hiring for projects in the Middle East. There are many opportunities in the region. Our team consists of frontend and backend developers data analysts and data scientists architects analysts and project managers and Databricks Data engineers. As...

  • Process Engineer I

    4 weeks ago


    abu dhabi, United Arab Emirates The Chemical Engineer Full time

    51,000 people. 49 countries. Over 120 office locations.We’re not just engineers. We’re a global team of data scientists, consultants, construction workers andinnovators all working to create a better tomorrow. Every day, we help customers in energy, chemicals andresources get one step closer to solving our planet’s toughest issues. Climate change. The...


  • Abu Dhabi, United Arab Emirates Talent Pal Full time

    Abu DhabiUnited Arab EmiratesBigData DevelopmentBCMIndustry15/04/2024Req.VR103447Projectdescription We are hiring forprojects in the Middle East. There are many opportunities in theregion. Our team consists of frontend and backend developers dataanalysts and data scientists architects analysts and projectmanagers and Databricks Data engineers. As a Senior...


  • abu dhabi, United Arab Emirates Talent Pal Full time

    We are hiring for projects in the Middle East. There are many opportunities in the region. Our team consists of frontend and backend developers data analysts and data scientists architects analysts and project managers and Databricks Data engineers. As a Senior Databricks Data Engineer you will play a crucial role within our data platform team focusing on...

  • AML Tuning

    4 weeks ago


    abu dhabi, United Arab Emirates Alpha Data Recruitment Full time

    Notice Period: ImmediateContinuous ImprovementSupport the Compliance Technology, Data and Reporting Department in identifying continuous improvement and sustainability improvement opportunities for the related systems, processes and practices.Manage the implementation of the AML Group compliance systems Road map as agreed with the Compliance management and...


  • Abu Dhabi, Abu Dhabi, United Arab Emirates Charterhouse UAE Full time

    ​Charterhouse is working with a government entity based in Abu Dhabi who is looking to hire a Senior Business Data Analyst. This role will play a crucial part in collecting, managing, and presenting strategic data to support decision-making within the organization. This position involves daily operations, continuous process improvement, and collaboration...


  • Abu Dhabi, United Arab Emirates Charterhouse UAE Full time

    ​Charterhouse is working with a government entity based in Abu Dhabi who is looking to hire a Senior Business Data Analyst. This role will play a crucial part in collecting, managing, and presenting strategic data to support decision-making within the organization. This position involves daily operations, continuous process improvement, and...


  • abu dhabi, United Arab Emirates Amazon Data Serv Emirates LLC Full time

    Facility Engineer – Data Centers | Amazon Jobs in Abu DhabiDescription:We are currently looking for an energetic, detail-oriented individual to join our Data Center Engineering Operations Team. This committed group works to maintain the critical physical infrastructure that supports our Data Center Organization within Amazon. Specifically, this team works...


  • Abu Dhabi, United Arab Emirates HTC Global Services Full time

    Good understanding of other Azure servicesExcellent knowledge of SQLGood understanding of Data Warehouse Architecture, Data Modelling and design conceptsExperience in Power BI, SFTP, Messaging, APIsLead and guide the data engineering team technically and share best practices with the teamExcellent analytical and organization skills.Effective working in a...


  • abu dhabi, United Arab Emirates First Abu Dhabi Bank Full time

    Job DescriptionJob PurposeRole is required to support the transformation programme which aims at modernizing the data platforms estate for the bank. Traditional on-premises data warehouse and data lake should be transformed to meet the scalability, agility and modern capabilities needs as the bank embarks on the journey to implement the industry best...

  • Data Engineer

    3 weeks ago


    abu dhabi, United Arab Emirates BlackStone eIT Full time

    BlackStone eIT Data Engineer Job DescriptionBlackStone eIT, a leading computer software company, is seeking a talented and experienced Data Engineer to join our dynamic team. As a Data Engineer at BlackStone eIT, you will be responsible for designing, developing, and maintaining scalable data infrastructure and systems. You will work closely with...


  • abu dhabi, United Arab Emirates HTC Global Services Full time

    Skills Required:Azure: Good understanding of other Azure servicesSQL: Excellent knowledge of SQLData Warehouse: Good understanding of Data Warehouse Architecture, Data Modelling, and design conceptsTools: Experience in Power BI, SFTP, Messaging, APIsResponsibilities:Lead and guide the data engineering team technicallyShare best practices with the teamUtilize...

  • Data Engineer

    4 weeks ago


    abu dhabi, United Arab Emirates Tech Junction Ltd Full time

    Job Type: 12 Months ExtendableResponsibilities: •Experience with handling large volumes of unstructured data (TBs ideally), looking for proficiency beyond structured databases like MySQL.•Experience with various tools such as Kibana, Power BI, and optionally, Neo4j for graph visualization.•Expert level of proficiency in Python programming.•Must...

  • Data Engineer

    1 week ago


    abu dhabi, United Arab Emirates Locai Full time

    Job Overview: We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. The ideal candidate should have a strong background in data engineering, with expertise in designing, developing, and maintaining robust data infrastructure. The Data Engineer will play a crucial role in building and optimizing our data pipeline architecture,...

  • Data Engineer

    4 days ago


    abu dhabi, United Arab Emirates Chainlink Labs Inc. Full time

    At Chainlink Labs, the Market Data, Data Platform, and Real-time Streaming Platform teams excel in delivering critical data insights, foundational datasets, and real-time data services, empowering the organization with advanced tools for operational and strategic excellenceMarket Data Team:The Market Data team at Chainlink is at the forefront of developing...


  • abu dhabi, United Arab Emirates First Gulf Bank PJSC Full time

    Sub Division: Data Analytics and Artificial IntelligenceDivision: GCOOCompany DescriptionJoin the UAE’s largest bank and one of the world’s largest and safest financial institutions. Our focus is to create value for our employees, customers, shareholders and communities to grow through differentiation, agility and innovation. We are looking for top...

  • Data Engineer

    4 weeks ago


    Abu Dhabi, United Arab Emirates Adecco Full time

    · Bachelor’s degree in computer science, Engineering, Mathematics, or a related field; or equivalent work experience.· 5+ years of experience in a Data Engineering role.· Proficiency in SQL and programming languages like Python, Java, and Scala.· Hands-on experience with big data technologies like Hadoop, Spark and...