Lead Data Engineer
4 weeks ago
Luxoft Abu Dhabi Emirate, United Arab Emirates
We are hiring for projects in the Middle East. Our team consists of frontend and backend developers, data analysts and data scientists, architects, analysts and project managers, and Databricks Data engineers. As a Lead Data Engineer, you will manage the development, implementation and maintenance of data infrastructure for a modern data platform.
Responsibilities- Platform Design: Collaborate with cross-functional teams to refine and design the architecture of our data platform.
- Platform Optimization: Continuously enhance the performance, scalability, and reliability of the data platform to meet evolving business requirements.
- Platform Maintenance: Implement monitoring, alerting, and maintenance processes to ensure the availability and health of the data platform.
- Data Governance: Establish and enforce data governance best practices to uphold data quality, security, and compliance standards.
- Platform Integration: Integrate new technologies seamlessly into the data platform ecosystem, ensuring compatibility and interoperability.
- Collaboration: Work closely with data engineers, data scientists, analysts, and business stakeholders to comprehend requirements and offer technical expertise and support.
- Documentation: Create and maintain comprehensive documentation for the data platform architecture, processes, and workflows.
- Mentorship: Provide guidance and mentorship to junior members of the data engineering team, nurturing their growth and development.
- Design, develop, and maintain data pipelines to ingest and transform raw data into Silver and Gold tables using Databricks.
- Build reusable data pipelines that can be utilized by the application development team.
- Collaborate with data consumers and other stakeholders to understand data requirements and deliver high-quality data solutions.
- Implement declarative ETL pipelines using Databricks, Delta Lake, and other relevant tools to ensure data is accurate, timely, and accessible.
- Optimize data models and pipelines for performance, scalability, and reliability.
- Model data from various sources, ensuring consistency and quality in the Silver layer.
- Curate data assets in the Silver layer to support data science and machine learning applications.
- Build and maintain the Gold layer to provide clean, aggregated, and business-ready data for reporting and analytics.
- Ensure data governance and security best practices are followed.
- Well versed with Databricks fine-tuning techniques for huge data sets.
- Document data processes, models, and pipelines for transparency and knowledge sharing.
- Well versed with DevOps practices for continuous integration and deployment.
- Writing Code, reviewing pull requests, providing constructive feedback and maintaining high code quality standards.
- Maintain a deep understanding of data engineering principles, best practices and emerging technologies to drive engineering initiatives.
- Overall 5-7 years of experience in data engineering and transformation on Cloud.
- 3+ Years of Very Strong Experience in Azure Data Engineering, Databricks.
- Expertise in supporting/developing lakehouse workloads at the enterprise level.
- Experience in PySpark is required.
- Developing and deploying the workloads to run on the Spark distributed computing.
- Candidate must possess at least a Graduate or bachelor's degree in Computer Science/Information Technology, Engineering (Computer/Telecommunication) or equivalent.
- Cloud deployment: Preferably Microsoft Azure.
- Experience in implementing the platform and application monitoring using Cloud native tools.
Languages
Seniority : Lead
Seniority level- Mid-Senior level
- Full-time
- Information Technology
- Industries: IT Services and IT Consulting
-
Senior Data Analyst
4 weeks ago
, , united arab emirates, united arab emirates Data Bene Full timeYour mission Data Analysis & Insight Generation: Lead complex analytical projects that directly impact business strategy and decision-making. Employ advanced analytics and machine learning techniques to uncover deep insights from data. Champion the importance of a data-driven culture, mentoring others in advanced analytics methodologies and...
-
Senior Data Analyst
4 weeks ago
, , united arab emirates, united arab emirates Data Bene Full timeYour mission Data Analysis & Insight Generation: Lead complex analytical projects that directly impact business strategy and decision-making. Employ advanced analytics and machine learning techniques to uncover deep insights from data. Champion the importance of a data-driven culture, mentoring others in advanced analytics methodologies and...
-
Lead Data Engineer
2 weeks ago
United Arab Emirates Emirates Global Aluminium (EGA) Full timeA Lead Data Engineer works closely with a multidisciplinary Agile team to build high quality data pipelines driving analytic solutions. These solutions will generate insights allowing to advance data-driven decision-making capabilities. This role requires deep understanding of data architecture, data engineering, data analysis, reporting, and a basic...
-
Lead Civil Structural Designer
4 weeks ago
, , united arab emirates, united arab emirates Lead Soluções Ambientais Full timeWood has been awarded a major project with ADNOC and, we are looking to hire a Lead Designer (Civil Structural) to be based at our Abu Dhabi Engineering Center. RESPONSIBILITIES Review Designs for O&G processing facilities and their associated infrastructure. Cooperate with other disciplines to ensure the timely realization of competitive, achievable and...
-
Lead Civil Structural Designer
4 weeks ago
, , united arab emirates, united arab emirates Lead Soluções Ambientais Full timeWood has been awarded a major project with ADNOC and, we are looking to hire a Lead Designer (Civil Structural) to be based at our Abu Dhabi Engineering Center. RESPONSIBILITIES Review Designs for O&G processing facilities and their associated infrastructure. Cooperate with other disciplines to ensure the timely realization of competitive, achievable and...
-
Lead Data Ops Engineer
2 weeks ago
United Arab Emirates Emirates Global Aluminium (EGA) Full timeThe Lead Data Ops Engineer will play a critical role in building, scaling, automating, and maintaining the company's Big Data infrastructure and Machine Learning operations (MLOps). This individual will work in close collaboration with our Data Architect, Data Science, Data Engineering, and IT teams to ensure the development, deployment, and scale of robust,...
-
Data Engineer
3 weeks ago
United Arab Emirates Involved Solutions Full timeOur client, a leading name in the insurance sector, is hiring a Data Engineer to drive their data innovation efforts.About the role:As a Data Engineer, you'll design and optimize data pipelines and systems to power business objectives. Your work will enable smarter decision-making and enhance efficiency across the organization. Build and maintain robust data...
-
Oil and Chemicals Insurance Risk Engineer
15 hours ago
United Arab Emirates The Chemical Engineer Full timeSalary: £75,000 - £115,000 (max) + housing allowance Location: UAE Company: The company is a leading player in the International high hazard Insurance sector, involved in cover associated with various sectors including oil and gas, refining, petrochemicals, LNG, metals, mining, and maritime. The organisation's Gulf hub is enjoying significant client...
-
Azure Data Engineer
4 weeks ago
, , united arab emirates, united arab emirates JobItUs Full timeWork Experience: 5.0 Year(s) To 10.0 Year(s) Responsibilities: Design, construct, install, test, and maintain highly scalable and robust data management systems. Understand and apply data warehousing concepts to design and implement robust data warehouse tables in line with business requirements. Build complex ETL/ELT processes for large-scale data migration...
-
Azure Data Engineer
4 weeks ago
, , united arab emirates, united arab emirates JobItUs Full timeWork Experience: 5.0 Year(s) To 10.0 Year(s) Responsibilities: Design, construct, install, test, and maintain highly scalable and robust data management systems. Understand and apply data warehousing concepts to design and implement robust data warehouse tables in line with business requirements. Build complex ETL/ELT processes for large-scale data migration...
-
Lead Civil Structural Designer
2 weeks ago
United Arab Emirates Lead Soluções Ambientais Full timeWood has been awarded a major project with ADNOC and, we are looking to hire a Lead Designer (Civil Structural) to be based at our Abu Dhabi Engineering Center. RESPONSIBILITIES Review Designs for O&G processing facilities and their associated infrastructure. Cooperate with other disciplines to ensure the timely realization of competitive, achievable and...
-
Data Engineer
15 hours ago
United Arab Emirates ENTERTAINER FZ LLC Full timeAbout Us The ENTERTAINER is a leading digital company dedicated to adding value for consumers by bringing them the best incentive offers globally. We are a 100% digital, data-driven tech company providing first-rate offers across renowned dining, leisure, entertainment, and hotel brands worldwide. The ENTERTAINER has grown with the aim of creating...
-
Data Engineer
4 weeks ago
, , united arab emirates, united arab emirates DuneSoft.ae Full timeA data engineer is an IT worker whose primary job is to prepare data for analytical or operational uses. These software engineers are typically responsible for building data pipelines to bring together information from different source systems. Overview You will be responsible for: Designing and implementing Big Data systems (e.g. data warehouses, data...
-
Data Engineer
4 weeks ago
, , united arab emirates, united arab emirates DuneSoft.ae Full timeA data engineer is an IT worker whose primary job is to prepare data for analytical or operational uses. These software engineers are typically responsible for building data pipelines to bring together information from different source systems. Overview You will be responsible for: Designing and implementing Big Data systems (e.g. data warehouses, data...
-
Data Science Lead
2 weeks ago
United Arab Emirates Michael Page Full timeAs the leading delivery company in the region, we have a great responsibility and opportunity to impact the lives of millions of customers, restaurant partners, and riders. To realize our potential, we need to advance our platform to become much more intelligent in how it understands and serves our users. As a data scientist on the algorithms track, your...
-
Data Engineer
2 weeks ago
United Arab Emirates Lune Full timeThe data engineer is responsible for designing, implementing, and maintaining the systems and processes necessary for efficient data collection, storage, and analysis. They work closely with the product team, data labeling team, and other stakeholders to understand data requirements and ensure that data pipelines are robust, scalable, and optimized for...
-
Senior Data
4 weeks ago
, , united arab emirates, united arab emirates Elkaso Full timeYour mission Are you passionate about leveraging data and AI to revolutionize the B2B food industry? Kaso is seeking a talented Senior Data & ML Engineer to join our team and help build a new innovative data product. Your profile What we are looking for: Strong expertise in Data Engineering, Data Modeling, and AI/ML Engineering Proven track record of...
-
Senior Data
4 weeks ago
, , united arab emirates, united arab emirates Elkaso Full timeYour mission Are you passionate about leveraging data and AI to revolutionize the B2B food industry? Kaso is seeking a talented Senior Data & ML Engineer to join our team and help build a new innovative data product. Your profile What we are looking for: Strong expertise in Data Engineering, Data Modeling, and AI/ML Engineering Proven track record of...
-
Lead HVAC Engineer
4 weeks ago
, abu dhabi emirate, united arab emirates, abu dhabi emirate, united arab emirates Descon Full timeDescon Abu Dhabi Emirate, United Arab Emirates Lead HVAC Engineer with 15 years of experience managing HVAC systems in large-scale Oil & Gas EPC projects. Responsible for design, installation, and commissioning of HVAC systems, ensuring integration with project requirements. Key Responsibilities: Lead HVAC design, installation, and commissioning. Coordinate...
-
Lead HVAC Engineer
4 weeks ago
, abu dhabi emirate, united arab emirates, abu dhabi emirate, united arab emirates Descon Full timeDescon Abu Dhabi Emirate, United Arab Emirates Lead HVAC Engineer with 15 years of experience managing HVAC systems in large-scale Oil & Gas EPC projects. Responsible for design, installation, and commissioning of HVAC systems, ensuring integration with project requirements. Key Responsibilities: Lead HVAC design, installation, and commissioning. Coordinate...