Senior Data Engineer

Dialog Axiata PLC
14 days ago
0 Applied
Expires on: Dec 13 2022

Ref.No 00001405

Description

Entry Requirements

  • Bachelor’s Degree in Computer Science, Engineering, IT, Business or relevant major with good programming and technical skills. Master’s degree in a relevant field will be preferred.
  • 2-5 years of experience within the field of Data Engineering, Data Management, implementing real-time ingestion/streaming, maintaining Big Data Lake/Warehouse and experience in ETL tasks on premise and cloud. 
  • Good understanding of Data lineage, Data Catalog, master data management, metadata management and Data Management tools.
  • Experience using ETL tools and developing complex pipelines through either Scala/Java/Python or opensource tools.
  • Expertise in SQL, Apache Spark, Nifi, Airflow, Kafka; Experience in designing and maintaining relational databases and non-SQL data storage. Data Engineering certifications will be an added advantage.
  • Experience in software development and optimizing code for performance and scalability will be an added advantage
Responsibilities
  • Design, develop and maintain Data Catalog, Metadata Management repository/module and Data Lineage through Data Management / Quality tools.
  • Ensure that Master data management capabilities meets quality requirement.
  • Report on data issues, data quality improvement initiatives and measurement metrics.
  • Perform Root Cause Analysis and effectively resolve critical issues within agreed timelines.
  • Collaborate with stakeholders, Data Governance team, Data Engineers/ IT team, Data Owners, Data Stewards, Data source administrators, platform and software developers to ensure standards, policy, measurement metrics are being met
  • Ensure that Data Management infrastructure, Policies & Procedures are kept up to date with Data Ecosystem;
  • Proactively identify gaps as part of Agile Data Quality Improvement initiatives
  • Collaborate with IT Data Engineering team on Commissioning new pipelines (New source systems, new applications, new products, change requests) and on ETL issues involving Data Quality
  • Translate business requirements to technical design, data models and manage data quality engineering tasks.
  • Work with stakeholders, data architects, data owners and stewards to implement data management process, business rules, business decisions, policy, and analyze gap as part of the organization’s data governance requirements.
  • Document, communicate progress and share improvement plan on Data Quality issues/reported incidents to stakeholders (Heads of Divisions, Data Scientists, ML Engineers, Analysts and Other Data Consumers)
  • Monitor, across data-lifecycle stages that privacy (data protection), business decision, access, rules, and regulatory requirements are adhered to consistently from sources, storages (hot and cold), metadata updates, job failure logs/data logs, and downstream reporting servers.
  • Facilitate data audits by auditors
Skills
Data Engineering
Data Management
ETL tools
SQL
Scala/Java/Python
Industry Sector