Data Engineer
- Job Ref: 9231
- Greater London
- IT
We are on the lookout for multiple Data Engineers for our client in the public sector for a 2 year contract. This role is hybrid and would require you to attend the office twice a week.
Responsibilities:
- You will be responsible for the design, implementation and maintenance of data pipelines and relevant automations.
- You will implement data transformations suited for follow-on data centric-applications (e.g. analysis, reports, and dashboards).
- The Data Engineer is aligned internally with the (Head) Data Architect, Big Data System Engineer and Data Scientist to guarantee consistency in the implementation of the overarching Data Strategy and relevant Data Policies (e.g. Data Protection, Data Security).
- The Data Engineer is a strongly technical figure with in-depth knowledge of distributed systems and computer science, proven experience in Big Data programming, data streaming, data storing and relevant tooling
Requirements:
- In-depth knowledge and experience in building and optimising
- Big Data pipelines with technologies such as NiFi, Airflow or Luigi, architectures and datasets
- In-depth knowledge and experience with Big Data Platforms (ideally Hadoop);
- Hands-on experience in working with relational and nonrelational databases
- Practical experience in root cause analysis on data and processes to identify opportunities for improvements;
- Practical experience in building processes supporting data transformation, data structures, metadata, dependency and workload management.
- Working knowledge and hands-on experience of message queuing, stream processing, highly scalable data-stores (ideally Elastic or Solr, Kafka, Hive, Hbase);
- Good knowledge of industry trends and emerging technologies;
- Ability to work as part of a team;
- Leadership skills, initiative and drive;
- Strong communication and interpersonal skills.
EU citizens, Stamp 4 and Spouse Visa holders will only be considered for this role.
Please apply belwo if interested :)