Back

Data Engineer – Spark/Scala

  • Job Ref: 7692
  • Dublin
  • IT

We are currently on the lookout for multiple Data Engineers include Lead, Senior and mid level as our clients look to grow the team here in Ireland.

The client who provides a cloud-based big data platform amongst other products is looking for Spark and Scala experience but is open to someone coming from a Python background. 

 Responsibilities include:

  • Actively participate in team technical discussions in all things data
  • Identify and address issues with data sets from multiple vendors
  • Identify and address code and data quality issues
  • Actively participate in code reviews and grooming sessions
  • Actively participate in technology architecture discussions for product development
  • Translate business requirements into strategy
  • Advocate for software best practices within your team as well as across engineering
  • Be ultra-responsive and capable of making instant decisions, always kicking the ball forward
  • Work on unique and interesting data challenges around architecting, building and managing pipelines that securely process hundreds of terabytes of data
  • Work closely with analysts and statisticians to ensure the validity of our processes

 

Key Requirements:

  • Bachelor's degree in Computer Science or a related field (or 4 additional years of relevant work experience)
  • A strong understanding of data structures, algorithms, and effective software design
  • Significant development experience with a major modern language (e.g. Scala, Python)
  • Skillful user of Apache Spark
  • Experience working with structured and unstructured data at scale and comfort with a variety of different stores (key-value, document, columnar, etc.) as well as traditional RDBMSs
  • Experience with the full development lifecycle, from ideation to running software in production
  • Excellent verbal and written communication skills; must work well in an agile, collaborative team environment

 

Beneficial:

  • Experience with AWS products (Redshift, EMR, S3, IAM, RDS, etc)
  • Experience with or interest in Databricks and any other tools that enable data processing at scale
  • Experience with automation tools such as Apache Airflow

Get in touch and I can provide you with more info. Email: [email protected] or call 0862247895.