Scala Big Data Engineer (Expert)

SCROLL DOWN

Kraków, Rzeszów, Kielce + REMOTE

Scala Big Data Engineer (Expert)

B2B: 24000 - 29000

PLN

UOP: 18000 - 22500

PLN

Would you like to wrangle terabytes of data in minutes using cutting-edge technologies? Would you like to join a team of talented and passionate Software Engineers? Developing your programming skills, building and maintaining data lakes, data platforms, and applications for various modern and highly-funded companies? Using FP and type safety to make sure your applications are correct and do that in a company developing Scala 3? Whether you’re an expert, or you are a Scala developer who would like to get into Big Data or a Data Engineer who wants to learn more about Scala (or both!) then we are looking for someone just like you.

Location

Kraków, Rzeszów, Kielce

Skills required

Scala

Expert

Spark

Expert

Hadoop or Cloud solutions

Advanced

English

Advanced

Available projects

Project Conway

Project scope

Developing a generic Big Data ingestion and transformation framework for a new, but highly-funded, UK-based, insurance company. We work across two teams: one responsible for developing the framework and extending the capabilities of the platform and another one focusing on using the framework to ingest hundred of datasets from various data sources.

There are lots of greenfield areas and possibilities to create something new, having a great effect on the whole company.

Tech stack

Scala, Spark (mostly Core, some SQL), AWS (EMR, EC2, S3, Codebuild)

Challenges

Creating a generic Big Data framework on top of Spark able to ingest and transform hundreds of datasets from various sources.
Making Apache Spark reliable and conforming to Functional Programming style.
Automating any possible part of the development process - the less manual work the better.

Team

Around 20 engineers (both Scala/Spark and Snowflake) in the team, of which about a half are from well known consultancies.
Most of the Engineers have Scala background and put a lot of focus on Functional Programming - some of the consultants have been in the Scala community for more than 10 years.

Project Sherlock

Project scope

We are developing an analytical platform (e-commerce industry), operating on a cluster of hundreds of machines, with hundreads of RAM’s terabytes, as well as thousands of cores. We integrate, process and analyze data. We need all this to run and test the applications we have developed that use state-of-the-art technologies from the Big Data world.

We are also optimizing complex machine learning applications (ML pipelines), improving the generation of huge analytical views (joining tables with millions of records in a few minutes is our speciality!). We have an enormous cluster available to test all the developed apps.

Tech Stack

Apache Spark (core, SQL, pySpark, Streaming, mllib), Scala, Kafka, Hive, HBase, Hadoop, Teradata, Azure, Jenkins, Ansible, SBT, GIT.

Team

Over thirty developers with experience in building Big Data solutions divided into teams of 4-6 people. We have a real influence on the selection of tools or the possibility of architectural changes.

What we expect in general

Able and eager to lead by example
Hands-on experience in designing and developing scalable, distributed, highly available solutions
Interest in solving challenging data engineering problems using state-of-the-art techniques
Grounded knowledge and understanding of data structures, algorithms, and distributed computing
Fluent with Scala and/or Spark
A functional Programming mindset is nice to have
Able to develop and maintain high-quality code
Can communicate with the business using good English (both written and verbal)
Experienced in using big data and cloud solutions (eg Hadoop, AWS, Azure)
Has a deep understanding of data-intensive distributed systems

What's on offer?

community

Building tech community in Krakow & Poland

We do our best to give back to the community by creating a space for the exchange of experience and ideas.

path

Transparent rules of getting promoted

We are proud we created a fair, transparent environment that gives opportunities for personal growth, a promotion and a pay rise, no matter what the team member’s starting point is.

knowledge

Virtusity - in-house training division

We're good at knowledge sharing. We provide internal workshops and rich knowledge base.

environment & atmosphere

Chillout zones

A table football, Mario Bros, Duck Hunt, comfy swings for you to relax.

environment & atmosphere

Equipment budget

you can get up to 800 PLN of refund for equipment you purchased as needed for home office work

work

Flexibility in projects

We make a room for your autonomy and provide you with mentoring and guidance form the first day.

work

Flexible hours

You decide when you work efficiently.

environment & atmosphere

Fresh fruits & snacks

Fruits, nuts and more of your favourite tasty treats.

events

Gossip pizza

Stay up-to-date with your co-workers and projects eating monthly gossip pizza.

work

Hybrid work model

We have the opportunity to work fully remotely, in the office or in the hybrid model. Just choose what suits you best!

events

Integration, sport events and tournaments

Sports? Other cool activities? We've got this covered.

knowledge

Language lessons

We’ll refund 50% of the price of English, German or Polish lessons you'd like to take.

environment & atmosphere

Macbook / Thinkpad

Depending on the project, best for you with all additional accessories needed.

environment & atmosphere

Office in the center of Kraków, Kielce or Rzeszów

We’re getting things done from an office in Kraków, Kielce and Rzeszów.

community

Open source initiatives

We contribute to open source projects and enhance software that is used by software developers around the world.

extras

Private healthcare

If you care, we care. Choose what’s best for you to meet your healthcare needs.

extras

Sport card

Want to stay healthy and fit? We provide you with the possibilities, the rest is in your hands.

environment & atmosphere

Stretching

Feeling tense or tired? Join in-house stretching sessions.

knowledge

VirtusLab Training Package

We give you 24 paid hours of work for your development. You can choose the training from our Virtusity portfolio or an external one (we'll cover up to 50% of its cost). If the course’s certificate is aligned with our core specifications - we’ll refund 100% of its price.

Show more benefits Hide benefits

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.

Necessary

Always Enabled

Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.

Cookie	Duration	Description
cookielawinfo-checbox-analytics	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checbox-functional	11 months	The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checbox-others	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-necessary	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-performance	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy	11 months	The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.

Marketing

Analytics

Others

Skills required

Available projects

What we expect in general

Application form