19 September 2017 / Jan Paw

Hadoop legacy

, , ,


In the previous blog post I explained the basic concepts of data lakes. Some core problems which can occur in data lakes were defined and I gave some hints to avoid them. Most of these pitfalls are caused by the traits of data lakes. Unfortunately, current Hadoop distributions can’t resolve them entirely. Additionally, the architecture […]

Read more

31 August 2017 / Bartłomiej Tomala

Navigating data lakes using Atlas

, , , , ,


Nowadays almost every company wants to have their own Big Data system to analyse client behaviour and optimise operating costs. One of the most popular solutions for implementing such systems is a Data Lake based on the Hadoop ecosystem. If you don’t know what exactly a Data Lake is, you can read about it in […]

Read more

14 August 2017 / Paweł Lipski

Meow! Start using Cats in your project right now


Introduction Cats is a library providing abstractions for functional programming in Scala. There are a couple of great posts and courses on Cats out there on the web (like Herding cats and a tutorial on Scala Exercises), but they tend to explore the categories/type classes implemented in the library rather than give practical ready-to-use examples […]

Read more

7 August 2017 / Jan Paw

Diving in the data lake

, , ,


Rapid growth of unstructured data is a serious business challenge for organizations. Data repositories, known as data lakes, have a great chance to play an important role in extracting valuable business information from enormous amounts of data. Storing and processing data on such a scale is a very complex and demanding task. Existing RDBMS-based systems […]

Read more

1 August 2017 / Mikołaj Kromka

Spark SQL under the hood – part I

, , ,


Summary & initial requirements If you use and have a basic understanding of the core concepts of the Apache Spark and Spark SQL (RDDs, DataFrames, Execution Plan, Jobs & Stages & Tasks, Scheduling), then after reading this blog post you should be able to answer the following questions: How do Spark SQL’s Datasets relate to […]

Read more

27 July 2017 / Krzysztof Romanowski

Zinc (sbt) friendly code

, , ,


In April I gave a presentation (50 Shades of Scala Compiler) at the ScalaUA conference. In the abstract, I promised to give some hints on how to write compiler-friendly code. The talk lasted only 40 minutes and Scala compilers have many, many shades (I almost forgot about Dotty!), so all the hints didn’t fit into […]

Read more

12 January 2017 / Paweł Dolega

Smooth operator with Slick 3


Or how to use custom / database specific operators in Slick 3.x. Couple of days ago we had our company Xmas dinner. As we are a software company we obviously talked mostly about technology (surprise!). One of the conversation we had was about a comparison: Slick vs Quill. It was about difficulties of adding customization. […]

Read more

16 November 2016 / Krzysztof Borowski

AkkaPaint – simplicity and power of Akka

,


Introduction   Once upon a time, there was complete chaos, where you could draw and instantly share anything like a boss. However, the days of Adobe Flash are long gone, so a new idea was born: make chaos great again! And this is how from chaos arose AkkaPaint The idea was pretty straightforward: create a […]

Read more

23 May 2016 / Adam Bartosik & Paweł Dolega

Immutable graphs with Neo4j


Problem: Imagine you have to manage hundreds of different projects, that could have several different actions or building blocks. These projects can be grouped into “monitoring groups” – and you have some people assigned to execute them, some others are assigned to monitor them. You have “cost centers”, progress and performance measures and all this […]

Read more

22 April 2016 / Bartosz Kowalik

ScalaMock: macros strike back

, ,


Introduction ScalaMock is a powerful mocking library written purely in Scala. It uses macros to create mocks. Macros and compile-time reflection allow to create type safe code or manipulate programs. It is almost mythological tool, the Holy Grail of programming languages. Almost all modern languages have it. All features of ScalaMock are listed on its […]

Read more

15 March 2016 / Krzysztof Romanowski

ScalaSphere 2016 – few impressions from tooling developer

, , ,


ScalaSphere: conference focused on technical problems and solutions ScalaSphere was one-of-a-kind conference focused on Scala tooling. Those who expected presentations full of theory or maths, might feel disappointed – conference was dedicated purely to Scala tooling. “I’ve never been on such deeply technical conference” was often heard and I agree with those comments. Quo vadis, […]

Read more

11 December 2015 / Jakub Kramarz

Playing around with Mesos on Vagrant

,


TL;DR 1. Here’s a repo: VirtusLab/mesos-on-vagrant, 2. Clone it, make all your RAM free and setup VirtualBox, 3. vagrant up, 4. Congratulations, you’ve 6-node (3 master, 3 slave nodes) “highly-available” cluster running on virtual machines. End notes – to begin with :-) As it’s the only part (despite of short descriptions of each service), I’ll […]

Read more

16 November 2015 / Marcin Rzeźnicki

Arrows, Monads and Kleisli – part II

, ,


In part I I showed how Kleisli arrows could be used to implement domain modeling. Arrows serve as a foundation for a ‘DSL’ in which one can implement typical scenarios that arise in business-logic code: decoupling flow control from domain code, dealing with errors etc. Much to the spirit of Railway Oriented Programming but implemented […]

Read more

28 October 2015 / Marcin Rzeźnicki

Arrows, Monads and Kleisli – part I

, , ,


During Scala Days Amsterdam I came across a concept of arrow. It was called a general interface to computation and it looked like a technique that could bridge code that is imperative in nature with proper functional programming. Later on I read about Railway Oriented Programming and found it really neat. I was compelled to […]

Read more

16 October 2015 / Bartosz Kowalik

Vals gotcha!


Immutability in Scala is great. A lot of concerns go away when you use it, but those that remain are not so clear to reason about. Look at the code below: My question is: what is the value of expression (new Bar).b? In this post I will explain why it is neither 1 nor 2. […]

Read more

9 October 2015 / Jakub Kramarz

Hacking platform drivers into Intel Edison


Unlike most of the development boards, Intel Edison uses binary, proprietary blob for delivering tables of devices connected to I2C and SPI buses. They’re loaded via Simple Firmware Interface (SFI) during kernel startup in Intel MID SFI initialization code (located in intel_mid_sfi.c) and compared to board data structure (board.c). “To summarise, in the need of […]

Read more

31 July 2015 / Michał Pociecha

Interview with S. Schäfer and S. Ochsenreither

, ,


In the previous interview I was talking with Rex Kerr about his work and contribution to Scala. This time I had a pleasure of asking several questions to two well-known (in the community) Scala ecosystem contributors from Karlsruhe. Simon Ochsenreither (soc) studies computer science and works on various Scala-related things in his free time, including […]

Read more

7 July 2015 / Krzysztof Romanowski

Tool-friendly Scala code. What? Why?

, ,


People complain a lot about Scala tooling. People complain a lot about weather. But is Scala tooling like weather? Something you can’t do anything about except from moving to different climate (language)? Luckily no. Scala tooling was created for Java (especially metadata handling that is vital for tools) and usually has only the JVM stack […]

Read more

7 June 2015 / Michał Pociecha

Interview with Rex Kerr. The first appendix to Dick Wall’s ‘Scala needs you!’

,


During Scala Days in SF, Dick Wall was trying to convince people to spend their spare time helping to make Scala language, compiler and documentation even better. He focused mostly on showing the Scala’s web page and available resources which can help newcomers. Also some time ago Adriaan Moors completely redesigned readme.md on github having […]

Read more

19 May 2015 / Jakub Kramarz

Intel Edison + Scala = BlinkOnboard

, , ,


For a long time programming embedded systems with limited hardware resources forced you to use native languages and run your firmware bare-metal. That effectively meant that being a Scala software house and doing embedded weren’t an easy job. Fortunately, more and more relatively high-performance platforms have been recently emerging. After playing for some time with […]

Read more

5 May 2015 / Marcin Rzeźnicki

Path dependent types – modeling algebraic structures has never been easier

,


Let’s start with small refreshment of what actually the dependent typing is. Using Wikipedia as a source: In computer science and logic, a dependent type is a type that depends on a value. Neat, huh? But what does it mean in practice? If you take a look at the Wikipedia’s list of languages that implement […]

Read more

22 April 2015 / Jakub Kramarz

ConductR: Typesafe’s move in lightweight containers and age of distributed clusters

, , ,


During my career as an administrator I’ve tried multiple well-known and developed a few home-grown ways of dealing with applications created in micro-services architecture. Every time I fell into pitfalls of these approaches: the problem of maintaining them in reactive way. ConductR is a Typesafe’s response to this problem, addressing full life cycle of an […]

Read more

17 April 2015 / Marcin Rzeźnicki

The world after LambdaDays and ReactDay 2015

, ,


Attending (or, even better, organizing) a conference is one of the best opportunities to predict what the future may bring. Knowing what the future may bring is undoubtedly the most important knowledge for any enterprise – especially when you’re in IT. We recognize this at VirtusLab and we’ve always been trying to be on top […]

Read more

25 March 2015 / Piotr Kukiełka

Debugging implicits

, ,


I’ve been recently asked what are my favorite and least favorite Scala features. Probably few options would be matching here, but I decided to give one answer for both questions: implicits. Why? Implicits are very simple and yet very powerful concept but with great power comes great responsibility. They can be used for implicit parameters, […]

Read more