Category: Technology

Deploying ETL on Amazon EMR

Amazon offers a lot of cloud-based services hence getting started with AWS can be tough. Which one to use? Services usually do not have expressive names and one might get even more confused when it comes to combining multiple services.

The idea was to deploy a Spring Boot application performing ETL (extract, transform, load). Any application can be deployed using EC2. But there is one special thing: it is a Spark application. Therefore we opted for EMR.

Read more...

Dependency inversion: how to mock time in tests?

Time is an integral part of our lives — the same for the applications we develop. From the software engineering perspective, time is just another dependency which we introduce to our systems. And that’s OK. The situation might start to be problematic when some parts of the system are more dependent on time. How to test this component? How to mock the time?

In this article, I’m going to show you our approach to develop and test components which depend on time. We’ll get rid of time-dependency by creating a proper abstraction and applying the dependency inversion principle. Ultimately, we’ll create a mechanism to mock time in tests. Ready?

Read more...

Real time chat application on Vue.js and Socket.io

A couple of weeks ago, I was asked to create a simple communicator for one of our clients. I had to dig deeper into the topic and I decided to create a very basic proof of concept, without all the shiny features, a straightforward chat application that implements “one-to-many” communication. I wanted to use technology that I already knew, so the client application was created in Vue.js using Vue-cli 3, and the server was created in Node.js. For development purposes, I used the Express web server, which is flexible and fast, just enough for MVP. What I’d like to show you is how fast we can achieve this goal in terms of configuration and coding.

Read more...

Machine Learining PoC: Keras vs. PyTorch vs. SciKit learn

Recently we’ve started working on an health & wellbeing app whose aim is to increase the overall quality of life of individuals. In the app, a Machine Learning module is applied to help understand the factors impacting the quality of life.

To build the module we used a large set of data from ca. 2500 real persons, who completed a complex questionnaire about their habits, behaviours and conditions. An additional question was asked about the perceived overall quality of life.

Read more...

UX versus UI. Do UX designers really exist?

User Experience (UX) and User Interface (UI) are among the most complicated, misused and misinterpreted terms in software development. In the last few years as an industry, we’ve been focusing on responsive and intuitive designs of interfaces in the contexts far beyond mobile apps. Non-responsive and unintuitive designs have been rightly ditched. That is why good UX is so important. The product must be well-designed if it is to attract users. In addition, to satisfy the user, a friendly and simple look is essential.

The terms UX and UI designs often go hand in hand, and are often confused. So what do they have in common and what are the differences? In both cases the aim is the same – to help the product succeed. But can it be achieved by focusing only on one of the areas?

Read more...

ThreadLocal and ExecutorService

Recently I have been working on some code in which I was using ThreadLocal variables. This give me possibility to chew over the ThreadLocal variables concept again. The basic idea of this approach is quite simple. Every thread has its own copy of a ThreadLocalMap. When you ask for the value of the ThreadLocal variable it will search for this value in the ThreadLocalMap variable instance of the current thread.

Read more...

Legacy code and faster automated tests

The larger a project, the harder it is to test the code, and with most of the tests being functional we have to face the problem: we finalize the task, the user story, or close the entire sprint and we have to wait for the tests for hours. This process can be vastly accelerated, even in legacy applications from the era of the PHP 4.

Above all, it is worth introducing unit tests into the new code extract dependencies outside the code and test what is actually being developed. The tests are supposed to check our logical code, not the library or ORM. The introduction of unit tests is, however, a long-term step that does not give immediate results, which is why I’d like to show you what can be done on top of that.

Firstly, it must be said that one could write a book about the subject and still barely scratch the surface, so I’ll keep this article very brief and only touch upon the practical aspects based on my own experience – assuming a more difficult scenario, when the project has a lot of tests using the MySQL database.

Read more...

We use cookies to enhance your user experience. By continuing to browse the site, you are agreeing to our use of cookies.
Read more