Data Pipelines with Apache Beam

Preface Many data pipeline frameworks offer very similar functionality. With this in mind, Google developed a unified data pipeline framework under the name Cloud Dataflow SDK. This framework was later donated to the Apache Software Foundation. It was then named Apache Beam. Let’s look at the following figure to understand Apache Beam better. Source: https://cloud.google.com/blog/products/gcp/dataflow-and-open-source-proposal-to-join-the-apache-incubator We create a single pipeline, which then allows us to do either Batch Processing or Stream Processing. The processing is realized with a Runtime of our choice. We can test our code locally with Direct Pipeline or send it to a cloud service such as Google Cloud Dataflow. This makes Apache Beam a very interesting framework to try. ...

Teamfight Tactics Monte-Carlo-Simulation with Flask and Heroku

Teamfight Tactics is a statistics-based video game by Riot Games. Probabilities for complex scenarios are hard to approximate, therefore I created a simulation-based solution to help players make decisions.

Making Meta's RocksDB distributed with UCX to support Infiniband Hardware

RocksDB is a performant embedded Key-Value Store. As part of my Master's Thesis, I created a distributed version of it. This article goes over the key concepts and what I consider the most interesting parts.

Source: https://www.mercedesamgf1.com/de/wallpaper/

Lewis Hamilton 2017-2021

An analysis of various race statistics in regard to Lewis Hamilton. Done exclusively as a preparation for job interviews. This post does not offer any real insights.

Creating a Multi-Process Python Bot to serve Pixiv Images with discord.py and SQLite

Preface Discord is a popular chatting platform and there are plenty of bots available, which can entertain you and your friends. There are bots out there, that post images from for example Reddit. The bots do not post the image itself, instead, they post the link to the image, and Discord itself will embed the image into the chat. Pixiv is a Japanese platform on which artists share their creations. At the time I created this bot, embedding images from Pixiv was blocked. One could only embed posts, which does not look as good as only embedding the actual image. In the following image, you can see the proper way on top, and the not wanted way on the bottom. ...

OpenJDK's Project Loom: User-Level Threads in Java

Preface Nowadays it is universally known that a Java thread corresponds to a Kernel-Level Thread when looking at it in a very simplified matter. Project Loom is trying to introduce User-Level Threads to the Java ecosystem. They call these User-Level Threads Virtual Threads. Virtual Threads are supposed to increase performance and be more resource efficient than just using Kernel-Level Threads. It is planned, that Virtual Threads become a drop-in replacement, which would be a free performance boost for any multithreaded Java application. This is particularly interesting because many Big-Data-Frameworks such as Hadoop or Spark are written in Java. ...

Building a Calorietracker WebApp with Flutter and Firebase

Preface Flutter is first and foremost a Framework to create cross-platform applications, but it can also be used to create WebApps. Since I don’t have any experience with the three big Web Frameworks: VueJS, React, and Angular, I thought Flutter would be a good fit for me. It also uses Dart instead Javascript. My initial impression was, that this is also an improvement. Since I have not made any proper experiences with other Web Frameworks or Javascript, I won’t be able to compare those to Flutter/Dart in this article. However, I will still talk about my impressions as someone with very little experience in Frontend. My interest is in the Backend and in that regard, we will look at Firebase and how Flutter integrates with it. ...