Data Pipelines with Apache Beam

Preface Many data pipeline frameworks offer very similar functionality. With this in mind, Google developed a unified data pipeline framework under the name Cloud Dataflow SDK. This framework was later donated to the Apache Software Foundation. It was then named Apache Beam. Let’s look at the following figure to understand Apache Beam better. Source: https://cloud.google.com/blog/products/gcp/dataflow-and-open-source-proposal-to-join-the-apache-incubator We create a single pipeline, which then allows us to do either Batch Processing or Stream Processing....

July 17, 2022 · 11 min · 2222 words · Hutan

Teamfight Tactics Monte-Carlo-Simulation with Flask and Heroku

Preface This preface is for people who don’t know Teamfight Tactics. Teamfight Tactics is a board game, in which you buy champions from a shop and place them on the board. There are 5 different tiers of champions: common, uncommon, rare, epic, and legendary. The champions of each tier cost 1, 2, 3, 4, and 5 gold respectively. With your board, you fight other people and the loser of the fight loses Health Points....

July 12, 2022 · 13 min · 2710 words · Hutan

Making Meta's RocksDB distributed with UCX to support Infiniband Hardware

Preface This project is my first C++ project and it is my Master’s Thesis. I had 6 months to create this project. However, at least 2 months of this time was spent writing the actual thesis and not coding. About 3 months were spent coding with an additional month for benchmarking. RocksDB RocksDB is open-source and successfully used in many projects. For example in Apache Flink, state is maintained during calculations with RocksDB....

July 7, 2022 · 24 min · 4972 words · Hutan
Source: https://www.mercedesamgf1.com/de/wallpaper/

Lewis Hamilton 2017-2021

Preface The 2021 season of Formula 1 is the first one I watched from start to finish and I am a fan of both Hamilton and Bottas ever since. I just finished my studies and I am about to have my first job interviews. To make the preparation for those interviews more interesting, I decided to crunch some numbers of the seasons during which Bottas was Hamilton’s teammate. These numbers aren’t particularly insightful on their own, but they can be used as a starting point for further deeper analysis in the future....

June 23, 2022 · 9 min · 1728 words · Hutan

Creating a Multi-Process Python Bot to serve Pixiv Images with discord.py and SQLite

Preface Discord is a popular chatting platform and there are plenty of bots available, which can entertain you and your friends. There are bots out there, that post images from for example Reddit. The bots do not post the image itself, instead, they post the link to the image, and Discord itself will embed the image into the chat. Pixiv is a Japanese platform on which artists share their creations. At the time I created this bot, embedding images from Pixiv was blocked....

June 3, 2022 · 9 min · 1723 words · Hutan