Helping An Online Gaming Company Migrate From Postgres To ClickHouse Cloud

Our client are a provider of software technology to the sports betting and gaming industries. They provide a white-labelled platform to a number of global partners who in turn offer entertainment products to end consumers.

Historically, the platform has used Postgres for analytics. However, as is often the case, their application has begun to slow down as they have captured more data and are carrying out more sophisticated analytics over that data.

Our client have a big data workload, consisting of ~300 million events per day which should be analysed in real time. They decided to use ClickHouse for its high performance when ingesting and analysing large data sets, and chose to use ClickHouse Cloud for fast time to market and low operational overhead.

The customer were experiencing the following challenges prior to the engagement:

• Postgres was slowing down under the load of high amounts of data and increasingly complex queries.
• Their application user experience was being impacted as data volumes grow.
• They wanted to provide more real-time analytics to their clients and partners over time as opposed to end of day analytics.
• They wanted to keep load off of their transactional database to maximise performance.
• They were lacking skills and experience with ClickHouse and ClickHouse Cloud.

We took the following approach to this project:

• Worked with the client to design and build a solution based on ClickHouse Cloud.
• It makes use of ClickPipes for Postgres CDC in order to integrate large data volumes from their Postgres database.
• When the data hits ClickHouse, it has to be broken into materialized views which pre-calculate the results of common queries. This was challenging because they have many billions of rows which are captured as people interact with their games.
• The system includes financial transaction metrics which must be accurate and retained for regulatory and compliance purposes.
• Introduced a single tenant solution where data is sharded into separate partner specific databases from a multi-tenant source dataset.