Executive Summary

• The customer is a well-established game publisher that partners with game developers and studios to deliver a player-first user experience through game technology.
• The customer had a lot of disparate data coming from different data sources that were not integrated.
• They wanted a customer data platform that allowed them to unify their data to leverage it in a meaningful way.
• Neal Analytics built a unified data model that gives the game publisher the ability to easily navigate through complex data and derive insights.
• The unified data model and analytics platform were built on Azure and support streaming data from games as well as data from external and legacy systems.
• The game publisher can now access unified customer metrics & predictive analytics to enhance marketing and player experience.

Introduction

Neal Analytics worked with a well-established game publisher that works with game studios and developers to help them throughout the entire lifecycle of a game – from development and successful launch to sustaining the game with live services.

They’re a full-service incubator for game developers and have a history of creating high-quality, player-driven games.

In order to maximize revenue and create the ultimate player journey, the company wanted more insight from their player data spread across multiple channels.

Challenge

The company’s data came from games that integrated with legacy systems as well as new systems. That caused a disparity in how data was stored and accessed.

Their data comprised of:

• Transactional data (databases and flat files)
• Event data (web logs and event producer)
• Streaming data (from devices and external sources like iTunes, Google, social media platforms, etc.)

Data coming from various sources posed the biggest challenge for this company in their quest to efficiently navigate their data to generate actionable insights.

Solution

Neal Analytics worked with the game publisher to create a unified data model. The solution was built in three stages:

1. Build a data platform to extract and store all incoming data
2. Develop the logic to take data from these raw formats and create a unified data model by understanding
   • Which datasets are relevant and needed?
   • Which datasets can be simplified or merged?
3. Set up the platform so that the company can pull integrated reports, dashboards, and analytics at set time intervals (hourly, weekly or monthly).

Functional Architecture

The solution developed by Neal Analytics can be broken down and understood from the following functional architecture:

To create a unified data model, Neal Analytics used multiple platforms to manage the various data sources.

Data Ingestion

Azure Event Hubs

Microsoft Azure Event Hubs is a fully managed, real-time data ingestion service that can constantly pull data from hundreds of thousands of sources. Neal Analytics used Event Hubs to stream, capture, and process streaming and event data from thousands of live events from a game into the platform.

The company’s partners provided connections to their games with Event Hubs, allowing Neal Analytics to capture all in-game user activity (such as each click a user made). This user activity (dataset) was streamed into Azure Blob storage.

Fivetran

Fivetran is an automated data integration tool that consists of fully automated data connectors to effortlessly centralize data from any source to any warehouse. Neal Analytics used Fivetran to ingest transactional data from social media platforms and other databases. Companies such as Facebook, Google, Apple iTunes, and more allow API-based connectors to pull data. Fivetran simplifies that by being able to manage all those connections from multiple sources in one place.

Unlike Event Hubs, Fivetran does not pull data in real-time. Instead, it pulls on an hourly or daily basis, depending on the source.  Fivetran can’t directly integrate to Blob storage, so it had to be connected to a database (Azure Synapse Analytics in this case) as a staging place. It can then push the data into Azure Blob storage from this database.

Data Storage and Management

Azure Blob Storage & Azure Data Factory

Azure Blob Storage is a service that stores large amounts of unstructured data. It stores all incoming data from various sources in a raw format.

To curate the raw data from Azure Blob Storage, Neal Analytics worked with the company to understand and identify core datasets for their analytics and reporting needs. In the process, over 150 tables were consolidated into 17 core tables, which were then used to create a unified data model.

The 17 core tables contained transactional data such as:

• What are the unique games?
• What are the unique players in a game?
• What are the in-game events?

The data from these core tables were then used to create a dataset that could look at information across multiple sources and merge it all into a “master table.”

For example, the company could use this master table to combine and reference data from multiple sources that stored information such as:

• Who are all the unique players in the game?
• When did they download the game?
• When did they install the game?
• What are all the unique IDs associated with a player from Zendesk, Facebook, iTunes, Google, etc.?

Using the master table built from relevant and core data, the company could create a unique user profile and leverage that to further customize the player’s journey in the app.

Microsoft Azure PlayFab

Neal Analytics used Microsoft Azure’s PlayFab to track players and games across all datasets. PlayFab integrates with various data sources and keeps a record of all the IDs that a player may have across platforms such as iTunes, Facebook, Google, etc.

PlayFab then maps these IDs for each player to a unique PlayFab ID, which acts as the master ID for tracking across datasets.

Game developers and publishers connect to a master platform to access player-related information linked to the PlayFab ID.

AppsFlyer

AppsFlyer is a mobile attrition and marketing analytics platform. Our solution primarily used AppsFlyer as an in-app activity monitoring service to provide insights and metrics into each player’s activity in the application.

Data Processing and Curation

Azure Databricks

Azure Databricks is a data analytics platform that leverages AI to support data science and machine learning operations (MLOps). The platform was used to develop transformations for the tables and reports according to the customer’s needs. Databricks was ideal for this solution thanks to its ability to enable large-scale data processing and standardized data transformation from unstructured data into the schema required for analysis. These features were especially useful for managing the vast quantities transactional, event, and streaming data to derive useful insights.

The new tables built-in Databricks were then pushed back to Blob Storage in a curated format which could be used for data modeling and reporting.

Data Modeling and Reporting

This solution utilized two reporting outputs from Azure Blob Storage.

1. Azure Blob Storage to Azure Data Explorer 

Azure Data Explorer (ADX) was a new tool when we worked with it. It was useful for the customer’s data due to dataset size, complexity, and storage requirements. ADX’s ability to rapidly reason over large datasets full of complex telemetry data was particularly valuable, as it allowed for complex calculations to be easily executed in ETL and saved out for analysis at scale. This, in turn, allowed our team to focus on developing advanced analytics and BI focused on the customer’s key use cases for increasing advertising revenue.   

2. Blob Storage to Machine Learning (ML), Advanced Analytics and Data Science

The next project stage was to create an advanced analytics and reporting platform for the company and its partners.

In order to run their own analytics on the platform (and increase value for the company’s customers who can utilize the platform), the game publisher wanted to create visual reports from data on:

• Customer lifetime value
• Player-specific deals and offers for in-app purchase

With machine learning, advanced analytics, and data science, the company can develop models to identify new ways to drive revenue and gain customer insight to further personalize marketing efforts.

Results

This game publisher needed a solution that could ingest and manage data from multiple sources in order to get a better understanding of their players and customers. They approached the problem optimally by:

1. Cleaning the data platform first to ensure it is scalable and unified
2. Then derive additional insights through machine learning and reporting

Neal Analytics worked with the company to build the end-to-end functional architecture, mapping disparate player data to create a simple, unified model to improve navigation and customer insights.

The unified model and data architecture empowered the company with a better view of their players based on demographics, transactional history, player IDs, and user activity data.

Our solution laid the foundation for the game publisher’s future data analytics, helping them achieve their goals of adding value to customers and maintain a player-first mentality.

For more on how Neal Analytics can build a holistic and unified view of your customers from data, check out our CustomerIQ Solution.