Why we need Data Product Management

Datenprodukte – Perspektiven und Praxis | Webinar | Alexander Weiner, Dr. Marc Feldmann

Challenge 1: Lack of user orientation

Without use, there is no benefit. In order to use data effectively, a user group must first be defined, and the data prepared accordingly. In concrete terms, this means preparing data in such a way that it corresponds to the intended use of the users. This applies to all data products, from raw and metadata to processed data and more complex products such as code assets, decision support dashboards, or AI and machine learning models.

Data value creation therefore begins with understanding the needs of a user group as best as possible and defining homogeneous and stable requirements. However, too little attention is often paid to understanding requirements through user research with direct user participation. As a result, data is maintained, processed, and made available, but it fails to meet user needs. In extreme cases, the users are not even known.

If requirements are not met or only partially met, users often resort to manual or outdated processes despite the new data product. Consistent data product management avoids these pitfalls by focusing on added value and user orientation.

Challenge 2: Lack of scalability throughout the lifecycle

Data value creation is a complex process that requires company-wide accepted and proven processes. Such processes must be scalable across a wide range of possible use cases. In practice, however, this scalability is often lacking.

This is because many use cases often fail to make it from the project stage to the product stage. Wish lists and requirements can be formulated, but a lack of user focus and validation steps to ensure added value often lead to costly mistakes in product development. As a result, solutions are developed at great expense, only to be abandoned before they are put into production.

Furthermore, further development and maintenance are neglected, especially in the later stages of the lifecycle. This is particularly frustrating when an existing “productive” data product is intended to serve as the basis for further data products/applications. This also applies, for example, to current approaches in the field of generative AI. Here, retrieval augmented generation (RAG) opens up new possibilities in knowledge management. However, even the best search function is useless if the underlying documents and data are not adequately maintained and are therefore out of date.

To ensure that these processes function independently of the requirements of a specific use case, clearly defined standards, roles, and process steps must be implemented within the organization. Only then can an organization benefit from important scale and learning curve effects. Data product management throughout the life cycle phases provides the basis for this.

Challenge 3: Lack of reusability

As explained in the second challenge, data products can also build on each other. The value of the underlying data products multiplies when data solutions that have already been developed are reused and combined across departments and teams.

However, ensuring this reusability is the next challenge that many organizations are unable to meet.

One reason for this is that it requires data (analytics) managers to change their perspective: away from thinking in terms of individual use cases and toward systemic thinking in terms of product portfolios. This change in perspective is important because it is the only way to ensure effective management of dependencies and synergies between data solutions – and the only way to create valuable, interoperable, and scalable data solutions.

A major obstacle here is creating transparency. Many organizations have built up data catalogs over the past few years. In reality, however, these catalogs struggle with various problems, such as high maintenance costs or incomplete records. From a portfolio perspective, prioritization is therefore also necessary in order to build a data map that focuses on the relevant data products.

Conclusion

The aspects described above are not new in the world of data, but have been the focus of various data management initiatives for many years. However, these challenges have not been adequately addressed to date. Even today, data value chains are not yet fully traceable, metadata is not available or maintained, and centralized solutions are not user-oriented enough.

Data product management opens up promising prospects for organizations to tackle the challenges mentioned above and significantly improve their own data value creation. In data product management, data and data solutions are consistently understood and managed as products. This represents a departure from the concept of data assets that are hoarded like treasure (and not shared!). This is also clear from our definition of a data product: The user-centric integration of data and information technology to support tasks in a way that creates real added value.