FOR SHARE - JULY SNAPSHOT - Marketplaces & Usage Accounting - Functional description
Early draft
Please be aware that the Data Spaces Blueprint content shared in these pages are a very early draft published on 2023-07-01. The current draft is incomplete and the content might still change.
SAVE-THE-DATE 01-10/09/2023: We will welcome your feedbacks to future improve the Data Spaces Blueprint during the Public consultation that will open on September the 1st 2023 until September the 10th. Please mark these dates in your calendar and get ready!
1. Introduction
This document aims to guide the expert groups through the process of writing functional specification for data space building blocks. A functional specification is a description of the functionality of a given building block, i.e. it describes how a building block should behave on a high-level. In the context of DSSC, functional specification serves as the Data Spaces Blueprint for creating interoperable and reusable components that can be used to build data spaces.
This document provides templates that the expert groups can use to create a functional specification for data space building blocks. By following the guidelines outlined in this document and using these templates, in particular, WP5 of the DSSC will be able to create clear and concise input for the Data Spaces Blueprint that captures the requirements and design for the Blueprint.
2. Functional Specification Template Building Blocks
2.1. Front matter
To ensure consistency, it is recommended to reuse the existing DSSC word template for papers with covers, imprint, and content[1]. However, it should be extended to include important details such as the name/number of the expert group, the names of the reviewers, the date it was created, and the date it was last updated. This will provide a structured format for the front matter of the document.
Title: [Title of the document]
Publisher: [Name of the publisher]
Copyright: [Copyright information]
Consortium: [Name of the consortium]
Contact: [Contact information]
Author(s): [Name(s) of the author(s)]
Expert Group: [Name of the expert group]
Reviewers: [Name(s) of the reviewers]
Created On: [Date the document was created]
Last Updated: [Date the document was last updated]
2.2. Introduction
A brief introduction of the building block
FROM THE SCOPE:
The overarching objective of this building block is to generate value out of sharing of data among participants in the data space, through marketplace capabilities delivered on top of the catalogue developed in previous building blocks of this pillar. In a nutshell, the marketplace helps customers to discover available data products and support the procurement process, as well as monetization of those data products. Notice that the general objective of generation of value out of data sharing can include other mechanisms beyond data monetization through marketplace, which should correspond to what is identified in the business building block of the data space. Therefore, this building block will also provide the capabilities to technically implement those mechanisms.
“Marketplace and Ussage Accounting” building block materializes one of the options to create value out of data, which is data trading and monetization through or different marketplaces, and the corresponding ussage accounting to track all transactions to enable revenue settlement required for billing and payment to involved participants. This building block relies on the data and services catalogue implemented in the building block “Publication and Discovery”, and by using the different features provided by other building blocks (interoperability, identity management, governance, etc ..) exploits those catalogues and facilitates the connection between providers and users of data products, the negotiation between them, the provisioning of the service, etc .. during the duration of the contract, and the storage of all the information generated during the process.
[Illustrative image or figure describing the building block]
Interrelationships: An overview of how the building block relates to other building blocks in the OpenDEI DSSC framework. This should include any dependencies, shared functionalities, and potential conflicts with other building oit.
2.3. Purpose of the Building Block
This section should provide an introduction to the technical specification of the data space building block. A technical specification is a detailed and comprehensive document that describes all technical produces related to the building block. The introduction should explains its purpose, use cases, and intended functionalities. In addition, the introduction of the document should also contain a summary of the problem, including the context, suggested solution and the key stakeholder.
A very brief introduction (max 5 sentences): Provide an introduction of the technical specification for a technical data space building block by explaining the context in which the building block will be used.
This building block materializes the creation of value out of the sharing of data among participants in the data space. Depending on the type of data space, there will be different mechanisms to generate value for the participants, that will be defined in the business building block and implemented through the technical building blocks, including this one. At this stage, the focus of this building block is to implement mechanisms for data trading and monetization, and associated functionalities to manage, monitor and audit the transactions (and everything needed to ensure the smoothnes of the provider and the user journey), through one or several marketplaces, or making use of corresponding data connectors, on top of the catalogue defined in the previous builidng block, so participants can trade with data products, make transactions and monetize them.
Purpose and problem: Explain the purpose of the building block and how it relates to the specific problem or need that it is intended to address.
To bring together data products providers and users, negotiate conditions for the delivery and use of the products, monitor the process and store all the relevant information
Use cases: Describe the potential use cases of the building block. It involves identifying and explain the various ways in which the building block can be used to solve specific problems or achieve certain goals.
Potential scenarios:
Marketplace (centralized or distributed)
Data connector (fully decentralized, p2p scenario)
Others??
The use cases can also come for the provider and user journeys:
Data product providers:
Subscription: this stage includes registration as a provider, registration of the specification of the data product and verification of compliance with criteria and standards of the data space, product catalogue and marketplace.
Reference: the service provider might establish visibility rules and/or update the product specifications and offerings.
Sell: the product is procured by a customer and a product order is created
Follow: to monitor consumption, provide after-sale support and leverage the experience to innovate and continuously strengthen the product offering.
Data product users:
Discover: Customers use available search and filter functionalities to discover certain service offerings.
Select: Customers select certain product offerings on the marketplace.
Contract: This stage contains the checkout process of the cart, a verification and validation and also signing the contract. Payment providers might be already involved in order that customers can provide their payment details.
Consume & Pay: Customers access the service or product and pay for the usage. The access and authorization to the actual service is part of another building block.
Advocate: This stage comprises different activities like sharing of feedback about the product/service, after sale services or promotion activities.
Suggested solution: Provide a suggested solution or approach to address the problem. It is important to be clear and concise, and to provide enough details to allow key stakeholders to understand how the approach of the solution will address the problem.
Key stakeholders: Identify all stakeholders involved in the development, implementation, and usage of the building block to ensure their needs and expectations are taken into account:
Data product providers, offer data products that can be consumed by receivers or users, including services providing access to data, application services that can gather and process data, infrastructure services supporting the deployment and execution of the previous ones, but also potentially third parties services (certification, audit, standardization, external billing and payment, …). This type of stakeholder can play the role of both data provider and data intermediary, as they are described in the DSSC glossary.
Data product receivers or users, including actors, looking for data products, and accessing the marketplace from multiple ways: through the catalogue itself, or through the different associated marketplaces. This stakeholder plays the role of data users or data receivers, as they are considered in DSSC Glossary.
Marketplace governance body, to ensure that the operation of the marketplace is run according to the data space governance framework. This should consider, at least, liability of published assets, QoS (Quality of Service) & SLA (Service Level Agreement), compliance with regulations (e.g.: GDPR https://eur-lex.europa.eu/eli/reg/2016/679/oj and AI act https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex:52021PC0206 ), and other ethical aspects. This body should be part or aligned with the data space governance body, as it is reflected in the DSSC Glossary.
Operators of the infrastructure, who will ensure the proper functioning of the technical infrastructure, including security aspects. This role corresponds to the data space enabler, as they are considered in the DSSC Glossary.
Marketplace providers, actors that offer marketplace capabilities on top of the data product catalogue provided by the previous building block. Notice that some data products may be made visible by multiple marketplace providers. On the other hand, a given marketplace provider may only offer a subset of the products listed in the shared catalogue. This role could be seen as a different type of data space enabler, as they are considered in the DSSC Glossary.
When to use the building block? When not?
|
Scenario 1 |
Scenario 2 |
Scenario 3 |
Some attributes |
|
|
Not recommended |
More attributes |
Mandatory |
Use this building block is recommended |
|
Even more attributes |
|
|
Not recommended |
2.4. Definitions and Vocabulary
In this section, all terms and vocabulary used throughout the technical specification are defined and explained, which is a subset of the DSSC Glossary and thus fully aligned with the content of the DSSC Glossary.
For terms that are included in the DSSC Glossary, please refer to that section for their definitions. If new terms are introduced, they should be provided with a criterion as its description. The idea is that different people (in the context of DSSC and data space initiatives) will make the same judgments when they use criteria in some (relevant) context - i.e. they would agree on whether or not something is (not) an instance (example) of the term. By following this process, it is also possible to propose generic terms to the DSSC Glossary that were not initially included. The DSSC Glossary team will analize and approve/reject the proposed terms.
Term |
Description |
|
Marketplace |
Marketplaces are B2B and B2C platforms hosting product listings from one or multiple sellers, facilitating them to meet customers and conduct business with them. A marketplace, as opposed to a catalogue, also exposes some kind of online transaction capabilities. A marketplace listing products from one single seller (who, usually, also governs the platform) is commonly referred to as an eCommerce platform. There can be different kind of marketplaces:
|
|
Data services |
Services that provide access to data |
|
Data products |
Provided and used by participants in the data space, those products comply with certain Product Specifications. A Product Specification has name, version, description, and other attributes including the set of characteristics that complying products will have. According to its Product Specifications, a product comprises one or more data services and requires a number of resources for functioning |
|
Application (app) services |
Services that gather and process data, and typically deliver data results |
|
Cloud/Edge infrastructure services |
Services supporting the deployment and execution of data/app services |
|
Service provider |
Service providers (IaaS, platform and app/data service providers) are organizations (public institutions or private companies) that offer service products that can be consumed by customers, such as other organizations or individuals. |
|
Service customers |
Individuals, business consumers or public entities procuring services and resources |
|
Marketplace operator |
Operator run a marketplace with any selection of possible features (e.g. catalogue, ordering, billing, payment, etc..). |
|
Billing and Payment service providers |
Work as gateways that rely on transaction logs to provide secure, transparent and trustful billing to consumers and payment to providers. |
|
Party |
An individual or an organisation |
|
2.5. Conceptual Model
This section should provide a high-level conceptual model of the data space building block, including all the concepts and relations between them. A conceptual model can be used to clarify and communicate these technical specifications. It establishes a shared understanding among key stakeholders. The conceptual model can be seen as a highlighted subset of the DSSC conceptual model, optionally enriched with BB-specific concepts. It may function as input for the conceptual model expert group.
A conceptual model represents (a part of) a single mental model, and typically consists of specifications for:
concepts, i.e. (named) classes of entities that have similar properties, e.g. 'data space', 'party', 'person', etc.
properties, i.e. (named) characteristics that can be attributed a value to every instance of a specific concept. For example, the concept 'person' can have a property named 'gender', with values such as 'M' (for male) or 'F' (female).
relations between such concepts, i.e. meaningful (characteristic) links that can exist between (instances of) these concepts, e.g. '[party] is the governance authority of [data space]' or '[person] is the mother of [person]'.
constraints, i.e. logical expressions composed of concepts, their properties and/or relations, that are expected to be (made) true. For example:
o every person has precisely one other person as its mother, and that person has gender 'F' (this is always true);
o every data space must have precisely one party that is its governance authority (if this is not true, it must be made true).
To define a concept, its properties, or a relation between concepts, a criterion should be provided as a description in the glossary table.
**[Illustrative image or figure describing the Conceptual Model]**
Conceptual model of the project (ongoing work): I think everyting is there, except the part of usage account
2.6. Functionality
This section should provide a detailed description of the functions that the data space building block should acquire, including its intended features and capabilities. It must clearly describe what the building block is expected to do and its anticipated input/output interactions with other building blocks.
To ensure that different components can work together seamlessly, it is important to clearly define any assumptions being made about the interactions with other building blocks. These assumptions should be documented and clearly communicated to all stakeholders involved in the development and implementation of both building blocks. Interactions with other building blocks can be thought of as synchronization points, where data or functionality is exchanged between components of a data space.
** added functionality content **
In the following, different management capabilities are described that a marketplace should provide in order to be able to manage the entire lifecycle of different elements of a marketplace like product catalogs, order processes or parties such as individuals and organisations.
Catalog management
A marketplace should provide a catalog management allowing the management of the entire lifecycle of elements in a catalog, the consultation of catalog elements during several processes such as ordering process, campaign management or sales management.
It should provide a standardised solution for rapidly adding products to an existing Catalog. It brings the capability for Service Providers to directly feed systems with the technical description of the products they propose to them.
The catalog management should provide elements to represent offerings on products that are orderable from the provider. These elements include the specification of the products, which is a detailed description of some of the attributes that will characterise products created when a product offering is successfully ordered (procured). It should also allow to associate multiple pricing models to a product offering and to add other characteristics of the product that gets created when a product offering is procured.
The catalog management should also allow to create collections of product offerings intended for a specific distribution channel and market segment and to group product offerings, service and resource candidates by categories. Categories can contain other categories and/or product offerings, resource or service candidates.
There should be also mechanisms that allow to retrieve product inventory information such as creation, partial or full update and retrieval of the representation of a product in the inventory. This should consist of a simple set of operations that interact with Inventory systems in a consistent manner. Usually, a product is created and modified following a product order but could be also ‘directly’ modified for administrative reasons. It also allows the notification of events related to the product lifecycle.
Besides the management of products and their related entities, there should be also a catalog management for resources, which are are physical or non-physical components (or some combination of these) within an enterprise's infrastructure or inventory, and services, represented by service specifications made available through service candidates that an organisation provides to the consumers (internal consumers like its employees or B2B customers or B2C customers)
Order management
The order management should provide a standardised mechanism for placing orders for products, resources and services with all of the necessary order parameters. It consists of a simple set of operations that interact with CRM/Order negotiation systems in a consistent manner. A product order is created based on a product offering that is defined in a catalog and it is a type of order which can be used to place an order between a party (customer or other interested party) and a service provider. The product offering comprises the specification of the product or bundle of products that will be created when the offering is procured and includes characteristics such as pricing, product options and target market.
Contract duration management
There should be standardised mechanisms for managing agreements, especially in the context of partnerships between partners. Here, an agreement represents a contract or arrangement, either written or verbal and sometimes enforceable by law, such as a service level agreement or a customer price agreement. An agreement involves a number of other business entities, such as products, services, and resources and/or their specifications.
Usage Management
There should be standardised mechanisms for usage management such as creation, update, retrieval, import and export of a collection of usages. These manage both rated and non-rated usage. For example, it allows a service provider to 1) retrieve usage generated by a partner service platform in order to rate it and 2) to provide rated usage to a partner for consumption follow up purposes. Here, usage can be described as an occurrence of employing a Product, Service, or Resource for its intended purpose, which is of interest to the business and can have charges applied to it. It should provide a detailed description of a usage event that are of interest to the business and can have charges applied to it.
Party and account management
Parties can be an individual or an organisation that has any kind of relationship with the enterprise.
An individual represents a single human being. The individual can be a customer, an employee or any other person that the organisation needs to store information about.
An organisation represents a group of people identified by shared interests or purpose. Examples include business, department and enterprise. Because of the complex nature of many businesses, both organisations and organisation units are represented by the same data.
Standardised mechanisms should allow to perform the management of parties and their roles such as creation, update, retrieval, deletion and notification of events.
In addition, there should be standardised mechanisms for the management of billing and settlement accounts, as well as for financial accounting (account receivable) either in B2B or B2B2C contexts. A billing account is used for billing purposes and it includes a description of the bill structure (frequency, presentation media, format and so on), whereas a settlement account is used for settlement purposes. A financial account is an account of money owed by a party to another entity in exchange for goods or services that have been delivered or used and aggregates the amounts of one or more party accounts (billing or settlement) owned by a given party.
FROM SCOPE (as proposed by Mike / TNO):
This is a nice overview of steps for the customers. However, I am missing the steps for the ‘rainy day scenario’. What if something goes wrong? E.g. escalation methods, refunds, warranty claims, DOA claims, product not arrived,
The purchase to pay processes have been standarized for a long time in OASIS UBL. Why not use these ‘steps’: Quotation, Order, OrderStatus, Despatch Advice, ….
OASIS Universal Business Language (UBL) TC | OASIS (oasis-open.org) UBL, the Universal Business Language, defines a royalty-free library of standard XML business documents supporting digitization of the commercial and logistical processes for domestic and international supply chains such as procurement, purchasing, transport, logistics, intermodal freight management, and other supply chain management functions.
** end added content **
According to providers and customers journeys defined in the previous section, this building block should offer the following basic functionalities (to be further developed during the functional specifications phase):
Catalogue(s) management: this functionality will provide tools to manage the existing catalogues from the different marketplaces, not considered in previous blocks.
Customized publication and discovery of services, allowing providers to establish visibility rules and/or update the product specifications and offerings, and customers to use search and filter functionalities to discover certain service offerings, and select certain product offerings on the marketplace
Product order management
Contract duration management, including the checkout process of the cart, a verification and validation and also signature of the contract and procurement of the product.
Payment for the usage of the service,
Data transactions / usage log / accounting (Transactions Ledger): to storage information associated with services specifications, services offerings, service orders, etc … The usage log will typically be used to calculate how much can be charged to customers and paid to providers or conduct audit logging for data transactions including provenance tracking. Additionally, it will be used by providers to monitor the service consumption by reporting and analytics features.
Aftersales: allowing sharing of feedback about the product/service, after sale services or promotion activities.
In this section, a table is used to define the details of each function within a data space building block. Each row of the table represents a single function, and the columns represent various aspects of the function, such as its description, interactions with other building blocks, constraints or limitations, and dependencies.
UC-1 |
<Use case name> |
Primary Actor(s) |
< primary actors that participate in this use case> |
Trigger |
<Condition/action that initiates/starts the use-case> |
Pre-conditions |
<Condition assumed to be true before the first step> |
Post-conditions |
<Condition after the use case is successfully executed > |
Main Success Scenario |
Make sure GOAL-ACHIEVED> |
Extensions |
If Condition, then Alternative Steps <List any extended steps/ scenarios that occur, other than the main success scenario.> |
Dependencies |
<relations to other building blocks> |
Special Requirements |
<Any system related special requirements needed to fulfill the use case> |
Open Questions |
<Notes and questions> |
2.7. Open Challenges – Future Roadmap
This section highlights any open challenges or future directions for the data space building block, including potential improvements. Potential challenges for a building block may arise in its interactions with other building blocks, making compliance a crucial consideration. This section should provide an overview of potential challenges and opportunities for future development of the building block.
2.8. Conclusions
Short statements about the current state-of-the-art, how this matches with the defined requirements and functionality (gap analysis), and some final remarks.
CONTENT TO BE REUSED
From DSBA Tech Convergence document: https://data-spaces-business-alliance.eu/wp-content/uploads/dlm_uploads/Data-Spaces-Business-Alliance-Technical-Convergence-V2.pdf
As reference and source of insipration, I copy here the part of the conceptual model from the DSBA convergence paper related to this aspect:
It is true that the DSBA convergence paper provides a lot of information, but in my opinion some of it falls more under “functional specifications” than under “scope”. I propose to focus on the scope, by defining (in principle): (1) Set-up the scene & boundaries, (2) objectives of the BB, (3) Units / sub-blocks / services, and (4) relations with other BBs:Daniel Alonso (Unlicensed) ok, but here it is just a two pages doc, so we need to make something short and then explain the rest in the Functional and Technical specifications, I agree
Set-up the scene & boundaries
What this BB is / is not about (for me, this is the actual purpose of defining the scope of the BB …)
What are functional specs (depth & breadth) → it will be useful for the next step: what we understand by functional specs
What are tech specs (depth & breadth) → idem previous point
Objectives / goals of this BB
For participants in the data space, to create value (just monetization?) out of sharing data (and other data driven assets??) …
Others??
Units / sub-blocks / services
Mandatories:
Federated catalogue (if not addressed in other BBs):
Datasets
Services (see last comment, from Mark (EGI / GATE) yesterday …)
Other (data driven) assets (??) (e.g. courses, etc??)
Marketplace(s) -> should we include the word “data”? (data driven marketplace): otherwise, we are not emphasizing that the marketplace is benefitting of being built on top of a marketplace
As added value, could the marketplace include potential combinations: e.g. what services can consume what data, what courses could complement the use of what services, … ? (as it also considered in DOME, last page …)
Governance (not sure if this should be part of the BB on governance … or the catalogue / marketplace should have a dedicated sub-block, since more stakeholders are now considered … Anyhow, it should be considered and linked with the governance BB):
Who is publishing what, under what conditions … who makes the decision and is finally responsible for what is published
Personal data (compliant with GDPR) and sensitive data / ethics … should be enforced by the “governance” and “legal” BBs.
Risk services / apps (according to AI act)
Who is being liable for what is published in the catalogue / marketplace?
Others aspects …
Log / audit -> (transactions ledger)
Charging / billing / payment
Optional
Federated marketplaces (DOME concept) → not considered in the conceptual model figure of DS in DSBA, altough it is in the text …
Relations with other BBs (I think this point is extremely important in this BB, since creating value is the ultimate objective of the DS, and all the rest BBs shoule be aimed to this … )
With those BBs from data value creation pillar (description and publishing of data / services), and the rest of technical BBs.
Governance of the data space (governance of the marketplace has to be aligned)
Business model of the data space
To be considered: it was raised in the technical TG (by Mark EGI / GATE project) the need to include as mandatory “access to data” services, either as a BB, sub-block, etc … in the blueprint. The typical steps of the data value chain (access, acquisition, cleaning, processing …) are not considered in the blueprint per se, either because we consider that the data space is built on top of a platform that alreay deals with those aspects, or because those services can be offered / published from the providers to the federated catalogue / marketplace (DOME model, etc … cloud providers on top of IaaS, …) and offered to users that want to access, process, .. etc .. to specific datasets. Anyway, something to be considered and discussed.