GKS Maturity Model

The Genomic Knowledge Standards work stream is developing semantic data exchange standards for federated genomic knowledge sharing. To address this, new technical specifications are required, such as the VRS standard, which must be developed and iterated upon through application across community implementations. This creates a tension between the need to create products with enough stability for initial community adoption, while ensuring that they can evolve with minimal disruption to interoperate smoothly across a diverse set of genomic knowledge resources. Mechanisms for communicating the stability, uptake, and development of technical specifications are therefore of paramount importance to addressing this balance.

A maturity model is a useful mechanism for communicating varying stability across product features (e.g. data classes or protocols) of a GKS standard. This is needed to help data producers at each stage of the adoption lifecycle decide on the appropriate time to engage and implement the standard. Product features that have progressed through the maturity model should have an associated progression of support from the GKS specification maintainers for message generation, translation, and validation tooling.

Here we define the maturity model and release process for developing and maintaining GKS standards, with the goal of enabling timely specification adoption by the community.

../_images/adoption_lifecycle.png

The Innovation Adoption Lifecycle.

The Innovation Adoption Lifecycle illustrates adoption rates (y-axis) for new technologies over time (x-axis). Innovators (leftmost on the time axis) are among the first to adopt a new technology, and laggards (rightmost) are among the last, reflecting the differing needs for innovation and stability by these community groups. Adopters in every category along the innovation adoption lifecycle benefit from communication about the maturity of technical specification components generated by the Genomic Knowledge Standards work stream. Communicating when a component is ready for implementation by groups along the innovation / stability spectrum is a primary goal of the maturity model, enabling adopters to engage at a time that is appropriate for their organizational needs.

Feature Maturity levels

It may be helpful to visualize the application of maturity levels by viewing the current Class Diagram.

../_images/maturity_levels.png

Product feature maturity level criteria and commitments.

Product feature maturity levels are to be reviewed and advanced by consensus among defined decision-makers following Work Stream and GA4GH processes, in consultation with the associated product group membership. Factors to be considered for product feature maturity advancement include the criteria specified in the above table, the degree of adoption observed in the community, feedback provided by adopters, and availability of specification maintainers to provide the level of support required.

Developing a Draft Product Feature

Decision-makers: Feature Developers, Product Leads

Criteria: Draft product feature development work should be based on real use cases across multiple environments (aligned with GA4GH Product Development 14.5). Requirements may result directly from a landscape analysis of the problem domain, or may emerge in the course of technical specification development. It is expected that the need for product features are first discussed in a community forum (e.g. GitHub Discussions, GKS Work Stream calls).

Process: Follow the GKS development-process. As part of this process, it is expected that consensus among the decision-makers was reached and major design decisions documented. Disagreements are resolved per Work Stream and GA4GH processes.

Advancing from Draft to Trial Use

Decision-makers: Feature Developers, Product Leads, Product Implementers

Criteria: Advancing a draft product feature to trial use should include at least two independent product implementers that commit to supporting the draft product feature once it has been advanced to trial use. At least one of these implementations must be open (aligned with GA4GH Product Development 14.8.3). Advancing a product feature to trial use also mandates a minor version increment at the next release. As part of this process, it is expected that consensus among the decision-makers was reached and major design decisions documented. Disagreement resolution is handled per Work Stream and GA4GH processes.

Process: A ballot release is created that describes draft models under evaluation for advancement to trial use. A survey is sent to all Product Implementers that have indicated they are implementing one or more features under evaluation for advance to Trial Use. This survey includes:

  1. Name of Product Implementer

  2. Selection of a previously described implementation

  3. If (or if multiple, which) product feature(s) are suitable for advance to Trial Use

  4. Comments on response (e.g. explicit endorsement or description of gaps)

There is a minimum 1-week review period for Product Implementers to respond, though this may be longer at the discretion of the product leads. More time for individual contributors may be permitted on request.

Advancing from Trial Use to Normative

Decision-makers: Feature Developers, Product Leads, Product Implementers, Work Stream Leads

Criteria: A normative model should have demonstrated interoperability of multiple data generation and data consumption implementations, and should include implementations beyond those used to advance a model to Trial Use. Advancing a product feature to normative also mandates a minor version increment at the next release. As part of this process, it is expected that consensus among the decision-makers was reached and major design decisions documented. Community consultation and disagreement resolution are handled per Work Stream and GA4GH processes.

Product Versioning and Releases

Versions are used to identify releases of the entire specification, not to individual product features. Technical specification development is intrinsically linked to policy surrounding major and minor version identification, which follow semantic versioning v2 practices for API versioning.

Versioning examples

Version syntax follows SemVer syntax. Examples of how product features at different maturity levels are applied to the SemVer major/minor/patch syntax as follows:

Major Version Increment

  • Backwards-incompatible changes to a normative product feature

  • Backwards-incompatible changes to property names of a previously-released normative data class

  • Backwards-incompatible changes to the definition of a previously-released normative data class

  • Backwards-incompatible changes to the digests of previously-released normative data class (as applicable)

  • Addition of required fields to previously-released normative data class

Minor Version Increment

  • Backwards-incompatible changes to a trial use product feature

  • Addition of optional fields to data models

  • Release of a new product feature at the trial use or normative level

  • Backwards-incompatible changes to property names of a previously-released trial use data class

  • Backwards-incompatible changes to the definition of a previously-released trial use data class

  • Backwards-incompatible changes to the digests of previously-released trial use data class (as applicable)

  • Addition of required fields to previously-released trial use data class

Patch Version Increment

  • A new product feature at the draft maturity level

  • Any changes made to draft product features

  • Addition of implementation guidance, tests, or other supporting product features that do not directly affect data compatibility

Versioning of approved GA4GH standards additionally follow the procedures for GA4GH Product Updates. Specifically, advancement of data classes to the trial use or normative levels must be accompanied by a minor release increment, and therefore may only be included in a release following an appropriate community and PRC consultation process (GA4GH Product Development 32).

Releases

In order to support continuous development of a technical specification, pre-release snapshots are allowed and must use the SemVer syntax for pre-releases. Pre-release snapshots may be created for purpose at any time by the product leads. Pre-release snapshots should use the following pre-release labels as version suffixes for the indicated purposes:

  • connect.<YYYY>-<MM>[.<N>] - for pre-releases to be evaluated at an upcoming GA4GH Connect meeting - date corresponds to the month when the start of GA4GH Connect occurs - expected 2x/yr - .<N> extension may be used, as needed, for clarifying updates

  • ballot.<YYYY>-<MM>[.<N>] - for community evaluation and review ahead of a minor or major version release - used for implementer review, product review committee, and steering committee ballots - N increments for successive rounds of review

  • snapshot.<YYYY>-<MM>[.<N>] - for use as needed for all other purposes - N increments for successive snapshots

These pre-release labels are appended to the major, minor, and patch components to create a pre-release version following the SemVer <MAJOR>.<MINOR>.<PATCH>-<LABEL> syntax. For example, a pre-release of VRS 2.0 for discussion at Spring 2024 Connect would have a version identifier like 2.0.0-connect.2024-04. Releases and pre-releases should use GitHub Releases for release packaging and tracking (see VRS releases).

Decision-maker roles

A role is assumed by a person in developing GKS technical specifications and other GA4GH products. There are several roles relevant to this document:

Feature Developers

Feature developers are members of a product group assigned to implement a product feature.

Product Implementers

Representatives of teams that will develop implementations. These typically include (but are not limited to) Driver Project representatives that have committed to developing the product as part of the GA4GH Product Development and Approval Process.

Product Leads

Designated leads of a product group, who are responsible for overseeing product development.

Work Stream Leads

These are domain experts assigned by the GA4GH to coordinate and lead the activities of a GA4GH Work Stream.