Skip to main content
x

INCF Best Practices Review Criteria

INCF Best Practices Review Criteria

Version 1.0
Approved, September 17, 2020
Authors:  INCF SBP Committee

Open

While many BPs may not be licensed, it is essential they are open. If they are not currently open, is it possible for INCF to help make them open?

1. Is the BP covered under an open license so that it is free to implement and reuse by all interested parties (including commercial)? (List of open source licenses)
2. What license is used?
3. Does the BP follow open development practices?
4. Where and how are the code/documents managed?
5. Any additional comments on the openness of the BP?

FAIR

Considers the BP from the point of view of some (not all) of the FAIR criteria (Wilkinson et al. 2016). Is the BP itself FAIR? Does it result in the production of FAIR research objects? Note that many of these may not apply. If so, leave blank or mark N/A.

1. BP recommends persistent identifiers where appropriate (F1))
2. BP promotes addition of rich metadata to research objects (F2)
3. BP promotes addition of appropriate PIDs to metadata (F3)
4. The BP recommends for authentication and authorization when required (A1.2)
5. BP promotes the use of vocabularies that follow the FAIR principles (I2)
6. BP promotes interoperability by recommending the use of relevant standards and documenting (I1)
7. BP promotes proper citation and attribution so its use can be documented and tracked (R1.2)
8. Does the BP have a clear versioning scheme and appropriate documentation?
9. Any additional comments on aspects of FAIR?

Design, Testing and implementation

These may not be as important for a BP as opposed to a standard, although proper design, testing, and implementation, as well as supporting tools may aid in adoption.

1. What is the technical expertise level required to implement this? Even if it is quite difficult, should it be implemented anyway?
2. Context:  Does BP provide an overview of why it was created, how it is organized and when it is appropriate for use?
3. Does the BP provide examples to aid in use?
4. What software artifacts/resources (files/scripts/libraries/tools) are available to support the BP? If there are not any, should there be?
      - Are the supporting software artifacts/resources covered under an open source license?
       - Are the supporting software resources well documented (documentation of I/O operations, programming interfaces, user interfaces, installation)?
       - Have the supporting software resources been deployed, is there any experience or references to their use by the community?
5. What is your overall assessment of the quality of the BP artifact?
6. Any additional comments on design, testing, and implementation?

Governance

Ongoing governance may be less important for endorsement, but documentation of how the BP was created is important.

1. What was the process for creating the BP? Is it meant to be the product of a community?  If so, how were the community involved?
2. Is the BP meant to be a living document, if so, how are updates handled?
3. If a governing model exists, Is the governing model document for maintenance and updates compatible with the INCF project governing model document and the open standards principles?
4. Is the BP actively supported by the community? If so, what is the evidence?
5. Does the BP provide tools for community feedback and support?
6. Any additional comments on governance?

Adoption and use

This category is different from standards, as best practices don’t necessarily need to be widely adopted or have a long history of use to quality as best practices. However, it is worth evaluating whether the  best practice has some community or authority behind it that might further its adoption.  

1. How easy or difficult would it be for a stakeholder to comply with these BPs?
2. Is the level of difficulty warranted? 
3. Is it appropriate for use beyond the group that developed the BP?
4. Is it possible to provide some concrete examples of use or promotion, e.g., publications where the use of the BP is cited; databases or other projects that have adopted or referenced the BP?
5. Is the BP appropriate for international use?
6. Any additional comments on adoption and use?

Stability and support

While Standards need ongoing stability and support, a BP just needs to be readily available. If it’s maintained on a website by an individual it may not be stable, but if in a publication, it would be.

1. Who is responsible for maintaining the BP? Is it necessary to maintain the BP?
2. Has the BP been published?
3. Does it have the necessary support for maintenance?
4. What is the plan for long term support, if applicable?
5. Are training and other supporting materials available?
6. If there are no means of support and the artifact is fragile, is this something INCF could host?
7. Any additional comments on sustainability and support?

Extensibility

If this BP is meant to be updated, then this is an important feature to review. Also, this is an area where we want to be informed about the potential future extensibility of this BP. Also note if the BP can be modified in response to community feedback.

1. Can the BP be extended to cover additional domains/use cases? 
2. If so, how is the process documented and managed?
3. Any additional comments on extensibility?

Comparison

This information is important to INCF and the community. It may be important for evaluating one BP vs. another, or where INCF may help facilitate interoperability between complementary BPs.

1. Are there other similar BP's available?
      - Are there specific points in them that can be cross referenced?
      - Are there other BPs that would require the BP producers to work out similarities/ differences (perhaps form a working group)?
2. If yes, how do they compare on key INCF criteria?
3. What are the key advantages of the BP when compared to other BPs?
4. Are there any recommendations that run counter to other BP recommendations?
5. Any additional comments on comparison with other BPs?