skip to content

about us

JISC Collections Evaluation Framework

There are four decision points during the process of resource acquisition. Three decision points before signature of agreement and then a further evaluation of benefits to the JISC community. There may be additional reviews where added assurance is needed and if priorities need to be reviewed in light of unexpected budget and resource constraints:

  1. Strategic Assessment : The main focus is on preparation and ensuring that potential resource acquisitions fit with the wider JISC Strategy, with the JISC Collections Strategy and that they would be possible within the annual budget constraints.
  2. Business Assessment is a process that will be carried out in two stages to establish that the resource meets user requirements, that it is affordable by the community and that it will provide value for money.
  3. Licensing Assessment is carried out after the endorsement by the JISC committees and prior to signature of the agreement. This process is to ensure that publishers and suppliers comply with JISC standards.
  4. Benefits Evaluation is repeated throughout the life of an agreement.

The questions below are intended as a tool to evaluate resources at the key decision points.

Each question relates to a critical aspect of the resource – either something that must be in place for the resource to succeed in meeting the needs of the JISC and its community, or a question that must be considered before progressing to the next stage.

The Strategic Assessment checklist: Part 1

The following questions apply to all collecting areas:

  1. Is it appropriate for the JISC to be involved in the acquisition of this resource now and what is the likely impact on our community if the JISC is not involved?
  2. Does delivery of this resource in an online format provide potential to enrich research, teaching and learning?
  3. Will it offer the potential for cost saving?
  4. Would the acquisition of this resource, complement, extend or add to the interoperability of existing resources?
  5. Are the human and financial resources required for this resource available at this time, taking account of the organisation's current commitments and capacity to deliver?
  6. The following questions are specific to the fit within the four collecting areas defined in the JISC Collections Strategy:

Heritage : Would the acquisition of this online resource provide the JISC community with access to rare or inaccessible or otherwise unaffordable material? Would the acquisition widen access to scholarly material or primary source material? Will it offer the possibility of efficiencies or space saving?

Innovation : Would the acquisition of this online resource enrich research teaching and learning in a new way? Would this resource be available to the JISC community without the support and or intervention of JISC?

Specialist : Does this resource have the potential to provide high value to a small group of users? Is it based in a subject area that has been relatively underrepresented by previous dataset licensing?

Mature : Does this resource meet the needs of the JISC community? Will the JISC add value to the agreement through negotiation and its application of the JISC banding and the terms and conditions of the JISC model licence?

NEXT STEPS

If the answers to the above checklist indicate that the resource fits within the strategic framework, JISC Collections will continue discussions and add this resource to the list of potential acquisitions for the JISC Portfolio. JISC Collections will proceed to the business justification stage of evaluation.

Business justification checklist: Part 2

The options identified and appraised are: affordability, achievability and value for money. To ensure the resource meets these criteria, we ask the following questions:

  1. Is the resource fit for purpose with a robust delivery service in place or would the JISC need to provide such a service in order to enable delivery to its community? If the latter is the case, will the development of such a service be affordable and sustainable in the longer term?
  2. Have we taken into account the wider context, including excellence of resource and how it compares with competing resources?
  3. Do we have internal/external authority and support? For example, do we have endorsement from the virtual subject advisors and the HE Academy in support of the acquisition?
  4. Has the publisher agreed in principle to comply with the relevant JISC Information Environment standards? Membership of the UK Access management federation and implementation of federation compliant technologies is essential and compliance with Open URL standards, COUNTER, WC3 standards are highly desirable.
  5. Has the publisher agreed in principle to comply with the terms and conditions of the JISC Model Licence, including governance of English law?
  6. Does the publishers/content owner's initial offer regarding institutional subscription fees seem affordable? What is the appropriate JISC financial model for this resource and is it for the JISC to provide subsidy or to take any financial risk?
  7. If JISC financial support is appropriate, can it be afforded in this financial year?

NEXT STEPS

If the answers to the above checklist are positive, JISC Collections facilitates further consultation to establish viability of the resource.

Resource viability checklist: Part 3

To establish resource viability we ask the following questions:

  1. Have stake holders been consulted, these include the working groups and the wider community?
  2. Do the results of wider consultation indicate that this resource is affordable to the JISC community?
  3. Is take-up likely to be wide enough to achieve sustainability and or value for money?
  4. In the case of a renewal of agreement, does the take-up and usage of the resource justify renewal – what is the impact on our community if we do not renew the agreement?

NEXT STEPS

If the answers to the above checklist are positive the JISC Collections will present a case to the JCS committee for endorsement.

Licensing Strategy checklist: Part 4

This evaluation follows endorsement from the JCS committee and takes place prior to the signature of contract.

  1. Is the business case still valid and unaffected by internal and external events or changes?
  2. Can we confirm the plans for managing implementation of service delivery through one of our data centres (if required) are achievable and that we have the resources we need?
  3. Has the publisher in actuality agreed to implement compliance with the relevant JISC Information Environment standards?
  4. Has the publisher agreed to the terms and conditions of the JISC Model Licence?
  5. Are we learning from experience and applying the lessons learned from this resource acquisition to other resources?

NEXT STEPS

If the answers to the above checklist are positive the JISC Collections will complete the contract to signature of agreement and announce the agreement to the JISC community.

Benefits Evaluation and ongoing management of the agreement checklist

  1. Is the business case justification for the acquisition realistic and is the subscription take-up in line with expectations? If subscription/use is not in line with expectations what mitigating actions are we taking?
  2. Is the Collections Team/is the publisher actively seeking to improve value for money?
  3. Is the Collections Team/is the publisher ready for the future with plans for future developments and enhancements?
  4. Is the service proving to be robust?
  5. Does the resource continue to meet user needs?
  6. Is it essential to maintaining research, teaching and learning activity in the sector?

NEXT STEPS

Prior to renewal of contract repeat evaluation process cycle.