Critical Success Factors

Definition

In the Re.ViCa project a critical success factor is defined as follows:

A critical success factor is a factor whose presence is necessary for an organisation to fulfil its mission - in other words, if it is not present then its absence will cause organisational and/or mission failure.

The majority of work on this in e-learning has been oriented either to large-scale failures, usually of consortium models focussed on distance learning, or on "hygiene" and KPI type success factors not critical success factors

Thus Re.ViCa has to produce a new synthesis.

A natural starting point is to start from a base of general theory and fuse the large-scale critical success factors with evidence from the benchmarking and quality arenas, concentrating on criteria that are critical not just useful ones.

The Wikipedia entry on critical success factor indicates that one should look in the following areas:

1. Money factors: positive cash flow, revenue growth, and profit margins.
2. Acquiring new customers and/or distributors - your future.
3. Customer satisfaction - how happy are they?
4. Quality - how good is your product and service?
5. Product or service development - what's new that will increase business with existing customers and attract new ones?
6. Intellectual capital - increasing what you know that's profitable.
7. Strategic relationships - new sources of business, products and outside revenue.
8. Employee attraction and retention - your ability to do extend your reach.
9. Sustainability - your personal ability to keep it all going

These have to be turned from management-speak into concepts understandable by and acceptable to higher education providers.

Approach

There have been many projects which have been looking for CSFs.

Initial scoping

In this project we first carried out desktop research and learned from other projects (for an overview and download of the reports and literature, see the project website) and came to a list of 99 CSFs.

First International Advisory Commitee meeting

In spring 2008 the first International Advisory Committee Meeting took place at the EDEN Annual Conference in Lisbon, Portugal. In this meeting the experts worked in teams on this list, bringing it back to 29 essential factors. This 29 CSFs for large e-learning initiatives are labeled into three categories. First we distinguish factors that are mainly on an organizational level, these are more often strategy-and management issues. The second level is the work floor level, dealing with issues that immediately effect the daily performance of people working in this e-learning initiative. The third and last level is the service level. This involves factors that somehow have an influence on (internal or external) clients of the e-learning initiative.

Second International Advisory Commitee meeting

In a second meeting, at the ONLINE EDUCA Annual Conference in Berlin, December 2008, we let the International Advisory Committee (N= 17) vote on the 29 CSFs, using an electronic voting system in which they could give an opinion about the factors whether they must be kept or removed from the list of . The categories to answer on were:

1. must be removed
2. should be removed
3. no view
4. should be kept
5. must be kept

After each voting there was the possibility to have an discussion on that criterion. The data collection resulted in a quantitative part (the voting) and an qualitative part (the discussion).

Third International Advisory Commitee meeting

A considerable amount of works has been done in preparation for the IAC meeting at the ICDE Conference, Maastricht, June 2009.

First, a close look was taken at all the feedback from IAC.

Second, four related schemes were analysed or re-analysed.

1. MegaTrends is the main study on large-scale virtual campuses done before Re.ViCa. Its final list of factors is quite short. It has several lessons for us.
2. PBP-VC is the main study on consortia-based virtual campuses done before Re.ViCa. It has a strong focus on quality issues and on good practice for managing consortia but among the details there are a few lessons for CSFs in the consortial area.
3. UNIQUe is a scheme for quality/accreditation in e-learning developed by a consortium involving EuroPACE. In some ways it is parallel to E-xcellence though possibly more oriented to on-campus and blended uses of e-learning. Despite many of the criteria being more focussed on quality, there are some lessons to be learned, including on rewriting certain Critical Success Factors.
4. E-xcellence is a scheme of benchmarking/quality for e-learning developed in 2005-2006 by a consortium led by EADTU. It is often felt to still be most relevant to distance teaching organisations. It has earlier been taken into consideration for our CSF work but this work was rechecked.

Third, related activity on benchmarking/quality of e-learning in the UK had generated some comments on criterion wording and some new criteria, which were taken into consideration.

These pieces of work led to 19 criteria (potential CSFs) that got more serious attention - of which 8 were potential new criteria altogether. In particular there were three criteria to do with collaboration that needed more detailed attention – drawing on the experience of PBP-VC.

Acting on earlier feedback and intense debate on whether some Critical Success Factors were indeed critical for all types of Virtual Campus, it was finally decided to split the list of Critical Success Factors into two parts:

1. a list of 17 Critical Success Factors relevant to success of e-learning in all types of Virtual Campus.
2. a list of 14 Key Success Factors - these are Critical Success Factors relevant to success of e-learning in one or more subsets (categories) of Virtual Campus - such as private for-profit providers, consortia, etc.

It should be remembered that the Critical Success Factors are drawn from a much larger scheme of benchmarking/quality for e-learning, which makes it easy to promote or demote Critical Success Factors and Key Success Factors as further case study and country report information becomes available. For more details on the underlying scheme see Pick&Mix.

Selected Critical Success Factors

The following table has been derived from the latest version of the leading UK system for benchmarking e-learning, ELDDA which is being and has been used by 24 higher education institutions in the UK over the last three years:

Critical Success factors List

1. Organizational Learning (OLG)- The institution is a learning organization on all core aspects of e-learning.
2. Leadership in e-Learning (LEL) - The capability of leaders to make decisions regarding e-learning is fully developed.
3. e-Learning Strategy (ELS) - The organization regularly updates it’s e-Learning Strategy. That strategy is integrated with an learning- and teaching strategy (and all other related strategies such as IT etc).
4. Management Style (HYB)- The management style is a hybrid of academic and corporate, accepted by staff.
5. Quality Assurance (QAS) - Conformance to external quality agency precepts for the country or region, and to institutional guidelines for e-learning within an overarching methodology of quality (for example EFQM or other)
6. Planning Annually (PLA) – There is an integrated annual planning process for e-learning that is integrated with overall course planning.
7. Staff Recognition and Reward (SRR) - All e-learning experts have been explicitly recognized and rewarded (in a financial way) appropriate to their contribution to the institution, with a regular appraisal process.
8. Collaboration for e-Learning (CFE) - The institution has a reasoned approach to collaboration at various levels to gain additional benefit from sharing e-learning material, methodologies and systems (for example within an OER approach or via other methods, not excluding payment).
9. Costs (CNL) – The institution uses a costing system based on principles of activity-based costing (and that is used throughout the institution).
10. Foresight (FOR) - The institution has look-ahead capability and for example developmental labs so that new styles of e-learning can be to some extent predicted and piloted.
11. Brand Management (BMG) - The institution has a reasoned approach to managing its brand.
12. Market Research (MRE) - Market research is done centrally and in or on behalf of all departments, and is aware of e-learning aspects; it is updated annually or prior to major program planning.
13. Selling (SEL) - The institution has widespread skill in selling e-learning and the theory to support the skills.
14. Decisions on Programs (DPG) – There is effective decision-making for e-learning across the whole institution, including variations when justified.
15. Decisions on Projects (DPR) - There is effective decision-making for e-learning across the whole institution and in departments.
16. Collaboration Roles (COL) - In each collaboration, the roles and responsibilities of each collaborative partner are clearly defined and the procedures always followed.
17. Dissemination Internal (DIN) - The institution has a systematic managed process of internal dissemination of good practice.
18. Academic Workload (AWK) – The work planning system recognizes the main differences that e-learning courses have from traditional.
19. Technical Support to Staff (TSS) - All staff engaged in the e-learning process have "nearby" fast-response technical support.
20. Security (SEC) - The institution has a system where security breaches are known to occur very rarely, and when they do they are fixed fast (which allows staff and students to carry out their authorized duties easily and efficiently).
21. Performance (PER) - All e-learning systems operate in their uptime within documented and accepted response guidelines.
22. Reliability (REL) -The e-learning system is highly reliable - typically 0.999 (99.9% availability on a 24x7x365 basis).
23. Student Understanding of System (SUS) - Students have good understanding of the rules governing assignment submission, feedback, plagiarism, costs, attendance, etc and always act on them.
24. Student Help Desk (SDH) - The institution’s Student Help Desk is deemed as best practice.
25. Student Satisfaction (SAT) - The institution has an annual Student Satisfaction survey which explicitly addresses the main e-learning issues of relevance to students.
26. Employer Engagement (EEN) - The institution has a managed approach to involvement of employers of students in creating or updating courses to be delivered to their employees which include appropriate amounts of e-learning.
27. Usability (USA) - All services usable, with internal evidence to justify this.
28. Training (TRG) - All staff is trained in use of the e-learning system, appropriate to job type – and retrained when needed.
29. Organization (ORG) - An organizational unit to support e-learning exists and that is fit for purpose – (typically with a Director-level institution manager in charge and links to support teams in departments).

---

> Main Page