Critical Success Factors

Definition

Approach

There have been many projects which have been looking for CSF’s. In this project we first carried out desktop research and learned from other projects (for an overview and download of the reports and literature, see the project website) and came to a list of 99 CSF’s. In spring 2008 the first International Advisory Committee Meeting took place at the EDEN Annual Conference in Lisbon, Portugal. In this meeting the experts worked in teams on this list, bringing it back to 29 essential factors. This 29 CSF’s for large e-learning initiatives are labeled into three categories. First we distinguish factors that are mainly on an organizational level, these are more often strategy-and management issues (see table 2). The second level is the work floor level, dealing with issues that immediately effect the daily performance of people working in this e-learning initiative (see table 3). The third and last level is the service level. This involves factors that somehow have an influence on (internal or external) clients (see table 4) of the e-learning initiative.

In an second meeting, at the ONLINE EDUCA Annual Conference in Berlin, December 2008, we let the International Advisory Committee (N= 17) vote on the 29 CSF’s, using an electronic voting system in which they could give an opinion about the factors whether they must be kept or removed from the list of . The categories to answer on were: 1. must be removed, 2. should be removed, 3. no view, 4. should be kept and 5. must be kept. After each voting there was the possibility to have an discussion on that criterion. The data collection resulted in a quantitative part (the voting) and an qualitative part (the discussion).

Critical Success Factors for Major E-Learning Initiatives

The following table has been derived from the latest version of the leading UK system for benchmarking e-learning, ELDDA,<a href="#_ftn1" name="_ftnref1" title="">[1]</a> which is being and has been used by 24 higher education institutions in the UK over the last three years/ (The former name of ELDDA is Pick&Mix.<a href="#_ftn2" name="_ftnref2" title="">[2]</a> ) It incorporates the author’s earlier synthesis of Critical Success Factors based on his analysis of large e-university initiatives.<a href="#_ftn3" name="_ftnref3" title="">[3]</a>

Criteria in blue are recent additions to ELDAA, mainly focussing on employer issues.

Criteria in green are from the Critical Success Factors synthesis and recent consideration of other large initiatives.

Readers are invited to comment on the criteria and grades.

The CSF grade is defined as follows:

Critical Success Factor
Key Success Factor
Irrelevant to success of major initiatives
Counter-productive to success of major initiatives.

To avoid too much detail, only the potential grades of 1 have been included in this version.

The table of criteria

*No.*	*Criterion name*	*Code*	*Best practice statement*	*MIT90s category*	*OBHE category*	CSF grade (1-3)
1	Adoption	ADO	All taken it up except some laggards.	Individuals (Staff)	Delivery
2	VLE stage	VLE	One VLE.	Technology	Delivery
3	Tools	TLS	Institution-wide use of several tools.	Technology	Delivery
4	Usability	USA	All services usable, with internal evidence to back this up.	Technology	Delivery	1
5	Accessibility	ACC	All e-learning material and services are accessible, and key components validated by external agencies.	Technology	Delivery
6	e-Learning Strategy	ELS	Regularly updated e-Learning Strategy, integrated with Learning and Teaching Strategy and all related strategies (e.g. Distance Learning, if relevant).	Strategy	Strategy development	1
7	Decisions on Projects	DPR	Effective decision-making for e-learning across the whole institution, including variations when justified.	Processes	Management and leadership	1
8	Pedagogy	PED	Pedagogic guidelines for the whole institution and acted on.	Individuals (Staff)	Staff
9	Learning Material	LMA	Institution-wide standards for learning material, which are adhered to and embedded at an early stage, e.g. by style sheets.	Processes	Delivery
10	Training	TRG	All staff trained in VLE use, appropriate to job type - and retrained when needed.	Individuals (Staff)	Staff	1
11	Academic Workload	AWK	Work planning system which recognises the main differences that e-learning courses have from traditional.	Processes	Resources and value for money	1
12	Costs	CNL	Activity-Based Costing or a system with equivalent functionality being used to some extent in all departments.	Processes	Resources and value for money	1
13	Planning Annually	PLA	Integrated annual planning process for e-learning integrated with overall course planning.	Processes	Management and leadership
14	Evaluation	EVA	Regular evaluation of all courses using a variety of measurement techniques and involving outside agencies where appropriate.	Processes	Communication, evaluation and review
15	Organisation	ORG	Central unit has Director-level institution manager in charge and links to support teams in departments.	Organisation	Management and leadership	1
16	Technical Support to Staff	TSS	All staff engaged in the e-learning process have "nearby" fast-response tech support.	Individuals	Delivery
17	Quality Assurance	QAS	Conformance to external quality agency precepts and local guidelines for e-learning within an overarching methodology (EFQM, etc).	Processes	Communication, evaluation and review	1
18	Staff Recognition and Reward	SRR	All informally recognised e-learning experts have been explicitly recognised in a financial way.	Individuals (Staff) (NOT Technology)	Staff	1
19	Decisions on Programmes	DPG	Effective decision-making for e-learning across the whole institution, including variations when justified.	Processes	Management and leadership	1
20	Quality Enhancement	QEN	Total integration of "traditional" quality enhancement work with e-learning.	Processes	Communication, evaluation and review
21	Adoption by Staff of Enhanced Learning	AEL	All staff except laggards are delivering "enhanced learning" via appropriate use of e-learning and can evidence their claims.	Processes	Staff
22	Leadership in e-Learning	LEL	The capability of leaders to make decisions regarding e-learning is fully developed.	Individuals (Leaders)	Management and leadership	1
23	Teaching 2.0	T2O	The institution is fully comfortable using web 2.0 tools where appropriate.	Technology	Staff
24	Collaboration for e-Learning	CFE	The institution has a reasoned approach to collaboration at various levels to gain additional benefit from sharing e-learning material, methodologies and systems.	External environment	Partnership and collaboration	1
25	Brand Management	BMG	The institution has a reasoned approach to managing its brand	External environment	Management and leadership	1
26	USP Management	USP	The institution has a reasoned approach to managing its Unique Selling Propositions	External environment	Management and leadership	1
27	Time to Market	TTM	Time to market is low compared with comparators	External environment	Management and leadership	1
28	Cost of Sales	CSL	Cost of sales is low compared with comparators	External environment	Management and leadership	1
29	Management Style	HYB	The management style is a hybrid of academic and corporate, accepted by staff	Individuals (Leaders)	Management and leadership	1
30	Language Choice	LAC	The choice of languages for teaching is evidence-based not e.g. politically determined	External environment	Management and leadership	1
50	Learning Outcomes	LOU	Learning outcomes across all courses are provably unaffected by the extent to which e-learning is a component of each course.	Individuals (Students)	Delivery
51	Uniformity	UNI	Institution offers the same service level (pedagogic and administrative) to all students irrespective of mode or location of study (including DL and WBL).	Technology	Delivery	1
52	Ubiquity	UBI	Institution offers a pervasive seamless network/service to all its students, on- and off-campus and via wireless on campus also.	Technology	Delivery
53	Reliability	REL	0.999 (99.9% availability)	Technology	Delivery	1
54	Performance	PER	All e-learning systems operate in all uptime within documented and accepted response guidelines.	Technology	Delivery
55	Foresight	FOR	Both look-ahead and lab, working in concert; at least one of these should be a sector leader.	External environment	Management and leadership	1
56	Selling	SEL	Widespread skill in selling e-learning and the theory to support the skills.	Processes	Management and leadership	1
57	IPR	IPR	IPR embedded and enforced in staff, consultant and supplier contracts.	Processes	Management and leadership
58	Market Research	MRE	Market research done centrally and in or on behalf of all departments, and aware of e-learning aspects; updated annually or prior to major programme planning.	External environment	Communication, evaluation and review	1
59	Competitor Research	CRE	Competitor research done centrally and in or on behalf of all departments, and fully aware of e-learning aspects.	External environment	Communication, evaluation and review	1
60	Security	SEC	A system where security breaches are known not to occur yet which allows staff and students to carry out their authorised duties easily and efficiently.	Technology	Delivery
61	Pedagogy Research	PRC	The institution is fully aware of outcomes of research which will enhance the experience of its students by suitable use of e-learning.		Communication, evaluation and review
62	Integration	INT	Seamless integration with total uniformity of data formats, interface and response time.	Technology	Delivery
63	Leverage	LEV	Annual student surveys and focus groups are used to determine skill levels and this is taken into account for programme proposals and e-learning support.	Individuals (Students)	Students
64	Plagiarism Avoidance	PAV	Sustained universal campaign to brief students against plagiarism in the e-learning context and to set more suitable assignments.	Individuals (Students)	Students
65	Plagiarism Detection	PDT	All departments operate an electronic system for the detection of plagiarism and an associated human process to confirm diagnoses and take corrective action.	Individuals (Students)	Students
66	Physical	PHY	Integration of e-learning strategy, plans and decisions with the space management processes.	Strategy	Strategy development
67	Risks	RSK	Projects and programmes integrate pro-active risk management at all levels.	Processes	Management and leadership
68	Research Out	ROU	Essentially RAE 4*.	Individuals (Staff)	Staff
69	Research In	RIN	Programme offerings via e-learning as equally as informed by research as offerings of a more traditional nature.	External environment	Delivery
70	Widening Participation	WPR	Integration of e-learning strategy, plans and decisions with Widening Participation processes, within a framework of cost- and income-awareness.	Strategy	Strategy development
71	Disadvantaged	DSA	Full integration of e-learning strategy, plans and decisions with support for disadvantaged students (other than disabled - see 05 - and WP - see 70).	Strategy	Strategy development
72	Personalisation	PRS	An approach to personalisation in e-learning which balances pedagogy, access and government desires within an agenda of cost-effectiveness, quality assurance and the grounded research literature.	Individuals (Students)	Students
73	Eco-Sustainability	ESU	A culture where sustainability is built into decision-making and operation of large e-learning projects.	Processes	Strategy development
79	Collaboration Roles	COL	The roles and responsibilities of each collaborative partner are clearly defined and the procedures always followed.	External environment	Partnership and collaboration	1
80	Computer Based Assessment	CBA	A systematic managed process is in place of using CBA across the institution in ways appropriate to each programme.	Technology	Students
81	Computer Managed Assessment	CMA	A systematic managed process is in place of using Computer Managed Assessment and feedback of assignments (including dissertations) across the institution in ways appropriate to each programme.	Technology	Delivery
82	Dissemination Internal	DIN	A systematic managed process of internal dissemination of good practice is in place.	Processes	Communication, evaluation and review	1
83	Staff Experience	SEP	All courses using significant e-learning measure the staff experience aspects of this.	Individuals (Staff)	Staff
84	Staff Satisfaction	SST	Annual Staff Satisfaction survey which explicitly addresses the main e-learning issues of relevance to staff (e.g. support, workload)	Individuals (Staff)	Staff
85	Employer Engagement	EEN	A managed approach to involvement of employers of students in creating or updating WBL courses to include appropriate amounts of e-learning.	Processes	Management and leadership
86	Employer Experience	EEP	All WBL courses using significant e-learning measure the employer experience aspects of this.	Processes	Management and leadership
87	Employer Satisfaction	EST	Annual Employer Satisfaction survey which explicitly addresses the main e-learning issues of relevance to employers of students on WBL courses.	Individuals (Students)	Management and leadership
89	Employability	EMP	The institution takes care to ensure that e-learning is used in a reasoned way to enhance the employability of students including in their own businesses.	Individuals (Students)	Delivery
90	Student Engagement in Design	SED	A managed approach to involvement of students in updating the design of e-learning courses across many courses.	Individuals (Students)	Students
91	Student Understanding of System	SUS	Students have good understanding of the rules governing assignment submission, feedback, plagiarism, costs, attendance, etc and always act on them.	Individuals (Students)	Students	1
92	Student Help Desk	SHD	Help Desk is deemed as best practice.	Individuals (Students)	Delivery	1
93	Student Experience	SXP	All courses using significant e-learning measure the student experience aspects of this.	Individuals (Students)	Students
94	Student Satisfaction	SAT	Annual Student Satisfaction survey which explicitly addresses the main e-learning issues of relevance to students.	Individuals (Students)	Students
95	e-Portfolios	EPO	Use of e-portfolios in all departments.	Processes	Students
96	Learning Objects	LOB	An approach to use of learning objects which balances pedagogy and technology within an agenda of cost-effectiveness, quality assurance and the grounded research literature.	Technology	Delivery
97	Open Educational Resources	OER	The institution has a reasoned approach to the use of OER and is using them across the institution.	Technology	Delivery
98	Benchmarking	BMK	The institution has recently benchmarked its e-learning and is working through the implications of the results.	Processes	Management and leadership
99	Organisational Learning	OLG	Institution is a learning organisation on all core aspects of e-learning.	Processes	Staff

<a href="#_ftnref1" name="_ftn1" title="">[1]</a> See <a href="http://www.matic-media.co.uk/ELDDA/ELDDA-2pt1-beta3b-wbl.xls">http://www.matic-media.co.uk/ELDDA/ELDDA-2pt1-beta3b-wbl.xls</a>.

<a href="#_ftnref2" name="_ftn2" title="">[2]</a> See <a href="http://elearning.heacademy.ac.uk/wiki/index.php/Pick%26Mix">http://elearning.heacademy.ac.uk/wiki/index.php/Pick%26Mix</a>.

<a href="#_ftnref3" name="_ftn3" title="">[3]</a> Lessons to be learned from the failure of the UK e-University, ODLAA, 2005, <a href="http://www.unisa.edu.au/odlaaconference/PDFs/32%20odlaa2005%20-%20bacsich.pdf">http://www.unisa.edu.au/odlaaconference/PDFs/32%20odlaa2005%20-%20bacsich.pdf</a>.

</body>

</html>

> Main Page

Critical Success Factors

Navigation menu

Search