ISD Knowledge Base / Analysis

This document contains the needs assessment concept map and explanation. A description for task analysis is included in another document.

Needs Assessment

Selected concepts in the concept map are hotlinks to explanations below. (Click here for a full size concept map image; opens in new window)

Needs Assessment concept map

The dimensions of the concept of needs assessment as depicted in the concept map above are described below:

"analysis prior to action"

the initial pursuit of information

the tasks and approach of needs assessment draws from qualitative research methods, strategies, and techniques

General Definition

The process of comparing a desired goal state with existing conditions

• Data is fundamental to all decision making
• Needs assessment requires considerable preplanning
• An understanding of the various dimensions of needs assessment is essential for effective planning and implementation

Atsusi Hirumi (University of Houston) writes: Systematic process of identifying alternative solutions to performance problems and/or opportunities

Allison Rossett (1982) writes: A typology for generating needs assessments. Journal of Instructional Development, 6 (1), 28-33, Fall 1982.
Needs assessment is a technique to understand a performance problem before trying to solve it. Also called front-end analysis, needs assessment is what developers undertake to find out what learners think and feel. It is a plan to find a solution to a given problem or to meet a need.

Back to Top

History = Kaufman and English (1979) and Dewey (1939) emphasized the importance of determining learner needs prior to developing curricumulum for that learner. See Kaufman later in this summary. McGehee and Thayer (1961) introduced the idea of levels of analysis, of repeated studies of increasing detail involving more and varied sources. This has become crucial to needs assessment.

Thomas Gilbert (1977) described needs assessment as: PIPs, Leisurely Theorems...a holistic, analytical type of needs assessment

Needs assessments are useful in every organizational environment: education, business & industry, military, government agencies, public & private institutions.

Instruction is not the answer to every challenge (performance problem) in the workplace.

A wide variety of interventions can be used to enhance performance, including, but not limited to instruction (job aids, selection strategies, incentive programs, job redesign).

Matching the appropriate interventions to the challenge/opportunity/problem is based on analysis.

Analysis is the best (most persuasive) manner in which to influence the organization.

Back to Top

5 Types of Needs (Burton & Merrill, 1977)

Normative
An individual or group has a deficiency and falls below the norm or expectations of some established group

Felt
Someone with financial backing and/or influence, "feels something ought to be done"

Expressed (Demand)
There is more "need" for the good, service, or article than there is supply

Comparative
No existing facilities or provisions (someone else has it and you don't)

Anticipated (Future)
A needs that does not exist at the current time, but is predicted for some future date

Phases of needs assessment

Identify and prioritize future goals
What SHOULD BE

Determine existing conditions and goals
What IS

Identify discrepancies between #1 and #2
These are the NEEDS

Prioritize #3

Back to Top

When is Needs Assessment Warranted?

A needs assessment is warranted when there is a difference between what IS and what SHOULD BE.

Reasons for needs assessment:

• Learning/performance/skill/KSAs (knowledge/skills/attitudes) goals not being met
  
• Inefficiency of current instruction/performance/KSAs
• Lack of appeal/effectiveness/efficiency of current instruction/performance/KSAs
• Ineffective instruction/performance/KSAs in an area
  
• Change in learner/client/management KSAs in work group population or external organizational environment
  
• New technology/strategy/management
  
• Before completing a strategic plan
  

Before beginning, it is wise to make a business case (rationale) for the investment required to conduct a needs assessment. Develop option scenarios, and if possible, include non-training interventions. Estimate costs associated with developing and delivering the appropriate training program(s). Assess the benefit to be derived from closing the performance gaps identified previously. Prepare a cost-benefit analysis to show how the expected contribution of your intervention solution supports the business objectives.

Needs assessments are useful in every organizational environment: education, business & industry, military, government agencies, public & private institutions.

Back to Top

Why conduct needs assessment?

Needs assessment are effective and efficient for addressing relevant components of problematic system that are affecting results. Usually organizations start with a solution based on faulty assumptions of the problem without a complete understanding of the dynamics of the entire system.

Rossett outlines five purposes for analysis (finding and disseminating detailed information):

Optimal performance
about optimal performance (who/what/how characterizes the exemplary performance?)

Actual performance
what's going on?

Feelings
how do learners feel about the topic/training about the topic/priority of the topic/confidence in the topic

Causes
Regarding what is causing the problem (Gilbert (1978), Mager & Pipe (1970) focused on causes of performance problems:

• employees lack skill or knowledge
  
• the environment gets in the way of job performance
  
• no, few, or improper incentives
  
• employees are unmotivated
  
  

Solutions
Solutions are related to the causes of the problem:

• Job aids, training, selection = employees lack skill or knowledge
• Improved tools, improved forms. workplace redesign, job redesign = the environment gets in the way of job performance
  
• Improved policies, better supervision, improved incentives = no, few, or improper incentives
  
• Training, information, coaching, better supervision = employees are unmotivated


How to select the needs?


Size of discrepancy
Pick the biggest problem — a comparison between optimal and actual gives the magnitude of the problem

Attainability
Can we do it? ã the liklihood of success is an important criteria for prioritizing

Perceived value
What does everyone else think is important? ã perceived value through rating and ranking

Back to Top

Kaufman's 6 Levels of Needs Assessment

Progressing from the most to the least amount of change

Alpha
["Anything goes"] Any and all reasonable means of change may be implemented

Characterized by a willingness of management to make possible ANY mode of learning, when performance deficits of employees are perceived as a lack of relevant skills rather than due to company policies or other causes of poor performance. Any reasonable way to learn, any required goals would be open for approval as a solution to the problem. Aim for the best possible solution and be willing to ignore present conventions.

Beta
["deBugging"] The existing training department and modes of training operations are retained, only the content and objectives of the training would be open to some degree of modification.

Working within the existing system.

Gamma
["Go for it"] Existing facilities will be used, some content and some objectives will be changed

Delta
["Determine, delegate, do it"] Existing facilities will be used, a little of the content, and a few of the objectives will be changed.

Epsilon
["Evaluate"] Existing facilities will be used, a little of the content will be changed, none of the objectives will be changed.

Zeta
["Zero in on fine tuning"] Existing facilities will be used, none of the content will be changed, none of the objectives will be changed, only the delivery can be changed.

Back to Top

Methods of Data Gathering

The methods of data gathering are based on qualitative research principles!

Key consultations
Specific interviews, usually w/managers. Beware of bias

Interviews
Gives a more broad perspective than key consultations, but time consuming

Group discussion
Faciliate focus groups or ogranizational groups to seek information and dissiminate findings

Questionnaire
Course evaluations, surveys, input in the form of of attitudes and beliefs

Specific directive

Written materials
Professional or company publications, records, abseteeism records, employee records, standardized test scores

Records/reports
company files

Observation
The researcher skilled in observation watches work performance

Back to Top

Define, Differentiate, Compose Question Types

This is defined by Rossett as a typology. The type of questions that need to be answered determine the type of questions the developer puts on an assessment instrument.

Type 1: Problem finding
"Is there a problem?", "If so, what is its nature?"
Type 1 questions get at finding problems and finding details of particular problems

Type 2: Problem selecting
"What are the priorities within the problem?" ã Prioritize identified problems
Type 2 questions press respondents to prioritize and select from among several needs or facets of one need.

Type 3: Knowledge/skill proving
Corroborate self-report information; ask about content related to the problem
Type 3 questions ask learners to perform as if there were no problem, as if they already knew how to accomplish something. Think of it as a pre-test

Type 4: Finding feelings
Type 4 questions ask learners about their feelings and attiudes toward the problem and/or training. The developer is trying to figure out the emotions and attitudes held by the learner.

Type 5: Cause finding
Type 5 questions ask the learner for the cause of the problem. This is the chance for the developer to find out what or who is contributing to the problem.

Back to Top

Conducting a Needs Assessment

Strategy
Training needs should be addressed at three levels: organizational, occupational, individual

Organizational level
Needs must be assessed at the highest level in the organization where business strategies are identified, cost-cutting policies are developed, vision/mission/goals and business processes are set, and budget/resource decisions are made.

Occupational level
Needs must be assessed at the occupational level to identify competencies and skill requirements (KSAs) of key occupational groups critical to the organizational mission.

Individual level
Needs must be assessed at the individual level to understand specific performance needs of employees and KSAs required to achieve performance goals. Gilbert's PIPs are used at this level to identify exemplar performance

Back to Top

Implementation

Reference: Training Needs Assessment Handbook: A Guide for Conducting a Multi-level Needs Assessment, published by US Office of Personnel Management (1994)

Plan

• Obtain key stakeholder buy-in and support. Identify those who will be affected by the needs assessment and promote the benefits of the process and its outcomes (persuasion)
  
• Set goals and objectives for the needs assessment. Define why the needs assessment is important to the organization and identify critical goals and objectives that need to be accomplished at each level of the assessment.
• Build assessment team. Determine the roles that should be represented and select/identify team members.
• Conduct preliminary research. Investigate other needs assessments conducted by the organization and the results of those assessments; also research obstacles that may impact data collection or other assessment activities.
  
• Prepare project plan. Develop a plan that defines staffing, equipment, and other resource requirements; timetables; data to be collected; specific methodologies to be used; and othe planning considerations.

Who is asking for a needs assessment or what is the driving factor for a front-end analysis?
performance problems, new stuff, mandates

Identify sources
who has the information that is needed?

Select tools/instruments
Using the typology for needs assessment outlined above to create any survey instruments.
See Methods of Data Gathering for possible tools.


Conduct

• Conduct the needs assessment in stages; build stages to better handle data and analysis
  
• Collect needs assessment data. Collect required data using the data collection techniques appropriate to the type of data being collected.
  
• Analyze the data. Compile and analyze the data at the organizational, occupational, individual levels to identify performance issues or problems, gaps, trends, or other relevant conclusions and observations.
  
• Assess peformance gaps and identify interventions. Assess performance gaps for the importance and determine the causes of the performance gaps; identify the kinds of interventions (training, organizational development, management systems) likley to resolve the peformance gap.

Back to Top

Evaluate: Kirkpatrick's 4 Levels

Evaluate the outcomes of the training or intervention strategy

Level 1: Reaction
Did the participants like it?
Correlative evaluation = a smiley/happy sheet evaluation regarding content, relevancy, format, methods, media, graphics, pacing, clarity, fun, comfort, location, etc.
Measured in training class.

Level 2: Learning
Did they learn what was taught?
Formative evaluation = Learners are evaluated throughout training to see how well they are acquiring KSAs; post-test for mastery skills. Can be used pre-training to establish benchmarking levels of the learners entering the training program.
Measured in training class.

Level 3: Behavior
Did they use what they learned back on the job?
How well did learners apply their new KSAs? = a measure of transfer of training
Summative evaluation = output or behavior measured in the workplace after a few weeks; evaluation by self or others

Level 4: Results
Did the intervention produce a return on investment?
How much did it cost relative to benefits
Measured in the workplace after a few weeks, looking for long term impact
Cost-benefit or pre- and post-training comparison of data already in the system (accident rate, new accounts, rejects, absenteeism, turnover)
Costs: course development, instructional materials, equipment, facilities, off-site expenses, salary, and lost productivity
Benefits: time savings (achiveing proficiency faster), better productivity, improved quality of work, and better personal performance
Level 4 is the most difficult to evaluate. It may be difficult to clearly measure cause-and-effect relationships between a training program and subsequent performance of its graduates back on the job. It is important for trainers to attempt the measure in terms of training dollars saved or earned as a result of training

Use Findings for Decision Making

When the results are a clear picture for the developer, decisions (recommendations) can be made about:

• whether the training is appropriate
  
• what kind of training is favored, for whom, and by whom
  
• what other supportive interventions will solve the problem, introduce the new system, or respond to the mandate
  
• strategies for involving other related professionals in the effort
  
• the content of courses, if training is judged appropriate
  
• how training and other interventions will be received by trainees, supervisors, customers, and others