BSCS Assessment Report Spring 2012

1. PROGRAM OUTCOMES ASSESSED

Outcome A: An ability to apply knowledge of computing and mathematics to solve problems

Outcome J: An ability to apply mathematical foundations, algorithmic principles, and computer science theory in the modeling and design of computer-based systems in a way that demonstrates comprehension of the tradeoffs involved in design choices

2. COURSES ASSESSED

Outcome A was assessed at the Advanced Level in CS 146 in all sections taught (Sections 2, 3, 4 and 5).

Outcome J was assessed at the Advanced Level in CS 146 in all sections taught (Sections 2, 3, 4 and 5).

Note: Section 1 was not offered.

3.1. Outcome A Analysis and Recommendation

This is the summary of data collected from all sections.

Performance Indicator

1

2

3

Beginning

Satisfactory

Exemplary

Calculate running time of a divide and conquer algorithm

does not know how to calculate running time

some correct steps in calculation, but incorrect solution

correct solution

Number of Students

15

8

81

Solve recurrence with Master Theorem (MT)

does not know how to apply MT

correct steps but incorrect solution

successfully use MT to find correct solution

Number of Students

24

51

29

Calculate running time of an algorithm given in pseudocode (for example sort algorithm)

does not know how to calculate running time

some correct steps in calculation, but incorrect solution

correct solution

Number of Students

43

22

41

Apply an operation in an instance of an advanced data structure

fail to apply operation

some correct steps in application of operation, but incomplete

performs operation correctly

Number of Students

23

12

69

The percentage of students achieving at least satisfactory performance on each indicator is:

Indicator 1: 85.6%

Indicator 2: 76.9%

Indicator 3: 60.6%

Indicator 4: 77.9%

Overall, the students did well.

The only indicator of concern is Indicator 3: Calculate running time of an algorithm given in pseudo-code. Only 60.6% of students were able to achieve at least satisfactory performance on this indicator. According to one instructor, finding the correct running time of a recursive algorithm is hard for students at this level. Many of them confuse it with the special case where the Master Theorem can be applied or fail to 'guess'/prove correctly the running time. One way to improve this outcome is to devote more explanation and practice questions to this topic in future.

3.2. Outcome J Analysis and Recommendation

This is the summary of data collected from all sections.

Performance Indicator

1

2

3

Beginning

satisfactory

exemplary

Given an instance of data (eg. graph) and problem (eg. MST, search, shortest path), state and explain which algorithm would be best

wrong algorithm chosen

answer shows understanding of trade-off of algorithm in terms of space and time

correct answer, correctly analyzes relative costs of algorithms for specific data and problem

Number of Students

23

23

59

Given pseudocode of two algorithms (eg. sort), describe most suitable data structure to use

wrong data structure chosen

answer shows understanding of time/space efficiency of data structure for algorithm

correct answer, correctly analyzes relative costs of data structures for algorithm

Number of Students

35

40

27

Given a "real-life" problem, figure out which algorithm or data structure would be most helpful

algorithm or data structure does not solve problem

can explain how algorithm or data structure can solve problem

can explain how algorithm or data structure can solve problem better than other algorithms or data structures

Number of Students

44

26

32

The percentage of students achieving at least satisfactory performance on each indicator is:

Indicator 1: 78.1%

Indicator 2: 65.7%

Indicator 3: 56.9%

The students did well in all indicators, except for Indicator 3, where only 56.9% of students achieved at least a satisfactory level of competence. Jeff Smith (the instructor for Section 3) performed further analysis on the students' performance and found that the errors were due to:

33%:  bad asymptotic results given, without explanation
23%:  problem not attempted, or barely attempted
11%:  steps of relevant algorithm stated incorrectly
9%:  only one data structure was analyzed
8%:  no data structure chosen, or incorrect or illegal choice given the asymptotic analysis
6%:  confused the number of vertices with the number of edges
6%:  incorrectly understood or applied the assumptions of the problem
4%:  other

The question of which is the best data structure to use is a difficult one, because it requires students to have a firm grasp of many different data structures, and also be able to apply each data structure to the given problem, and analyze their space and time requirements. One way to improve students' performance is to give an example of a different case in class, before assessing students on this indicator, so that students have practice on how to apply, compare and analyze different data structures.