1. PROGRAM OUTCOMES ASSESSED

Outcome A: An ability to apply knowledge of computing and mathematics to solve problems

Outcome J: An ability to apply mathematical foundations, algorithmic principles, and computer science theory in the modeling and design of computer-based systems in a way that demonstrates comprehension of the tradeoffs involved in design choices 

 

2. COURSES ASSESSED

Outcome A was assessed at the Advanced Level in CS 146 in all sections taught (Sections 2, 3, 4 and 5).

Outcome J was assessed at the Advanced Level in CS 146 in all sections taught (Sections 2, 3, 4 and 5).

Note: Section 1 was not offered.

 

3. RESULTS AND ANALYSIS

3.1. Outcome A Analysis and Recommendation

 

This is the summary of data collected from all sections.

Performance Indicator	1	2	3
	Beginning	Satisfactory	Exemplary
Calculate running time of a divide and conquer algorithm	does not know how to calculate running time	some correct steps in calculation, but incorrect solution	correct solution
Number of Students	15	8	81
Solve recurrence with Master Theorem (MT)	does not know how to apply MT	correct steps but incorrect solution	successfully use MT to find correct solution
Number of Students	24	51	29
Calculate running time of an algorithm given in pseudocode (for example sort algorithm)	does not know how to calculate running time	some correct steps in calculation, but incorrect solution	correct solution
Number of Students	43	22	41
Apply an operation in an instance of an advanced data structure	fail to apply operation	some correct steps in application of operation, but incomplete	performs operation correctly
Number of Students	23	12	69

 

The percentage of students achieving at least satisfactory performance on each indicator is:

Indicator 1: 85.6%

Indicator 2: 76.9%

Indicator 3: 60.6%

Indicator 4: 77.9%

Overall, the students did well.

The only indicator of concern is Indicator 3: Calculate running time of an algorithm given in pseudo-code. Only 60.6% of students were able to achieve at least satisfactory performance on this indicator. According to one instructor, finding the correct running time of a recursive algorithm is hard for students at this level. Many of them confuse it with the special case where the Master Theorem can be applied or fail to 'guess'/prove correctly the running time. One way to improve this outcome is to devote more explanation and practice questions to this topic in future.

 

3.2. Outcome J Analysis and Recommendation

This is the summary of data collected from all sections.

Performance Indicator	1	2	3
	Beginning	satisfactory	exemplary
Given an instance of data (eg. graph) and problem (eg. MST, search, shortest path), state and explain which algorithm would be best	wrong algorithm chosen	answer shows understanding of trade-off of algorithm in terms of space and time	correct answer, correctly analyzes relative costs of algorithms for specific data and problem
Number of Students	23	23	59
Given pseudocode of two algorithms (eg. sort), describe most suitable data structure to use	wrong data structure chosen	answer shows understanding of time/space efficiency of data structure for algorithm	correct answer, correctly analyzes relative costs of data structures for algorithm
Number of Students	35	40	27
Given a "real-life" problem, figure out which algorithm or data structure would be most helpful	algorithm or data structure does not solve problem	can explain how algorithm or data structure can solve problem	can explain how algorithm or data structure can solve problem better than other algorithms or data structures
Number of Students	44	26	32

 

The percentage of students achieving at least satisfactory performance on each indicator is:

Indicator 1: 78.1%

Indicator 2: 65.7%

Indicator 3: 56.9%

The students did well in all indicators, except for Indicator 3, where only 56.9% of students achieved at least a satisfactory level of competence. Jeff Smith (the instructor for Section 3) performed further analysis on the students' performance and found that the errors were due to:

   33%:  bad asymptotic results given, without explanation
    23%:  problem not attempted, or barely attempted
    11%:  steps of relevant algorithm stated incorrectly
     9%:  only one data structure was analyzed
     8%:  no data structure chosen, or incorrect or illegal choice given the asymptotic analysis
     6%:  confused the number of vertices with the number of edges
     6%:  incorrectly understood or applied the assumptions of the problem
     4%:  other

The question of which is the best data structure to use is a difficult one, because it requires students to have a firm grasp of many different data structures, and also be able to apply each data structure to the given problem, and analyze their space and time requirements. One way to improve students' performance is to give an example of a different case in class, before assessing students on this indicator, so that students have practice on how to apply, compare and analyze different data structures.