BSCS Assessment Report Spring 2011

 

As dictated by the assessment schedule, the following Program Outcomes were assessed:

BSCS.OC7

The ability to make ethical judgments about the design and use of computing systems

BSCS.OC13

The ability to explain the structure and function of computer hardware components and their influence on the design of software systems

BSCS.OC14

The ability to explain the structure and function of operating systems, their interactions with computer hardware components, and their influence on the design of software systems

According to the Courses/Outcomes Matrix,

  1. BSCS.OC7 is to be assessed in Philosophy 134 at the Intermediate level,
  2. BSCS.OC13 is to be assessed in CS 47 at the Beginner level, in CS 149 at the Intermediate level and in CS 147 at the Advanced level, and
  3. BSCS.OC14 is to be assessed in CS 149 at the Advanced level.

 

The Assessment Reports for the Program Outcomes are given below:

 

1. Program Outcome BSCS.OC7

The ability to make ethical judgments about the design and use of computing systems

1.1. Assessment

BSCS.OC7 was assessed at the Intermediate level in Philosophy 134.

1.2. Tasks

Students who pass Philosophy 134 should be able to:

  1. Identify, describe and use tools to respond to moral issues in software design, and
  2. Identify, describe and use tools to respond to the moral issues in software use

 

1.3. Results of Assessing BSCS.OC7 in Phil 134 at the Intermediate level

 The following table shows the result summary provided by the Philosophy Department.

 

 

Philosophy 134, Spring 2011

 

I. Moral Dimensions of software design

# of students scoring 6 (out of 6) points

# of students scoring 5

# of students scoring 4

# of students scoring 3

Q1

26

37

7

2

Q2

28

33

11

0

Q3

44

19

9

0

II. Moral Dimensions of Software Use

 

 

 

 

 

Q1

30

31

9

2

Q2

29

34

9

0

Q3

38

25

9

0

 

1.3.1. Description of Assessment instrument and strategy 

We used two cases with three questions for each on the final exam in two sections of  Philosophy 134 with a total of 78 students. Students answered the question in an essay format. The first case was designed to assess the students’ sensitivity to the moral dimension of software design, the second to software use.  For each case, they were asked to identify the moral issue, then to describe tools for responding to it, and finally they were asked to use those tools in crafting a response. Each case was worth 6 points-2 points for each question.  The two cases are below.

1.3.1.1. Moral Dimensions of Software Design

The creators of the website PleaseRobMe used Twitter posts of Foursquare gamers to determine where the gamers lived & when the gamers weren’t home, then posted lists of empty home locations. In the game Foursquare, gamers earn points by using GPS to post their locations and activities around their city. They could opt to push their locations to Twitter.  The site developers initially claimed they were merely trying to alert social network users to the dangers of posting personal info online.  "The website is not a tool for burglary.  The point we're getting at is that not long ago it was questionable to share your full name on the internet. We've gone past that point by 1,000 miles." http://news.bbc.co.uk/2/hi/technology/8521598.stm

Please answer these three questions about this case: 1) What is a key moral issue here?  2) What concepts could you use to develop a response?  3) Using these concepts, what do you think should be done and why? 

1.3.1.2. Moral Dimensions of Software Use

The debate over digital privacy flamed higher this week with news that Apple’s popular iPhones and iPads store users’ GPS coordinates for a year or more. Phones that run Google’s Android software also store users’ location data. And not only is the data stored — allowing anyone who can get their hands on the device to piece together a chillingly accurate profile of where you’ve been — but it’s also transmitted back to the companies to use for their own research.

http://www.usatoday.com/tech/news/2011-04-23-smartphone-tracking.htm

Please answer these three questions about this issue: 1) What is a key moral issue here?  2) What concepts could you use to develop a response?  3) Using these concepts, what do you think should be done and why? 

 

Based on the report from the Philosophy Department, The following table shows the summary of how the students did in performing the tasks.

 

Task 1: Moral Dimensions of Software Design

6

5

4

3

% > 3

Q1

26

37

7

2

92.30769

Q2

28

33

11

0

92.30769

Q3

44

19

9

0

92.30769

Overall

 

 

 

 

92.30769

Task 2: Moral Dimensions of Software Use

 

 

 

 

 

Q1

30

31

9

2

92.30769

Q2

29

34

9

0

92.30769

Q3

38

25

9

0

92.30769

Overall

 

 

 

 

92.30769

 

1.4. Analysis

Overall 92.3% of the students were able to perform both tasks. We are satisfied with this result.

1.5. Recommendations

None.

 

2. Program Outcome BSCS.OC13

The ability to explain the structure and function of computer hardware components and their influence on the design of software systems

2.1. Assessment

BSCS.OC13 was assessed at the Beginner level in CS 47 taught in different sections by Tom Howell and Johnny Martin.

BSCS.OC13 was assessed at the Intermediate level in CS 149 taught in different sections by Robert Chun and Terry Allen.

BSCS.OC13 was assessed at the Advanced level in CS 147 taught by Sin-Min Lee.

 

2.2.1. Tasks for Assessment of BSCS.OC13 in CS 47 at the Beginner level

 

Students who pass CS 47 should be able to perform the following tasks:

  Task 1:

Demonstrate beginning level understanding of integer arithmetic hardware by writing a program that does efficient multiplication of a multi-precision integer by a machine integer. 

  Task 2:

Demonstrate beginning level understanding of floating point arithmetic hardware by writing programs that decompose a floating point number into sign, exponent, and significand and another to reconstitute the floating point number from its components.

  Task 3:

Demonstrate beginning level understanding of the stack management hardware and software supporting parameter passing and return of function results in a procedural programming language.

  Task 4:

Demonstrate beginning level capability to measure the running times of programs using the “read time stamp counter” instruction.  Measure the running time of a program as a function of a suitable input parameter.

 

2.2.2. Results of Assessing BSCS.OC13 in CS 47 at the Beginner level

The assessment of BSCS.OC13 was completed by Tom Howell in CS 47 Section 1 and the results submitted are given in this report.

The assessment of BSCS.OC13 was completed by Johnny Martin in CS 47 Sections 2 and 3 and the results submitted are in this report.

The overall percentage of passing students able to complete each task is given in the following table:

 

 

Section 1

Sections 2, 3

Task 1

86%

83%

Task 2

68%

66%

Task 3

59%

38%

Task 4

N/A

100%

 

The overall percentage of passing students for a task for a section is calculated by taking the simple average percentage of passing students for all questions supporting that task.

 

2.3.1. Tasks for Assessment of BSCS.OC13 in CS 149 at the Intermediate level

Students who pass CS 149 should be able to perform the following tasks:

Task 1:

Describe a Memory Hierarchy and explain why a Memory Hierarchy, consisting of different types of memory technologies is needed.  Describe each of the three main memory technologies in the hierarchy (Random Access, Direct Access, and Sequential Access) including their speed and cost characteristics, along with their advantages and disadvantages.  (Assessed with an exam question)

Task 2:

Explain the role and operation of cache memory including the concepts of hit ratio and temporal and spatial locality of references.  Compute the effective memory speedup that can be attained from using a cache when given a stipulated hit ratio along with the access times of the cache memory and main memory.  (Assessed with an exam question)

Task 3:

Explain the difference in expected hardware memory performance (and therefore, execution speed) between a software program that is highly modular in structure versus that of one which is not modular in structure and makes extensive use of Go_To statements.  (Assessed with an exam question)

 

2.3.2. Results for Assessment of BSCS.OC13 in CS 149 at the Intermediate level

 

The assessment of BSCS.OC13 was completed by Robert Chun in CS 149 Sections 1 and 2.

The assessment of BSCS.OC13 was completed by Terry Allen in CS 149 Section 3.

The overall percentage of passing students able to complete each task is given in the following table:

 

 

Sections 1 and 2

Section 3

Task 1

94%

39%

Task 2

87%

30%

Task 3

90%

31%

 

The overall percentage of passing students for a task for a section is calculated by taking the simple average percentage of passing students for all questions supporting that task.

 

2.4.1. Tasks for Assessment of BSCS.OC13 in CS 147 at the Advanced level

Students who pass CS 147 should be able to perform the following tasks:

Task 1: Given the inputs of a latch or flip-flop in a timing diagram, draw the outputs.

Task 2: Given the delays of components in a datapath, the schematics of the datapath and control, and the instruction set, calculate the minimum clock cycle time for the datapath and control.

Task 3: Given the characteristics (size, line size, associativity) of a cache, write the bits used to index the cache, and the bits used to match the tag, or given the contents of a cache, and a memory access, determine whether data is found on the cache.

Task 4: Given the parameters of virtual memory (page size, directory page size, entry size, physical memory size etc.), and given a virtual memory address, state which bits are used to index the Translation Lookaside Buffer, which bits are used to index the directory page, and which bits are used to index the page table, also determine if the required page table and page are in main memory.

 

2.4.2. Results for Assessment of BSCS.OC13 in CS 147 at the Advanced level

No assessment report was submitted by Sin-Min Lee who was supposed to assess this Program Outcome.

 

2.5. Analysis

 The most obvious concern is with the low rates of satisfaction of students from CS 149 Section 3. The instructor notes that "the direct feedback from the students was indeed that the exam questions were too tough.  One student said that they 'were certain they knew all the concepts', but found they couldn't apply them in an exam setting."

Thus, the main problems are that the exam questions were extremely difficult and the students are also weak in mathematics coming into the class.

 

2.6. Recommendations

One instructor suggests providing more homework questions to help students understand and absorb the content better. More calculation questions will be assigned on homework assignments for future semesters so that students will have more practice and be more comfortable with such questions before they encounter them in the exam.

 

3. Program Outcome BSCS.OC14

The ability to explain the structure and function of operating systems, their interactions with computer hardware components, and their influence on the design of software systems

 

3.1. Assessment

BSCS.OC14 was assessed at the Advanced level in CS 149 taught in different sections by Robert Chun and Terry Allen.

 

3.2. Tasks for Assessment of BSCS.OC14 in CS 149 at the Advanced level

Students who pass CS 149 should be able to perform the following tasks:

Task 1:

Describe the tradeoffs between the different Scheduling Algorithms that can be used by an Operating System to manage the CPU using criteria such as fairness, potential for indefinite postponement (starvation), throughput, and turnaround time.  (Assessed with an exam question)

Task 2:

Emulate the different Scheduling Algorithms that can be used by an Operating System to schedule a list of jobs to be run on the CPU such as: FIFO, Shortest Job First, Highest Priority First, and Round Robin.  (Assessed with an exam question)

Task 3:

Explain what Deadlock is, including the four conditions necessary for it to occur in a system.  (Assessed with an exam question)

Task 4:

Explain what a Race Condition is, including how it can be prevented by enforcing mutual exclusion on critical sections via different OS-assisted mechanisms such as semaphores.  Demonstrate proper usage of semaphores in pseudo-code to accomplish process synchronization or mutual exclusion on critical sections.  (Assessed with an exam question)

 

3.3. Results for Assessment of BSCS.OC14 in CS 149 at the Advanced level

The assessment of BSCS.OC14 was completed by Robert Chun in CS 149 Sections 1 and 2.

The assessment of BSCS.OC14 was completed by Terry Allen in CS 149 Section 3.

The overall percentage of passing students able to complete each task is given in the following table:

 

 

Sections 1 and 2

Section 3

Task 1

94%

69%

Task 2

97%

38%

Task 3

90%

20%

Task 4

84%

18%

 

The overall percentage of passing students for a task for a section is calculated by taking the simple average percentage of passing students for all questions supporting that task. 

 

3.4. Analysis

The most obvious concern is with the low rates of satisfaction of students from CS 149 Section 3 for Tasks 2, 3 and 4. The instructor notes that "the direct feedback from the students was indeed that the exam questions were too tough.  One student said that they 'were certain they knew all the concepts', but found they couldn't apply them in an exam setting."

Thus, the main problems are that the exam questions were extremely difficult and the students are also weak in mathematics coming into the class.

 

3.5. Recommendations

One instructor suggests providing more homework questions to help students understand and absorb the content better. More calculation questions will be assigned on homework assignments for future semesters so that students will have more practice and be more comfortable with such questions before they encounter them in the exam.