Computer Science Assessment Process
for
Computer Science BA, Computer Science BS, and Software Engineering BS
degree programs
Computer Science Department
State University of New York at Oswego
Last update: October 2011
1. Educational Objectives
Our departmental mission statement (last updated in 1996) reads:
The Mission of the Computer Science Department at Oswego is to
provide students with a learning environment which will nurture their
growth into competent software developers, computer scientists, and
information system designers. We strive to maintain modern computing
resources, modern software systems and languages, and a diverse
faculty capable of engaging students in meaningful learning activities
steeped in classic computer science yet applicable to contemporary
problems associated with computer and information science.
Our expectations of students graduating from our programs match
those of the ACM in its current
Curricula Recommendations (section 4.2): Graduates possess an
appreciation of the interplay between theory and practice, are
adaptable, attend to rigorous thinking, are familiar with common
themes and principles, have significant project experience, and can
adopt a system-level perspective. Our main differences from typical
programs and common curriculum recommendations stem from a greater
focus on project-based teaching and learning. Additional
information about the structure of our programs may be found in the
CS department's
2009 Self-Study.
2. Learning Outcomes
We expect graduates to demonstrate knowledge and abilities within
each of the six principal outcomes listed below. Each category
defines core competencies, with details that may vary over time to
reflect the continually evolving fields of computing. The three
degree programs, Computer Science BA, Computer Science BS, and
Software Engineering BS share core outcomes, but vary in depth
within categories, as discussed below.
- Programming. Constructing software that meets its
requirements; employing commonly applicable data structures and
algorithms; using and creating APIs, components, services, and
applications; using programming tools, operating systems, and
networked services in the construction of programs.
- Formalisms. Manipulating formal systems including logic,
analytic and discrete mathematics, and models of computation, that
may be used to represent properties of problems and solutions
encountered in computing.
- Systems. Understanding the structure and properties of
common architectural elements of physical computing systems
(including arithmetic units, memory, persistent storage,
communication, and networks), along with those of layered software
systems (including instruction sets, operating systems, runtime
systems, and virtual machines) that impact the structure,
functionality, and performance of programs executing on them.
- Design. Knowing of, selecting and employing the
underlying algorithms, patterns, styles, and design techniques
applicable in the construction of software solutions to common
problems; developing software in a variety of programming
languages, computational styles (e.g., object-oriented,
functional) and modes (e.g., parallel, event-based, networked);
using abstraction, decomposition and composition techniques (e.g.,
modules, frameworks) to create larger software components and
programs from smaller ones; adapting designs to target systems
with varying architectures, resources, features, and limitations.
- Analysis. Using formalisms to understand software
systems and evaluate their correctness, performance, and resource
(memory, processors, communication, etc) requirements; using
empirical analytic methods (including testing, simulation, and
measurement) to augment formal methods; evaluating the impact of
software on its users and society, including matters of usability,
security and privacy based on an understanding of the ethical
standards of the profession.
- Process. Planning, developing, and delivering software
projects, especially those developed by teams of developers;
performing scheduling, quality assurance, and requirements
collection; applying analytic techniques to software verification,
testing, and maintenance; communicating knowledge to others in the
course of collaboration, tutoring, presentations, and
documentation; engaging in ongoing professional development,
life-long learning and process improvement.
Depth Requirements
The CS BS and SE BS programs impose further depth requirements in
outcomes, corresponding to their greater numbers of courses required
(rather than allowed as electives) compared to the CS BA program:
- CS BS Depth. Designing, constructing, and analyzing
solutions based on algorithms, protocols, representations, and
related design techniques, including new or experimental
ideas, applicable to problems encountered in a signficant area
of computing such as artificial intelligence, middleware,
embedded systems, or gaming.
- SE BS Depth. Managing software development lifecycles,
assigning development tasks and roles, performing requirements
analysis, estimation, configuration, and system maintenance and
evolution; systematically applying quality assurance;
understanding problems and solutions encountered in the
development of large scale systems entailing networking,
concurrency, and/or persistent storage, that are developed and
maintained by multiple groups.
Relationship to ABET and ACM
While sometimes expressed differently, program outcomes are consistent
with the "a-k" criteria used by
ABET for computing and engineering
programs, as noted below in brackets. These are in turn consistent
with current
ACM
curricular recommendations. The ACM guidelines include more
detailed outcomes that we use to guide assessment of component
learning objectives (see Section 3).
- an ability to apply knowledge of mathematics, science, and
engineering [design, analysis]
- an ability to design and conduct experiments, as well as to
analyze and interpret data [analysis]
- an ability to design a system, component, or process to meet
desired needs within realistic constraints [design]
- an ability to function on multi-disciplinary teams [process]
- an ability to identify, formulate, and solve engineering
problems[programming, design, analysis]
- an understanding of professional and ethical responsibility
[analysis]
- an ability to communicate effectively [process]
- the broad education necessary to understand the impact of
engineering solutions in a global, economic, environmental, and
societal context [analysis]
- a recognition of the need for, and an ability to engage in
life-long learning [process]
- a knowledge of contemporary issues [process]
- an ability to use the techniques, skills, and modern
engineering tools necessary for engineering practice. [programming]
3. Assessment
Our program outcomes cover knowledge and abilities that span
multiple courses. Appendix I lists more specific per-course learning
objectives that together are used to assess the more general
outcomes. Our primary assessment relies on relevant student work
products (programming projects and associated artifacts) and exams,
quizes, or test items from these courses. When applicable, these
rubrics match detailed objectives recommended in current ACM
guideline documents (mainly
the
2008 update). In keeping with the project-based focus of our
curriculum, we rely on work products when possible, to better assess
that students can effectively use knowlegde in developing software.
In addition to course-based assessment, our upcoming SE BS program
evaluation will also entail surveys of graduates and their
employers, as well as consultation with the Engineering Advisory
Committee.
Historical note: Our department initially used a single
comprehensive capstone exam. However, the results were too
contaminated by sampling and recency effects to serve as a reliable
instrument. The department does not yet have a full evaluation
using the current process.
4. Evaluation
Our assessment process is designed to be consistent with
ABET recommendations, and to be
useful in the continuous improvement of our programs. We plan to
perform a full ABET-compliant assessment of at least the SE BS
program within the next two years. This effort will then be used
to further improve curriculum and assessment.
Appendix I : Per-Course Objectives
The following (in-progress) lists specific learning objectives from
each course contributing to associated program outcomes. The listed
objectives do not necessarily cover all requirements of any given
course, only those that contribute to program outcomes. Some sets
of objectives overlap because not all courses are required of all
students, yet any set of electives must together provide minimal
compliance. In our project-based curriculum, the entries for most
electives are similar because they entail development of significant
software projects, collaboration, written and/or oral reports, and
they cover current trends in their fields and their impact.
Graduates across all programs take approximately ten courses
involving programming; each contributes about 10% of the total
coverage, but most of the contributions of earlier courses are
prerequisite to those in later courses where they are measured. We
do not include here include cognate requirements in Math and Science
or courses colisted with other programs,
Key for bracketed course annotations:
C: Required for CS BA and BS programs
S: Required for SE program
A: One of two choices for requirements (applies only to CSC222/322)
T: Required for a CS or SE BS concentrate
E: free elective
Courses in common core for all programs
- CSC212 CS1 [C,S}
- 1. Programming
- Produce programs entailing elementary constructs
including: variables, expressions, conditionals, loops,
functions, methods, objects, and arrays
- Produce correct conditional statements for a collection
of related conditions
- Produce an iterative structure that makes progress and
correctly terminates in the presence of the required
condition
- Implement a program that can traverse a recursive data
structure
- Use an integrated development environment (IDE) to
build and run a program
- 2. Formalisms
- Simplify a given boolean expression
- 3. Systems
- 4. Design
- Design a collection of classes with associated
attributes and methods that collectively contribute to a
solution
- Identify the appropriate parameters and return types
necessary to support method operation
- Correctly specify bounds on user input
- 5. Analysis
- Understand the impact of infinite loops on available
computing resources
- 6. Process
- Package a solution as a deployable program.
- Cite the work of others that informs/contributes to
their finished product
- CSC221 Foundations of CS [C, S]
- 1. Programming
- 2. Formalisms
- Formulate and manipulate logic expressions for a variety
of applications
- Describe and recognize basic proof techniques
- Understand, define and manipulate discrete structures
such as functions, relations, and sets
- Understand recursion as a basic paradigm for computing
with functions
- Perform proofs by mathematical induction
- Understand formal definitions of machine models.
- Prove the equivalence of languages described by finite
state machines and regular expressions.
- Construct pushdown automata and the equivalent context
free grammars.
- Prove the equivalence of languages described by pushdown
automata and context free grammars
- Use the pumping lemma and closure properties to prove
particular problems cannot be solved by finite
automata.
- Describe characteristics of a Turing machine.
- Explain the theoretical limits on computational
solutions of undecidable and inherently complex
problems.
- 3. Systems
- 4. Design
- Design finite automata to recognize string patterns
- Construct finite automata with specific
properties and the equivalent regular expressions.
- Describe simple recursive data structure and
algorithms
- 5. Analysis
- Understand induction as a basic paradigm for proving
properties of recursive functions
- 6. Process
- CSC241 CS2 [C, S]
- 1. Programming
- Implement programs that entail common data types including
lists, stacks, queues, trees.
- Implement programs that entail sorting and searching algorithms.
- 2. Formalisms
- Explain why big-O analysis ignores constant factors
- Explain the relation between nested loops and polynomial
complexity.
- 3. Systems
- 4. Design
- Design classes and APIs for collections and aggregates
- Use existing APIs in the design of application programs
- Design a recursive solution to a given problem
- Use an event-driven framework to design a program supporting
user interaction
- 5. Analysis
- Characterize the time complexity of common data structures
using big-O notation.
- Emprirically compare the performance of multiple solutions
to a common problem
- 6. Process
- Develop at least one program in multiple stages, each
with added requirements, functionality, and tests.
- CSC365 Data Structures and Algorithms [C, S]
- 1. Programming
- Produce programs that entail the construction and/or
use of balanced trees, hash tables, persistent data
structures, and graph algorithms.
- 2. Formalisms
- Characterize asymptotic properties of algorithms with
constant, linear, polynomial, logarithmic, and exponential
complexity.
- 3. Systems
- Describe the properties of file systems and storage media
supporting persistent data structures
- 4. Design
- Design APIs for representing aggregated data.
- Describe alternative representation strategies
for trees, graphs, and tables, the algorithms they support,
and their suitability for various applications
- Describe general categories of algorithms including
brute-force combinatorial, greedy, divide-and-conquer,
and stream (or String) processing.
- Describe common graph algorithms, including those for
traversal, path, and spanning tree computations
- 5. Analysis
- Analyze the time and space complexity, and empirical
tradeoffs of alternative data structures with similar
functionality.
- Identify differences among best, average, and
worst case behaviors of an algorithm.
- 6. Process
- Develop at least one layered program that entails
construction and testing of a component providing a data
representation, and an application that uses it.
- CSC344 Programming Languages [C, S]
- 1. Programming
- Process some representation of code for some purpose,
such as an interpreter, a serializer, an expression
optimizer, or a documentation-generator.
- Write programs using multiple encapsulation mechanisms,
including function closures and object-oriented
interfaces, in multiple programming languages.
- Implement components that avoid assigning to mutable state or
considering object identity.
- Write event handlers for use in reactive systems, such as GUIs.
- 2. Formalisms
- Describe the syntax of a fragment of a programming language
using a context free grammar
- Describe a fragment of a type system in terms of
Set operations.
- 3. Systems
- Explain benefits and limitations of language support for
automatic memory management (garbage collection), and
reason about memory-management errors including memory
leaks and dangling-pointer dereferences.
- 4. Design
- Define and use operations on aggregates (loops,
iterators, mapping operators, etc) using idioms supported
in multiple programming languages
- Contrast object-oriented inheritance (code-sharing and
overriding) from subtyping (the idea of a subtype behaving like a
supertype).
- 5. Analysis
- Rigorously determine that a version of a component (a)
without mutable state (b) with mutable state, ensures a given
property or result.
- Distinguish a language definition (what constructs mean) from a
particular language implementation (compiler vs. interpreter,
run-time representation of data objects, etc.).
- Resolve a question about an unfamiliar situation (for
example, about naming scopes) by reference to a language
specification document.
- 6. Process
- Explain the benefits and limitations of static typing
in different modes of software development, including prototyping
and quality assurance.
- Describe criteria for choosing a programming language
for a given software development project.
- CSC380 Software Engineering [C, S]
- 1. Programming
- Effectively use the API of a target platform to
deliver required functionality
- Effectively use collaboration tools such as wikis and
code repositories
- 2. Formalisms
- Characterize the different models supported
by UML or related modeling notations.
- 3. Systems
- 4. Design
- Use the Unified Modeling Language (UML) to produce
diagrams of classes and object interaction in their
systems
- Generate Use Cases that describe user interaction with
the system
- Describe the challenges associated with scaling the
team project if the users are increased by several orders
of magnitude
- 5. Analysis
- Create a test plan to effectively isolate defects and
verify correct operation of the final project
- Produce a correct UML interaction (State, Sequence,
Communication, etc.) diagram given a problem
specification
- Identify and use accepted practices for code testing
and inspection
- Identify and describe the stakeholders associated with
software projects and their motivations
- Identify the security technologies (encryption, access
controls, etc.) necessary for their projects
- Prepare a usability testing protocol that appropriately
involves human subjects
- 6. Process
- Produce software working in a group of developers
- Produce a requirements document that articulates
verifiable system attributes
- Accurately record efforts contributing to a
project
- Produce a Build Plan that outlines a series of
incremental deliverables
- Articulate the differences among accepted development
paradigms such as Top down, Bottom up, Agile
- Provide regular updates to project sponsor on the
status of the development
- Prepare and present regular builds demostrating
incremental functionality
- Present a final oral report that demonstrates
functionality of the project
- Produce a written narrative describing the project and
reflecting on the process.
- CSC222 Computer Organization [A]
- 1. Programming
- Produce programs written in an assembly language
for a given instruction set that entail elementary constructs
including: variables, conditionals, loops, and subroutines.
- 2. Formalisms
- Describe given computer arithmetic operations using boolean algebra
-
- 3. Systems
- Describe the basic units and architecture of Von Neumann machines
and the steps entailed in executing programs
- Describe the design choices surrounding pipelining,
superscalar CPUs and related CPU design features and their
impact on program execution
- Describe the design choices surrounding caches and
memory hierarchies and their impact on program execution
- Describe the design choices surrounding DMA,
interrupts and related IO control
- Describe the tradeoffs and interactions between
hardware and software control of system functionality
such as memory address management
- Describe the differences between non-virtualized
and virtualized operationing systems, and their impact
on program execution.
- 4. Design
- 5. Analysis
- 6. Process
- CSC322 Systems Programming [A]
- 1. Programming
- Produce programs written in a low-level systems
programming language (such as C) entailing elementary
constructs including: variables, expression on bits,
conditionals, loops, and operating system calls.
- Implement a program that manages its own memory and/or
IO processing with minimal reliance on system or run-time
support.
- Implement a program that entails control flow (for
example, interrupts, traps, non-local jumps) not typically
supported in higher-level languages.
- 2. Formalisms
- 3. Systems
- Describe the underlying representation choices for given
data types (including pointers)
- Describe the common components and structure of common
run-time support systems including linkers, buffered IO,
and shared libraries.
- 4. Design
- Construct the API (for example as a C header file)
for a component with given functionality.
- 5. Analysis
- Measure the performance of programs that have similar
functionality but rely on alternative system functions.
- 6. Process
Courses required for a CSBS concentrate or SEBS
- CSC338 Robotics [T]
- 1. Programming
- 2. Formalisms
- 3. Systems
- 4. Design
- 5. Analysis
- 6. Process
- CSC366 Comp Models of Cognitive Process [T]
- 1. Programming
- 2. Formalisms
- 3. Systems
- 4. Design
- 5. Analysis
- 6. Process
- CSC375 Parallel Computing [T]
- 1. Programming
- Use mutual exclusion to avoid a given race condition
- Write a program that correctly terminates when all of a set of
concurrent tasks have completed
- Use a properly synchronized queue to buffer data passed among
activities
- 2. Formalisms
- 3. Systems
- 4. Design
- Distinguish using computational resources for a faster answer
from managing efficient access to a shared resource
- Explain when and why multicast or event-based messaging can be
preferable to alternatives
- Describe the relative merits of optimistic versus conservative
concurrency control under different rates of contention among
updates
- Explain why checks for preconditions, and actions based on these
checks, must share the same unit of atomicity to be effective
- Describe at least one design technique for avoiding liveness
failures in programs using multiple locks or semaphores
- 5. Analysis
- Give an example of a scenario in which blocking message sends can
deadlock
- Give an example of an ordering of accesses among concurrent
activities that is not sequentially consistent
- Give an example of a scenario in which an attempted optimistic
update may never complete
- Distinguish data races from higher level races
- Explain why synchronization is necessary in a specific
multithreaded program
- Describe how data distribution/layout can affect an algorithm’s
communication costs
- Write a test program that can reveal a concurrent programming
error; for example, missing an update when two activities both
try to increment a variable
- 6. Process
- CSC416 AI Programming Languages [T]
- 1. Programming
- 2. Formalisms
- 3. Systems
- 4. Design
- 5. Analysis
- 6. Process
- CSC420 GUIs [T]
- 1. Programming
- 2. Formalisms
- 3. Systems
- 4. Design
- 5. Analysis
- 6. Process
- CSC435 Web Services [T]
- 1. Programming
- 2. Formalisms
- 3. Systems
- 4. Design
- 5. Analysis
- 6. Process
- CSC436 Networked Systems [T]
- 1. Programming
- 2. Formalisms
- 3. Systems
- 4. Design
- 5. Analysis
- 6. Process
- CSC444 Compiler Construction [T]
- 1. Programming
- Implement a scanner and parser for a complete small
language, and/or modify/extend one for a full production
language.
- Implement static analyses, including type-checking and
symbol resolution to identify binding occurrences.
- Implement code generation for a given target -- a
machine, virtual machine, or a lower-level programming
language.
- 2. Formalisms
- 3. Systems
- 4. Design
- Design software accommodating the essential steps for
compiling source code to a target representation.
- Use formal grammars to specify the syntax of languages
- Use declarative tools to design or generate parsers and scanners
- Identify parsing issues introduced by ambiguity,
associativity, precedence
- Design an abstract syntax representation supporting a
set of static analyses.
- Identify static semantics issues introduced by various
scoping rules and type systems.
- Characterize target-platform characteristics including
registers, instructions, bytecodes, dynamic memory management,
layouts for objects and activation records
- Represent program dynamics as data flow using basic
blocks, control-flow graphs,static single assignment, and/or
alternative approaches
- Design data flow solutions for problems including
available expression analysis and liveness analysis
- Understand alternative approaches to implementing
non-elementatry control flow (such as exceptions) and
invocation (such as object-oriented method dispatching).
- 5. Analysis
- Validate conformance of a given compiler action to a
given property required in a language specification,
using formal and/or empirical techniques.
- 6. Process
- Deliver a working compiler in stages.
- CSC445 Computer Networks [T]
- 1. Programming
- Write a program that performs any required marshalling and
conversion into message units such as packets to communicate
interesting data among two hosts, and measure its performance
when run on different hosts or networks.
- Implement a simple server; for example a spell checking service
- Implement a reliable protocol (for example, a subset of
TCP) using an unreliable one (for example, UDP).
- Implement a distributed program (for example, a shared
whiteboard) entailing group communication.
- 2. Formalisms
- 3. Systems
- 4. Design
- Explain why synchronization constructs such as simple locks are
not useful in the presence of distributed faults
- Give examples of problems for which consensus algorithms such as
leader election are required
- Explain why no distributed system can be simultaneously
consistent, available, and partition tolerant
- Explain the tradeoffs among overhead, scalability, and
fault-tolerance when choosing a stateless design versus others
for a given service
- Describe the scalability challenges associated with a service
growing to accommodate many clients, as well as those associated
with a service only transiently having many clients
- 5. Analysis
- Distinguish network faults from other kinds of failures
- Measure the observed throughput and response latency across hosts
in a given network
- 6. Process
- CSC454 System Simulation and Virtual Worlds [T]
- 1. Programming
- 2. Formalisms
- 3. Systems
- 4. Design
- 5. Analysis
- 6. Process
- CSC455 Computer Game Programming [T]
- 1. Programming
- Implement a computer game.
- 2. Formalisms
- 3. Systems
- 4. Design
- Design a computer game.
- Understand the historical origins of computer games and
their impact on game design
- Understand interaction mechanisms and their use in
games.
- 5. Analysis
- 6. Process
- Apply knowledge of game development practice in a team
that includes non-programmers
- CSC459 Database Systems [T]
- 1. Programming
- 2. Formalisms
- 3. Systems
- 4. Design
- 5. Analysis
- 6. Process
- CSC466 AI [T]
- 1. Programming
- 2. Formalisms
- 3. Systems
- 4. Design
- 5. Analysis
- 6. Process
- CSC470 Computer Graphics [T]
- 1. Programming
- 2. Formalisms
- 3. Systems
- 4. Design
- 5. Analysis
- 6. Process
- CSC480 Software Design [S]
- 1. Programming
- 2. Formalisms
- 3. Systems
- 4. Design
- 5. Analysis
- 6. Process
- CSC495 SE Project I [S]
- 1. Programming
- 2. Formalisms
- 3. Systems
- 4. Design
- 5. Analysis
- 6. Process
- CSC496 SE Project II [S]
- 1. Programming
- 2. Formalisms
- 3. Systems
- 4. Design
- 5. Analysis
- 6. Process
Free Electives
- CSC320 Numerical Methods [E]
- 1. Programming
- 2. Formalisms
- 3. Systems
- 4. Design
- 5. Analysis
- 6. Process
- CSC332 Cryptology [E]
- 1. Programming
- 2. Formalisms
- 3. Systems
- 4. Design
- 5. Analysis
- 6. Process
- CSC443 Operating Systems [E]
- 1. Programming
- 2. Formalisms
- 3. Systems
- 4. Design
- 5. Analysis
- 6. Process
- CSC465 Algorithms [E]
- 1. Programming
- 2. Formalisms
- 3. Systems
- 4. Design
- 5. Analysis
- 6. Process
|