如何做好软件工程方面研究--mini教程，Mary shaw(CMU)

软件工程研究问题的类型

问题类型

例子

Method or means of

development

How can we do/create/modify/evolve (or automate doing) X?

What is a better way to do/create/modify/evolve X?

Method for analysis

or evaluation

How can I evaluate the quality/correctness of X?

How do I choose between X and Y?

Design, evaluation, or

analysis of a

particular instance

How good is Y? What is property X of artifact/method Y?

What is a (better) design, implementation, maintenance, or adaptation for application X?

How does X compare to Y?

What is the current state of X / practice of Y?

Generalization or

characterization

Given X, what will Y (necessarily) be?

What, exactly, do we mean by X? What are its important characteristics?

What is a good formal/empirical model for X?

What are the varieties of X, how are they related?

Feasibility study or

exploration

Does X even exist, and if so what is it like?

Is it possible to accomplish X at all?

Types of software engineering research results

Type of result

Examples

Procedure or

technique

• New or better way to do some task, such as design, implementation, maintenance,
• measurement, evaluation, selection from alternatives; includes techniques for
• implementation, representation, management, and analysis; a technique should be
• operational—not advice or guidelines, but a procedure

Qualitative or

descriptive model

• Structure or taxonomy for a problem area; architectural style, framework, or design pattern;
• non-formal domain analysis, well-grounded checklists, well-argued informal
• generalizations, guidance for integrating other results, well-organized interesting observations

Empirical model

• Empirical predictive model based on observed data

Analytic model

• Structural model that permits formal analysis or automatic manipulation

Tool or notation

• Implemented tool that embodies a technique; formal language to support a technique or model
• (should have a calculus, semantics, or other basis for computing or doing inference

Specific solution,

prototype, answer,

or judgment

• Solution to application problem that shows application of SE principles – may be design,
• prototype, or full implementation; careful analysis of a system or its development, result of
• a specific analysis, evaluation, or comparison

Report

• Interesting observations, rules of thumb, but not sufficiently general or systematic to rise to the
• level of a descriptive model.

Awful

▼

我完全一般的解决了...

（除非你真的做到了）

Bad

▼

I worked on galumphing.(or studied, investigated, sought,explored)

Poor

▼

I worked on improving galumphing.(or contributed to, participated in,helped with)

Good

▲

l  I showed the feasibility of composing blitzing with flitzing.

l  I significantly improved the accuracy of the standard detector.(or proved, demonstrated, created,established, found, developed)

Better

▲

l  I automated the production of flitz tables from specifications.

l  With a novel application of the blivet transform, I achieved a 10% increase in speed and a 15% improvement in coverage over the standard method.

Awful

▼

The galumphing problem has attracted much attention [3,8,10,18,26,32,37]

Bad

▼

Smith [36] and Jones [27] worked on galumphing.

Poor

▼

Smith [36] addressed galumphing by blitzing, whereas Jones [27] took a flitzing approach.

Good

▲

Smith’s blitzing approach to galumphing[36] achieved 60% coverage [39].Jones [27] achieved 80% by flitzing,but only for pointer-free cases [16].

Better

▲

Smith’s blitzing approach to galumphing[36] achieved 60% coverage [39].Jones [27] achieved 80% by flitzing,but only for pointer-free cases [16].We modified the blitzing approach to use the kernel representation of flitzing and achieved 90% coverage while

relaxing the restriction so that only cyclic data structures are prohibited.

Types of software engineering research validation

Type of validation

Examples

Analysis

I have analyzed my result and find it satisfactory through rigorous analysis, e.g. …

For a formal model … rigorous derivation and proof

For an empirical model … data on use in controlled situation

For a controlled experiment … carefully designed experiment with statistically significant

results

Evaluation

Given the stated criteria, my result...

For a descriptive model … adequately describes phenomena of interest …

For a qualitative model … accounts for the phenomena of interest…

For an empirical model … is able to predict … because …, or

… generates results that fit actual data …

Includes feasibility studies, pilot projects

Experience

My result has been used on real examples by someone other than me, and the evidence of its

correctness/usefulness/effectiveness is …

For a qualitative model … narrative

For an empirical model or tool … data, usually statistical, on practice

For a notation or technique … comparison of systems in actual use

Example

Here’s an example of how it works on

For a technique or procedure …a "slice of life" example based on a real system …

For a technique or procedure …a system that I have been developing …

For a technique or procedure … a toy example, perhaps motivated by reality

The "slice of life" example is most likely to be convincing, especially if accompanied by an

explanation of why the simplified example retains the essence of the problem being solved.

Toy or textbook examples often fail to provide persuasive validation, (except for standard

examples used as model problems by the field).

Persuasion

I thought hard about this, and I believe passionately that ...

For a technique … if you do it the following way, then …

For a system … a system constructed like this would …

For a model … this example shows how my idea works

Validation purely by persuasion is rarely sufficient for a research paper. Note, though, that if the

original question was about feasibility, a working system, even without analysis, can suffice

Blatant assertion

No serious attempt to evaluate result. This is highly unlikely to be acceptable