Evaluation

Evaluation is an ongoing, repetitive process that underpins every aspect of the Systems Engineering Process.

Evaluation is defined as making a judgement about the amount, number, or value of something. We frequently think of evaluation as an assessment or grade. There is no assessment or grade assigned to an evaluation.

Evaluation is where you stop and think about how well your work is going. You look at your ideas, research, and decisions and ask "Is this actually working?", and "Why or why not?".

Good evaluation uses evidence (tests, results, feedback, mistakes) to explain what went well, what didn’t, and what you would change next time. It’s not just saying “it was good” or “it didn’t work” — it’s explaining why and how you know.

There is a quote from a famous education specialist, John Dewey:

$$ \text{We do not learn from experience; we learn from reflecting on experience.} $$

Evaluation is just reflecting on our experiences and practices.

Framework

There are countless frameworks and models for evaluation. The framework provided here is a similar to the Gibbs Reflective Cycle, and the Reflective Learning Cycle by Juliana Kaya Prpic of the University of Melbourne. Just like the Systems Engineering Process, this is not the only way to reflect, but just one way.

You can use this structure to help write your evaluations or reflections on the parts of the Systems Engineering Process that you have completed.

The evaluation / reflective cycle provided includes 5 stages.

• Description. What happened?
• Feelings. How you were feeing?
• Understanding. Why it happened?
• Lessons Learned. What does this mean?
• Action. What happens next?

Description

Here is essentially an info dump on what happened. Explain what you set out to do, what you did, what the outcomes were. Avoid providing personal analysis like "it was bad", "it was great" as those ideas come later.

• What happened?
• What did you try for the first time?
• What did you do for the nth time?
• What shortcuts did you take?

Feelings

Feelings link us to experience. If you're not frustrated and over it at some point, then you're not engineering. Understanding how you were feeling will help you make sense of your own actions and thought processes. Were you happy, sad, frustrated, angry, tired? Tired and frustrated? Exhausted but proud? Write it down, but this is not venting or your angsty teen diary. These are feeling that will inform our understanding.

• How did you feel at the start of the task?
• How did your feelings change as the task progressed?
• What moments made you feel confident, frustrated, stressed, or proud?
• How did your emotions affect the decisions you made?
• Did any feelings help you work better? How?
• Did any feelings slow you down or cause mistakes?
• How did you respond when something went wrong?
• How did working alone or with others affect how you felt?
• Looking back, do you think your emotional response was helpful or unhelpful?

Understanding

Take time to understand what happened and explore the whole experience. This is the most important part of the evaluation process. Ask and answer questions that elicit meaning from the experience.

• Why do you think this happened the way it did?
• What factors influenced the outcome? (time, skills, resources, decisions)
• What assumptions did you make at the start?
• How did your choices affect the result?
• How did your emotions affect the result?
• What parts of the system/process worked as expected?
• What parts did not behave the way you expected?
• How did other people, tools, or constraints influence the situation?
• What evidence do you have to support your explanation?

Lessons Learned

We want to walk away from our experiences better. Stronger. Smarter. More patient. More detailed. More... whatever you need to make an improvement.

• What did this experience reveal about how you work?
• What patterns do you notice in your decisions or mistakes?
• What surprised you about the outcome?
• What assumptions were challenged or proven wrong?
• What strengths did this experience reveal?
• What weaknesses became clear through this experience?
• What did you learn about the system, process, or materials that you didn’t know before?
• What did this experience teach you about planning, testing, or managing time?
• How has your understanding of the task or problem changed?
• What is one key insight you will take away from this experience?

Action

Where to next?

• What will you change next time?
• What will you keep doing?
• What specific step will you take first?
• What will you test or check earlier next time?
• What skills or knowledge will you work on improving?
• What resources or support will you use next time?
• How will you know if your change has worked?
• When will you apply this action (next lesson, next build, next project)?

Do not simply write "I'll try harder". Give yourself a clear action, timeframe, and specific outcome.

Types of Reflection

There are different sizes of evaluations that you will write throughout the project.

They are weekly/ongoing, checkpoint/milestone, and final. Each serves a different purpose, and informs the next part of the process.

Weekly / Ongoing

Checkpoint / Milestone

Final Evaluation