MSc: Applied statistics and experiments in science and engineering
Applied statistics and experiments in science and engineering
- Course name: Applied statistics and experiments in science and engineering
- Code discipline:
- Subject area:
Short Description
This course covers the following concepts: Goal-Question-Metric approach; Experimental design; Basics of statistics.
Prerequisites
Prerequisite subjects
Prerequisite topics
Course Topics
| Section | Topics within the section |
|---|---|
| Concept of measuring |
|
| Fundamentals of statistics |
|
Intended Learning Outcomes (ILOs)
What is the main purpose of this course?
The main purpose of this course is to present the fundamentals of empirical methods and fundamental statistics to the future software engineers and data scientists, on one side providing the scientific fundamentals of the disciplines, and on the other anchoring the theoretical concepts on practices coming from the world of software development and engineering. As a side product, the course also refreshes the basics of statistics, providing the basis for more advanced statistical courses in the following semester(s) of study.
ILOs defined at three levels
Level 1: What concepts should a student know/remember/explain?
By the end of the course, the students should be able to ...
- Remember the fundamentals of statistics and probability theory
- Remember the basic models for experimentation and quasi-experimentation
- Remember the specifics and purpose of different measurement scales
- Distinguish between random variable and random process
- Explain the difference between the correlation and causation
Level 2: What basic practical skills should a student be able to perform?
By the end of the course, the students should be able to ...
- the value of experimentation for software engineers and data scientists
- the basic concepts of an hypothesis
- the concept of correlation
- the fundamental laws in statistics
- the concept of Goal-Question-Metric approach
Level 3: What complex comprehensive skills should a student be able to apply in real-life scenarios?
By the end of the course, the students should be able to ...
- Apply Goal-Question-Metric approach in practice
- Apply the fundamental principles of experimental design
- Apply reduction to quasi-experimentation experimental design
- Apply statistics and probability theory in practice
- Apply hypothesis testing technique in software analysis
Grading
Course evaluation
The evaluation is based on two parts:
- A student stating the grade that s/he wants to have
- An assessment based on a sequence of components based on the grade that the student wants
Notice that failing any component means failing the entire course, so aiming at a higher grade might mean that the student will fail the course, while if s/he aimed at a lower s/he would have passed.
| Component | Actual points | Required for Grade3 |
|---|---|---|
| Class participation | 15 | A, B, C |
| Quizzes (weekly evaluations) | 20 | A, B, C |
| Midterm at the end of the 8th Lecture | 20 | A, B, C |
| Final Written Exam at the end of the course | 25 | A, B |
| Final Project with regular deliverables2 | 20 | A |
1 Absences from a quiz will trigger a 0, however, 3 lowest grades will be disregarded from the computation of the average of this component.
2 Requires a report describing rigorously all the work done, the report needs to be written incrementally in Overleaf.
3 The student needs to pass the corresponding components for each grade to get it. Specifically, the student who wants grade C, then s/he needs to pass only the first two components whereas the student who wants grade B, needs to pass the first three components. The student who wants grade A, needs to pass all components. In case of any situation, the instructors can ask students to come to an oral exam to finalize the grade.
Each component apart from weekly evaluations will be assessed on a scale 0-10, where 6 is the minimum passing grade. In case of exceptional work a 10 cum laude will be assigned, with a numeric value from 10 to 13 at the discretion of the instructor. The weekly evaluations will be initially graded on a scale 0-100 weekly and then the overall grade will be assembled on a scale 0-10. However, the grаdes will not be a simple linear combination, since to get a passing grade, each of the parts of the course evaluation need to be performed in at least a sufficient way, that is, obtaining a grade of at least 6 out of 10 in each; failing to achieve a sufficient grade in any of the parts will trigger a failure in the overall course.
Course grading range4
| Actual range | |
|---|---|
| A. Excellent | 95-100 |
| B. Good | 70-94 |
| C. Satisfactory | 50-69 |
| D. Poor | 0-49 |
4 Failing any part of the evaluation will trigger a failure in the entire course
Retakes
Retakes will be run as comprehensive oral exam, where the student will be assessed the acquired knowledge coming from the textbooks, the lectures, the tutorials, the labs, and the additional required reading material, as supplied by the instructor. During such comprehensive oral the student could be asked to solve exercises and to explain theoretical and practical aspects of the course.
Recommendations for students on how to succeed in the course
Resources, literature and reference materials
Open access resources
- Donald T. Campbell and Julian C. Stanley. Experimental and Quasi-Experimental Designs for Research. Rand McNally College Publishing, 1963
- Larry Wasserman. All of Statistics: A Concise Course in Statistical Inference. Springer Texts in Statistics. Springer, New York, 2004. ISBN 978-1-4419-2322-6. doi: 10.1007/978-0-387-21736-9
- Oliver Laitenberger and Dieter Rombach. Lecture Notes on Empirical Software Engineering. chapter (Quasi) Experimental Studies in Industrial Settings, pages 167–227. World Scientific Publishing Co., Inc., River Edge, NJ, USA, 2003. ISBN 981-02-4914-4
- Rini van Solingen and Egon Berghout. The Goal/Question/Metric Method: a practical guide for quality improvement of software development. The McGraw-Hill Companies, Cambridge, England, 1999. ISBN 077-709553-7.
- Andrea Janes and Giancarlo Succi. Lean Software Development in Action. Springer, Heidelberg, Germany, 2014. ISBN 978-3-662-44178-7. doi: 10.1007/978-3-642-00503-9
Closed access resources
Software and tools used within the course
Teaching Methodology: Methods, techniques, & activities
Activities and Teaching Methods
| Learning Activities | Section 1 | Section 2 |
|---|---|---|
| Midterm evaluation | 1 | 1 |
| Testing (written or computer based) | 1 | 1 |
| Discussions | 1 | 1 |
Formative Assessment and Course Activities
Ongoing performance assessment
Section 1
| Activity Type | Content | Is Graded? |
|---|---|---|
| Question | What are the phases of GQM? How are they connected to each other? What are steps of GQM method? | 1 |
| Question | What does SWOT mean? | 1 |
| Question | What is the measurement? | 1 |
| Question | How measurement can help us to understand, control and improve development process?? | 1 |
| Question | What does Representation Condition mean? | 1 |
| Question | What is the Measurement Scale? | 1 |
| Question | What are characteristics of a good measurement? What is the difference between validity and reliability? | 1 |
| Question | Which benefits the GQM provides to you as a Software Engineer / Data Scientist? | 0 |
| Question | Imagine your goal is to ”increase availability of some software system”. Provide Questions and Metrics for this goal. | 0 |
| Question | What is measurement Reliability and measurement Validity? What are the differences between the two? Provide an example of reliable, but invalid measurement and an example of valid, but unreliable measurement | 0 |
| Question | Which Measurement Scales do you know? What are the differences between them? Provide examples for each of them. | 0 |
| Question | Provide an example of Representation Condition | 0 |
| Question | Create metrics that measures your study progress, outline the properties of such metrics in terms of subjective vs. objective, direct vs. indirect, etc; detail how you will collect your metrics, concretely and check your metric on reliability & validity | 0 |
Section 2
| Activity Type | Content | Is Graded? |
|---|---|---|
| Question | Describe the three approaches to probability. | 1 |
| Question | Write the fundamental theorem of algebra. | 1 |
| Question | Write the general structure of the OLS equation for one variable. | 1 |
| Question | Write the general structure of the OLS equation for the case of multiple independent variables. | 1 |
| Question | What is the connection between the correlation coefficient and the coefficient of determination? What does each of them show? | 1 |
| Question | State and prove the Law of Large Numbers | 1 |
| Question | State and prove the Central Limit Theorem | 1 |
| Question | Explain what are H0 and H1 in hypothesis testing | 1 |
| Question | Explain the role of the Bonferroni inequality in hypothesis testing | 1 |
| Question | Define and provide examples of sample space, events and probability measure | 0 |
| Question | Write the formula for the coefficients of the simple linear regression. Explain the mathematical procedure you do to derive them and derive them | 0 |
| Question | Fully deduce the value of the coefficient in OLS equation for multiple independent variables | 0 |
| Question | Calculate the correlation between two functions and explain its meaning | 0 |
| Question | Calculate the Pearson coefficient for the given functions | 0 |
| Question | Write and prove Markov’s inequality. Write and prove Chebyshev’s inequality. How these theorems related to the LLN? | 0 |
| Question | Deduce the MGF for normal distribution | 0 |
| Question | Provide a concrete example of a test, detailing both H0 and H1 | 0 |
| Question | State and prove the Bonferroni inequality | 0 |
Final assessment
Section 1
Section 2
- State the Fenton Measurement theory and explain what is Representation condition?
- Define the steps needed to elaborate a GQM
- Describe the Taylor
- Describe the fundamental theorem of Algebra
- Based on the concept the Taylor theorem and the fundamental theorem of Algebra, explain whether the number of datapoints should be equal, smaller or larger than the number of independent variables (features) and why
- Explain how the GQM can be used to define appropriate number of variables
- For the given function compute its mean, mode, median, standard deviation
- Define the OLS linear regression in the case of one and multiple variables and deduce their parameters
- For the given two functions compute their Covariance, Pearson’s correlation coefficient and describe results
- State and prove weak and strong formulation of LLN
- State Lindeberg–Lévy formulation of CLT and prove it
- Compute LOC, MCC, FI, FO for given code. Describe how to apply MCC metrics and meaning of FI-FO output
- For the given module compute the CK metrics for its classes
The retake exam
Section 1
Section 2