Difference between revisions of "MSc: Empirical Methods"

From IU
Jump to navigation Jump to search
 
(7 intermediate revisions by 2 users not shown)
Line 1: Line 1:
  +
 
= Empirical Methods =
  
= Empirical Methods =
  +
* '''Course name''': Empirical Methods
  +
* '''Code discipline''':
  +
* '''Subject area''':
   
  +
== Short Description ==
−
* <span>'''Course name:'''</span> Empirical Methods
  
  +
This course covers the following concepts: Goal-Question-Metric approach; Experimental design; Basics of statistics.
   
−
== Course Characteristics ==
 +
== Prerequisites ==
   
−
=== Key concepts of the class ===
 +
=== Prerequisite subjects ===
   
−
* Goal-Question-Metric approach
  
−
* Experimental design
  
−
* Basics of statistics
  
   
  +
=== Prerequisite topics ===
−
=== What is the purpose of this course? ===
  
   
−
The main purpose of this course is to present the fundamentals of empirical methods and fundamental statistics to the future software engineers and data scientists, on one side providing the scientific fundamentals of the disciplines, and on the other anchoring the theoretical concepts on practices coming from the world of software development and engineering. As a side product, the course also refreshes the basics of statistics, providing the basis for more advanced statistical courses in the following semester(s) of study.
  
   
−
== Prerequisites ==
 +
== Course Topics ==
  +
{| class="wikitable"
−
This course will benefit from the knowledge of fundamental arithmetics and [http://www.math.utah.edu/online/Math1210/Polycalcnotes.pdf polynomial calculus], and well as knowing [https://youtu.be/cEvgcoyZvB4?t=325 logarithms and exponentiation]. Also [https://www.youtube.com/playlist?list=PLZHQObOWTQDP5CVelJJ1bNDouqrAhVPev this whole playlist] can be helpful.
  
  +
|+ Course Sections and Topics
  +
|-
  +
! Section !! Topics within the section
  +
|-
  +
| Concept of measuring ||
  +
# Measurement: concept, definition and fundamentals of measurement
  +
# Goal-Question-Metric approach
  +
# Representational theory of measurement
  +
# Measurement scales and functions that can be applied to scales
  +
# Experimental designs
  +
|-
  +
| Fundamentals of statistics ||
  +
# Basic concepts of probability theory
  +
# Random variable and random process
  +
# Linear regression
  +
# Correlation and convolution
  +
# Moments and moment generating functions
  +
# Law of Large Numbers
  +
# Central Limit Theorem
  +
# Hypothesis testing
  +
|}
  +
== Intended Learning Outcomes (ILOs) ==
   
  +
=== What is the main purpose of this course? ===
−
Fundamental arithmetics and polynomial calculus, logarithms:
  
  +
The main purpose of this course is to present the fundamentals of empirical methods and fundamental statistics to the future software engineers and data scientists, on one side providing the scientific fundamentals of the disciplines, and on the other anchoring the theoretical concepts on practices coming from the world of software development and engineering. As a side product, the course also refreshes the basics of statistics, providing the basis for more advanced statistical courses in the following semester(s) of study.
   
  +
=== ILOs defined at three levels ===
−
* [https://eduwiki.innopolis.university/index.php/BSc:_Mathematical_Analysis_I CSE201] — Mathematical Analysis I
  
−
* [https://eduwiki.innopolis.university/index.php/BSc:_Mathematical_Analysis_II CSE203] — Mathematical Analysis II
  
−
* [https://eduwiki.innopolis.university/index.php/BSc:_Analytic_Geometry_And_Linear_Algebra_I1 CSE202] — Analytical Geometry and Linear Algebra I
  
−
* [https://eduwiki.innopolis.university/index.php/BSc:_Analytic_Geometry_And_Linear_Algebra_II CSE204] — Analytic Geometry And Linear Algebra II
  
−  
−
== Course Objectives Based on Bloom’s Taxonomy ==
  
−  
−
=== What should a student remember at the end of the course? ===
  
−  
−
By the end of the course, the students should be able to:
  
   
  +
==== Level 1: What concepts should a student know/remember/explain? ====
  +
By the end of the course, the students should be able to ...
 
* Remember the fundamentals of statistics and probability theory
  
* Remember the fundamentals of statistics and probability theory
 
* Remember the basic models for experimentation and quasi-experimentation
  
* Remember the basic models for experimentation and quasi-experimentation
Line 37: Line 54:
 
* Explain the difference between the correlation and causation
  
* Explain the difference between the correlation and causation
   
−
=== What should a student be able to understand at the end of the course? ===
 +
==== Level 2: What basic practical skills should a student be able to perform? ====
  +
By the end of the course, the students should be able to ...
−  
−
By the end of the course, the students should be able to understand:
  
−  
 
* the value of experimentation for software engineers and data scientists
  
* the value of experimentation for software engineers and data scientists
 
* the basic concepts of an hypothesis
  
* the basic concepts of an hypothesis
Line 47: Line 62:
 
* the concept of Goal-Question-Metric approach
  
* the concept of Goal-Question-Metric approach
   
−
=== What should a student be able to apply at the end of the course? ===
 +
==== Level 3: What complex comprehensive skills should a student be able to apply in real-life scenarios? ====
−  
 
By the end of the course, the students should be able to ...
  
By the end of the course, the students should be able to ...
−  
 
* Apply Goal-Question-Metric approach in practice
  
* Apply Goal-Question-Metric approach in practice
 
* Apply the fundamental principles of experimental design
  
* Apply the fundamental principles of experimental design
 
* Apply reduction to quasi-experimentation experimental design
  
* Apply reduction to quasi-experimentation experimental design
 
* Apply statistics and probability theory in practice
  
* Apply statistics and probability theory in practice
−
* Apply hypothesis testing technique in software analysis
 +
* Apply hypothesis testing technique in software analysis
  +
== Grading ==
   
−
=== Course evaluation ===
 +
=== Course grading range ===
  +
{| class="wikitable"
−  
  +
|+
−
The course has two major forms of evaluations:
  
−  
−
* a standard evaluation,
  
−
* for very motivated students, an alternative form of evaluation.
  
−  
−
The '''standard evaluation''' follows.
  
−  
−
<div id="tab:EMCourseGrading1">
  
−  
−
{|style="border-spacing: 2px; border: 1px solid darkgray;"
  
−
|+ Course grade breakdown
  
−
!
  
−
!
  
−
!align="center"| '''Points'''
  
 
|-
  
|-
  +
! Grade !! Range !! Description of performance
−
| Labs/seminar classes (weekly evaluations)
  
−
| 20 <sup>1</sup>
  
 
|-
  
|-
  +
| A. Excellent || 85-100 || -
−
| Interim performance assessment (class participation)
  
−
| 5
  
 
|-
  
|-
  +
| B. Good || 65-84 || -
−
| Midterm
  
−
| 30
  
 
|-
  
|-
  +
| C. Satisfactory || 51-64 || -
−
| Final exam
  
−
| 45
 +
|-
  +
| D. Poor || 0-50 || -
 
|}
  
|}
   
  +
=== Course activities and grading breakdown ===
−
</div>
  
  +
{| class="wikitable"
−  
  +
|+
−
<sup>1</sup> Of which 10 class tests and 10 for lab tests. Absences from a test will trigger a 0, however, the 3 lowest grades will be disregarded from the computation of the average of this component.
  
−  
−
The '''alternative evaluation''' follows.
  
−  
−
<div id="tab:EMCourseGrading2">
  
−  
−
{|style="border-spacing: 2px; border: 1px solid darkgray;"
  
−
|+ Course grade breakdown. Alternative form assumes attendance to all lecture and always a grade above 95% in tests on average (minus 3).
  
−
!
  
−
!
  
−
!align="center"| '''Points'''
  
 
|-
  
|-
  +
! Activity Type !! Percentage of the overall course grade
−
| Labs/seminar classes (weekly evaluations)
  
−
| 20 <sup>1</sup>
  
 
|-
  
|-
  +
| Quiz during each lecture (weekly evaluations) || 11.5
−
| Interim performance assessment (class participation)
  
−
| 5
  
 
|-
  
|-
  +
| Labs classes (weekly evaluations) || 11.5
−
| Midterm
  
−
| 5
  
 
|-
  
|-
  +
| Midterm || 17
−
| Project
 
  +
|-
−
| 70 <sup>2</sup>
  
  +
| Final exam || 60
 
|}
  
|}
   
  +
=== Recommendations for students on how to succeed in the course ===
−
</div>
  
   
−
<sup>1</sup> Of which 10 class tests and 10 for lab tests. Absences from a test will trigger a 0, however, the 3 lowest grades will be disregarded from the computation of the average of this component.
  
−  
−
<sup>2</sup> Requiers a paper describing rigorously the individual experiment, the paper needs to be written incrementally in Overleaf.
  
−  
−
'''In both cases''' each component apart from weekly reviews and tests will be assessed on a scale 0-10, where 6 is the minimum passing grade. In case of exceptional work a 10 cum laude will be assigned, with a numeric value from 10 to 13 at the discretion of the instructor. The weekly reviews component will be initially graded on a scale 0-2 weekly and then the overall grade will be assembled on a scale 0-10.
  
−  
−
The grading, though, is not a simple linear combination of the components above. In particular:
  
−  
−
* failing any part of the evaluation will trigger a failure in the entire course,
  
−
* if there are not failing components, the final grade will be computed as a weighted average of the components above approximated at the highest second digit and then rounded to the closest integer.
  
−  
−
=== Retakes ===
  
−  
−
Retakes will be run as comprehensive oral exam, where the student will be assessed the acquired knowledge coming from the textbooks, the lectures, the labs, and the additional required reading material, as supplied by the instructor. During such comprehensive oral the student could be asked to solve exercises and to explain theoretical and practical aspects of the course.
  
−  
−
=== Grades range ===
  
−  
−
{|style="border-spacing: 2px; border: 1px solid darkgray;"
  
−
|+ Course grading range
  
−
!
  
−
!align="center"| '''Range'''
  
−
|-
  
−
| A. Excellent
  
−
|align="center"| 95-100
  
−
|-
  
−
| B. Good
  
−
|align="center"| 75-94
  
−
|-
  
−
| C. Satisfactory
  
−
|align="center"| 55-74
  
−
|-
  
−
| D. Poor
  
−
|align="center"| 0-54
  
−
|}
  
   
−
=== Resources and reference material ===
 +
== Resources, literature and reference materials ==
   
  +
=== Open access resources ===
 
* Donald T. Campbell and Julian C. Stanley. Experimental and Quasi-Experimental Designs for Research. Rand McNally College Publishing, 1963
  
* Donald T. Campbell and Julian C. Stanley. Experimental and Quasi-Experimental Designs for Research. Rand McNally College Publishing, 1963
 
* Larry Wasserman. All of Statistics: A Concise Course in Statistical Inference. Springer Texts in Statistics. Springer, New York, 2004. ISBN 978-1-4419-2322-6. doi: 10.1007/978-0-387-21736-9
  
* Larry Wasserman. All of Statistics: A Concise Course in Statistical Inference. Springer Texts in Statistics. Springer, New York, 2004. ISBN 978-1-4419-2322-6. doi: 10.1007/978-0-387-21736-9
Line 159: Line 113:
 
* Andrea Janes and Giancarlo Succi. Lean Software Development in Action. Springer, Heidelberg, Germany, 2014. ISBN 978-3-662-44178-7. doi: 10.1007/978-3-642-00503-9
  
* Andrea Janes and Giancarlo Succi. Lean Software Development in Action. Springer, Heidelberg, Germany, 2014. ISBN 978-3-662-44178-7. doi: 10.1007/978-3-642-00503-9
   
−
== Course Sections ==
 +
=== Closed access resources ===
   
−
The main sections of the course and approximate hour distribution between them is as follows:
  
   
  +
=== Software and tools used within the course ===
−
{| style="border-spacing: 2px; border: 1px solid darkgray;"
  
  +
−
|+ '''Course Sections'''
  
  +
= Teaching Methodology: Methods, techniques, & activities =
−
!align="center"| '''Section'''
  
  +
−
! '''Section Title'''
  
−
!align="center"| '''Teaching Hours'''
 +
== Activities and Teaching Methods ==
  +
{| class="wikitable"
  +
|+ Activities within each section
 
|-
  
|-
  +
! Learning Activities !! Section 1 !! Section 2
−
|align="center"| 1
  
−
| Concept of Hypothesis Testing and Experimentation
  
−
|align="center"| 12
  
 
|-
  
|-
  +
| Midterm evaluation || 1 || 1
−
|align="center"| 2
  
  +
|-
−
| Fundamentals of statistics
  
  +
| Testing (written or computer based) || 1 || 1
−
|align="center"| 24
  
−
|}
 +
|-
  +
| Discussions || 1 || 1
−  
  +
|}
−
=== Section 1 ===
  
  +
== Formative Assessment and Course Activities ==
−  
−
==== Section title: ====
 
−
Concept of measuring
 
−  
−
==== Topics covered in this section: ====
  
−  
−
* Measurement: concept, definition and fundamentals of measurement
  
−
* Goal-Question-Metric approach
  
−
* Representational theory of measurement
  
−
* Measurement scales and functions that can be applied to scales
  
−
* Experimental designs
  
−  
−
==== What forms of evaluation were used to test students’ performance in this section? ====
  
   
  +
=== Ongoing performance assessment ===
−
<div id="tab:EMSectionEval1">
  
   
  +
==== Section 1 ====
−
{| style="border-spacing: 2px; border: 1px solid darkgray;"
  
  +
{| class="wikitable"
−
|''' '''
  
  +
|+
−
! '''Yes/No'''
  
 
|-
  
|-
  +
! Activity Type !! Content !! Is Graded?
−
| Development of individual parts of software product code
  
−
|align="center"| 0
  
 
|-
  
|-
  +
| Question || What are the phases of GQM? How are they connected to each other? What are steps of GQM method? || 1
−
| Homework and group projects
  
−
|align="center"| 0
  
 
|-
  
|-
  +
| Question || What does SWOT mean? || 1
−
| Midterm evaluation
  
−
|align="center"| 1
  
 
|-
  
|-
  +
| Question || What is the measurement? || 1
−
| Testing (written or computer based)
  
−
|align="center"| 1
  
 
|-
  
|-
  +
| Question || How measurement can help us to understand, control and improve development process?? || 1
−
| Reports
  
−
|align="center"| 0
  
 
|-
  
|-
  +
| Question || What does Representation Condition mean? || 1
−
| Essays
  
−
|align="center"| 0
  
 
|-
  
|-
  +
| Question || What is the Measurement Scale? || 1
−
| Oral polls
  
−
|align="center"| 0
  
 
|-
  
|-
  +
| Question || What are characteristics of a good measurement? What is the difference between validity and reliability? || 1
−
| Discussions
  
  +
|-
−
|align="center"| 1
  
  +
| Question || Which benefits the GQM provides to you as a Software Engineer / Data Scientist? || 0
−
|}
  
  +
|-
−  
  +
| Question || Imagine your goal is to ”increase availability of some software system”. Provide Questions and Metrics for this goal. || 0
−  
  +
|-
−
</div>
  
  +
| Question || What is measurement Reliability and measurement Validity? What are the differences between the two? Provide an example of reliable, but invalid measurement and an example of valid, but unreliable measurement || 0
−  
  +
|-
−
==== Typical questions for ongoing performance evaluation within this section ====
  
  +
| Question || Which Measurement Scales do you know? What are the differences between them? Provide examples for each of them. || 0
−  
−
# What are the phases of GQM? How are they connected to each other? What are steps of GQM method?
  
−
# What does SWOT mean?
  
−
# What is the measurement?
  
−
# How measurement can help us to understand, control and improve development process??
  
−
# What does Representation Condition mean?
  
−
# What is the Measurement Scale?
  
−
# What are characteristics of a good measurement? What is the difference between validity and reliability?
  
−  
−
==== Typical questions for seminar classes (labs) within this section ====
  
−  
−
# Which benefits the GQM provides to you as a Software Engineer / Data Scientist?
  
−
# Imagine your goal is to ”increase availability of some software system”. Provide Questions and Metrics for this goal.
  
−
# What is measurement Reliability and measurement Validity? What are the differences between the two? Provide an example of reliable, but invalid measurement and an example of valid, but unreliable measurement
  
−
# Which Measurement Scales do you know? What are the differences between them? Provide examples for each of them.
  
−
# Provide an example of Representation Condition
  
−
# Create metrics that measures your study progress, outline the properties of such metrics in terms of subjective vs. objective, direct vs. indirect, etc; detail how you will collect your metrics, concretely and check your metric on reliability &amp; validity
  
−  
−
=== Section 2 ===
  
−  
−
==== Section title: ====
  
−  
−
Fundamentals of statistics
  
−  
−
==== Topics covered in this section: ====
  
−  
−
* Basic concepts of probability theory
  
−
* Random variable and random process
  
−
* Linear regression
  
−
* Correlation and convolution
  
−
* Moments and moment generating functions
  
−
* Law of Large Numbers
  
−
* Central Limit Theorem
  
−
* Hypothesis testing
  
−  
−
==== What forms of evaluation were used to test students’ performance in this section? ====
  
−  
−
<div id="tab:EMSectionEval2">
  
−  
−
{| style="border-spacing: 2px; border: 1px solid darkgray;"
  
−
|''' '''
  
−
! '''Yes/No'''
  
 
|-
  
|-
  +
| Question || Provide an example of Representation Condition || 0
−
| Development of individual parts of software product code
  
−
|align="center"| 0
  
 
|-
  
|-
  +
| Question || Create metrics that measures your study progress, outline the properties of such metrics in terms of subjective vs. objective, direct vs. indirect, etc; detail how you will collect your metrics, concretely and check your metric on reliability & validity || 0
−
| Homework and group projects
  
  +
|}
−
|align="center"| 0
  
  +
==== Section 2 ====
  +
{| class="wikitable"
  +
|+
 
|-
  
|-
  +
! Activity Type !! Content !! Is Graded?
−
| Midterm evaluation
  
−
|align="center"| 1
  
 
|-
  
|-
  +
| Question || Describe the three approaches to probability. || 1
−
| Testing (written or computer based)
  
−
|align="center"| 1
  
 
|-
  
|-
  +
| Question || Write the fundamental theorem of algebra. || 1
−
| Reports
  
−
|align="center"| 0
  
 
|-
  
|-
  +
| Question || Write the general structure of the OLS equation for one variable. || 1
−
| Essays
  
−
|align="center"| 0
  
 
|-
  
|-
  +
| Question || Write the general structure of the OLS equation for the case of multiple independent variables. || 1
−
| Oral polls
  
−
|align="center"| 0
  
 
|-
  
|-
  +
| Question || What is the connection between the correlation coefficient and the coefficient of determination? What does each of them show? || 1
−
| Discussions
  
  +
|-
−
|align="center"| 1
  
  +
| Question || State and prove the Law of Large Numbers || 1
−
|}
  
  +
|-
−  
  +
| Question || State and prove the Central Limit Theorem || 1
−  
  +
|-
−
</div>
  
  +
| Question || Explain what are H0 and H1 in hypothesis testing || 1
−
==== Typical questions for ongoing performance evaluation within this section ====
  
  +
|-
−  
  +
| Question || Explain the role of the Bonferroni inequality in hypothesis testing || 1
−
# Describe the three approaches to probability.
  
  +
|-
−
# Write the fundamental theorem of algebra.
  
  +
| Question || Define and provide examples of sample space, events and probability measure || 0
−
# Write the general structure of the OLS equation for one variable.
  
  +
|-
−
# Write the general structure of the OLS equation for the case of multiple independent variables.
  
  +
| Question || Write the formula for the coefficients of the simple linear regression. Explain the mathematical procedure you do to derive them and derive them || 0
−
# What is the connection between the correlation coefficient and the coefficient of determination? What does each of them show?
  
  +
|-
−
# State and prove the Law of Large Numbers
  
  +
| Question || Fully deduce the value of the coefficient in OLS equation for multiple independent variables || 0
−
# State and prove the Central Limit Theorem
  
  +
|-
−
# Explain what are H0 and H1 in hypothesis testing
  
  +
| Question || Calculate the correlation between two functions and explain its meaning || 0
−
# Explain the role of the Bonferroni inequality in hypothesis testing
  
  +
|-
−  
  +
| Question || Calculate the Pearson coefficient for the given functions || 0
−
==== Typical questions for seminar classes (labs) within this section ====
  
  +
|-
−  
  +
| Question || Write and prove Markov’s inequality. Write and prove Chebyshev’s inequality. How these theorems related to the LLN? || 0
−
# Define and provide examples of sample space, events and probability measure
  
  +
|-
−
# Write the formula for the coefficients of the simple linear regression. Explain the mathematical procedure you do to derive them and derive them
  
  +
| Question || Deduce the MGF for normal distribution || 0
−
# Fully deduce the value of the coefficient in OLS equation for multiple independent variables
  
  +
|-
−
# Calculate the correlation between two functions and explain its meaning
  
  +
| Question || Provide a concrete example of a test, detailing both H0 and H1 || 0
−
# Calculate the Pearson coefficient for the given functions
  
  +
|-
−
# Write and prove Markov’s inequality. Write and prove Chebyshev’s inequality. How these theorems related to the LLN?
  
  +
| Question || State and prove the Bonferroni inequality || 0
−
# Deduce the MGF for normal distribution
  
  +
|}
−
# Provide a concrete example of a test, detailing both H0 and H1
  
  +
=== Final assessment ===
−
# State and prove the Bonferroni inequality
  
  +
'''Section 1'''
−  
−
==== Test questions for final assessment in the course ====
  
   
  +
'''Section 2'''
 
# State the Fenton Measurement theory and explain what is Representation condition?
  
# State the Fenton Measurement theory and explain what is Representation condition?
 
# Define the steps needed to elaborate a GQM
  
# Define the steps needed to elaborate a GQM
Line 337: Line 227:
 
# Compute LOC, MCC, FI, FO for given code. Describe how to apply MCC metrics and meaning of FI-FO output
  
# Compute LOC, MCC, FI, FO for given code. Describe how to apply MCC metrics and meaning of FI-FO output
 
# For the given module compute the CK metrics for its classes
  
# For the given module compute the CK metrics for its classes
  +
  +
=== The retake exam ===
  +
'''Section 1'''
  +
  +
'''Section 2'''

Latest revision as of 16:03, 19 December 2022

Empirical Methods

  • Course name: Empirical Methods
  • Code discipline:
  • Subject area:

Short Description

This course covers the following concepts: Goal-Question-Metric approach; Experimental design; Basics of statistics.

Prerequisites

Prerequisite subjects

Prerequisite topics

Course Topics

Course Sections and Topics
Section Topics within the section
Concept of measuring
  1. Measurement: concept, definition and fundamentals of measurement
  2. Goal-Question-Metric approach
  3. Representational theory of measurement
  4. Measurement scales and functions that can be applied to scales
  5. Experimental designs
Fundamentals of statistics
  1. Basic concepts of probability theory
  2. Random variable and random process
  3. Linear regression
  4. Correlation and convolution
  5. Moments and moment generating functions
  6. Law of Large Numbers
  7. Central Limit Theorem
  8. Hypothesis testing

Intended Learning Outcomes (ILOs)

What is the main purpose of this course?

The main purpose of this course is to present the fundamentals of empirical methods and fundamental statistics to the future software engineers and data scientists, on one side providing the scientific fundamentals of the disciplines, and on the other anchoring the theoretical concepts on practices coming from the world of software development and engineering. As a side product, the course also refreshes the basics of statistics, providing the basis for more advanced statistical courses in the following semester(s) of study.

ILOs defined at three levels

Level 1: What concepts should a student know/remember/explain?

By the end of the course, the students should be able to ...

  • Remember the fundamentals of statistics and probability theory
  • Remember the basic models for experimentation and quasi-experimentation
  • Remember the specifics and purpose of different measurement scales
  • Distinguish between random variable and random process
  • Explain the difference between the correlation and causation

Level 2: What basic practical skills should a student be able to perform?

By the end of the course, the students should be able to ...

  • the value of experimentation for software engineers and data scientists
  • the basic concepts of an hypothesis
  • the concept of correlation
  • the fundamental laws in statistics
  • the concept of Goal-Question-Metric approach

Level 3: What complex comprehensive skills should a student be able to apply in real-life scenarios?

By the end of the course, the students should be able to ...

  • Apply Goal-Question-Metric approach in practice
  • Apply the fundamental principles of experimental design
  • Apply reduction to quasi-experimentation experimental design
  • Apply statistics and probability theory in practice
  • Apply hypothesis testing technique in software analysis

Grading

Course grading range

Grade Range Description of performance
A. Excellent 85-100 -
B. Good 65-84 -
C. Satisfactory 51-64 -
D. Poor 0-50 -

Course activities and grading breakdown

Activity Type Percentage of the overall course grade
Quiz during each lecture (weekly evaluations) 11.5
Labs classes (weekly evaluations) 11.5
Midterm 17
Final exam 60

Recommendations for students on how to succeed in the course

Resources, literature and reference materials

Open access resources

  • Donald T. Campbell and Julian C. Stanley. Experimental and Quasi-Experimental Designs for Research. Rand McNally College Publishing, 1963
  • Larry Wasserman. All of Statistics: A Concise Course in Statistical Inference. Springer Texts in Statistics. Springer, New York, 2004. ISBN 978-1-4419-2322-6. doi: 10.1007/978-0-387-21736-9
  • Oliver Laitenberger and Dieter Rombach. Lecture Notes on Empirical Software Engineering. chapter (Quasi) Experimental Studies in Industrial Settings, pages 167–227. World Scientific Publishing Co., Inc., River Edge, NJ, USA, 2003. ISBN 981-02-4914-4
  • Rini van Solingen and Egon Berghout. The Goal/Question/Metric Method: a practical guide for quality improvement of software development. The McGraw-Hill Companies, Cambridge, England, 1999. ISBN 077-709553-7.
  • Andrea Janes and Giancarlo Succi. Lean Software Development in Action. Springer, Heidelberg, Germany, 2014. ISBN 978-3-662-44178-7. doi: 10.1007/978-3-642-00503-9

Closed access resources

Software and tools used within the course

Teaching Methodology: Methods, techniques, & activities

Activities and Teaching Methods

Activities within each section
Learning Activities Section 1 Section 2
Midterm evaluation 1 1
Testing (written or computer based) 1 1
Discussions 1 1

Formative Assessment and Course Activities

Ongoing performance assessment

Section 1

Activity Type Content Is Graded?
Question What are the phases of GQM? How are they connected to each other? What are steps of GQM method? 1
Question What does SWOT mean? 1
Question What is the measurement? 1
Question How measurement can help us to understand, control and improve development process?? 1
Question What does Representation Condition mean? 1
Question What is the Measurement Scale? 1
Question What are characteristics of a good measurement? What is the difference between validity and reliability? 1
Question Which benefits the GQM provides to you as a Software Engineer / Data Scientist? 0
Question Imagine your goal is to ”increase availability of some software system”. Provide Questions and Metrics for this goal. 0
Question What is measurement Reliability and measurement Validity? What are the differences between the two? Provide an example of reliable, but invalid measurement and an example of valid, but unreliable measurement 0
Question Which Measurement Scales do you know? What are the differences between them? Provide examples for each of them. 0
Question Provide an example of Representation Condition 0
Question Create metrics that measures your study progress, outline the properties of such metrics in terms of subjective vs. objective, direct vs. indirect, etc; detail how you will collect your metrics, concretely and check your metric on reliability & validity 0

Section 2

Activity Type Content Is Graded?
Question Describe the three approaches to probability. 1
Question Write the fundamental theorem of algebra. 1
Question Write the general structure of the OLS equation for one variable. 1
Question Write the general structure of the OLS equation for the case of multiple independent variables. 1
Question What is the connection between the correlation coefficient and the coefficient of determination? What does each of them show? 1
Question State and prove the Law of Large Numbers 1
Question State and prove the Central Limit Theorem 1
Question Explain what are H0 and H1 in hypothesis testing 1
Question Explain the role of the Bonferroni inequality in hypothesis testing 1
Question Define and provide examples of sample space, events and probability measure 0
Question Write the formula for the coefficients of the simple linear regression. Explain the mathematical procedure you do to derive them and derive them 0
Question Fully deduce the value of the coefficient in OLS equation for multiple independent variables 0
Question Calculate the correlation between two functions and explain its meaning 0
Question Calculate the Pearson coefficient for the given functions 0
Question Write and prove Markov’s inequality. Write and prove Chebyshev’s inequality. How these theorems related to the LLN? 0
Question Deduce the MGF for normal distribution 0
Question Provide a concrete example of a test, detailing both H0 and H1 0
Question State and prove the Bonferroni inequality 0

Final assessment

Section 1

Section 2

  1. State the Fenton Measurement theory and explain what is Representation condition?
  2. Define the steps needed to elaborate a GQM
  3. Describe the Taylor
  4. Describe the fundamental theorem of Algebra
  5. Based on the concept the Taylor theorem and the fundamental theorem of Algebra, explain whether the number of datapoints should be equal, smaller or larger than the number of independent variables (features) and why
  6. Explain how the GQM can be used to define appropriate number of variables
  7. For the given function compute its mean, mode, median, standard deviation
  8. Define the OLS linear regression in the case of one and multiple variables and deduce their parameters
  9. For the given two functions compute their Covariance, Pearson’s correlation coefficient and describe results
  10. State and prove weak and strong formulation of LLN
  11. State Lindeberg–Lévy formulation of CLT and prove it
  12. Compute LOC, MCC, FI, FO for given code. Describe how to apply MCC metrics and meaning of FI-FO output
  13. For the given module compute the CK metrics for its classes

The retake exam

Section 1

Section 2

Retrieved from "https://eduwiki.innopolis.university/index.php?title=MSc:_Empirical_Methods&oldid=7992"