BSc: Introduction to Optimization.F22

Introduction to Optimization

• Course name: Introduction to Optimization
• Code discipline: CSE???
• Subject area: Data Science and Artificial Intelligence

Short Description

The course outlines the classification and mathematical foundations of optimization methods, and presents algorithms for solving linear and nonlinear optimization. The purpose of the course is to introduce students to methods of linear and convex optimization and their application in solving problems of linear, network, integer, and nonlinear optimization. Course starts with linear programming and moving on to more complex problems. primal and dual simplex methods, network flow algorithms, branch and bound, interior point methods, Newton and quasi-Newton methods, and heuristic methods. Current states, literature, techniques, theories, and methodologies are presented and discussed during the semester.

Prerequisites

Prerequisite subjects

• CSE201: Mathematical Analysis I
• CSE202: Analytical Geometry and Linear Algebra I
• CSE203: Mathematical Analysis II
• CSE204: Analytical Geometry and Linear Algebra II

Prerequisite topics

• Basic programming skills.
• OOP, and software design.

Course Topics

Course Sections and Topics

Section	Topics within the section
Linear programming	Optimization Applications Formulation of Optimization Models Graphical Solution of Linear Programs Simplex Method Linear Programming Duality and Sensitivity Analysis
Nonlinear programming	Nonlinear Optimization Theory Steepest Descent and Conjugate Gradient Methods Newton and Quasi-Newton Methods Quadratic Programming
Extensions	Network Models Dynamic Programming Multi-objective Programming Optimization Heuristics

Intended Learning Outcomes (ILOs)

What is the main purpose of this course?

The main purpose of this course is to enable a student to go from an idea to implementation of different optimization algorithms to solve problems in different fields of studies like machine and deep learning, etc.

ILOs defined at three levels

We specify the intended learning outcomes at three levels: conceptual knowledge, practical skills, and comprehensive skills.

Level 1: What concepts should a student know/remember/explain?

By the end of the course, the students should be able to ...

• remember the different classifications and mathematical foundations of optimization methods
• remember basic properties of the corresponding mathematical objects
• remember the fundamental concepts, laws, and methods of linear and convex optimization
• distinguish between different types of optimization methods

Level 2: What basic practical skills should a student be able to perform?

By the end of the course, the students should be able to ...

• Funderstand the basic concepts of optimization problems
• evaluate the correctness of problem statements
• explain which algorithm is suitable for solving problems
• evaluate the correctness of results

Level 3: What complex comprehensive skills should a student be able to apply in real-life scenarios?

By the end of the course, the students should be able to ...

• understand the basic foundations behind optimization problems.
• classify optimization problems.
• choose a proper algorithm to solve optimization problems.
• validate algorithms that students choose to solve optimization problems.

Grading

Course grading range

Grade	Range	Description of performance
A. Excellent	90-100	-
B. Good	75-89	-
C. Satisfactory	60-74	-
D. Fail	0-59	-

Course activities and grading breakdown

Activity Type	Percentage of the overall course grade
Midterm	25
2 Intermediate tests	30 (15 for each)
Final exam	30
Final presentation	15

Recommendations for students on how to succeed in the course

• Participation is important. Attending lectures is the key to success in this course.
• Review lecture materials before classes to do well.
• Reading the recommended literature is optional, and will give you a deeper understanding of the material.

Resources, literature and reference materials

Open access resources

• Convex Optimization – Boyd and Vandenberghe. Cambridge University Press.

Closed access resources

• Engineering Optimization: Theory and Practice, by Singiresu S. Rao, John Wiley and Sons.
• Bertsimas, Dimitris, and John Tsitsiklis. Introduction to Linear Optimization. Belmont, MA: Athena Scientific, 1997. ISBN: 9781886529199.

Software and tools used within the course

• MATLAB
• Python
• Excel

Activities and Teaching Methods

Teaching and Learning Methods within each section

Teaching Techniques	Section 1	Section 2	Section 3
Problem-based learning (students learn by solving open-ended problems without a strictly-defined solution)	0	0	0
Project-based learning (students work on a project)	1	1	1
Modular learning (facilitated self-study)	0	0	0
Differentiated learning (provide tasks and activities at several levels of difficulty to fit students needs and level)	1	1	1
Contextual learning (activities and tasks are connected to the real world to make it easier for students to relate to them)	0	0	0
Business game (learn by playing a game that incorporates the principles of the material covered within the course)	0	0	0
Inquiry-based learning	0	0	0
Just-in-time teaching	0	0	0
Process oriented guided inquiry learning (POGIL)	0	0	0
Studio-based learning	0	0	0
Universal design for learning	0	0	0
Task-based learning	0	0	0

Activities within each section

Learning Activities	Section 1	Section 2	Section 3
Lectures	1	1	1
Interactive Lectures	1	1	1
Lab exercises	1	1	1
Experiments	0	0	0
Modeling	0	0	0
Cases studies	0	0	0
Development of individual parts of software product code	0	0	0
Individual Projects	1	1	1
Group projects	0	0	0
Flipped classroom	0	0	0
Quizzes (written or computer based)	0	0	0
Peer Review	0	0	0
Discussions	1	1	1
Presentations by students	1	1	1
Written reports	0	0	0
Simulations and role-plays	0	0	0
Essays	0	0	0
Oral Reports	0	0	0

Formative Assessment and Course Activities

Final assessment

Section 1

1. Grading criteria for the final project presentation:
2. Problem: short clear statement on what you are solving, and why it’s important.
3. User: should be a specific user, can start from generic and then show how you narrowed it.
4. Solution: how do you target the problem, what were the initial assumptions/hypotheses
5. Elicitation process: interviews, how many people, what questions you asked, what you learnt.

Section 2

1. Arriving at MVP: how you chose features, describe prototyping and learning from it, when did you launch, and how it went.
2. Team and development process: how it evolved, what were the challenges, what fixes you made to keep progressing.
3. Product demo: make it clear what your current product progress is.

Section 3

1. Hypothesis-driven development: how did you verify value and understandability of your product, what were the main hypotheses you had to check through MVP.
2. Measuring product: what metrics you chose, why, what funnels did you set for yourself, and what was the baseline for your MVP.
3. Experimentation: What usability tests and experiments you conducted, what did you learn, how did it affect your funnels and metrics.

The retake exam

Section 1

1. Grading criteria for the final project presentation:
2. Problem: short clear statement on what you are solving, and why it’s important.
3. User: should be a specific user, can start from generic and then show how you narrowed it.
4. Solution: how do you target the problem, what were the initial assumptions/hypotheses
5. Elicitation process: interviews, how many people, what questions you asked, what you learnt.

Section 2

1. Arriving at MVP: how you chose features, describe prototyping and learning from it, when did you launch, and how it went.
2. Team and development process: how it evolved, what were the challenges, what fixes you made to keep progressing.
3. Product demo: make it clear what your current product progress is.

Section 3

1. Hypothesis-driven development: how did you verify value and understandability of your product, what were the main hypotheses you had to check through MVP.
2. Measuring product: what metrics you chose, why, what funnels did you set for yourself, and what was the baseline for your MVP.