MSc: Computational Intelligence

Computational intelligence

• Course name: Computational intelligence
• Code discipline: R-02
• Subject area: Computer Science and Engineering

Short Description

This course covers the following concepts: Convex Optimization; Global Optimization; Machine Learning and function approximation.

Prerequisites

Prerequisite subjects

• CSE202 — Analytical Geometry and Linear Algebra I
• CSE204 — Analytic Geometry And Linear Algebra II: SVD, least squares, pseudoinverse, semidefinite matrices, solving systems of linear eq., dot product, norms. Basic Geometry (ellipsoids, planes, normal, tangent).
• Basic multivariable Calculus (extremum and minimum of a function, derivatives, Jacobians)
• Programming (Python/Matlab)

Course Topics

Course Sections and Topics

Section	Topics within the section
Introduction to optimization methods	Optimization problem types. Constraint types. Cost function (Reward function) types. Practical examples of optimization problems. Basic optimization algorithms and their limitations. Lagrange multipliers. Gradient descent.
Convex Optimization.	Convex sets. Convex functions. Convex optimization problems. Properties of the convex optimizations. Linear Programming Quadratic Programming. Second Order Cone Programming. Semidefinite Programming. Vertical Stability of a bipedal robot as an optimization. Quadrotor path planning as an optimization. Controller design as an optimization. CVX.
Global Optimization	Properties of non-convex problems. Problems with multiple solutions. Problems with weak local minima. Problems with computationally expensive gradients. Path planning on non-convex maps. Controller design as a non-convex problem. Relaxations. Mixed integer programming. PSO. RRT. Genetic Algorithm. Machine Learning.

Intended Learning Outcomes (ILOs)

What is the main purpose of this course?

Computational Intelligence serves as a combined multidisciplinary subject, encompassing a wide range of topics. These include numeric optimization, especially convex optimization, which is the necessary and required topic for most of the modern engineering and scientific work. The course also covers global non-convex optimization methods, which are important instruments in a number of areas: product design and manufacturing, control, and others. Basic information from a number of other areas, such as machine learning, are added to complete the picture of modern intelligent computational tools that serve the same set of goals.

ILOs defined at three levels

Level 1: What concepts should a student know/remember/explain?

By the end of the course, the students should be able to ...

• Convexity, Convex Sets, Convex optimization
• Quadratic programming, Second Order Cone Programming, Semidefinite Programming
• Optimization in Controller design, Optimization in path planning, Optimization in Mechanical Engineering
• Nonlinear non-convex optimization, RRT algorithm, Genetic Algorithm, Particle Swarm Optimization, Function approximation.

Level 2: What basic practical skills should a student be able to perform?

By the end of the course, the students should be able to ...

• Structure of optimization problems
• Convexity criteria
• Properties of convex optimization
• Types of problems that cab be transformed into Linear Programs
• Types of problems that cab be transformed into Quadratic Programs
• Types of problems that cab be transformed into Second Order Cone Programs
• Types of problems that cab be transformed into Semidefinite Programs
• Types of non-convex problems that can be approximated by a convex relaxation
• Mixed-integer Optimization
• Path planning as an optimization
• Controller design as an optimization
• Optimal parameter choice
• RRT implementation
• PSO implementation

Level 3: What complex comprehensive skills should a student be able to apply in real-life scenarios?

By the end of the course, the students should be able to ...

• Design Control using Convex Optimization.
• Implement iterative non-linear controllers with inequality constraints.
• Find optimal parameter choice for processes
• proof convexity of a problem
• program optimization in CVX
• make RRT-based algorithms, understand their limitations
• make PSO-based algorithms, understand their limitations
• make GA-based algorithms, understand their limitations

Grading

Course grading range

Grade	Range	Description of performance
A. Excellent	85-100	-
B. Good	70-84	-
C. Satisfactory	50-69	-
D. Poor	0-49	-

Course activities and grading breakdown

Activity Type	Percentage of the overall course grade
Labs/seminar classes	30
Interim performance assessment	20
Exams	50

Resources, literature and reference materials

Open access resources

• Engelbrecht, A.P., 2007. Computational intelligence: an introduction. John Wiley & Sons.
• Pedrycz, W., 1997. Computational intelligence: an introduction. CRC press.
• Konar, A., 2006. Computational intelligence: principles, techniques and applications. Springer Science & Business Media.

Teaching Methodology: Methods, techniques, & activities

Activities and Teaching Methods

Activities within each section

Learning Activities	Section 1	Section 2	Section 3
Homework and group projects	1	1	1
Testing (written or computer based)	1	0	0
Reports	1	1	1
Midterm evaluation	0	1	0
Discussions	0	1	0

Formative Assessment and Course Activities

Ongoing performance assessment

Section 1

Activity Type	Content	Is Graded?
Question	Describe an engineering problem as an optimization problem.	1
Question	What is the domain of an optimization problem?	1
Question	What is the difference between Convex and Non-convex optimization?	1
Question	Formulate a linear system with multi-dimensional solution space and inequality constraints.	0
Question	Solve a linear system with inequality constraints via gradient descent.	0

Section 2

Activity Type	Content	Is Graded?
Question	Provide examples of convex problems.	1
Question	What is a convex domain?	1
Question	Examples of problems with a convex cost but non-convex domain?	1
Question	What is the difference between quadratic and conic programming?	1
Question	What is the hierarchy of convex optimization problems?	1
Question	What solvers can solve quadratic programs?	1
Question	What solvers can solve SOCP?	1
Question	What solvers can solve LP?	1
Question	What solvers can solve SDP?	1
Question	What kinds of inequality constraints make the problem non-convex?	0
Question	What kinds of inequality constraints make the problem non-feasible?	0
Question	Show an example of a problem where the cost is a 2-norm. Show that it is a SOCP.	0
Question	Show an example of a problem where the cost is a 2-norm. Prove that it can be equivalently solved as a QP.	0
Question	Solve trajectory planning for a quadrotor as a SOCP.	0
Question	Solve vertical stability check for a biped as a QP.	0
Question	implement ZMP trajectory planning as a QP.	0
Question	implement LTI controller design as a SDP.	0

Section 3

Activity Type	Content	Is Graded?
Question	Provide examples of non-convex problems?	1
Question	Why non-convex problems can have multiple solutions?	1
Question	What are local minima?	1
Question	What is global optimization?	1
Question	Provide an example of a non-convex path planning problems?	1
Question	What are the limitations of PSO?	1
Question	Implement PSO for a parameter optimization problem.	0
Question	Make a comparative study of PSO, GA and Random Search.	0
Question	What is the difference between random search, Sobol sequences-based methods and PSO? Show it in the numerical examples.	0
Question	Implement RRT	0
Question	Implement GA	0

Final assessment

Section 1

Section 2

Section 3

1. Solve minimum distance to a plane problem, when the domain is non-convex.
2. Find optimal controller parameters for a given trajectory of a non-linear system.
3. Which non-linear algorithms can solve problems with non-convex domains?

The retake exam

Section 1

Section 2

Section 3