BSc: Nature Inspired Computing

Nature Inspired Computing

• Course name: Nature Inspired Computing
• Course number: CSE340

Course characteristics

Key concepts of the class

• Turing machines, computational complexity
• Approximation algorithms vs heuristics
• Evolutionary Algorithms
• Classification
• Optimization

What is the purpose of this course?

The main purpose of this is to introduce methods and algorithms inspired by naturally occurring phenomena and applying them to optimization, design and learning problems. The course focuses on learning implementation and analysis of such algorithms. These include evolutionary computation, swarm intelligence and neural networks.

Course Objectives Based on Bloom’s Taxonomy

What should a student remember at the end of the course?

By the end of the course, the students should be able to

• Basic concepts in Swarm intelligence concepts
• Basic concepts in Global optimization
• Basic concepts in Classification methods

What should a student be able to understand at the end of the course?

By the end of the course, the students should be able to

• Computational complexity
• Methods of computing computational complexity
• Methods of optimization in swarm intelligence

What should a student be able to apply at the end of the course?

By the end of the course, the students should be able to apply:

• Reductions
• Empirical Algorithms
• Evolutionary Algorithms
• Co-evolutionary Algorithms
• Genetic Programming

Course evaluation

The course has two major forms of evaluations:

Course grade breakdown

Component	Points
Labs/seminar classes	20
Interim performance assessment (class participation)	30
Exams	50

Grades range

Course grading range

A. Excellent	90-100
B. Good	75-89
C. Satisfactory	60-74
D. Poor	0-55

Resources and reference material

• Textbook: Bio-Inspired Artificial Intelligence: Theories, Methods, and Technologies, D. Floreano and C. Mattiussi (2008), MIT Press.
• Textbook: Siddique, N. and Adeli, H., 2015. Nature inspired computing: an overview and some future directions. Cognitive computation, 7(6), pp.706-714.
• Textbook: Coello, C.C., Dhaenens, C. and Jourdan, L. eds., 2009. Advances in multi-objective nature inspired computing (Vol. 272). Springer.
• Textbook: Khosravy, M., Gupta, N., Patel, N. and Senjyu, T. eds., 2020. Frontier applications of nature inspired computation. Springer Nature.

Course Sections

The main sections of the course and approximate hour distribution between them is as follows:

Course Sections

Section	Section Title	Teaching Hours
1	basic concepts of Nature Inspired Computing	12
2	Computational complexity, ANN	14
3	Optimization, PSO	10

Section 1

Section title

Basic concepts of Nature Inspired Computing

Topics covered in this section

• Course Introduction
• Approximation algorithms vs heuristics
• Characteristics of Natural Systems/Algorithms
• Cellular Automata
• Evolutionary Algorithms
• Dynamic Environments
• Genetic Programming
• Symbolic Regression

What forms of evaluation were used to test students’ performance in this section?

	Yes/No
Development of individual parts of software product code	1
Homework and group projects	1
Midterm evaluation	1
Testing (written or computer based)	1
Reports	0
Essays	0
Oral polls	0
Discussions	1

Typical questions for ongoing performance evaluation within this section

1. What is an approximation?
2. What is a heuristics?
3. What are the main Characteristics of Natural Systems/Algorithms?
4. What are Cellular Automata?
5. Give examples of Evolutionary Algorithms

Typical questions for seminar classes (labs) within this section

1. Provide an example of approximation?
2. What is the difference between Approximation algorithms and heuristics
3. Write an Evolutionary Algorithm as a pseudocode
4. Write a Genetic Algorithm as a pseudocode
5. What are the characteristics of Genetic Algorithms

Test questions for final assessment in this section

1. Genetic Algorithms can be used for:

   a. Optimization (correct)

   b. Feedback Control

   c. Automatic Differentiation

   d. Numeric Geometry

2. Applications of evolutionary algorithms do not include

   a. Robotics

   b. Design

   c. Optimization

   d. Measurements (correct)

3. Which of the following algorithms suffers from local minima of the objective function

   a. Genetic algorithm

   b. Evolutionary algorithm

   c. Gradient descent (correct)

   d. All three of the above

Section 2

Section title

Computational complexity, ANN

Topics covered in this section

• Turing machines, computational complexity
• NP-hardness
• Reductions
• Learning Classifier Systems
• Co-evolutionary Algorithms
• Fitness Landscape Analysis
• Immunocomputing
• Swarms/Flocking
• Feedforward and Recurrent Networks, Backpropagation
• Spiking Neural Networks
• Self Organizing Maps
• Deep Learning

What forms of evaluation were used to test students’ performance in this section?

	Yes/No
Development of individual parts of software product code	1
Homework and group projects	1
Midterm evaluation	1
Testing (written or computer based)	1
Reports	0
Essays	0
Oral polls	0
Discussions	1

Typical questions for ongoing performance evaluation within this section

1. What are Turing machines?
2. What is computational complexity?
3. What are artificial neural networks?
4. What are Recurrent ANN?
5. What is deep learning?

Typical questions for seminar classes (labs) within this section

1. Give examples of algorithms of different computational complexities
2. Give example of how one computes computational complexity
3. What is a basic structure of an ANN?
4. What is a fully connected ANN?
5. Draw structure of LSTM RNN

Test questions for final assessment in this section

1. Which of the following are neural networks architectures?

   a. ANN, RNN, CNN

   b. RNN, CNN, LSTM

   c. RNN, CNN, fully connected (correct)

   d. Reinforcement learning, Supervised learning

2. Which types of machine learning can be used to learn policies for agents acting in dynamic environments, where marked datasets are not available?

   a. Reinforcement learning, evolutionary algorithms (correct)

   b. YOLO

   c. Cauterization

   d. Supervised learning

3. What distinguishes deep learning?

   a. The number of layers of the network, and the methods of gradient propagation aiming at lowering the effects of the vanishing gradients (correct)

   b. Fully-connected layers

   c. LSTM cells

   d. Auto-encoding layers

Section 3

Section title

Optimization, PSO

Topics covered in this section

• Ant Colony Optimization
• Ant Clustering Algorithm
• Particle Swarm Optimization
• Multiobjectiveness
• Foraging Algorithms
• Harmony Search
• Hebbian Learning, Boltzmann Machines

What forms of evaluation were used to test students’ performance in this section?

	Yes/No
Development of individual parts of software product code	1
Homework and group projects	1
Midterm evaluation	1
Testing (written or computer based)	1
Reports	0
Essays	0
Oral polls	0
Discussions	1

Typical questions for ongoing performance evaluation within this section

1. What is a Foraging Algorithm
2. What is PSO?
3. What is a multi-objective optimization?
4. What is Ant Colony Optimization?

Typical questions for seminar classes (labs) within this section

1. Write PSO as a diagram
2. Give example of a problem that can be solved using PSO
3. Write a code implementing PSO for a problem with two variables
4. How does the number of particles influence PSO performance?
5. How do local minima influence PSO performance?

Test questions for final assessment in this section

1. Which of the following can be used to solve optimization problems?

   a. PSO, Genetic algorithm (correct)

   b. CNN, PSO

   c. CNN

   d. RNN, PSO

2. Which of the following can be used to find acceptable parameters of a process, given inequality constraints?

   a. PSO, Genetic algorithm (correct)

   b. CNN, PSO

   c. CNN

   d. RNN, PSO