Difference between revisions of "IU:TestPage"

From IU
Jump to navigation Jump to search
Line 1: Line 1:
   
  +
= Mathematical Analysis I =
−
= Practical Machine Learning and Deep Learning =
  
−
* '''Course name''': Practical Machine Learning and Deep Learning
 +
* '''Course name''': Mathematical Analysis I
 
* '''Code discipline''':
 
* '''Code discipline''':
  +
* '''Subject area''': Differentiation; Integration; Series
−
* '''Subject area''': Practical aspects of deep learning (DL); Practical applications of DL in Natural Language Processing, Computer Vision and generation.
  
   
 
== Short Description ==
  
== Short Description ==
Line 11: Line 11:
   
 
=== Prerequisite subjects ===
  
=== Prerequisite subjects ===
  +
−
* CSE202 — Analytical Geometry and Linear Algebra I / []: Manifolds "Linear Alg./Calculus: Manifolds
  
−
* CSE203 — Mathematical Analysis II: Basics of optimisation
  
−
* CSE201 — Mathematical Analysis I: integration and differentiation.
  
−
* CSE103 — Theoretical Computer Science: Graph theory basics, Spectral decomposition.
  
−
* CSE206 — Probability And Statistics: Multivariate normal dist.
  
−
* CSE504 — Digital Signal Processing: convolution, cross-correlation"
  
   
 
=== Prerequisite topics ===
  
=== Prerequisite topics ===
Line 27: Line 22:
 
! Section !! Topics within the section
  
! Section !! Topics within the section
 
|-
  
|-
−
| Review. CNNs and RNNs ||
+
| Sequences and Limits ||
  +
# Sequences. Limits of sequences
−
# Image processing, FFNs, CNNs
  
  +
# Limits of sequences. Limits of functions
−
# Training Deep NNs
  
  +
# Limits of functions. Continuity. Hyperbolic functions
−
# RNNs, LSTM, GRU, Embeddings
  
−
# Bidirectional RNNs
  
−
# Seq2seq
  
−
# Encoder-Decoder Networks
  
−
# Attention
  
−
# Memory Networks
  
 
|-
  
|-
  +
| Differentiation ||
−
| Team Data Science Processes ||
 
  +
# Derivatives. Differentials
−
# Team Data Science Processes
  
  +
# Mean-Value Theorems
−
# Team Data Science Roles
  
  +
# l’Hopital’s rule
−
# Team Data Science Tools (MLFlow, KubeFlow)
  
  +
# Taylor Formula with Lagrange and Peano remainders
−
# CRISP-DM
  
  +
# Taylor formula and limits
−
# Productionizing ML systems
  
  +
# Increasing / decreasing functions. Concave / convex functions
 
|-
  
|-
−
| VAEs, GANs ||
+
| Integration and Series ||
  +
# Antiderivative. Indefinite integral
−
# Autoencoders
  
  +
# Definite integral
−
# Variational Autoencoders
  
  +
# The Fundamental Theorem of Calculus
−
# GANs, DCGAN
  
  +
# Improper Integrals
  +
# Convergence tests. Dirichlet’s test
  +
# Series. Convergence tests
  +
# Absolute / Conditional convergence
  +
# Power Series. Radius of convergence
  +
# Functional series. Uniform convergence
 
|}
 
|}
 
== Intended Learning Outcomes (ILOs) ==
  
== Intended Learning Outcomes (ILOs) ==
   
 
=== What is the main purpose of this course? ===
  
=== What is the main purpose of this course? ===
  +
understand key principles involved in differentiation and integration of functions, solve problems that connect small-scale (differential) quantities to large-scale (integrated) quantities, become familiar with the fundamental theorems of Calculus, get hands-on experience with the integral and derivative applications and of the inverse relationship between integration and differentiation.
−
The course is about the practical aspects of deep learning. In addition to frontal lectures, the flipped classes and student project presentations will be organized. During lab sessions the working language is Python. The primary framework for deep learning is PyTorch. Usage of TensorFlow and Keras is possible, usage of Docker is highly appreciated.
  
   
 
=== ILOs defined at three levels ===
  
=== ILOs defined at three levels ===
Line 58: Line 55:
 
==== Level 1: What concepts should a student know/remember/explain? ====
  
==== Level 1: What concepts should a student know/remember/explain? ====
 
By the end of the course, the students should be able to ...
  
By the end of the course, the students should be able to ...
  +
* Derivative. Differential. Applications
−
* to apply deep learning methods to effectively solve practical (real-world) problems;
  
  +
* Indefinite integral. Definite integral. Applications
−
* to work in data science team;
  
  +
* Sequences. Series. Convergence. Power Series
−
* to understand of principles and a lifecycle of data science projects.
  
   
 
==== Level 2: What basic practical skills should a student be able to perform? ====
  
==== Level 2: What basic practical skills should a student be able to perform? ====
 
By the end of the course, the students should be able to ...
  
By the end of the course, the students should be able to ...
  +
* Derivative. Differential. Applications
−
* to understand modern deep NN architectures;
  
  +
* Indefinite integral. Definite integral. Applications
−
* to compare modern deep NN architectures;
  
  +
* Sequences. Series. Convergence. Power Series
−
* to create a prototype of a data-driven product.
  
  +
* Taylor Series
   
 
==== Level 3: What complex comprehensive skills should a student be able to apply in real-life scenarios? ====
  
==== Level 3: What complex comprehensive skills should a student be able to apply in real-life scenarios? ====
 
By the end of the course, the students should be able to ...
  
By the end of the course, the students should be able to ...
  +
* Take derivatives of various type functions and of various orders
−
* to apply techniques for efficient training of deep NNs;
  
  +
* Integrate
−
* to apply methods for data science team organisation;
  
  +
* Apply definite integral
−
* to apply deep NNs in NLP and computer vision.
 
  +
* Expand functions into Taylor series
  +
* Apply convergence tests
 
== Grading ==
  
== Grading ==
   
Line 109: Line 109:
   
 
=== Open access resources ===
  
=== Open access resources ===
  +
* Zorich, V. A. “Mathematical Analysis I, Translator: Cooke R.” (2004)
−
* Goodfellow et al. Deep Learning, MIT Press. 2017
  
−
* Géron, Aurélien. Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems. 2017.
  
−
* Osinga, Douwe. Deep Learning Cookbook: Practical Recipes to Get Started Quickly. O’Reilly Media, 2018.
  
   
 
=== Closed access resources ===
  
=== Closed access resources ===
Line 125: Line 123:
 
|-
  
|-
 
! Learning Activities !! Section 1 !! Section 2 !! Section 3
  
! Learning Activities !! Section 1 !! Section 2 !! Section 3
−
|-
  
−
| Development of individual parts of software product code || 1 || 1 || 1
  
 
|-
  
|-
 
| Homework and group projects || 1 || 1 || 1
  
| Homework and group projects || 1 || 1 || 1
 
|-
  
|-
−
| Midterm evaluation || 1 || 1 || 1
 +
| Midterm evaluation || 1 || 1 || 0
 
|-
  
|-
 
| Testing (written or computer based) || 1 || 1 || 1
  
| Testing (written or computer based) || 1 || 1 || 1
Line 146: Line 142:
 
! Activity Type !! Content !! Is Graded?
  
! Activity Type !! Content !! Is Graded?
 
|-
  
|-
  +
| Question || A sequence, limiting value || 1
−
| Question || Suppose you use Batch Gradient Descent and you plot the validation error at every epoch. If you notice that the validation error consistently goes up, what is likely going on? How can you fix this? || 1
  
 
|-
  
|-
  +
| Question || Limit of a sequence, convergent and divergent sequences || 1
−
| Question || Is it a good idea to stop Mini-batch Gradient Descent immediately when the validation error goes up? || 1
  
 
|-
  
|-
−
| Question || List the optimizers that you know (except SGD) and explain one of them || 1
 +
| Question || Increasing and decreasing sequences, monotonic sequences || 1
 
|-
  
|-
−
| Question || Describe Xavier (or Glorot) initialization. Why do you need it? || 1
 +
| Question || Bounded sequences. Properties of limits || 1
 
|-
  
|-
−
| Question || Name advantages of the ELU activation function over ReLU. || 0
 +
| Question || Theorem about bounded and monotonic sequences. || 1
 
|-
  
|-
  +
| Question || Cauchy sequence. The Cauchy Theorem (criterion). || 1
−
| Question || Can you name the main innovations in AlexNet, compared to LeNet-5? What about the main innovations in GoogLeNet and ResNet? || 0
  
 
|-
  
|-
−
| Question || What is the difference between LSTM and GRU cells? || 0
 +
| Question || Limit of a function. Properties of limits. || 1
  +
|-
  +
| Question || The first remarkable limit. || 1
  +
|-
  +
| Question || The Cauchy criterion for the existence of a limit of a function. || 1
  +
|-
  +
| Question || Second remarkable limit. || 1
  +
|-
  +
| Question || Find a limit of a sequence || 0
  +
|-
  +
| Question || Find a limit of a function || 0
 
|}
 
|}
 
==== Section 2 ====
  
==== Section 2 ====
Line 166: Line 172:
 
! Activity Type !! Content !! Is Graded?
  
! Activity Type !! Content !! Is Graded?
 
|-
  
|-
  +
| Question || A plane curve is given by <math>{\displaystyle x(t)=-{\frac {t^{2}+4t+8}{t+2}}}</math> , <math>{\textstyle y(t)={\frac {t^{2}+9t+22}{t+6}}}</math> . Find<br>the asymptotes of this curve;<br>the derivative <math>{\textstyle y'_{x}}</math> . || 1
−
| Question || What is CRISP-DM? || 1
  
 
|-
  
|-
  +
| Question || Derive the Maclaurin expansion for <math>{\textstyle f(x)={\sqrt[{3}]{1+e^{-2x}}}}</math> up to <math>{\textstyle o\left(x^{3}\right)}</math> . || 1
−
| Question || What is TDSP? || 1
  
 
|-
  
|-
−
| Question || How to use MLflow? || 1
 +
| Question || Differentiation techniques: inverse, implicit, parametric etc. || 0
 
|-
  
|-
−
| Question || What is TensorBoard? || 1
 +
| Question || Find a derivative of a function || 0
 
|-
  
|-
−
| Question || How to apply Kubeflow in practice? || 1
 +
| Question || Apply Leibniz formula || 0
 
|-
  
|-
−
| Question || Explain issues in distributed learning of deep NNs. || 0
 +
| Question || Draw graphs of functions || 0
 
|-
  
|-
−
| Question || How do you organize your data science project? || 0
 +
| Question || Find asymptotes of a parametric function || 0
−
|-
  
−
| Question || Recall a checklist for organization of a typical data science project. || 0
  
 
|}
 
|}
 
==== Section 3 ====
  
==== Section 3 ====
Line 188: Line 192:
 
! Activity Type !! Content !! Is Graded?
  
! Activity Type !! Content !! Is Graded?
 
|-
  
|-
  +
| Question || Find the indefinite integral <math>{\textstyle \displaystyle \int x\ln \left(x+{\sqrt {x^{2}-1}}\right)\,dx}</math> . || 1
−
| Question || What is an Autoencoder? Can you list the structure and types of Autoencoders? || 1
  
−
|-
  
−
| Question || Can you describe ways to train Stacked AEs? || 1
  
−
|-
  
−
| Question || What is Denoising AE? Can you describe what is sparsity loss and why it can be useful? || 1
  
−
|-
  
−
| Question || Can you make a distinction between AE and VAE? || 1
  
−
|-
  
−
| Question || If an autoencoder perfectly reconstructs the inputs, is it necessarily a good autoencoder? How can you evaluate the performance of an autoencoder? || 0
  
 
|-
  
|-
  +
| Question || Find the length of a curve given by <math>{\textstyle y=\ln \sin x}</math> , <math>{\textstyle {\frac {\pi }{4}}\leqslant x\leqslant {\frac {\pi }{2}}}</math> . || 1
−
| Question || How do you tie weights in a stacked autoencoder? What is the point of doing so? || 0
  
 
|-
  
|-
  +
| Question || Find all values of parameter <math>{\textstyle \alpha }</math> such that series <math>{\textstyle \displaystyle \sum \limits _{k=1}^{+\infty }\left({\frac {3k+2}{2k+1}}\right)^{k}\alpha ^{k}}</math> converges. || 1
−
| Question || What about the main risk of an overcomplete autoencoder? || 0
  
 
|-
  
|-
−
| Question || How the loss function for VAE is defined? What is ELBO? || 0
 +
| Question || Integration techniques || 0
 
|-
  
|-
−
| Question || Can you list the structure and types of a GAN? || 0
 +
| Question || Integration by parts || 0
 
|-
  
|-
−
| Question || How would you train a GAN? || 0
 +
| Question || Calculation of areas, lengths, volumes || 0
 
|-
  
|-
−
| Question || How would you estimate the quality of a GAN? || 0
 +
| Question || Application of convergence tests || 0
 
|-
  
|-
−
| Question || Can you describe cost function of a Discriminator? || 0
 +
| Question || Calculation of Radius of convergence || 0
 
|}
 
|}
 
=== Final assessment ===
  
=== Final assessment ===
 
'''Section 1'''
  
'''Section 1'''
  +
# Find limits of the following sequences or prove that they do not exist:
−
# Explain what the Teacher Forcing is.
  
  +
# <math>{\displaystyle a_{n}=n-{\sqrt {n^{2}-70n+1400}}}</math> ;
−
# Why do people use encoder–decoder RNNs rather than plain sequence-to-sequence RNNs for automatic translation?
  
  +
# <math>{\textstyle d_{n}=\left({\frac {2n-4}{2n+1}}\right)^{n}}</math> ;
−
# How could you combine a convolutional neural network with an RNN to classify videos?
  
  +
# <math>{\textstyle x_{n}={\frac {\left(2n^{2}+1\right)^{6}(n-1)^{2}}{\left(n^{7}+1000n^{6}-3\right)^{2}}}}</math> .
 
'''Section 2'''
  
'''Section 2'''
  +
# Find a derivative of a (implicit/inverse) function
−
# Can you explain what it means for a company to be ML-ready?
  
  +
# Apply Leibniz formula Find <math>{\textstyle y^{(n)}(x)}</math> if <math>{\textstyle y(x)=\left(x^{2}-2\right)\cos 2x\sin 3x}</math> .
−
# What a company can do to become ML-ready / Data driven?
  
  +
# Draw graphs of functions
−
# Can you list approaches to structure DS-teams? Discuss their advantages and disadvantages.
  
  +
# Find asymptotes
−
# Can you list and define typical roles in a DS team?
  
  +
# Apply l’Hopital’s rule
−
# What do you think about practical aspects of processes and roles in Data Science projects/teams?
  
  +
# Find the derivatives of the following functions:
  +
<math>{\textstyle f(x)=\log _{|\sin x|}{\sqrt[{6}]{x^{2}+6}}}</math> ;
  +
<math>{\textstyle y(x)}</math> that is given implicitly by <math>{\textstyle x^{3}+5xy+y^{3}=0}</math> .
 
'''Section 3'''
  
'''Section 3'''
  +
# Find the following integrals:
−
# Can you make a distinction between Variational approximation of density and MCMC methods for density estimation?
  
  +
# <math>{\textstyle \int {\frac {{\sqrt {4+x^{2}}}+2{\sqrt {4-x^{2}}}}{\sqrt {16-x^{4}}}}\,dx}</math> ;
−
# What is DCGAN? What is its purpose? What are main features of DCGAN?
  
  +
# <math>{\textstyle \int 2^{2x}e^{x}\,dx}</math> ;
−
# What is your opinion about Word Embeddings? What types do you know? Why are they useful?
  
  +
# <math>{\textstyle \int {\frac {dx}{3x^{2}-x^{4}}}}</math> .
−
# How would you classify different CNN architectures?
  
  +
# Use comparison test to determine if the following series converge.
−
# How would you classify different RNN architectures?
  
  +
<math>{\textstyle \sum \limits _{k=1}^{\infty }{\frac {3+(-1)^{k}}{k^{2}}}}</math> ;
−
# Explain attention mechanism. What is self-attention?
  
  +
# Use Cauchy criterion to prove that the series <math>{\textstyle \sum \limits _{k=1}^{\infty }{\frac {k+1}{k^{2}+3}}}</math> is divergent.
−
# Explain the Transformer architecture. What is BERT?
  
  +
# Find the sums of the following series:
  +
# <math>{\textstyle \sum \limits _{k=1}^{\infty }{\frac {1}{16k^{2}-8k-3}}}</math> ;
  +
# <math>{\textstyle \sum \limits _{k=1}^{\infty }{\frac {k-{\sqrt {k^{2}-1}}}{\sqrt {k^{2}+k}}}}</math> .
   
 
=== The retake exam ===
  
=== The retake exam ===

Revision as of 15:41, 28 April 2022

Mathematical Analysis I

  • Course name: Mathematical Analysis I
  • Code discipline:
  • Subject area: Differentiation; Integration; Series

Short Description

Prerequisites

Prerequisite subjects

Prerequisite topics

Course Topics

Course Sections and Topics
Section Topics within the section
Sequences and Limits
  1. Sequences. Limits of sequences
  2. Limits of sequences. Limits of functions
  3. Limits of functions. Continuity. Hyperbolic functions
Differentiation
  1. Derivatives. Differentials
  2. Mean-Value Theorems
  3. l’Hopital’s rule
  4. Taylor Formula with Lagrange and Peano remainders
  5. Taylor formula and limits
  6. Increasing / decreasing functions. Concave / convex functions
Integration and Series
  1. Antiderivative. Indefinite integral
  2. Definite integral
  3. The Fundamental Theorem of Calculus
  4. Improper Integrals
  5. Convergence tests. Dirichlet’s test
  6. Series. Convergence tests
  7. Absolute / Conditional convergence
  8. Power Series. Radius of convergence
  9. Functional series. Uniform convergence

Intended Learning Outcomes (ILOs)

What is the main purpose of this course?

understand key principles involved in differentiation and integration of functions, solve problems that connect small-scale (differential) quantities to large-scale (integrated) quantities, become familiar with the fundamental theorems of Calculus, get hands-on experience with the integral and derivative applications and of the inverse relationship between integration and differentiation.

ILOs defined at three levels

Level 1: What concepts should a student know/remember/explain?

By the end of the course, the students should be able to ...

  • Derivative. Differential. Applications
  • Indefinite integral. Definite integral. Applications
  • Sequences. Series. Convergence. Power Series

Level 2: What basic practical skills should a student be able to perform?

By the end of the course, the students should be able to ...

  • Derivative. Differential. Applications
  • Indefinite integral. Definite integral. Applications
  • Sequences. Series. Convergence. Power Series
  • Taylor Series

Level 3: What complex comprehensive skills should a student be able to apply in real-life scenarios?

By the end of the course, the students should be able to ...

  • Take derivatives of various type functions and of various orders
  • Integrate
  • Apply definite integral
  • Expand functions into Taylor series
  • Apply convergence tests

Grading

Course grading range

Grade Range Description of performance
A. Excellent 90-100 -
B. Good 75-89 -
C. Satisfactory 60-74 -
D. Poor 0-59 -

Course activities and grading breakdown

Activity Type Percentage of the overall course grade
Labs/seminar classes 20
Interim performance assessment 30
Exams 50

Recommendations for students on how to succeed in the course

Resources, literature and reference materials

Open access resources

  • Zorich, V. A. “Mathematical Analysis I, Translator: Cooke R.” (2004)

Closed access resources

Software and tools used within the course

Teaching Methodology: Methods, techniques, & activities

Activities and Teaching Methods

Activities within each section
Learning Activities Section 1 Section 2 Section 3
Homework and group projects 1 1 1
Midterm evaluation 1 1 0
Testing (written or computer based) 1 1 1
Discussions 1 1 1

Formative Assessment and Course Activities

Ongoing performance assessment

Section 1

Activity Type Content Is Graded?
Question A sequence, limiting value 1
Question Limit of a sequence, convergent and divergent sequences 1
Question Increasing and decreasing sequences, monotonic sequences 1
Question Bounded sequences. Properties of limits 1
Question Theorem about bounded and monotonic sequences. 1
Question Cauchy sequence. The Cauchy Theorem (criterion). 1
Question Limit of a function. Properties of limits. 1
Question The first remarkable limit. 1
Question The Cauchy criterion for the existence of a limit of a function. 1
Question Second remarkable limit. 1
Question Find a limit of a sequence 0
Question Find a limit of a function 0

Section 2

Activity Type Content Is Graded?
Question A plane curve is given by , . Find
the asymptotes of this curve;
the derivative .		 1
Question Derive the Maclaurin expansion for up to . 1
Question Differentiation techniques: inverse, implicit, parametric etc. 0
Question Find a derivative of a function 0
Question Apply Leibniz formula 0
Question Draw graphs of functions 0
Question Find asymptotes of a parametric function 0

Section 3

Activity Type Content Is Graded?
Question Find the indefinite integral . 1
Question Find the length of a curve given by , . 1
Question Find all values of parameter such that series converges. 1
Question Integration techniques 0
Question Integration by parts 0
Question Calculation of areas, lengths, volumes 0
Question Application of convergence tests 0
Question Calculation of Radius of convergence 0

Final assessment

Section 1

  1. Find limits of the following sequences or prove that they do not exist:
  2.  ;
  3.  ;
  4. .

Section 2

  1. Find a derivative of a (implicit/inverse) function
  2. Apply Leibniz formula Find if .
  3. Draw graphs of functions
  4. Find asymptotes
  5. Apply l’Hopital’s rule
  6. Find the derivatives of the following functions:

 ; that is given implicitly by . Section 3

  1. Find the following integrals:
  2.  ;
  3.  ;
  4. .
  5. Use comparison test to determine if the following series converge.

 ;

  1. Use Cauchy criterion to prove that the series is divergent.
  2. Find the sums of the following series:
  3.  ;
  4. .

The retake exam

Section 1

Section 2

Section 3

Retrieved from "https://eduwiki.innopolis.university/index.php?title=IU:TestPage&oldid=6587"