Difference between revisions of "IU:TestPage"

From IU
Jump to navigation Jump to search
Line 1: Line 1:
   
  +
= Big Data Technologies and Analytics =
−
= Theoretical Mechanics =
  
−
* '''Course name''': Theoretical Mechanics
 +
* '''Course name''': Big Data Technologies and Analytics
−
* '''Code discipline''':
+
* '''Code discipline''': N/A
−
* '''Subject area''': Mechanics; mathematical modeling and calculating of mechanical systems.
 +
* '''Subject area''': P.1 Short Description
   
 
== Short Description ==
  
== Short Description ==
−
This course covers the following concepts: Mechanics: Physical principles and methods for calculating kinematic, static and dynamic problems of mechanics.
 +
This course covers the following concepts: Advanced distributed data organization; Advanced distributed data processing.
   
 
== Prerequisites ==
  
== Prerequisites ==
   
 
=== Prerequisite subjects ===
  
=== Prerequisite subjects ===
  +
−
* CSE203 — Mathematical Analysis II: Linear algebra, vectors and matrices, partial derivatives.
  
−
* CSE205 — Differential Equations: ODE.
  
   
 
=== Prerequisite topics ===
  
=== Prerequisite topics ===
Line 23: Line 22:
 
! Section !! Topics within the section
  
! Section !! Topics within the section
 
|-
  
|-
−
| Kinematics ||
+
| Introduction ||
  +
# What is Big Data
−
# Introduction to theoretical mechanics
  
−
# Kinematics of a particle
 +
# Characteristics of Big Data
  +
# Technologies
−
# Translatory and rotational motion of a rigid body
  
  +
# Virtualization and cloud computing
−
# Plane motion of a rigid body
  
  +
|-
−
# Spherical motion of a rigid body
  
  +
| File systems and resource managers ||
−
# Motion of a free rigid body
  
  +
# HDFS
−
# Resultant motion
  
  +
# YARN
 
|-
  
|-
−
| Statics ||
+
| Batch Processing ||
  +
# Distributed batch processing
−
# Basic concepts and principles of Statics
  
  +
# MapReduce model
−
# Parallel forces and couples
  
  +
# Applications
−
# Equilibrium of a rigid body system in 2D
  
  +
# Tasks management
−
# Equilibrium of a rigid body system in 3D
  
  +
# Patterns
−
# Friction
  
−
# Center of gravity
  
 
|-
  
|-
−
| Dynamics ||
+
| Stream Processing ||
  +
# CAP theorem
−
# Particle dynamics
  
  +
# Distributed storage and computation
−
# Theorem of the motion of the center of mass of a system
  
  +
# Distributed Stream Processing
−
# Theorem of the change in the linear momentum of a system
  
  +
# Usage patterns
−
# Theorem of the change in the angular momentum of a system
  
−
# Some cases of rigid body motion.
  
−
# D’Alambert’s principle
  
−
# Mechanical work and power
  
−
# Theorem of the change in the kinetic energy of a system
  
−
# The theory of impact
  
−
# Oscillations
  
 
|-
  
|-
−
| Analytical mechanics ||
+
| Analytics ||
  +
# Architecture
−
# Constraints and their classification
  
  +
# Use cases
−
# Generalized coordinates
  
  +
# SparkML
−
# Generalized forces
  
  +
# GraphX
−
# The D’Alembert-Lagrange’s principle
  
−
# The principle of virtual work
  
−
# The General Equation of dynamics
  
−
# Lagrange’s equations
  
−
# The Hamilton’s equations
  
 
|}
 
|}
 
== Intended Learning Outcomes (ILOs) ==
  
== Intended Learning Outcomes (ILOs) ==
   
 
=== What is the main purpose of this course? ===
  
=== What is the main purpose of this course? ===
  +
Nowadays companies need to manage vast amounts of data on a daily basis. Storing, sorting, accessing and analyzing obtaining synthetic information is considered one of the great challenges of the 21st century and and being effective in this may make the difference between success and failure. In order to gain a competitive advantage, Big Data and Analytics professionals are able to extract useful information from data and increase the Return Of Investments. In this course, students will be exposed to the key technologies and techniques, including R and Apache Spark, in order to analyze large-scale data sets and uncover valuable business information.
−
The purpose of the course is to give basic and advanced knowledge on theoretical mechanics. The course covers kinematics of a particle and a rigid body, statics of rigid bodies, particle dynamics, dynamics of a system, analytical mechanics. The objective of the course is to give knowledge and skills which can be used further for calculating of kinematics, statics and dynamics of mechanical parts of robots and studying advanced courses on robotics.
  
   
 
=== ILOs defined at three levels ===
  
=== ILOs defined at three levels ===
Line 71: Line 60:
 
==== Level 1: What concepts should a student know/remember/explain? ====
  
==== Level 1: What concepts should a student know/remember/explain? ====
 
By the end of the course, the students should be able to ...
  
By the end of the course, the students should be able to ...
  +
* Understanding of big data applications.
−
* Methods for describing the laws of motion of a particle and a solid,
  
  +
* Algorithms for the statistical analysis of big data
−
* Methods for calculating the speeds and accelerations of points and bodies included in a mechanical system,
  
  +
* Fundamental principles of predictive analytics
−
* Methods for studying the equilibrium of mechanical systems,
  
−
* Methods for creating differential equations of motion of a particle and a solid,
  
−
* Methods for creating differential equations of motion of a mechanical system based on the classical approach,
  
−
* Methods for creating differential equations of motion of a mechanical system based on methods of analytical mechanics.
  
   
 
==== Level 2: What basic practical skills should a student be able to perform? ====
  
==== Level 2: What basic practical skills should a student be able to perform? ====
 
By the end of the course, the students should be able to ...
  
By the end of the course, the students should be able to ...
−
* How to draw up and use calculation schemes,
 +
* How to process batch data
  +
* How to process stream data
−
* What calculation methods can be used to solve a specific problem,
  
  +
* Advanced design of distributed architectures
−
* What calculation methods are appropriate to use when solving a specific problem,
  
  +
* Advanced design of distributed algorithms
−
* What limitations and errors are imposed by a specific method when solving a problem.
  
   
 
==== Level 3: What complex comprehensive skills should a student be able to apply in real-life scenarios? ====
  
==== Level 3: What complex comprehensive skills should a student be able to apply in real-life scenarios? ====
 
By the end of the course, the students should be able to ...
  
By the end of the course, the students should be able to ...
  +
* Write a program for batch processing
−
* Analyze and explain mechanical phenomena based on the laws and theorems of theoretical mechanics,
  
  +
* Write a program for stream processing
−
* Apply the basic laws and methods of theoretical mechanics to solving technical problems,
  
  +
* Design distributed processing pipelines
−
* Create mathematical models, evaluate their value and the relativity of their limits of application.
 
  +
* Desing distributed algorithms
 
== Grading ==
  
== Grading ==
   
Line 113: Line 100:
 
! Activity Type !! Percentage of the overall course grade
  
! Activity Type !! Percentage of the overall course grade
 
|-
  
|-
−
| Labs/seminar classes || 10
 +
| Labs/seminar classes || 30
 
|-
  
|-
−
| Interim performance assessment || 50
 +
| Interim performance assessment || 30
 
|-
  
|-
 
| Exams || 40
  
| Exams || 40
Line 126: Line 113:
   
 
=== Open access resources ===
  
=== Open access resources ===
−
* S. Targ Theoretical Mechanics. A short course, 1968
 +
* Slides and material provided during the course.
  +
* F. Provost and T. Fawcett. Data Science for Business. O’Reilly, 2013
−
* D. Deleanu Theoretical mechanics. Theory and applications / Dumitru Deleanu – Constanta: Nautica, 2012
  
  +
* Matthew North. Data Mining for the Masses, Second Edition: with implementations in RapidMiner and R. CreateSpace Independent Publishing Platform, 2012
−
* Stephen T. Thornton and Jerry B. Marion Classical Dynamics of Particles and Systems. 5th edition, 2004
  
  +
* Tom White. Hadoop: The Definitive Guide. O’Reilly Media, Inc., 2012
−
* Meshchersky I.V. Collection of Problems in Theoretical Mechanics 2014
  
  +
* Seema Acharya and Subhashini Chellappan. Big data and analytics. WileyIndia, 2016
−
* Prof . Dr. Ing. Vasile Szolga Theoretical Mechanics, 2010
  
−
* S.M. Targ Kratki kurs teoreticheskoi mechaniki, 1986 - in Russian
  
−
* A.I. Lurie Analiticheskaya mechanika, 1961 - in Russian
  
−
* Sbornik kursovych rabot po teoreticheskoi mechanike. A.A.Yablonski, 2000 - in Russian
  
−
* Meshchersky I.V. Sbornik zadach po teoreticheskoi mechanike, 1986 - in Russian
  
   
 
=== Closed access resources ===
  
=== Closed access resources ===
Line 147: Line 130:
 
|+ Activities within each section
  
|+ Activities within each section
 
|-
  
|-
−
! Learning Activities !! Section 1 !! Section 2 !! Section 3 !! Section 4
 +
! Learning Activities !! Section 1 !! Section 2 !! Section 3 !! Section 4 !! Section 5
 
|-
  
|-
−
| Homework and group projects || 1 || 1 || 1 || 1
 +
| Testing (written or computer based) || 1 || 1 || 1 || 1 || 1
 
|-
  
|-
−
| Midterm evaluation || 1 || 1 || 0 || 0
 +
| Discussions || 1 || 1 || 1 || 1 || 1
 
|-
  
|-
−
| Testing (written or computer based) || 1 || 1 || 1 || 1
 +
| Development of individual parts of software product code || 0 || 1 || 1 || 1 || 1
  +
|-
  +
| Homework and group projects || 0 || 1 || 1 || 1 || 1
  +
|-
  +
| Midterm evaluation || 0 || 1 || 1 || 1 || 1
 
|}
 
|}
 
== Formative Assessment and Course Activities ==
  
== Formative Assessment and Course Activities ==
Line 165: Line 152:
 
! Activity Type !! Content !! Is Graded?
  
! Activity Type !! Content !! Is Graded?
 
|-
  
|-
  +
| Question || Describe the 6 Vs || 1
−
| Question || Calculate of the kinematic parameters of the particle according to the given laws of motion, it is required to determine: <br> particle trajectory, <br> particle velocity, <br> particle acceleration and its normal and tangential components, <br> radius of curvature of the trajectory. || 1
  
 
|-
  
|-
  +
| Question || Describe the technologies to support big data || 1
−
| Question || Calculate of the kinematic parameters of the planar mechanism, determine: <br> velocity of specific points of the mechanism and angular velocity of the links of the mechanism using the method of instantaneous velocity centers, <br> velocity of specific points of the mechanism and angular velocity of the links of the mechanism using the analytical method, <br> acceleration of specific points of the mechanism and angular accelerations of the links of the mechanism. || 1
  
 
|-
  
|-
  +
| Question || Design the structure of a cloud architecture for big data || 0
−
| Question || Calculate of the kinematics of the complex motion of a point, determine: <br> transport, relative and absolute velocity of the point, <br> transport, relative, Coriolis and absolute acceleration of the point. || 1
  
 
|-
  
|-
  +
| Question || Give examples of the 6 Vs in real systems || 0
−
| Question || Calculate of the kinematics of gears, determine the gear ratio, angular velocities and angular accelerations of links, velocities and accelerations of specific points of links for: <br> gearbox with fixed axles, <br> planetary gearbox with parallel axes, <br> planetary gearbox with intersecting axes. || 1
  
−
|-
  
−
| Question || Make a synthesis of the laws of motion of a point and a solid, taking into account given conditions and restrictions. || 0
  
−
|-
  
−
| Question || Do a kinematic analysis of complex planar mechanisms with a large number of links. || 0
  
−
|-
  
−
| Question || Do a kinematic analysis of complex planar mechanisms with several degrees of freedom. || 0
  
−
|-
  
−
| Question || Do a kinematic analysis of spatial mechanisms. || 0
  
−
|-
  
−
| Question || Do a kinematic analysis of the complex motion of a solid body. || 0
  
 
|}
 
|}
 
==== Section 2 ====
  
==== Section 2 ====
Line 189: Line 166:
 
! Activity Type !! Content !! Is Graded?
  
! Activity Type !! Content !! Is Graded?
 
|-
  
|-
−
| Question || Derive equilibrium equations for a system of concurrent forces. || 1
 +
| Question || Describe the characteristics of the different nodes of HDFS || 1
 
|-
  
|-
−
| Question || Derive equilibrium equations for a solid in 2D. || 1
 +
| Question || How files and blocks are managed || 1
 
|-
  
|-
−
| Question || Derive equilibrium equations for a system of two or more solids in 2D. || 1
 +
| Question || Describe the resource manager || 1
 
|-
  
|-
−
| Question || Derive equilibrium equations for a solid in 3D. || 1
 +
| Question || Describe the lifecycle of an application || 1
 
|-
  
|-
−
| Question || Apply equilibrium equations to calculate the reactions of supports and forces in the rods of a truss in 2D. || 0
 +
| Question || Describe and compare the scheduling approaches || 1
 
|-
  
|-
−
| Question || Apply equilibrium equations to calculate the reactions of supports of a solid body in 2D. || 0
 +
| Question || Configure a HDFS cluster || 0
 
|-
  
|-
  +
| Question || Build a HDFS client || 0
−
| Question || Apply equilibrium equations for calculating the reactions of supports of a system of bodies in 2D. || 0
  
 
|-
  
|-
−
| Question || Apply equilibrium equations to calculate the reactions of supports of a solid body in 3D. || 0
 +
| Question || Use a HDFS command line || 0
 
|-
  
|-
−
| Question || Investigate the equilibrium of a system of bodies taking into account friction. || 0
 +
| Question || Configure YARN || 0
  +
|-
  +
| Question || Evaluate the overall performance of YARN || 0
 
|}
 
|}
 
==== Section 3 ====
  
==== Section 3 ====
Line 213: Line 192:
 
! Activity Type !! Content !! Is Graded?
  
! Activity Type !! Content !! Is Graded?
 
|-
  
|-
−
| Question || Derive and solve the differential equations of rectilinear and curvilinear motion of a particle. || 1
 +
| Question || Describe the MapReduce model || 1
 
|-
  
|-
  +
| Question || Describe tasks management || 1
−
| Question || Derive and solve differential equations based on the theorem on the motion of the center of mass of a system. || 1
  
 
|-
  
|-
  +
| Question || Describe patterns of usage || 1
−
| Question || Derive and solve differential equations based on the theorem on the change in the angular momentum of a system. || 1
  
 
|-
  
|-
  +
| Question || Solve with MapReduce a specific problem || 0
−
| Question || Derive and solve the differential equations of rectilinear and curvilinear motion of bodies that form a system with one degree of freedom. || 1
  
 
|-
  
|-
  +
| Question || Implement a usage pattern || 0
−
| Question || Derive and solve differential equations of motion based on the D’Alembert’s principle. || 1
  
  +
|}
  +
==== Section 4 ====
  +
{| class="wikitable"
  +
|+
 
|-
  
|-
  +
! Activity Type !! Content !! Is Graded?
−
| Question || Derive and solve the differential equation based on the theorem on the change in kinetic energy. || 1
  
 
|-
  
|-
  +
| Question || Analyze the CAP theorem || 1
−
| Question || Apply the differential equations of a particle to study the motion of a body in a field of gravity under the influence of air resistance. || 0
  
 
|-
  
|-
−
| Question || Apply the differential equations of a particle to study oscillations. || 0
 +
| Question || Define the kinds of data storage available || 1
 
|-
  
|-
  +
| Question || Characteristics of stream processing || 1
−
| Question || Apply the theorem on the motion of the center of mass of a system to determine the dynamic reactions of the support of the mechanism. || 0
  
 
|-
  
|-
  +
| Question || Describe the usage patterns || 1
−
| Question || Apply the theorem on the change in the angular momentum of a system to study the gyroscopic effect. || 0
  
 
|-
  
|-
−
| Question || Apply the D’Alembert’s principle to determine the dynamic reactions of the supports of a mechanical system || 0
 +
| Question || Build a program to solve a problem with stream processing || 0
 
|-
  
|-
  +
| Question || Interact with a NoSQL database || 0
−
| Question || Apply the kinetic energy change theorem to determine the velosity of bodies of a mechanical system. || 0
  
 
|}
 
|}
−
==== Section 4 ====
 +
==== Section 5 ====
 
{| class="wikitable"
  
{| class="wikitable"
 
|+
 
|+
Line 243: Line 226:
 
! Activity Type !! Content !! Is Graded?
  
! Activity Type !! Content !! Is Graded?
 
|-
  
|-
−
| Question || What are generalized coordinates? || 1
 +
| Question || Features of SparkML || 1
−
|-
  
−
| Question || What are cyclic coordinates? || 1
  
 
|-
  
|-
−
| Question || Derive the differential equations of a mechanical system based on the principle of virtual work || 1
 +
| Question || Features of GraphX || 1
 
|-
  
|-
  +
| Question || Write a program using SparkML || 0
−
| Question || Derive the differential equations of a mechanical system based on the General Equation of dynamics || 1
  
 
|-
  
|-
−
| Question || Derive the differential equations of a mechanical system based on Lagrange’s equations || 1
 +
| Question || Write a program using GraphX || 0
−
|-
  
−
| Question || Apply the principle of virtual work to study the laws of motion of a mechanical system with one degree of freedom. || 0
  
−
|-
  
−
| Question || Apply the General Equation of dynamics to study the laws of motion of a mechanical system with one degree of freedom. || 0
  
−
|-
  
−
| Question || Apply the Lagrange’s equations to study the laws of motion of a mechanical system with several degrees of freedom. || 0
  
−
|-
  
−
| Question || Apply the Lagrange’s equations to study the oscillations of a mechanical system with two degrees of freedom. || 0
  
 
|}
 
|}
 
=== Final assessment ===
  
=== Final assessment ===
 
'''Section 1'''
  
'''Section 1'''
  +
# Design the structure of a cloud architecture for a specific analytics type
−
# Describe the vector, coordinate, and natural methods of specifying particle motion. Show the transition from one method to another. Find the velocity and acceleration of the particle in various methods.
  
  +
# Give examples of the 6 Vs in real systems
−
# Define the angular velocity vector and the angular acceleration vector of the body. Prove the independence of these vectors from the choice of the pole. Use the Euler vector formula to find the velocities and accelerations of points of a rotating rigid body and a rigid body making plane motion.
  
−
# Describe ways to set the orientation of a solid in space, including Euler angles, Tight-Brian angles, quaternions. Show the methodology for determining the angular velocity vector in these cases.
  
−
# Show the methodology of kinematic analysis of planar mechanisms, including the method of composing the equations of motion for the points of the mechanism, the theorems on the velocities and accelerations of body points in plane motion, and the instantaneous center of velocity method.
  
−
# Show the methodology of kinematic analysis of the complex motion of a particle, the theorems on the addition of velocities and accelerations for complex motion of a particle.
  
 
'''Section 2'''
  
'''Section 2'''
  +
# Configure a HDFS cluster with some specific replication approaches
−
# Explain the basic axioms of statics.
  
  +
# Build a HDFS client
−
# Demonstrate the methods for determining the moment of force about a point and about an axis.
  
  +
# Evaluate the performance of a specific configuration
−
# Demonstrate the methods for transformation a couple of forces.
  
  +
# Compare the different schedules
−
# Demonstrate the method for determining the principal vector and the principal moment of the force system.
  
−
# Describe the method for transformation a force system to the simplest possible form.
  
 
'''Section 3'''
  
'''Section 3'''
  +
# Describe the advantages and disadvantages of the MapReduce model
−
# Formulate the theorem of the motion of the center of mass of a system. Show for what problems this theorem is effective.
  
  +
# Solve a task designing the solution using MapReduce
−
# Formulate the D’Alambert’s principle. Show for what problems the calculation method based on this principle is effective.
  
  +
# Solve a task designing the solution using a composition of usage patterns
−
# Describe the concept of force field. Show the method for determining the work of a force at a movement of a particle in a potential force field.
  
−
# Describe the processes that occur upon impact and methods for calculating the law of motion of the body upon impact.
  
 
'''Section 4'''
  
'''Section 4'''
  +
# Identify problems and solutions related to the CAP theorem
−
# Formulate the the principle of virtual work. Show for what problems the calculation method based on this principle is effective.
  
  +
# Compare solutions with batch and stream processing approaches
−
# Formulate the the D’Alembert-Lagrange’s principle. Show for problems tasks the calculation method based on this principle is effective.
  
  +
# Design a system using a NoSQL database
−
# Demonstrate methods for calculating generalized forces.
  
  +
'''Section 5'''
−
# Show the different forms of writing the Lagrange equations and explain in which cases each of these forms will be more convenient.
  
  +
# Extend the SparkML library with a custom algorithm
−
# Demonstrate the principles of choosing the most convenient method for solving a given specific problem of mechanics.
  
  +
# Extend the GraphX library with a custom algorithm
   
 
=== The retake exam ===
  
=== The retake exam ===
Line 294: Line 263:
   
 
'''Section 4'''
  
'''Section 4'''
  +
  +
'''Section 5'''

Revision as of 13:23, 10 November 2022

Big Data Technologies and Analytics

  • Course name: Big Data Technologies and Analytics
  • Code discipline: N/A
  • Subject area: P.1 Short Description

Short Description

This course covers the following concepts: Advanced distributed data organization; Advanced distributed data processing.

Prerequisites

Prerequisite subjects

Prerequisite topics

Course Topics

Course Sections and Topics
Section Topics within the section
Introduction
  1. What is Big Data
  2. Characteristics of Big Data
  3. Technologies
  4. Virtualization and cloud computing
File systems and resource managers
  1. HDFS
  2. YARN
Batch Processing
  1. Distributed batch processing
  2. MapReduce model
  3. Applications
  4. Tasks management
  5. Patterns
Stream Processing
  1. CAP theorem
  2. Distributed storage and computation
  3. Distributed Stream Processing
  4. Usage patterns
Analytics
  1. Architecture
  2. Use cases
  3. SparkML
  4. GraphX

Intended Learning Outcomes (ILOs)

What is the main purpose of this course?

Nowadays companies need to manage vast amounts of data on a daily basis. Storing, sorting, accessing and analyzing obtaining synthetic information is considered one of the great challenges of the 21st century and and being effective in this may make the difference between success and failure. In order to gain a competitive advantage, Big Data and Analytics professionals are able to extract useful information from data and increase the Return Of Investments. In this course, students will be exposed to the key technologies and techniques, including R and Apache Spark, in order to analyze large-scale data sets and uncover valuable business information.

ILOs defined at three levels

Level 1: What concepts should a student know/remember/explain?

By the end of the course, the students should be able to ...

  • Understanding of big data applications.
  • Algorithms for the statistical analysis of big data
  • Fundamental principles of predictive analytics

Level 2: What basic practical skills should a student be able to perform?

By the end of the course, the students should be able to ...

  • How to process batch data
  • How to process stream data
  • Advanced design of distributed architectures
  • Advanced design of distributed algorithms

Level 3: What complex comprehensive skills should a student be able to apply in real-life scenarios?

By the end of the course, the students should be able to ...

  • Write a program for batch processing
  • Write a program for stream processing
  • Design distributed processing pipelines
  • Desing distributed algorithms

Grading

Course grading range

Grade Range Description of performance
A. Excellent 90-100 -
B. Good 75-89 -
C. Satisfactory 60-74 -
D. Poor 0-59 -

Course activities and grading breakdown

Activity Type Percentage of the overall course grade
Labs/seminar classes 30
Interim performance assessment 30
Exams 40

Recommendations for students on how to succeed in the course

Resources, literature and reference materials

Open access resources

  • Slides and material provided during the course.
  • F. Provost and T. Fawcett. Data Science for Business. O’Reilly, 2013
  • Matthew North. Data Mining for the Masses, Second Edition: with implementations in RapidMiner and R. CreateSpace Independent Publishing Platform, 2012
  • Tom White. Hadoop: The Definitive Guide. O’Reilly Media, Inc., 2012
  • Seema Acharya and Subhashini Chellappan. Big data and analytics. WileyIndia, 2016

Closed access resources

Software and tools used within the course

Teaching Methodology: Methods, techniques, & activities

Activities and Teaching Methods

Activities within each section
Learning Activities Section 1 Section 2 Section 3 Section 4 Section 5
Testing (written or computer based) 1 1 1 1 1
Discussions 1 1 1 1 1
Development of individual parts of software product code 0 1 1 1 1
Homework and group projects 0 1 1 1 1
Midterm evaluation 0 1 1 1 1

Formative Assessment and Course Activities

Ongoing performance assessment

Section 1

Activity Type Content Is Graded?
Question Describe the 6 Vs 1
Question Describe the technologies to support big data 1
Question Design the structure of a cloud architecture for big data 0
Question Give examples of the 6 Vs in real systems 0

Section 2

Activity Type Content Is Graded?
Question Describe the characteristics of the different nodes of HDFS 1
Question How files and blocks are managed 1
Question Describe the resource manager 1
Question Describe the lifecycle of an application 1
Question Describe and compare the scheduling approaches 1
Question Configure a HDFS cluster 0
Question Build a HDFS client 0
Question Use a HDFS command line 0
Question Configure YARN 0
Question Evaluate the overall performance of YARN 0

Section 3

Activity Type Content Is Graded?
Question Describe the MapReduce model 1
Question Describe tasks management 1
Question Describe patterns of usage 1
Question Solve with MapReduce a specific problem 0
Question Implement a usage pattern 0

Section 4

Activity Type Content Is Graded?
Question Analyze the CAP theorem 1
Question Define the kinds of data storage available 1
Question Characteristics of stream processing 1
Question Describe the usage patterns 1
Question Build a program to solve a problem with stream processing 0
Question Interact with a NoSQL database 0

Section 5

Activity Type Content Is Graded?
Question Features of SparkML 1
Question Features of GraphX 1
Question Write a program using SparkML 0
Question Write a program using GraphX 0

Final assessment

Section 1

  1. Design the structure of a cloud architecture for a specific analytics type
  2. Give examples of the 6 Vs in real systems

Section 2

  1. Configure a HDFS cluster with some specific replication approaches
  2. Build a HDFS client
  3. Evaluate the performance of a specific configuration
  4. Compare the different schedules

Section 3

  1. Describe the advantages and disadvantages of the MapReduce model
  2. Solve a task designing the solution using MapReduce
  3. Solve a task designing the solution using a composition of usage patterns

Section 4

  1. Identify problems and solutions related to the CAP theorem
  2. Compare solutions with batch and stream processing approaches
  3. Design a system using a NoSQL database

Section 5

  1. Extend the SparkML library with a custom algorithm
  2. Extend the GraphX library with a custom algorithm

The retake exam

Section 1

Section 2

Section 3

Section 4

Section 5

Retrieved from "https://eduwiki.innopolis.university/index.php?title=IU:TestPage&oldid=7751"