MSc: Big Data Technologies And Analytics - Revision history

R.sirgalina at 08:33, 29 August 2022

2022-08-29T08:33:08Z

M.petrishchev: /* Course Objectives Based on Bloom’s Taxonomy */

2022-04-26T07:53:01Z

Course Objectives Based on Bloom’s Taxonomy

← Older revision		Revision as of 07:53, 26 April 2022
Line 18:		Line 18:
	== Course Objectives Based on Bloom’s Taxonomy ==		== Course Objectives Based on Bloom’s Taxonomy ==

−	=== - What should a student remember at the end of the course? ===	+	=== What should a student remember at the end of the course? ===

	* Understanding of big data applications.		* Understanding of big data applications.

M.petrishchev: /* Course Objectives Based on Bloom’s Taxonomy */

2022-04-25T07:13:06Z

Course Objectives Based on Bloom’s Taxonomy

← Older revision		Revision as of 07:13, 25 April 2022
Line 24:		Line 24:
	* Fundamental principles of predictive analytics		* Fundamental principles of predictive analytics

−	=== - What should a student be able to understand at the end of the course? ===	+	=== What should a student be able to understand at the end of the course? ===

	By the end of the course, the students should be able to ...		By the end of the course, the students should be able to ...
Line 33:		Line 33:
	* Advanced design of distributed algorithms		* Advanced design of distributed algorithms

−	=== - What should a student be able to apply at the end of the course? ===	+	=== What should a student be able to apply at the end of the course? ===

	By the end of the course, the students should be able to ...		By the end of the course, the students should be able to ...

M.petrishchev at 07:00, 25 April 2022

2022-04-25T07:00:08Z

← Older revision		Revision as of 07:00, 25 April 2022
Line 16:		Line 16:
	Nowadays companies need to manage vast amounts of data on a daily basis. Storing, sorting, accessing and analyzing obtaining synthetic information is considered one of the great challenges of the 21st century and and being effective in this may make the difference between success and failure. In order to gain a competitive advantage, Big Data and Analytics professionals are able to extract useful information from data and increase the Return Of Investments. In this course, students will be exposed to the key technologies and techniques, including R and Apache Spark, in order to analyze large-scale data sets and uncover valuable business information.		Nowadays companies need to manage vast amounts of data on a daily basis. Storing, sorting, accessing and analyzing obtaining synthetic information is considered one of the great challenges of the 21st century and and being effective in this may make the difference between success and failure. In order to gain a competitive advantage, Big Data and Analytics professionals are able to extract useful information from data and increase the Return Of Investments. In this course, students will be exposed to the key technologies and techniques, including R and Apache Spark, in order to analyze large-scale data sets and uncover valuable business information.

−	=== Course ~~objectives~~ ~~based~~ on Bloom’s ~~taxonomy~~ ===	+	== Course Objectives Based on Bloom’s Taxonomy ==

	=== - What should a student remember at the end of the course? ===		=== - What should a student remember at the end of the course? ===

R.akhmetzyanov: /* Course Characteristics */

2022-04-21T07:54:32Z

Course Characteristics

← Older revision		Revision as of 07:54, 21 April 2022
Line 4:		Line 4:
	* <span>'''Course number:'''</span> N/A		* <span>'''Course number:'''</span> N/A

		+	== Prerequisites ==
	== Course Characteristics ==		== Course Characteristics ==

M.petrishchev: M.petrishchev moved page MSc:Big Data Technologies And Analytics to MSc: Big Data Technologies And Analytics

2022-04-15T07:24:17Z

M.petrishchev moved page MSc:Big Data Technologies And Analytics to MSc: Big Data Technologies And Analytics

← Older revision	Revision as of 07:24, 15 April 2022
(No difference)

M.petrishchev: M.petrishchev moved page MSc:BigDataTechnologiesAndAnalytics.F21 to MSc:Big Data Technologies And Analytics

2022-04-10T12:29:05Z

M.petrishchev moved page MSc:BigDataTechnologiesAndAnalytics.F21 to MSc:Big Data Technologies And Analytics

← Older revision	Revision as of 12:29, 10 April 2022
(No difference)

I.konyukhov: I.konyukhov moved page MSc:BigDataTechnologiesAndAnalytics to MSc:BigDataTechnologiesAndAnalytics.F21

2021-11-14T12:43:36Z

I.konyukhov moved page MSc:BigDataTechnologiesAndAnalytics to MSc:BigDataTechnologiesAndAnalytics.F21

← Older revision	Revision as of 12:43, 14 November 2021
(No difference)

10.90.136.11: Created page with "= Big Data Technologies and Analytics = * '''Course name:''' Big Data Technologies and Analytics * '''Course number:''' N/A == Course Characteristi..."

2021-07-30T11:20:32Z

Created page with "= Big Data Technologies and Analytics = * '''Course name:''' Big Data Technologies and Analytics * '''Course number:''' N/A == Course Characteristi..."

New page

= Big Data Technologies and Analytics =

* '''Course name:''' Big Data Technologies and Analytics
* '''Course number:''' N/A

== Course Characteristics ==

=== Key concepts of the class ===

* Advanced distributed data organization
* Advanced distributed data processing

=== What is the purpose of this course? ===

Nowadays companies need to manage vast amounts of data on a daily basis. Storing, sorting, accessing and analyzing obtaining synthetic information is considered one of the great challenges of the 21st century and and being effective in this may make the difference between success and failure. In order to gain a competitive advantage, Big Data and Analytics professionals are able to extract useful information from data and increase the Return Of Investments. In this course, students will be exposed to the key technologies and techniques, including R and Apache Spark, in order to analyze large-scale data sets and uncover valuable business information.

=== Course objectives based on Bloom’s taxonomy ===

=== - What should a student remember at the end of the course? ===

* Understanding of big data applications.
* Algorithms for the statistical analysis of big data
* Fundamental principles of predictive analytics

=== - What should a student be able to understand at the end of the course? ===

By the end of the course, the students should be able to ...

* How to process batch data
* How to process stream data
* Advanced design of distributed architectures
* Advanced design of distributed algorithms

=== - What should a student be able to apply at the end of the course? ===

By the end of the course, the students should be able to ...

* Write a program for batch processing
* Write a program for stream processing
* Design distributed processing pipelines
* Desing distributed algorithms

=== Course evaluation ===

{|
|+ Course grade breakdown
!
!
!align="center"| '''Proposed points'''
|-
| Labs/seminar classes
| 20
|align="center"| 30
|-
| Interim performance assessment
| 30
|align="center"| 30
|-
| Exams
| 50
|align="center"| 40
|}

If necessary, please indicate freely your course’s features in terms of students’ performance assessment.

=== Grades range ===

{|
|+ Course grading range
!
!
!align="center"| '''Proposed range'''
|-
| A. Excellent
| 90-100
|align="center"|
|-
| B. Good
| 75-89
|align="center"|
|-
| C. Satisfactory
| 60-74
|align="center"|
|-
| D. Poor
| 0-59
|align="center"|
|}

If necessary, please indicate freely your course’s grading features.

=== Resources and reference material ===

* Slides and material provided during the course.
* F. Provost and T. Fawcett. Data Science for Business. O’Reilly, 2013
* Matthew North. Data Mining for the Masses, Second Edition: with implementations in RapidMiner and R. CreateSpace Independent Publishing Platform, 2012
* Tom White. Hadoop: The Definitive Guide. O’Reilly Media, Inc., 2012
* Seema Acharya and Subhashini Chellappan. Big data and analytics. WileyIndia, 2016

== Course Sections ==

The main sections of the course and approximate hour distribution between them is as follows:

{|
|+ Course Sections
!align="center"| '''Section'''
! '''Section Title'''
!align="center"| '''Teaching Hours'''
|-
|align="center"| 1
| Introduction
|align="center"| 2
|-
|align="center"| 2
| File systems and resource managers
|align="center"| 8
|-
|align="center"| 3
| Batch Processing
|align="center"| 8
|-
|align="center"| 4
| Stream Processing
|align="center"| 8
|-
|align="center"| 5
| Analytics
|align="center"| 4
|}

=== Section 1 ===

==== Section title: ====

Introduction

=== Topics covered in this section: ===

* What is Big Data
* Characteristics of Big Data
* Technologies
* Virtualization and cloud computing

=== What forms of evaluation were used to test students’ performance in this section? ===

<div class="tabular">

|a|c| & '''Yes/No''' 
Development of individual parts of software product code & 0 
Homework and group projects & 0 
Midterm evaluation & 0 
Testing (written or computer based) & 1 
Reports & 0 
Essays & 0 
Oral polls & 0 
Discussions & 1 

</div>
=== Typical questions for ongoing performance evaluation within this section ===

# Describe the 6 Vs
# Describe the technologies to support big data

=== Typical questions for seminar classes (labs) within this section ===

# Design the structure of a cloud architecture for big data
# Give examples of the 6 Vs in real systems

=== Test questions for final assessment in this section ===

# Design the structure of a cloud architecture for a specific analytics type
# Give examples of the 6 Vs in real systems

=== Section 2 ===

==== Section title: ====

File systems and resource managers

==== Topics covered in this section: ====

* HDFS
* YARN

=== What forms of evaluation were used to test students’ performance in this section? ===

<div class="tabular">

|a|c| & '''Yes/No''' 
Development of individual parts of software product code & 1 
Homework and group projects & 1 
Midterm evaluation & 1 
Testing (written or computer based) & 1 
Reports & 0 
Essays & 0 
Oral polls & 0 
Discussions & 1 

</div>
=== Typical questions for ongoing performance evaluation within this section ===

# Describe the characteristics of the different nodes of HDFS
# How files and blocks are managed
# Describe the resource manager
# Describe the lifecycle of an application
# Describe and compare the scheduling approaches

==== Typical questions for seminar classes (labs) within this section ====

# Configure a HDFS cluster
# Build a HDFS client
# Use a HDFS command line
# Configure YARN
# Evaluate the overall performance of YARN

==== Test questions for final assessment in this section ====

# Configure a HDFS cluster with some specific replication approaches
# Build a HDFS client
# Evaluate the performance of a specific configuration
# Compare the different schedules

=== Section 3 ===

==== Section title: ====

Batch Processing

==== Topics covered in this section: ====

* Distributed batch processing
* MapReduce model
* Applications
* Tasks management
* Patterns

=== What forms of evaluation were used to test students’ performance in this section? ===

<div class="tabular">

|a|c| & '''Yes/No''' 
Development of individual parts of software product code & 1 
Homework and group projects & 1 
Midterm evaluation & 1 
Testing (written or computer based) & 1 
Reports & 0 
Essays & 0 
Oral polls & 0 
Discussions & 1 

</div>
=== Typical questions for ongoing performance evaluation within this section ===

# Describe the MapReduce model
# Describe tasks management
# Describe patterns of usage

==== Typical questions for seminar classes (labs) within this section ====

# Solve with MapReduce a specific problem
# Implement a usage pattern

==== Test questions for final assessment in this section ====

# Describe the advantages and disadvantages of the MapReduce model
# Solve a task designing the solution using MapReduce
# Solve a task designing the solution using a composition of usage patterns

=== Section 4 ===

==== Section title: ====

Stream Processing

==== Topics covered in this section: ====

* CAP theorem
* Distributed storage and computation
* Distributed Stream Processing
* Usage patterns

=== What forms of evaluation were used to test students’ performance in this section? ===

<div class="tabular">

|a|c| & '''Yes/No''' 
Development of individual parts of software product code & 1 
Homework and group projects & 1 
Midterm evaluation & 1 
Testing (written or computer based) & 1 
Reports & 0 
Essays & 0 
Oral polls & 0 
Discussions & 1 

</div>
=== Typical questions for ongoing performance evaluation within this section ===

# Analyze the CAP theorem
# Define the kinds of data storage available
# Characteristics of stream processing
# Describe the usage patterns

==== Typical questions for seminar classes (labs) within this section ====

# Build a program to solve a problem with stream processing
# Interact with a NoSQL database

==== Test questions for final assessment in this section ====

# Identify problems and solutions related to the CAP theorem
# Compare solutions with batch and stream processing approaches
# Design a system using a NoSQL database

=== Section 5 ===

==== Section title: ====

Analytics

==== Topics covered in this section: ====

* Architecture
* Use cases
* SparkML
* GraphX

=== What forms of evaluation were used to test students’ performance in this section? ===

<div class="tabular">

|a|c| & '''Yes/No''' 
Development of individual parts of software product code & 1 
Homework and group projects & 1 
Midterm evaluation & 1 
Testing (written or computer based) & 1 
Reports & 0 
Essays & 0 
Oral polls & 0 
Discussions & 1 

</div>
=== Typical questions for ongoing performance evaluation within this section ===

# Features of SparkML
# Features of GraphX

==== Typical questions for seminar classes (labs) within this section ====

# Write a program using SparkML
# Write a program using GraphX

==== Test questions for final assessment in this section ====

# Extend the SparkML library with a custom algorithm
# Extend the GraphX library with a custom algorithm