Difference between revisions of "MSc: Big Data Technologies And Analytics"

From IU
Jump to navigation Jump to search
 
Line 1: Line 1:
  +
 
= Big Data Technologies and Analytics =
  
= Big Data Technologies and Analytics =
  +
* '''Course name''': Big Data Technologies and Analytics
  +
* '''Code discipline''': N/A
  +
* '''Subject area''':
   
  +
== Short Description ==
−
* <span>'''Course name:'''</span> Big Data Technologies and Analytics
  
  +
This course covers the following concepts: Advanced distributed data organization; Advanced distributed data processing.
−
* <span>'''Course number:'''</span> N/A
  
   
 
== Prerequisites ==
  
== Prerequisites ==
−
== Course Characteristics ==
  
   
−
=== Key concepts of the class ===
 +
=== Prerequisite subjects ===
   
−
* Advanced distributed data organization
  
−
* Advanced distributed data processing
  
   
  +
=== Prerequisite topics ===
−
=== What is the purpose of this course? ===
  
   
  +
  +
== Course Topics ==
  +
{| class="wikitable"
  +
|+ Course Sections and Topics
  +
|-
  +
! Section !! Topics within the section
  +
|-
  +
| Introduction ||
  +
# What is Big Data
  +
# Characteristics of Big Data
  +
# Technologies
  +
# Virtualization and cloud computing
  +
|-
  +
| File systems and resource managers ||
  +
# HDFS
  +
# YARN
  +
|-
  +
| Batch Processing ||
  +
# Distributed batch processing
  +
# MapReduce model
  +
# Applications
  +
# Tasks management
  +
# Patterns
  +
|-
  +
| Stream Processing ||
  +
# CAP theorem
  +
# Distributed storage and computation
  +
# Distributed Stream Processing
  +
# Usage patterns
  +
|-
  +
| Analytics ||
  +
# Architecture
  +
# Use cases
  +
# SparkML
  +
# GraphX
  +
|}
  +
== Intended Learning Outcomes (ILOs) ==
  +
  +
=== What is the main purpose of this course? ===
 
Nowadays companies need to manage vast amounts of data on a daily basis. Storing, sorting, accessing and analyzing obtaining synthetic information is considered one of the great challenges of the 21st century and and being effective in this may make the difference between success and failure. In order to gain a competitive advantage, Big Data and Analytics professionals are able to extract useful information from data and increase the Return Of Investments. In this course, students will be exposed to the key technologies and techniques, including R and Apache Spark, in order to analyze large-scale data sets and uncover valuable business information.
  
Nowadays companies need to manage vast amounts of data on a daily basis. Storing, sorting, accessing and analyzing obtaining synthetic information is considered one of the great challenges of the 21st century and and being effective in this may make the difference between success and failure. In order to gain a competitive advantage, Big Data and Analytics professionals are able to extract useful information from data and increase the Return Of Investments. In this course, students will be exposed to the key technologies and techniques, including R and Apache Spark, in order to analyze large-scale data sets and uncover valuable business information.
   
  +
=== ILOs defined at three levels ===
−
== Course Objectives Based on Bloom’s Taxonomy ==
  
−  
−
=== What should a student remember at the end of the course? ===
  
   
  +
==== Level 1: What concepts should a student know/remember/explain? ====
  +
By the end of the course, the students should be able to ...
 
* Understanding of big data applications.
  
* Understanding of big data applications.
 
* Algorithms for the statistical analysis of big data
  
* Algorithms for the statistical analysis of big data
 
* Fundamental principles of predictive analytics
  
* Fundamental principles of predictive analytics
   
−
=== What should a student be able to understand at the end of the course? ===
 +
==== Level 2: What basic practical skills should a student be able to perform? ====
−  
 
By the end of the course, the students should be able to ...
  
By the end of the course, the students should be able to ...
−  
 
* How to process batch data
  
* How to process batch data
 
* How to process stream data
  
* How to process stream data
Line 33: Line 71:
 
* Advanced design of distributed algorithms
  
* Advanced design of distributed algorithms
   
−
=== What should a student be able to apply at the end of the course? ===
 +
==== Level 3: What complex comprehensive skills should a student be able to apply in real-life scenarios? ====
−  
 
By the end of the course, the students should be able to ...
  
By the end of the course, the students should be able to ...
−  
 
* Write a program for batch processing
  
* Write a program for batch processing
 
* Write a program for stream processing
  
* Write a program for stream processing
 
* Design distributed processing pipelines
  
* Design distributed processing pipelines
−
* Desing distributed algorithms
 +
* Desing distributed algorithms
  +
== Grading ==
   
−
=== Course evaluation ===
 +
=== Course grading range ===
  +
{| class="wikitable"
−  
−
{|
 +
|+
−
|+ Course grade breakdown
  
−
!
  
−
!
  
−
!align="center"| '''Proposed points'''
  
 
|-
  
|-
  +
! Grade !! Range !! Description of performance
−
| Labs/seminar classes
  
−
| 20
  
−
|align="center"| 30
  
 
|-
  
|-
  +
| A. Excellent || 90-100 || -
−
| Interim performance assessment
  
−
| 30
  
−
|align="center"| 30
  
 
|-
  
|-
  +
| B. Good || 75-89 || -
−
| Exams
  
−
| 50
 +
|-
  +
| C. Satisfactory || 60-74 || -
−
|align="center"| 40
  
  +
|-
  +
| D. Poor || 0-59 || -
 
|}
  
|}
   
  +
=== Course activities and grading breakdown ===
−
If necessary, please indicate freely your course’s features in terms of students’ performance assessment.
  
  +
{| class="wikitable"
−  
  +
|+
−
=== Grades range ===
  
−  
−
{|
  
−
|+ Course grading range
  
−
!
  
−
!
  
−
!align="center"| '''Proposed range'''
  
 
|-
  
|-
  +
! Activity Type !! Percentage of the overall course grade
−
| A. Excellent
  
−
| 90-100
  
−
|align="center"|
  
 
|-
  
|-
  +
| Labs/seminar classes || 30
−
| B. Good
  
−
| 75-89
  
−
|align="center"|
  
 
|-
  
|-
  +
| Interim performance assessment || 30
−
| C. Satisfactory
  
−
| 60-74
  
−
|align="center"|
  
 
|-
  
|-
−
| D. Poor
 +
| Exams || 40
−
| 0-59
  
−
|align="center"|
  
 
|}
  
|}
   
  +
=== Recommendations for students on how to succeed in the course ===
−
If necessary, please indicate freely your course’s grading features.
  
   
−
=== Resources and reference material ===
  
   
  +
== Resources, literature and reference materials ==
  +
  +
=== Open access resources ===
 
* Slides and material provided during the course.
  
* Slides and material provided during the course.
 
* F. Provost and T. Fawcett. Data Science for Business. O’Reilly, 2013
  
* F. Provost and T. Fawcett. Data Science for Business. O’Reilly, 2013
Line 100: Line 119:
 
* Seema Acharya and Subhashini Chellappan. Big data and analytics. WileyIndia, 2016
  
* Seema Acharya and Subhashini Chellappan. Big data and analytics. WileyIndia, 2016
   
−
== Course Sections ==
 +
=== Closed access resources ===
  +
   
  +
=== Software and tools used within the course ===
−
The main sections of the course and approximate hour distribution between them is as follows:
  
  +
  +
= Teaching Methodology: Methods, techniques, & activities =
   
  +
== Activities and Teaching Methods ==
−
{|
  
  +
{| class="wikitable"
−
|+ Course Sections
  
  +
|+ Activities within each section
−
!align="center"| '''Section'''
  
−
! '''Section Title'''
  
−
!align="center"| '''Teaching Hours'''
  
 
|-
  
|-
  +
! Learning Activities !! Section 1 !! Section 2 !! Section 3 !! Section 4 !! Section 5
−
|align="center"| 1
  
−
| Introduction
  
−
|align="center"| 2
  
 
|-
  
|-
  +
| Testing (written or computer based) || 1 || 1 || 1 || 1 || 1
−
|align="center"| 2
  
−
| File systems and resource managers
  
−
|align="center"| 8
  
 
|-
  
|-
  +
| Discussions || 1 || 1 || 1 || 1 || 1
−
|align="center"| 3
  
−
| Batch Processing
  
−
|align="center"| 8
  
 
|-
  
|-
  +
| Development of individual parts of software product code || 0 || 1 || 1 || 1 || 1
−
|align="center"| 4
  
−
| Stream Processing
  
−
|align="center"| 8
  
 
|-
  
|-
  +
| Homework and group projects || 0 || 1 || 1 || 1 || 1
−
|align="center"| 5
  
  +
|-
−
| Analytics
  
  +
| Midterm evaluation || 0 || 1 || 1 || 1 || 1
−
|align="center"| 4
  
−
|}
 +
|}
  +
== Formative Assessment and Course Activities ==
   
−
=== Section 1 ===
 +
=== Ongoing performance assessment ===
−  
−
==== Section title: ====
  
−  
−
Introduction
  
−  
−
=== Topics covered in this section: ===
  
−  
−
* What is Big Data
  
−
* Characteristics of Big Data
  
−
* Technologies
  
−
* Virtualization and cloud computing
  
−  
−
=== What forms of evaluation were used to test students’ performance in this section? ===
  
−  
−
<div class="tabular">
  
−  
−
<span>|a|c|</span> &amp; '''Yes/No'''<br />
  
−
Development of individual parts of software product code &amp; 0<br />
  
−
Homework and group projects &amp; 0<br />
  
−
Midterm evaluation &amp; 0<br />
  
−
Testing (written or computer based) &amp; 1<br />
  
−
Reports &amp; 0<br />
  
−
Essays &amp; 0<br />
  
−
Oral polls &amp; 0<br />
  
−
Discussions &amp; 1<br />
  
−  
−  
−  
−
</div>
  
−
=== Typical questions for ongoing performance evaluation within this section ===
  
−  
−
# Describe the 6 Vs
  
−
# Describe the technologies to support big data
  
−  
−
=== Typical questions for seminar classes (labs) within this section ===
  
−  
−
# Design the structure of a cloud architecture for big data
  
−
# Give examples of the 6 Vs in real systems
  
−  
−
=== Test questions for final assessment in this section ===
  
   
  +
==== Section 1 ====
  +
{| class="wikitable"
  +
|+
  +
|-
  +
! Activity Type !! Content !! Is Graded?
  +
|-
  +
| Question || Describe the 6 Vs || 1
  +
|-
  +
| Question || Describe the technologies to support big data || 1
  +
|-
  +
| Question || Design the structure of a cloud architecture for big data || 0
  +
|-
  +
| Question || Give examples of the 6 Vs in real systems || 0
  +
|}
  +
==== Section 2 ====
  +
{| class="wikitable"
  +
|+
  +
|-
  +
! Activity Type !! Content !! Is Graded?
  +
|-
  +
| Question || Describe the characteristics of the different nodes of HDFS || 1
  +
|-
  +
| Question || How files and blocks are managed || 1
  +
|-
  +
| Question || Describe the resource manager || 1
  +
|-
  +
| Question || Describe the lifecycle of an application || 1
  +
|-
  +
| Question || Describe and compare the scheduling approaches || 1
  +
|-
  +
| Question || Configure a HDFS cluster || 0
  +
|-
  +
| Question || Build a HDFS client || 0
  +
|-
  +
| Question || Use a HDFS command line || 0
  +
|-
  +
| Question || Configure YARN || 0
  +
|-
  +
| Question || Evaluate the overall performance of YARN || 0
  +
|}
  +
==== Section 3 ====
  +
{| class="wikitable"
  +
|+
  +
|-
  +
! Activity Type !! Content !! Is Graded?
  +
|-
  +
| Question || Describe the MapReduce model || 1
  +
|-
  +
| Question || Describe tasks management || 1
  +
|-
  +
| Question || Describe patterns of usage || 1
  +
|-
  +
| Question || Solve with MapReduce a specific problem || 0
  +
|-
  +
| Question || Implement a usage pattern || 0
  +
|}
  +
==== Section 4 ====
  +
{| class="wikitable"
  +
|+
  +
|-
  +
! Activity Type !! Content !! Is Graded?
  +
|-
  +
| Question || Analyze the CAP theorem || 1
  +
|-
  +
| Question || Define the kinds of data storage available || 1
  +
|-
  +
| Question || Characteristics of stream processing || 1
  +
|-
  +
| Question || Describe the usage patterns || 1
  +
|-
  +
| Question || Build a program to solve a problem with stream processing || 0
  +
|-
  +
| Question || Interact with a NoSQL database || 0
  +
|}
  +
==== Section 5 ====
  +
{| class="wikitable"
  +
|+
  +
|-
  +
! Activity Type !! Content !! Is Graded?
  +
|-
  +
| Question || Features of SparkML || 1
  +
|-
  +
| Question || Features of GraphX || 1
  +
|-
  +
| Question || Write a program using SparkML || 0
  +
|-
  +
| Question || Write a program using GraphX || 0
  +
|}
  +
=== Final assessment ===
  +
'''Section 1'''
 
# Design the structure of a cloud architecture for a specific analytics type
  
# Design the structure of a cloud architecture for a specific analytics type
 
# Give examples of the 6 Vs in real systems
  
# Give examples of the 6 Vs in real systems
  +
'''Section 2'''
−  
−
=== Section 2 ===
  
−  
−
==== Section title: ====
  
−  
−
File systems and resource managers
  
−  
−
==== Topics covered in this section: ====
  
−  
−
* HDFS
  
−
* YARN
  
−  
−
=== What forms of evaluation were used to test students’ performance in this section? ===
  
−  
−
<div class="tabular">
  
−  
−
<span>|a|c|</span> &amp; '''Yes/No'''<br />
  
−
Development of individual parts of software product code &amp; 1<br />
  
−
Homework and group projects &amp; 1<br />
  
−
Midterm evaluation &amp; 1<br />
  
−
Testing (written or computer based) &amp; 1<br />
  
−
Reports &amp; 0<br />
  
−
Essays &amp; 0<br />
  
−
Oral polls &amp; 0<br />
  
−
Discussions &amp; 1<br />
  
−  
−  
−  
−
</div>
  
−
=== Typical questions for ongoing performance evaluation within this section ===
  
−  
−
# Describe the characteristics of the different nodes of HDFS
  
−
# How files and blocks are managed
  
−
# Describe the resource manager
  
−
# Describe the lifecycle of an application
  
−
# Describe and compare the scheduling approaches
  
−  
−
==== Typical questions for seminar classes (labs) within this section ====
  
−  
−
# Configure a HDFS cluster
  
−
# Build a HDFS client
  
−
# Use a HDFS command line
  
−
# Configure YARN
  
−
# Evaluate the overall performance of YARN
  
−  
−
==== Test questions for final assessment in this section ====
  
−  
 
# Configure a HDFS cluster with some specific replication approaches
  
# Configure a HDFS cluster with some specific replication approaches
 
# Build a HDFS client
  
# Build a HDFS client
 
# Evaluate the performance of a specific configuration
  
# Evaluate the performance of a specific configuration
 
# Compare the different schedules
  
# Compare the different schedules
  +
'''Section 3'''
−  
−
=== Section 3 ===
  
−  
−
==== Section title: ====
  
−  
−
Batch Processing
  
−  
−
==== Topics covered in this section: ====
  
−  
−
* Distributed batch processing
  
−
* MapReduce model
  
−
* Applications
  
−
* Tasks management
  
−
* Patterns
  
−  
−
=== What forms of evaluation were used to test students’ performance in this section? ===
  
−  
−
<div class="tabular">
  
−  
−
<span>|a|c|</span> &amp; '''Yes/No'''<br />
  
−
Development of individual parts of software product code &amp; 1<br />
  
−
Homework and group projects &amp; 1<br />
  
−
Midterm evaluation &amp; 1<br />
  
−
Testing (written or computer based) &amp; 1<br />
  
−
Reports &amp; 0<br />
  
−
Essays &amp; 0<br />
  
−
Oral polls &amp; 0<br />
  
−
Discussions &amp; 1<br />
  
−  
−  
−  
−
</div>
  
−
=== Typical questions for ongoing performance evaluation within this section ===
  
−  
−
# Describe the MapReduce model
  
−
# Describe tasks management
  
−
# Describe patterns of usage
  
−  
−
==== Typical questions for seminar classes (labs) within this section ====
  
−  
−
# Solve with MapReduce a specific problem
  
−
# Implement a usage pattern
  
−  
−
==== Test questions for final assessment in this section ====
  
−  
 
# Describe the advantages and disadvantages of the MapReduce model
  
# Describe the advantages and disadvantages of the MapReduce model
 
# Solve a task designing the solution using MapReduce
  
# Solve a task designing the solution using MapReduce
 
# Solve a task designing the solution using a composition of usage patterns
  
# Solve a task designing the solution using a composition of usage patterns
  +
'''Section 4'''
−  
−
=== Section 4 ===
  
−  
−
==== Section title: ====
  
−  
−
Stream Processing
  
−  
−
==== Topics covered in this section: ====
  
−  
−
* CAP theorem
  
−
* Distributed storage and computation
  
−
* Distributed Stream Processing
  
−
* Usage patterns
  
−  
−
=== What forms of evaluation were used to test students’ performance in this section? ===
  
−  
−
<div class="tabular">
  
−  
−
<span>|a|c|</span> &amp; '''Yes/No'''<br />
  
−
Development of individual parts of software product code &amp; 1<br />
  
−
Homework and group projects &amp; 1<br />
  
−
Midterm evaluation &amp; 1<br />
  
−
Testing (written or computer based) &amp; 1<br />
  
−
Reports &amp; 0<br />
  
−
Essays &amp; 0<br />
  
−
Oral polls &amp; 0<br />
  
−
Discussions &amp; 1<br />
  
−  
−  
−  
−
</div>
  
−
=== Typical questions for ongoing performance evaluation within this section ===
  
−  
−
# Analyze the CAP theorem
  
−
# Define the kinds of data storage available
  
−
# Characteristics of stream processing
  
−
# Describe the usage patterns
  
−  
−
==== Typical questions for seminar classes (labs) within this section ====
  
−  
−
# Build a program to solve a problem with stream processing
  
−
# Interact with a NoSQL database
  
−  
−
==== Test questions for final assessment in this section ====
  
−  
 
# Identify problems and solutions related to the CAP theorem
  
# Identify problems and solutions related to the CAP theorem
 
# Compare solutions with batch and stream processing approaches
  
# Compare solutions with batch and stream processing approaches
 
# Design a system using a NoSQL database
  
# Design a system using a NoSQL database
  +
'''Section 5'''
  +
# Extend the SparkML library with a custom algorithm
  +
# Extend the GraphX library with a custom algorithm
   
−
=== Section 5 ===
 +
=== The retake exam ===
  +
'''Section 1'''
   
−
==== Section title: ====
 +
'''Section 2'''
   
  +
'''Section 3'''
−
Analytics
  
   
  +
'''Section 4'''
−
==== Topics covered in this section: ====
  
   
  +
'''Section 5'''
−
* Architecture
  
−
* Use cases
  
−
* SparkML
  
−
* GraphX
  
−  
−
=== What forms of evaluation were used to test students’ performance in this section? ===
  
−  
−
<div class="tabular">
  
−  
−
<span>|a|c|</span> &amp; '''Yes/No'''<br />
  
−
Development of individual parts of software product code &amp; 1<br />
  
−
Homework and group projects &amp; 1<br />
  
−
Midterm evaluation &amp; 1<br />
  
−
Testing (written or computer based) &amp; 1<br />
  
−
Reports &amp; 0<br />
  
−
Essays &amp; 0<br />
  
−
Oral polls &amp; 0<br />
  
−
Discussions &amp; 1<br />
  
−  
−  
−  
−
</div>
  
−
=== Typical questions for ongoing performance evaluation within this section ===
  
−  
−
# Features of SparkML
  
−
# Features of GraphX
  
−  
−
==== Typical questions for seminar classes (labs) within this section ====
  
−  
−
# Write a program using SparkML
  
−
# Write a program using GraphX
  
−  
−
==== Test questions for final assessment in this section ====
  
−  
−
# Extend the SparkML library with a custom algorithm
  
−
# Extend the GraphX library with a custom algorithm
  

Latest revision as of 11:33, 29 August 2022

Big Data Technologies and Analytics

  • Course name: Big Data Technologies and Analytics
  • Code discipline: N/A
  • Subject area:

Short Description

This course covers the following concepts: Advanced distributed data organization; Advanced distributed data processing.

Prerequisites

Prerequisite subjects

Prerequisite topics

Course Topics

Course Sections and Topics
Section Topics within the section
Introduction
  1. What is Big Data
  2. Characteristics of Big Data
  3. Technologies
  4. Virtualization and cloud computing
File systems and resource managers
  1. HDFS
  2. YARN
Batch Processing
  1. Distributed batch processing
  2. MapReduce model
  3. Applications
  4. Tasks management
  5. Patterns
Stream Processing
  1. CAP theorem
  2. Distributed storage and computation
  3. Distributed Stream Processing
  4. Usage patterns
Analytics
  1. Architecture
  2. Use cases
  3. SparkML
  4. GraphX

Intended Learning Outcomes (ILOs)

What is the main purpose of this course?

Nowadays companies need to manage vast amounts of data on a daily basis. Storing, sorting, accessing and analyzing obtaining synthetic information is considered one of the great challenges of the 21st century and and being effective in this may make the difference between success and failure. In order to gain a competitive advantage, Big Data and Analytics professionals are able to extract useful information from data and increase the Return Of Investments. In this course, students will be exposed to the key technologies and techniques, including R and Apache Spark, in order to analyze large-scale data sets and uncover valuable business information.

ILOs defined at three levels

Level 1: What concepts should a student know/remember/explain?

By the end of the course, the students should be able to ...

  • Understanding of big data applications.
  • Algorithms for the statistical analysis of big data
  • Fundamental principles of predictive analytics

Level 2: What basic practical skills should a student be able to perform?

By the end of the course, the students should be able to ...

  • How to process batch data
  • How to process stream data
  • Advanced design of distributed architectures
  • Advanced design of distributed algorithms

Level 3: What complex comprehensive skills should a student be able to apply in real-life scenarios?

By the end of the course, the students should be able to ...

  • Write a program for batch processing
  • Write a program for stream processing
  • Design distributed processing pipelines
  • Desing distributed algorithms

Grading

Course grading range

Grade Range Description of performance
A. Excellent 90-100 -
B. Good 75-89 -
C. Satisfactory 60-74 -
D. Poor 0-59 -

Course activities and grading breakdown

Activity Type Percentage of the overall course grade
Labs/seminar classes 30
Interim performance assessment 30
Exams 40

Recommendations for students on how to succeed in the course

Resources, literature and reference materials

Open access resources

  • Slides and material provided during the course.
  • F. Provost and T. Fawcett. Data Science for Business. O’Reilly, 2013
  • Matthew North. Data Mining for the Masses, Second Edition: with implementations in RapidMiner and R. CreateSpace Independent Publishing Platform, 2012
  • Tom White. Hadoop: The Definitive Guide. O’Reilly Media, Inc., 2012
  • Seema Acharya and Subhashini Chellappan. Big data and analytics. WileyIndia, 2016

Closed access resources

Software and tools used within the course

Teaching Methodology: Methods, techniques, & activities

Activities and Teaching Methods

Activities within each section
Learning Activities Section 1 Section 2 Section 3 Section 4 Section 5
Testing (written or computer based) 1 1 1 1 1
Discussions 1 1 1 1 1
Development of individual parts of software product code 0 1 1 1 1
Homework and group projects 0 1 1 1 1
Midterm evaluation 0 1 1 1 1

Formative Assessment and Course Activities

Ongoing performance assessment

Section 1

Activity Type Content Is Graded?
Question Describe the 6 Vs 1
Question Describe the technologies to support big data 1
Question Design the structure of a cloud architecture for big data 0
Question Give examples of the 6 Vs in real systems 0

Section 2

Activity Type Content Is Graded?
Question Describe the characteristics of the different nodes of HDFS 1
Question How files and blocks are managed 1
Question Describe the resource manager 1
Question Describe the lifecycle of an application 1
Question Describe and compare the scheduling approaches 1
Question Configure a HDFS cluster 0
Question Build a HDFS client 0
Question Use a HDFS command line 0
Question Configure YARN 0
Question Evaluate the overall performance of YARN 0

Section 3

Activity Type Content Is Graded?
Question Describe the MapReduce model 1
Question Describe tasks management 1
Question Describe patterns of usage 1
Question Solve with MapReduce a specific problem 0
Question Implement a usage pattern 0

Section 4

Activity Type Content Is Graded?
Question Analyze the CAP theorem 1
Question Define the kinds of data storage available 1
Question Characteristics of stream processing 1
Question Describe the usage patterns 1
Question Build a program to solve a problem with stream processing 0
Question Interact with a NoSQL database 0

Section 5

Activity Type Content Is Graded?
Question Features of SparkML 1
Question Features of GraphX 1
Question Write a program using SparkML 0
Question Write a program using GraphX 0

Final assessment

Section 1

  1. Design the structure of a cloud architecture for a specific analytics type
  2. Give examples of the 6 Vs in real systems

Section 2

  1. Configure a HDFS cluster with some specific replication approaches
  2. Build a HDFS client
  3. Evaluate the performance of a specific configuration
  4. Compare the different schedules

Section 3

  1. Describe the advantages and disadvantages of the MapReduce model
  2. Solve a task designing the solution using MapReduce
  3. Solve a task designing the solution using a composition of usage patterns

Section 4

  1. Identify problems and solutions related to the CAP theorem
  2. Compare solutions with batch and stream processing approaches
  3. Design a system using a NoSQL database

Section 5

  1. Extend the SparkML library with a custom algorithm
  2. Extend the GraphX library with a custom algorithm

The retake exam

Section 1

Section 2

Section 3

Section 4

Section 5

Retrieved from "https://eduwiki.innopolis.university/index.php?title=MSc:_Big_Data_Technologies_And_Analytics&oldid=7460"