Difference between revisions of "IU:TestPage"
R.sirgalina (talk | contribs) Tag: Manual revert |
R.sirgalina (talk | contribs) |
||
Line 1: | Line 1: | ||
+ | = Introduction to Big Data = |
||
− | = Business Development, Sales and Marketing in IT Industry = |
||
− | * '''Course name''': |
+ | * '''Course name''': Introduction to Big Data |
− | * '''Code discipline''': |
+ | * '''Code discipline''': N/A |
− | * '''Subject area''': |
+ | * '''Subject area''': |
== Short Description == |
== Short Description == |
||
− | This course |
+ | This course covers the following concepts: Distributed data organization; Distributed data processing. |
− | These are the parts that are linked with each other - it is very difficult to sell without marketing support and it is very difficult to achieve results with marketing efforts only. |
||
− | Marketing part, starting from defining things like developing marketing strategy for the companies, finally offers practical tools of digital marketing. We will explore new digital reality and its impact on IT business. We will learn success stories of real businesses and how companies are adapting to the new changing landscape. |
||
− | The second part of the course covers important things for every company's success – the sales process. Understand how to attract customers in negotiations, how to “get to yes” getting great deals, how to control the sales funnel – you will get the understanding how it works and try it in practice. |
||
== Prerequisites == |
== Prerequisites == |
||
=== Prerequisite subjects === |
=== Prerequisite subjects === |
||
+ | |||
− | * HSS310 |
||
=== Prerequisite topics === |
=== Prerequisite topics === |
||
+ | |||
− | * Basic IT industry knowledge |
||
− | * Basic marketing knowledge |
||
== Course Topics == |
== Course Topics == |
||
Line 26: | Line 22: | ||
! Section !! Topics within the section |
! Section !! Topics within the section |
||
|- |
|- |
||
− | | |
+ | | Introduction || |
− | # |
+ | # What is Big Data |
+ | # Characteristics of Big Data |
||
− | # Product-centric marketing |
||
+ | # Data Structures |
||
− | # Customer-centric marketing |
||
+ | # Types of Analytics |
||
− | # Developing Marketing Strategy |
||
+ | |- |
||
+ | | Hadoop || |
||
+ | # Data storage |
||
+ | # Clustering |
||
+ | # Design decisions |
||
+ | # Scaling |
||
+ | # Distributed systems |
||
+ | # The ecosystem |
||
+ | |- |
||
+ | | HDFS || |
||
+ | # Distributed storage |
||
+ | # Types of nodes |
||
+ | # Files and blocks |
||
+ | # Replication |
||
+ | # Memory usage |
||
+ | |- |
||
+ | | MapReduce || |
||
+ | # Distributed processing |
||
+ | # MapReduce model |
||
+ | # Applications |
||
+ | # Tasks management |
||
+ | # Patterns |
||
|- |
|- |
||
− | | |
+ | | YARN || |
+ | # Resource manager |
||
− | # Brand&Presentation |
||
+ | # Components |
||
− | # Analytics |
||
+ | # Run an application |
||
− | # Content |
||
+ | # Schedules |
||
− | # SMM |
||
− | # Context advertising |
||
− | # E-mail marketing |
||
|- |
|- |
||
+ | | Optimizing Data Processing || |
||
− | | Sales || |
||
− | # |
+ | # CAP theorem |
+ | # Distributed storage and computation |
||
− | # B2B |
||
+ | # Batch Processing |
||
− | # B2C |
||
+ | # Stream Processing |
||
− | # Negotiations |
||
+ | # Usage patterns |
||
+ | # NoSQL databases |
||
|- |
|- |
||
+ | | Spark || |
||
− | | Final Project Presentation || |
||
+ | # Architecture |
||
− | # Presentation of marketing&sales strategy and tactics for startup |
||
+ | # Use cases |
||
+ | # Job scheduling |
||
+ | # Data types |
||
+ | # SparkML |
||
+ | # GraphX |
||
|} |
|} |
||
== Intended Learning Outcomes (ILOs) == |
== Intended Learning Outcomes (ILOs) == |
||
=== What is the main purpose of this course? === |
=== What is the main purpose of this course? === |
||
+ | Software systems are increasingly based on large amount of data that come from a wide range of sources (e.g., logs, sensors, user-generated content, etc.). However, data are useful only if it can be analyzed properly to extract meaningful information can be used (e.g., to take decisions, to make predictions, etc.). This course provides an overview of the state-of-the-art technologies, tools, architectures, and systems constituting the big data computing solutions landscape. Particular attention will be given to the Hadoop ecosystem that is widely adopted in the industry. |
||
− | This course aims to give students the skills of developing a winning marketing strategy for a startup, as well as the skills to implement marketing strategy using real digital-marketing tools and sales tactics for a startup product. |
||
=== ILOs defined at three levels === |
=== ILOs defined at three levels === |
||
Line 58: | Line 81: | ||
==== Level 1: What concepts should a student know/remember/explain? ==== |
==== Level 1: What concepts should a student know/remember/explain? ==== |
||
By the end of the course, the students should be able to ... |
By the end of the course, the students should be able to ... |
||
+ | * The most common structures of distributed storage. |
||
− | * Develop naming, presentation, and product offer |
||
+ | * Batch processing techniques |
||
− | * Use digital marketing tools |
||
+ | * Stream processing techniques |
||
− | * Use CRM |
||
+ | * Basic distributed data processing algorithms |
||
− | * Sell its product |
||
+ | * Basic tools to address specific processing needs |
||
==== Level 2: What basic practical skills should a student be able to perform? ==== |
==== Level 2: What basic practical skills should a student be able to perform? ==== |
||
By the end of the course, the students should be able to ... |
By the end of the course, the students should be able to ... |
||
+ | * The basis of the CAP theorem |
||
− | * Skills of market type identification |
||
+ | * The structure of the MapReduce |
||
− | * Skills in developing naming, presentations, product offerings |
||
+ | * How to process batch data |
||
− | * Skills of context advertising |
||
+ | * How to process stream data |
||
− | * Skills of SMM doing |
||
+ | * The characteristics of a NoSQL database |
||
− | * Skills of content marketing |
||
− | * Skills of e-mail marketing |
||
==== Level 3: What complex comprehensive skills should a student be able to apply in real-life scenarios? ==== |
==== Level 3: What complex comprehensive skills should a student be able to apply in real-life scenarios? ==== |
||
By the end of the course, the students should be able to ... |
By the end of the course, the students should be able to ... |
||
+ | * Use a NoSQL database |
||
− | * Skills for valuation the market environment |
||
+ | * Write a program for batch processing |
||
− | * Skills how to find the right addressable market for its product |
||
+ | * Write a program for stream processing |
||
− | * Skills of web analytics |
||
− | * Skills of CRM using |
||
− | * Sales skills to various types of clients |
||
== Grading == |
== Grading == |
||
Line 87: | Line 108: | ||
! Grade !! Range !! Description of performance |
! Grade !! Range !! Description of performance |
||
|- |
|- |
||
− | | A. Excellent || 90-100 || |
+ | | A. Excellent || 90-100 || - |
|- |
|- |
||
− | | B. Good || 75-89 || |
+ | | B. Good || 75-89 || - |
|- |
|- |
||
− | | C. Satisfactory || 60-74 || |
+ | | C. Satisfactory || 60-74 || - |
|- |
|- |
||
− | | D. |
+ | | D. Poor || 0-59 || - |
|} |
|} |
||
Line 102: | Line 123: | ||
! Activity Type !! Percentage of the overall course grade |
! Activity Type !! Percentage of the overall course grade |
||
|- |
|- |
||
− | | |
+ | | Labs/seminar classes || 30 |
|- |
|- |
||
− | | Interim performance assessment |
+ | | Interim performance assessment || 30 |
|- |
|- |
||
− | | |
+ | | Exams || 40 |
|} |
|} |
||
=== Recommendations for students on how to succeed in the course === |
=== Recommendations for students on how to succeed in the course === |
||
+ | |||
− | The student is recommended the following scheme of preparation for classes:<br>Marketing and sales are much more about hypothesis testing and math, than creativity. Therefore, it is so important for students to try the acquired knowledge in real practice, doing small tasks after each lecture.<br>Finally, we will try to assemble a working strategy for a startup from these tasks.<br>Moreover:<br>Participation is important. Showing up is the key to success in this course.<br>Reading the recommended literature is optional, and will give you a deeper understanding of the material. |
||
== Resources, literature and reference materials == |
== Resources, literature and reference materials == |
||
=== Open access resources === |
=== Open access resources === |
||
+ | * Slides and material provided during the course. |
||
− | * Андрей Кравченко. Неидеальная стратегия для идеальной компании. |
||
+ | * Vignesh Prajapati. Big Data Analytics with R and Hadoop. Packt Publishing, 2013 |
||
− | * Peter Fader. Customer Centricity. |
||
+ | * Jules J. Berman. Principles of Big Data: Preparing, Sharing, and Analyzing Complex Information. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2013 |
||
=== Closed access resources === |
=== Closed access resources === |
||
+ | |||
− | * Viktor Pelevin. Empire V. |
||
− | * W. Chan Kim, Renee Mauborgne. Blue Ocean Strategy. |
||
− | * Eric ries. Lean startup. |
||
− | * Simon Kingsnorth. Digital Marketing Strategy. |
||
− | * Chet Holmes. The Ultimate Sales Machine. |
||
=== Software and tools used within the course === |
=== Software and tools used within the course === |
||
+ | |||
− | * Standard office tools for Tables, Text and Presentation |
||
= Teaching Methodology: Methods, techniques, & activities = |
= Teaching Methodology: Methods, techniques, & activities = |
||
Line 133: | Line 151: | ||
|+ Teaching and Learning Methods within each section |
|+ Teaching and Learning Methods within each section |
||
|- |
|- |
||
− | ! Teaching Techniques !! Section 1 !! Section 2 !! Section 3 !! Section 4 |
+ | ! Teaching Techniques !! Section 1 !! Section 2 !! Section 3 !! Section 4 !! Section 5 !! Section 6 !! Section 7 |
|- |
|- |
||
− | | |
+ | | Testing (written or computer based) || 1 || 1 || 1 || 1 || 1 || 1 || 1 |
|- |
|- |
||
− | | |
+ | | Discussions || 1 || 1 || 1 || 1 || 1 || 1 || 1 |
|- |
|- |
||
− | | |
+ | | Development of individual parts of software product code || 1 || 1 || 1 || 1 || 1 || 1 || 1 |
|- |
|- |
||
− | | |
+ | | Homework and group projects || 1 || 1 || 1 || 1 || 1 || 1 || 1 |
+ | |- |
||
+ | | Midterm evaluation || 1 || 1 || 1 || 1 || 1 || 1 || 1 |
||
|} |
|} |
||
{| class="wikitable" |
{| class="wikitable" |
||
|+ Activities within each section |
|+ Activities within each section |
||
|- |
|- |
||
− | ! Learning Activities !! Section 1 !! Section 2 !! Section 3 !! Section 4 |
+ | ! Learning Activities !! Section 1 !! Section 2 !! Section 3 !! Section 4 !! Section 5 !! Section 6 !! Section 7 |
|- |
|- |
||
− | | |
+ | | Question || 0 || 1 || 0 || 0 || 0 || 0 || 0 |
+ | |} |
||
+ | == Formative Assessment and Course Activities == |
||
+ | |||
+ | === Ongoing performance assessment === |
||
+ | |||
+ | ==== Section 1 ==== |
||
+ | {| class="wikitable" |
||
+ | |+ |
||
|- |
|- |
||
+ | ! Activity Type !! Content !! Is Graded? |
||
− | | Interactive Lectures || 1 || 1 || 1 || 0 |
||
|- |
|- |
||
− | | |
+ | | Question || Describe the 6 Vs || 1 |
|- |
|- |
||
− | | |
+ | | Question || Describe the types of analytics || 1 |
|- |
|- |
||
+ | | Question || Design the structure of a DB to address a specific analytics type || 0 |
||
− | | Individual Projects || 1 || 1 || 1 || 1 |
||
|- |
|- |
||
− | | |
+ | | Question || Give examples of the 6 Vs in real systems || 0 |
+ | |} |
||
+ | ==== Section 2 ==== |
||
+ | {| class="wikitable" |
||
+ | |+ |
||
|- |
|- |
||
+ | ! Activity Type !! Content !! Is Graded? |
||
− | | Discussions || 1 || 1 || 1 || 1 |
||
|- |
|- |
||
− | | |
+ | | Question || Describe the Hadoop ecosystem || 1 |
|- |
|- |
||
− | | |
+ | | Question || Structure of an Hadoop cluster || 1 |
|- |
|- |
||
− | | |
+ | | Question || Describe the scaling techniques || 1 |
|- |
|- |
||
− | | |
+ | | Question || Configure a basic Hadoop node || 0 |
|- |
|- |
||
− | | |
+ | | Question || Configure a basic Hadoop cluster || 0 |
|} |
|} |
||
+ | ==== Section 3 ==== |
||
− | == Formative Assessment and Course Activities == |
||
− | |||
− | === Ongoing performance assessment === |
||
− | |||
− | ==== Section 1 ==== |
||
{| class="wikitable" |
{| class="wikitable" |
||
|+ |
|+ |
||
Line 182: | Line 210: | ||
! Activity Type !! Content !! Is Graded? |
! Activity Type !! Content !! Is Graded? |
||
|- |
|- |
||
+ | | Question || Describe the characteristics of the different nodes || 1 |
||
− | | after lecture assignments || Define target audience and describe type of market for your product. || 1 |
||
|- |
|- |
||
+ | | Question || How files and blocks are managed || 1 |
||
− | | after lecture assignments || Make 3 cusdev with potential/existing customers of your product. || 1 |
||
|- |
|- |
||
+ | | Question || How memory is managed || 1 |
||
− | | after lecture assignments || Develop your marketing strategy and present it in-class. || 1 |
||
+ | |- |
||
+ | | Question || How replication works || 1 |
||
+ | |- |
||
+ | | Question || Configure a HDFS cluster || 0 |
||
+ | |- |
||
+ | | Question || Configure different replication approaches || 0 |
||
+ | |- |
||
+ | | Question || Build a HDFS client || 0 |
||
+ | |- |
||
+ | | Question || Use a HDFS command line || 0 |
||
|} |
|} |
||
− | ==== Section |
+ | ==== Section 4 ==== |
{| class="wikitable" |
{| class="wikitable" |
||
|+ |
|+ |
||
Line 194: | Line 232: | ||
! Activity Type !! Content !! Is Graded? |
! Activity Type !! Content !! Is Graded? |
||
|- |
|- |
||
+ | | Question || Describe the MapReduce model || 1 |
||
− | | after lecture assignments || Write a marketing article about your product or technology in the informational style manner. || 1 |
||
|- |
|- |
||
+ | | Question || Describe tasks management || 1 |
||
− | | after lecture assignments || Create a landing page for your product and connect it to Yandex Metrica or Google Analytics. || 1 |
||
|- |
|- |
||
+ | | Question || Describe patterns of usage || 1 |
||
− | | after lecture assignments || Create a semantic core for your product and determine the current positions on your landing page. Determine key marketing metrics, including conversion rate, on your landing page. || 1 |
||
+ | |- |
||
+ | | Question || Solve with MapReduce a specific problem || 0 |
||
+ | |- |
||
+ | | Question || Implement a usage pattern || 0 |
||
|} |
|} |
||
− | ==== Section |
+ | ==== Section 5 ==== |
{| class="wikitable" |
{| class="wikitable" |
||
|+ |
|+ |
||
Line 206: | Line 248: | ||
! Activity Type !! Content !! Is Graded? |
! Activity Type !! Content !! Is Graded? |
||
|- |
|- |
||
+ | | Question || Describe the resource manager || 1 |
||
− | | after lecture assignments || Create the sales funnel of your product and present it in-class. || 1 |
||
|- |
|- |
||
+ | | Question || Describe the lifecycle of an application || 1 |
||
− | | after lecture assignments || Create the budget for your marketing and sales activities and approve it with management. || 1 |
||
|- |
|- |
||
− | | |
+ | | Question || Describe and compare the scheduling approaches || 1 |
+ | |- |
||
+ | | Question || Compare the performance of the different schedules in different load conditions || 0 |
||
+ | |- |
||
+ | | Question || Configure YARN || 0 |
||
+ | |- |
||
+ | | Question || Evaluate the overall performance of YARN || 0 |
||
+ | |} |
||
+ | ==== Section 6 ==== |
||
+ | {| class="wikitable" |
||
+ | |+ |
||
+ | |- |
||
+ | ! Activity Type !! Content !! Is Graded? |
||
+ | |- |
||
+ | | Question || Analyze the CAP theorem || 1 |
||
+ | |- |
||
+ | | Question || Define the kinds of data storage available || 1 |
||
+ | |- |
||
+ | | Question || Characteristics of batch processing || 1 |
||
+ | |- |
||
+ | | Question || Characteristics of stream processing || 1 |
||
+ | |- |
||
+ | | Question || Describe the usage patterns || 1 |
||
+ | |- |
||
+ | | Question || Compare NoSQL databases || 1 |
||
+ | |- |
||
+ | | Question || Build a program to solve a problem with batch processing || 0 |
||
+ | |- |
||
+ | | Question || Build a program to solve a problem with stream processing || 0 |
||
+ | |- |
||
+ | | Question || Interact with a NoSQL database || 0 |
||
+ | |} |
||
+ | ==== Section 7 ==== |
||
+ | {| class="wikitable" |
||
+ | |+ |
||
+ | |- |
||
+ | ! Activity Type !! Content !! Is Graded? |
||
+ | |- |
||
+ | | Question || Describe the architecture of Spark || 1 |
||
+ | |- |
||
+ | | Question || Describe the types of schedulers || 1 |
||
+ | |- |
||
+ | | Question || Different characteristics of the data types || 1 |
||
+ | |- |
||
+ | | Question || Features of SparkML || 1 |
||
+ | |- |
||
+ | | Question || Features of GraphX || 1 |
||
+ | |- |
||
+ | | Question || Analyze the performance of different schedulers || 0 |
||
+ | |- |
||
+ | | Question || Write a program exploiting the features of each data type || 0 |
||
+ | |- |
||
+ | | Question || Write a program using SparkML || 0 |
||
+ | |- |
||
+ | | Question || Write a program using GraphX || 0 |
||
|} |
|} |
||
− | ==== Section 4 ==== |
||
− | |||
=== Final assessment === |
=== Final assessment === |
||
'''Section 1''' |
'''Section 1''' |
||
+ | # Design the structure of a DB to address a specific analytics type |
||
− | # For the final assessment, students have to prepare a full project of marketing and sales promotion of their IT product and present it on the exam. The project should contain the next parts: |
||
+ | # Give examples of the 6 Vs in real systems |
||
− | # The idea of your product/service. |
||
− | # Define your market. |
||
− | # Analise what type of market. |
||
− | # Target segment, who should we talk to? |
||
− | # What is your main message(s)? |
||
− | # What should we do to achieve the addressable market? |
||
− | # Brand promotion, knowledge, interest, coverage, sales etc. |
||
− | # Media design. |
||
− | # How should we say it? Creative strategy&content. |
||
− | # Channel (media) strategy. |
||
− | # How do we reach them? Evidence on a real case. |
||
− | # Budget. |
||
− | # Money for promotion. |
||
− | # How to close deals. Evidence on a real case. |
||
− | # Measurement. |
||
− | # How we control the result. Evidence on a real case. |
||
'''Section 2''' |
'''Section 2''' |
||
+ | # Identify the Hadoop components useful to address a specific problem. |
||
− | |||
+ | # Configure an multi-node Hadoop system. |
||
'''Section 3''' |
'''Section 3''' |
||
+ | # Configure a HDFS cluster with some specific replication approaches |
||
− | |||
+ | # Build a HDFS client |
||
'''Section 4''' |
'''Section 4''' |
||
+ | # Describe the advantages and disadvantages of the MapReduce model |
||
− | |||
+ | # Solve a task designing the solution using MapReduce |
||
+ | # Solve a task designing the solution using a composition of usage patterns |
||
+ | '''Section 5''' |
||
+ | # Evaluate the performance of a specific configuration |
||
+ | # Compare the different schedules |
||
+ | '''Section 6''' |
||
+ | # Identify problems and solutions related to the CAP theorem |
||
+ | # Compare solutions with batch and stream processing approaches |
||
+ | # Design a system using a NoSQL database |
||
+ | '''Section 7''' |
||
+ | # Compare the performance of different schedules with different loads |
||
+ | # Extend the SparkML library with a custom algorithm |
||
+ | # Extend the GraphX library with a custom algorithm |
||
=== The retake exam === |
=== The retake exam === |
||
'''Section 1''' |
'''Section 1''' |
||
+ | |||
− | # .3 The retake exam. |
||
− | # For the retake, students have to implement a product and follow the guidelines of the course. There has to be a meeting before the retake itself to plan and agree on the product ideas, and to answer questions. |
||
'''Section 2''' |
'''Section 2''' |
||
Line 249: | Line 342: | ||
'''Section 4''' |
'''Section 4''' |
||
+ | |||
+ | '''Section 5''' |
||
+ | |||
+ | '''Section 6''' |
||
+ | |||
+ | '''Section 7''' |
Revision as of 15:57, 15 August 2022
Introduction to Big Data
- Course name: Introduction to Big Data
- Code discipline: N/A
- Subject area:
Short Description
This course covers the following concepts: Distributed data organization; Distributed data processing.
Prerequisites
Prerequisite subjects
Prerequisite topics
Course Topics
Section | Topics within the section |
---|---|
Introduction |
|
Hadoop |
|
HDFS |
|
MapReduce |
|
YARN |
|
Optimizing Data Processing |
|
Spark |
|
Intended Learning Outcomes (ILOs)
What is the main purpose of this course?
Software systems are increasingly based on large amount of data that come from a wide range of sources (e.g., logs, sensors, user-generated content, etc.). However, data are useful only if it can be analyzed properly to extract meaningful information can be used (e.g., to take decisions, to make predictions, etc.). This course provides an overview of the state-of-the-art technologies, tools, architectures, and systems constituting the big data computing solutions landscape. Particular attention will be given to the Hadoop ecosystem that is widely adopted in the industry.
ILOs defined at three levels
Level 1: What concepts should a student know/remember/explain?
By the end of the course, the students should be able to ...
- The most common structures of distributed storage.
- Batch processing techniques
- Stream processing techniques
- Basic distributed data processing algorithms
- Basic tools to address specific processing needs
Level 2: What basic practical skills should a student be able to perform?
By the end of the course, the students should be able to ...
- The basis of the CAP theorem
- The structure of the MapReduce
- How to process batch data
- How to process stream data
- The characteristics of a NoSQL database
Level 3: What complex comprehensive skills should a student be able to apply in real-life scenarios?
By the end of the course, the students should be able to ...
- Use a NoSQL database
- Write a program for batch processing
- Write a program for stream processing
Grading
Course grading range
Grade | Range | Description of performance |
---|---|---|
A. Excellent | 90-100 | - |
B. Good | 75-89 | - |
C. Satisfactory | 60-74 | - |
D. Poor | 0-59 | - |
Course activities and grading breakdown
Activity Type | Percentage of the overall course grade |
---|---|
Labs/seminar classes | 30 |
Interim performance assessment | 30 |
Exams | 40 |
Recommendations for students on how to succeed in the course
Resources, literature and reference materials
Open access resources
- Slides and material provided during the course.
- Vignesh Prajapati. Big Data Analytics with R and Hadoop. Packt Publishing, 2013
- Jules J. Berman. Principles of Big Data: Preparing, Sharing, and Analyzing Complex Information. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2013
Closed access resources
Software and tools used within the course
Teaching Methodology: Methods, techniques, & activities
Activities and Teaching Methods
Teaching Techniques | Section 1 | Section 2 | Section 3 | Section 4 | Section 5 | Section 6 | Section 7 |
---|---|---|---|---|---|---|---|
Testing (written or computer based) | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
Discussions | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
Development of individual parts of software product code | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
Homework and group projects | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
Midterm evaluation | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
Learning Activities | Section 1 | Section 2 | Section 3 | Section 4 | Section 5 | Section 6 | Section 7 |
---|---|---|---|---|---|---|---|
Question | 0 | 1 | 0 | 0 | 0 | 0 | 0 |
Formative Assessment and Course Activities
Ongoing performance assessment
Section 1
Activity Type | Content | Is Graded? |
---|---|---|
Question | Describe the 6 Vs | 1 |
Question | Describe the types of analytics | 1 |
Question | Design the structure of a DB to address a specific analytics type | 0 |
Question | Give examples of the 6 Vs in real systems | 0 |
Section 2
Activity Type | Content | Is Graded? |
---|---|---|
Question | Describe the Hadoop ecosystem | 1 |
Question | Structure of an Hadoop cluster | 1 |
Question | Describe the scaling techniques | 1 |
Question | Configure a basic Hadoop node | 0 |
Question | Configure a basic Hadoop cluster | 0 |
Section 3
Activity Type | Content | Is Graded? |
---|---|---|
Question | Describe the characteristics of the different nodes | 1 |
Question | How files and blocks are managed | 1 |
Question | How memory is managed | 1 |
Question | How replication works | 1 |
Question | Configure a HDFS cluster | 0 |
Question | Configure different replication approaches | 0 |
Question | Build a HDFS client | 0 |
Question | Use a HDFS command line | 0 |
Section 4
Activity Type | Content | Is Graded? |
---|---|---|
Question | Describe the MapReduce model | 1 |
Question | Describe tasks management | 1 |
Question | Describe patterns of usage | 1 |
Question | Solve with MapReduce a specific problem | 0 |
Question | Implement a usage pattern | 0 |
Section 5
Activity Type | Content | Is Graded? |
---|---|---|
Question | Describe the resource manager | 1 |
Question | Describe the lifecycle of an application | 1 |
Question | Describe and compare the scheduling approaches | 1 |
Question | Compare the performance of the different schedules in different load conditions | 0 |
Question | Configure YARN | 0 |
Question | Evaluate the overall performance of YARN | 0 |
Section 6
Activity Type | Content | Is Graded? |
---|---|---|
Question | Analyze the CAP theorem | 1 |
Question | Define the kinds of data storage available | 1 |
Question | Characteristics of batch processing | 1 |
Question | Characteristics of stream processing | 1 |
Question | Describe the usage patterns | 1 |
Question | Compare NoSQL databases | 1 |
Question | Build a program to solve a problem with batch processing | 0 |
Question | Build a program to solve a problem with stream processing | 0 |
Question | Interact with a NoSQL database | 0 |
Section 7
Activity Type | Content | Is Graded? |
---|---|---|
Question | Describe the architecture of Spark | 1 |
Question | Describe the types of schedulers | 1 |
Question | Different characteristics of the data types | 1 |
Question | Features of SparkML | 1 |
Question | Features of GraphX | 1 |
Question | Analyze the performance of different schedulers | 0 |
Question | Write a program exploiting the features of each data type | 0 |
Question | Write a program using SparkML | 0 |
Question | Write a program using GraphX | 0 |
Final assessment
Section 1
- Design the structure of a DB to address a specific analytics type
- Give examples of the 6 Vs in real systems
Section 2
- Identify the Hadoop components useful to address a specific problem.
- Configure an multi-node Hadoop system.
Section 3
- Configure a HDFS cluster with some specific replication approaches
- Build a HDFS client
Section 4
- Describe the advantages and disadvantages of the MapReduce model
- Solve a task designing the solution using MapReduce
- Solve a task designing the solution using a composition of usage patterns
Section 5
- Evaluate the performance of a specific configuration
- Compare the different schedules
Section 6
- Identify problems and solutions related to the CAP theorem
- Compare solutions with batch and stream processing approaches
- Design a system using a NoSQL database
Section 7
- Compare the performance of different schedules with different loads
- Extend the SparkML library with a custom algorithm
- Extend the GraphX library with a custom algorithm
The retake exam
Section 1
Section 2
Section 3
Section 4
Section 5
Section 6
Section 7