Deprecated: mysql_connect(): The mysql extension is deprecated and will be removed in the future: use mysqli or PDO instead in /home/content/77/8880177/html/ipartner/db.php on line 2
Big Data, Data Science Training - Combo Course -iPartner
loading...

Big Data, Data Science Training - Combo Course

  1. Big Data, Data Science Training - Combo Course
237 hr / 3910*
(* including all taxes.)


Key Features
SMALL
BATCHES
MENTORING
BY EXPERTS
FLEXIBLE
SCHEDULE
LEARN
BY DOING
GOAL
ORIENTED



Course Agenda


  • What is Big Data?
  • Factors constituting Big Data
  • Hadoop and its Ecosystem
  • Map Reduce -Concepts of Map, Reduce, Ordering, Concurrency, Shuffle, Reducing, Concurrency
  • Hadoop Distributed File System (HDFS) Concepts and its Importance
  • Deep Dive in Map Reduce – Execution Framework, Partitioner, Combiner, Data Types, Key pairs
  • HDFS Deep Dive – Architecture, Data Replication, Name Node, Data Node, Data Flow
  • Parallel Copying with DISTCP, Hadoop Archives
  • How to develop Map Reduce Application, writing unit test
  • Best Practices for developing and writing, Debugging Map Reduce applications
  • Joining Data sets in Map Reduce
  • Hadoop API’s
  • Introduction to Hadoop Yarn
  • Difference between Hadoop 1.0 and 2.0

1. Introduction to Pig

  • What Is Pig?
  • Pig’s Features
  • Pig Use Cases
  • Interacting with Pig

2. Basic Data Analysis with Pig

  • Pig Latin Syntax
  • Loading Data
  • Simple Data Types
  • Field Definitions
  • Data Output
  • Viewing the Schema
  • Filtering and Sorting Data
  • Commonly-Used Functions
  • Hands-On Exercise: Using Pig for ETL Processing

3. Processing Complex Data with Pig

  • Complex/Nested Data Types
  • Grouping
  • Iterating Grouped Data
  • Hands-On Exercise: Analyzing Data with Pig
  • Apache Spark- Introduction, Consistency, Availability, Partition
  • Unified Stack Spark
  • Spark Components
  • Comparison with Hadoop – Scalding example, mahout, storm, graph
  • Explain python example
  • Show installing a spark
  • Explain driver program
  • Explaining spark context with example
  • Define weakly typed variable
  • Combine scala and java seamlessly.
  • Explain concurrency and distribution.
  • Explain what is trait.
  • Explain higher order function with example.
  • Define OFI scheduler.
  • Advantages of Spark
  • Example of Lamda using spark
  • Explain Mapreduce with example
  • How ETL tools work in Big data Industry
  • Connecting to HDFS from ETL tool and moving data from Local system to HDFS
  • Moving Data from DBMS to HDFS
  • Working with Hive with ETL Tool
  • Creating Map Reduce job in ETL tool
  • End to End ETL PoC showing Hadoop integration with ETL tool.

  • Namenode/Datanode directory structures and files
  • File system image and Edit log
  • The Checkpoint Procedure
  • Namenode failure and recovery procedure
  • Safe Mode
  • Metadata and Data backup
  • Potential problems and solutions / what to look for
  • Adding and removing nodes
  • ZOOKEEPER Introduction
  • ZOOKEEPER use cases
  • ZOOKEEPER Services
  • ZOOKEEPER data Model
  • Znodes and its types
  • Znodes operations
  • Znodes watches
  • Znodes reads and writes
  • Consistency Guarantees
  • Cluster management
  • Leader Election
  • Distributed Exclusive Lock
  • Important points
  • Why Oozie?
  • Installing Oozie
  • Running an example
  • Oozie- workflow engine
  • Example M/R action
  • Word count example
  • Workflow application
  • Workflow submission
  • Workflow state transitions
  • Oozie job processing
  • Oozie- HADOOP security
  • Why Oozie security?
  • Job submission to hadoop
  • Multi tenancy and scalability
  • Time line of Oozie job
  • Coordinator
  • Bundle
  • Layers of abstraction
  • Architecture
  • Use Case 1: time triggers
  • Use Case 2: data and time triggers
  • Use Case 3: rolling window
  • Apache Flume
  • Big data ecosystem
  • Physically distributed Data sources
  • Changing structure of Data
  • Closer look
  • Anatomy of Flume
  • Core concepts
  • Event
  • Clients
  • Agents
  • Source
  • Channels
  • Sinks
  • Interceptors
  • Channel selector
  • Sink processor
  • Data ingest
  • Agent pipeline
  • Transactional data exchange
  • Routing and replicating
  • Why channels?
  • Use case- Log aggregation
  • Adding flume agent
  • Handling a server farm
  • Data volume per agent
  • Example describing a single node flume deployment
  • IMPALA Overview: Goals
  • User view of Impala: Overview
  • User view of Impala: SQL
  • User view of Impala: Apache HBase
  • Impala architecture
  • Impala state store
  • Impala catalogue service
  • Query execution phases
  • Comparing Impala to Hive
  • Why Hadoop testing is important
  • Unit testing
  • Integration testing
  • Performance testing
  • Diagnostics
  • Nightly QA test
  • Benchmark and end to end tests
  • Functional testing
  • Release certification testing
  • Security testing
  • Scalability Testing
  • Commissioning and Decommissioning of Data Nodes Testing
  • Reliability testing
  • Release testing

  • Understanding the Requirement, preparation of the Testing Estimation, Test Cases, Test Data, Test bed creation, Test Execution, Defect Reporting, Defect Retest, Daily Status report delivery, Test completion.
  • ETL testing at every stage (HDFS, HIVE, HBASE) while loading the input (logs/files/records etc) using sqoop/flume which includes but not limited to data verification, Reconciliation.
  • User Authorization and Authentication testing (Groups, Users, Privileges etc)
  • Report defects to the development team or manager and driving them to closure.
  • Consolidate all the defects and create defect reports.
  • Validating new feature and issues in Core Hadoop.
  • Report defects to the development team or manager and driving them to closure.
  • Consolidate all the defects and create defect reports.
  • Validating new feature and issues in Core Hadoop
  • Responsible for creating a testing Framework called MR Unit for testing of Map-Reduce programs.
  • Automation testing using the OOZIE.
  • Data validation using the query surge tool.
  • Test plan for HDFS upgrade
  • Test automation and result
  • Sealed traits
  • Case classes
  • Constant pattern in case classes
  • Wild card pattern
  • Variable pattern
  • Constructor pattern
  • Tuple pattern
  • Array in scala
  • List in scala
  • Difference between list and list buffer
  • Array buffer
  • Queue in scala
  • Dequeue in scala
  • Mutable queue in scala
  • Stacks in scala
  • Sets and maps in scala
  • Tuples
  • Different import types
  • Selective imports
  • Testing-Assertions
  • Scala test case- scala test fun. Suite
  • Junit test in scala
  • Interface for Junit via Junit 3 suite in scala test
  • SBT
  • Directory structure for packaging scala application

Learn & Get

  • Excel in the concepts of Hadoop Distributed File System (HDFS)
  • Implement HBase and MapReduce Integration
  • Understand Data science Project Life Cycle, Data Acquisition and Data Collection
  • Execute various Machine Learning Algorithms
  • Understand Apache Hadoop2.7 Framework and Architecture
  • Learn to write complex MapReduce programs in both MRv1 and Mrv2
  • Design and develop applications involving large data using Hadoop Ecosystem
  • Understand Prediction and Analysis Segmentation through Clustering
  • Set up Hadoop infrastructure with single and multi-node clusters using Amazon ec2 (CDH4)
  • Monitor a Hadoop cluster and execute routine administration procedures
  • Understand basic distributed concepts and Storm Architecture
  • Learn Hadoop Distributed Computing, Big Data features, Legacy architecture of Real-time System
  • Know the Logic Dynamics, Components and Topology in Storm
  • Learn Scala and its programming implementation
  • Implement Spark on a cluster
  • Understand the difference between Apache Spark and Hadoop
  • Get deep insights into the functioning of Scala

Payment Method

PAYMENT METHODS
You need to pay through PayPal. We accept both Debit and Credit Card for transaction.
SCHOLARSHIPS
We subsidize our fees by 10% for military personnel, and college students with exceptional records. To apply for a scholarship, email info@ipartner.ca
FREQUENTLY ASKED QUESTIONS
In our iPartner self-paced training program, you will receive the training assessments, recorded sessions, course materials, Quizzes, related softwares and assignments. The courses are designed in such a way that you will the get real world exposure; the solid understanding of every concept that allows you to get the most from the online training experience and you will be able to apply the information and skills in the workplace. After the successful completion of your training program, you can take quizzes which enable you to check your level of knowledge and also enables you to clear your relevant certification at higher marks/grade where you will be able to work on the technologies independently.
In Self-paced courses, the learners are able to conduct hands-on exercises and produce learning deliverables entirely on their own at any convenient time without a facilitator whereas in the Online training courses, a facilitator will be available for answering queries at a specific time to be dedicated for learning. During your self-paced learning, you can learn more effectively when you interact with the content that is presented and a great way to facilitate this is through review questions and quizzes that strengthen key concepts. In case if you face any unexpected challenges while learning, we will arrange a live class with our trainer.
All Courses from iPartner are highly interactive to provide good exposure to learners and gives them a real time experience. You can learn only at a time where there are no distractions, which leads to effective learning. The costs of self-paced training are 75% cheaper than the online training. You will offer lifetime access hence you can refer it anytime during your project work or job.
Yes, at the top of the page of course details you can see sample videos.
As soon as you enroll to the course, your LMS (The Learning Management System) Access will be Functional. You will immediately get access to our course content in the form of a complete set of previous class recordings, PPTs, PDFs, assignments and access to our 24*7 support team. You can start learning right away.
24/7 access to video tutorials and Email Support along with online interactive session support with trainer for issue resolving.
Yes, You can pay difference amount between Online training and Self-paced course and you can be enrolled in next online training batch.
Please send an email. You can join our Live chat for instant solution.
We will provide you the links of the software to download which are open source and for proprietary tools, we will provide you the trail version if available.
You will have to work on a training project towards the end of the course. This will help you understand how the different components of courses are related to each other.
Classes are conducted via LIVE Video Streaming, where you get a chance to meet the instructor by speaking, chatting and sharing your screen. You will always have the access to videos and PPT. This would give you a clear insight about how the classes are conducted, quality of instructors and the level of Interaction in the class.
Yes, we do keep launching multiple offers that best suits your needs. Please email us at: info@ipartner.ca and we will get back to you with exciting offers.
We will help you with the issue and doubts regarding the course. You can attempt the quiz again.
Sure! Your feedbacks are greatly appreciated. Please connect with us on the email support - info@ipartner.ca.