• Courses

    About Courses

    • Onsite
    • IT & Software
    • Database
    • Programming Language
    • Kids Programming
    Azure Data Factory

    Azure Data Factory

    R699.00 R299.00
    Read More
  • Features
    • Portfolio
    • Forums
    • About Us
  • Events
  • Blog
  • Contact
      • Cart

        0
    Have any question?
    (+27) 81 405 0333
    info@mindqsystems.co.za
    RegisterLogin
    MindQSystemsMindQSystems
    • Courses

      About Courses

      • Onsite
      • IT & Software
      • Database
      • Programming Language
      • Kids Programming
      Azure Data Factory

      Azure Data Factory

      R699.00 R299.00
      Read More
    • Features
      • Portfolio
      • Forums
      • About Us
    • Events
    • Blog
    • Contact
        • Cart

          0

      IT & Software

      • Home
      • All courses
      • IT & Software
      • Hadoop

      Hadoop

      User Avatar
      MindQ
      IT & Software, Online
      (0 review)
      R699.00 R299.00
      019-maintenance
      • Overview
      • Curriculum
      • Reviews

      Hadoop is one of the most popular big data frameworks in use today. Its popularity has actually made it synonymous with big data. Hadoop was one of the first frameworks of its kind which is one of the reasons why its adoption is so widespread.

      Hadoop was created in 2006 by Yahoo. It was based on the Google File System and MapReduce. The company started using Hadoop on a 1000 node cluster the following year. Hadoop would then go on to be released as an open source project to Apache Software Foundation in 2008.

      With Hadoop, it’s possible to store big data in a distributed environment. This allows for the data to be processed parallely. Hadoop has two core components that include the HDFS or Hadoop distributed File System which stores data of various formats across a cluster.

      YARN is the second component and it’s tasked with resource management. It’s the component that handles all of the processing activities by allocating resources and scheduling tasks. It allows parallel processing over the data.

      Advantages

      Scalability

      The ease at which Hadoop can be scaled up is one of its biggest advantages. The framework is based on the principle of horizontal scalability. It allows storage and distribution of big data across many different servers that operate in parallel.

      It’s easy to add nodes to a Hadoop cluster on the fly which speeds up the scale at which the cluster size can grow.

      Open Source

      Hadoop is open source. What that means is that the source code for Hadoop is available for free. Anyone can take the code and modify it to suit their specific requirement without any issues. This is one of the reasons why Hadoop remains such a widely used big data framework.

      Performance

      Hadoop is capable of processing very large amounts of data at incredible speeds, made possible by its distributed processing and storage architecture. The input data files are divided into blocks that are then stored over several nodes.

      The tasks that are submitted by the user are also divided into sub-tasks that are assigned to worker nodes and are run in parallel.

      Disadvantages

      Perturbed by small data

      Hadoop is widely regarded as strictly a big data framework even though there are some other frameworks that work just as well with small data. Hadoop doesn’t run into issues even when handling a small number of very large files but it can’t deal with a large number of small files.

      Any file that’s smaller than Hadoop’s block size, which can either be 128MB or 256B, can overload the Namenode and disrupt the framework’s function.

      Security concerns

      Hadoop has remained one of the most widely used big data frameworks despite the fact that there exist some security concerns.

      Those concerns largely stem from the fact that Hadoop is written in Java which happens to be a very common programming language. It’s relatively easier for cyber criminals to exploit Java vulnerabilities.

      Higher processing overhead

      Hadoop is a batch processing engine at its core. All of the data is read via the disk and written to it as well. This can make the read and write operations quite expensive, particularly when the framework deals with petabytes of data.

      This processing overhead happens because Hadoop is unable to perform in-memory calculations.

      Course Features

      • Lectures 87
      • Quizzes 0
      • Duration 50 hours
      • Skill level All levels
      • Language English
      • Students 0
      • Certificate No
      • Assessments Yes
      CoursesOnlineHadoop
      • Hadoop 14

        • Lecture1.1
          Session 1
        • Lecture1.2
          Session 2
        • Lecture1.3
          Session 3
        • Lecture1.4
          Session 4
        • Lecture1.5
          Session 5
        • Lecture1.6
          Session 6
        • Lecture1.7
          Session 7
        • Lecture1.8
          Session 8
        • Lecture1.9
          Session 9
        • Lecture1.10
          Session 10
        • Lecture1.11
          Session 11
        • Lecture1.12
          Session 12
        • Lecture1.13
          Session 13
        • Lecture1.14
          Session 14
      • HIVE 15

        • Lecture2.1
          Session 1
        • Lecture2.2
          Session 2
        • Lecture2.3
          Session 3
        • Lecture2.4
          Session 4
        • Lecture2.5
          Session 5
        • Lecture2.6
          Session 6
        • Lecture2.7
          Session 7
        • Lecture2.8
          Session 8
        • Lecture2.9
          Session 9
        • Lecture2.10
          Session 10
        • Lecture2.11
          Session 11
        • Lecture2.12
          Session 12
        • Lecture2.13
          Session 13
        • Lecture2.14
          Session 14
        • Lecture2.15
          Session 15
      • Map Reduce 19

        • Lecture3.1
          Session 1
        • Lecture3.2
          Session 2
        • Lecture3.3
          Session 3
        • Lecture3.4
          Session 4
        • Lecture3.5
          Session 5
        • Lecture3.6
          Session 6
        • Lecture3.7
          Session 7
        • Lecture3.8
          Session 8
        • Lecture3.9
          Session 9
        • Lecture3.10
          Session 10
        • Lecture3.11
          Session 11
        • Lecture3.12
          Session 12
        • Lecture3.13
          Session 13
        • Lecture3.14
          Session 14
        • Lecture3.15
          Session 15
        • Lecture3.16
          Session 16
        • Lecture3.17
          Session 17
        • Lecture3.18
          Session 18
        • Lecture3.19
          Session 19
      • Apache Pig 7

        • Lecture4.1
          Session 2
        • Lecture4.2
          Session 1
        • Lecture4.3
          Session 3
        • Lecture4.4
          Session 4
        • Lecture4.5
          Session 5
        • Lecture4.6
          Session 6
        • Lecture4.7
          Session 7
      • Scala 26

        • Lecture5.1
          Session 1
        • Lecture5.2
          Session 2
        • Lecture5.3
          Session 3
        • Lecture5.4
          Session 4
        • Lecture5.5
          Session 5
        • Lecture5.6
          Session 6
        • Lecture5.7
          Session 7
        • Lecture5.8
          Session 8
        • Lecture5.9
          Session 9
        • Lecture5.10
          Session 10
        • Lecture5.11
          Session 11
        • Lecture5.12
          Session 12
        • Lecture5.13
          Session 13
        • Lecture5.14
          Session 14
        • Lecture5.15
          Session 15
        • Lecture5.16
          Session 16
        • Lecture5.17
          Session 17
        • Lecture5.18
          Session 18
        • Lecture5.19
          Session 19
        • Lecture5.20
          Session 20
        • Lecture5.21
          Session 21
        • Lecture5.22
          Session 22
        • Lecture5.23
          Session 23
        • Lecture5.24
          Session 24
        • Lecture5.25
          Session 25
        • Lecture5.26
          Session 26
      • SQOOP 6

        • Lecture6.1
          Session 1
        • Lecture6.2
          Session 2
        • Lecture6.3
          Session 3
        • Lecture6.4
          Session 4
        • Lecture6.5
          Session 5
        • Lecture6.6
          Session 6

      Reviews

      Average Rating

      0
      0 rating

      Detailed Rating

      5
      0%
      4
      0%
      3
      0%
      2
      0%
      1
      0%
      • Overview
      • Curriculum
      • Reviews
      R699.00 R299.00
      • Share:

      You May Like

      Azure Data Factory Read More
      MindQ

      Azure Data Factory

      0
      0
      R699.00 R299.00
      Powershell Read More
      MindQ

      Powershell

      0
      0
      R699.00 R299.00
      Salesforce Read More
      MindQ

      Salesforce

      0
      0
      R699.00 R299.02
      Oracle 11g Read More
      MindQ

      Oracle 11g

      0
      0
      R699.00 R299.00
      SAS Read More
      MindQ

      SAS

      0
      0
      R699.00 R299.00

      Leave A Reply Cancel reply

      Your email address will not be published. Required fields are marked *

      All Courses

      • Database
      • Frontend
      • IT & Software
      • Kids Programming
      • Live Tutorial
      • Online
      • Onsite
      • Programming Language
      • Technology

      Latest Courses

      Azure Data Factory

      Azure Data Factory

      R699.00 R299.00
      Powershell

      Powershell

      R699.00 R299.00
      Salesforce

      Salesforce

      R699.00 R299.02

      (+27) 81 405 0333

      info@mindqsystems.co.za

      Company

      • About Us
      • Blog
      • Contact

      Links

      • Courses
      • Events
      • Gallery
      • FAQs

      Support

      • Contact Us
      • Forums

      Copyright 2020 MindQ Systems South Africa

      • Privacy
      • Terms

      Login with your site account

      Lost your password?

      Not a member yet? Register now

      Register a new account

      Are you a member? Login now

      Modal title

      Message modal