Skip to content
BENIBIX Learn

Hadoop Distributed File System (HDFS)

Data⚙️ ProcessingDistributed Processing

Also known as HDFS

Hadoop's file store. It cuts big files up and keeps three copies.

Think of it like

It is like tearing a big book into parts. You keep three copies of each part in three homes.

Example

A 1 GB log file is cut into 8 blocks. Each block sits on three of the cluster's computers.

Why it matters

It made it cheap to keep huge files safe on ordinary computers. Data lakes on cloud storage took its place in most new work.

Definition

HDFS is the file system of Hadoop, which spreads huge files over many computers. It cuts each big file into blocks, 128 MB each by default. It stores each block on three computers by default, so one failed disk loses nothing. A main server, the NameNode, keeps track of where every block lives. It is built for reading big files from start to end, not for small, quick changes.

Common mistake

Storing millions of tiny files. Each one costs the NameNode memory, and it can run out.

Key takeaways

  1. 1
    Big blocks. 128 MB by default.
  2. 2
    Three copies. On different computers.
  3. 3
    NameNode. Knows where each block is.