[HDDS-3816] Erasure Coding in Ozone (draft)

Authors: Uma Maheswara Rao Gangumalla, Marton Elek, Stephen O'Donnell
2020-06-30

 

Summary
Use Erasure Coding algorithm for efficient storage

Abstract

Support Erasure Coding for read and write pipeline of Ozone.

Status

The design doc describes two main methods to implement EC:

  • Container level, async Erasure Coding, to encode closed containers in the background
  • Block level, striped Erasure Coding

Second option can work only with new, dedicated write-path. Details of possible implementation will be included in the next version.

Link

https://issues.apache.org/jira/secure/attachment/13006245/Erasure%20Coding%20in%20Apache%20Hadoop%20Ozone.pdf