Insight Scality: Cloud Storage Technology

[ 0 ] Juli 15, 2011 |

Behind Scality’s management interface is a sophisticated key/value store with very rich metadata-handling capabilities.

Scality is built on three key concepts

    RING — the core of Scality technology is Scality’s patented RING technology. RING intelligently manages the What?, When?, Where?, Why?, and How?, of storage and retrieval.

    • Inspired by peer-to-peer technology, RING is hardened to provide carrier-grade service availability and data reliability.
    • It is composed of a number of nodes, typically off-the-shelf servers.
    • Each node on the RING is responsible for its own piece of the overall storage puzzle.
    • Every node is functionally equivalent. A node can disappear or you can add a new node and the system will rebalance automatically without intervention.
    • There is no need for a master database or directory.
    • Every node constantly monitors a limited number of its peers, and automatically rebalances replication and load to make the system completely self-healing

    Connectors — an connector is the piece of software that communicates with your application. Scality supports multiple accessors— HTTP REST, RS2, Zimbra, Open-Xchange, Dovecot and more—and you can develop your own purpose-built accessor with Scality’s API tools.

    BizIOD — an IO daemon developed by Scality that communicates directly with specific storage hardware. BizIOD supports most existing storage hardware technologies, either directly attached – SAS, SCSI, SATA, or network attached – iSCSI, Fiber Channel, NFS v3 and NFS v4, Dispersed Storage and more.

How Scality works

Any store or retrieve request for an object goes through an “accessor”. The accessor is close to your application, and it decides to how to handle the request. For example, depending on specific data attributes, the accessor may assign a storage class—the data needs to survive one, two or more disk failures. The accessor makes intelligent decisions about how it handles requests for larger files, such as a movie, and smaller objects like an email. It may decide to stream the former, for example.

Accessors contact one of the nodes on the RING—it doesn’t matter which node because that node will recommend the node to contact next. The protocol is such that the algorithm will converge in one hop for a ring of 10 nodes, two hops for a RING of 100 nodes, and three hops for 1000 nodes. Each node can typically handle 10-to-50 TB of storage. Within three hops, one node can address a massive 50 petabyte of storage within ≤ 20 ms latency on a typical gigabit LAN.

Once the correct node has been reached, the object will be passed to BizIOD, which places it on the storage hardware. The storage hardware can be memory (for caching), solid-state disk (for very high throughput); it can be directly-attached storage or network-attached storage. Several BizIOD can co-exist on the same RING node.

  • The RING nodes also manage which objects should be off-loaded to Tier Two storage, according to a sophisticated algorithm that takes into account read/write patterns for that object, as well as information about the object type.
  • RING implements a proven compression/de-duplication algorithm when data is pushed to Tier Two. Data is still accessible through the accessor, which means the process is completely transparent for the application.
  • The RING system can be set to never delete information and supports undelete. Typically, to add throughput (IOPS) you simply add Tier One RING nodes.
  • To add capacity you add Tier Two storage. Tier Two storage can be implemented with another RING sub-system with a different hardware configuration, a NAS-based system, or with technologies such as Dispersed Storage which deliver sub 1,000 USD/TB price points with very high reliability.

Additional functionality

  • RING — RING manages the number of replicas requested for each object. There is no concept of master, every replica is functionally equivalent. When a replica disappears, the RING system automatically detects it and recreates it without external intervention
  • Accessor — when there are multiple copies of a given object, the accessor automatically load balances, reading across all replicas and reconciling versions transparently
  • BizIOD — there is one BizIOD running per physical piece of storage hardware. If a given disk is causing problems, it only impacts that BizIOD daemon, not the entire RING node

Source: http://www.scality.com

Category: Services

René Büst

About the Author ()

Rene Buest is Senior Analyst and Cloud Practice Lead at Crisp Research, covering cloud computing, IT infrastructure, open source and Internet of Things. Prior to that he was Principal Analyst at New Age Disruption and member of the worldwide Gigaom Research Analyst Network. Rene Buest is top cloud computing blogger in Germany and one of the worldwide top 50 bloggers in this area. In addition, he is one of the world’s top cloud computing influencers and belongs to the top 100 cloud computing experts on Twitter and Google+. Since the mid-90s he is focused on the strategic use of information technology in businesses and the IT impact on our society as well as disruptive technologies. Rene Buest is the author of numerous professional cloud computing and technology articles. He regularly writes for well-known IT publications like Computerwoche, CIO Magazin, LANline as well as Silicon.de and is cited in German and international media – including New York Times, Forbes Magazin, Handelsblatt, Frankfurter Allgemeine Zeitung, Wirtschaftswoche, Computerwoche, CIO, Manager Magazin and Harvard Business Manager. Furthermore Rene Buest is speaker and participant of experts rounds. He is founder of CloudUser.de and writes about cloud computing, IT infrastructure, technologies, management and strategies. He holds a diploma in computer engineering from the Hochschule Bremen (Dipl.-Informatiker (FH)) as well as a M.Sc. in IT-Management and Information Systems from the FHDW Paderborn.

Leave a Reply

You must be logged in to post a comment.