In March 2006, Amazon launched a service that quietly broke a decades-old assumption about how computers should store files. Instead of folders inside folders inside folders, Simple Storage Service (S3) threw everything into one giant flat space and handed each item a unique key. Today, that model holds exabytes of photos, backups, logs, and machine-learning data across the internet, and it powers a huge share of the web’s static content.
The idea is simple, but the details trip people up. This guide walks through what object storage actually is, how it differs from the file and block systems most people grew up with, and where it shines and where it does not.
What object storage actually is
Object storage keeps data as discrete objects inside a flat namespace called a bucket (or container). Each object bundles three things: the data itself, rich metadata (custom tags, content type, timestamps, access rules), and a unique identifier called a key. There are no directories. A key like `photos/2024/trip/beach.jpg` looks like a path, but the slashes are just characters in a name, not real folders.
That flat structure is the whole point. Because there is no hierarchy to traverse, the system can spread objects across thousands of servers and find any one of them by key alone. Scale becomes a matter of adding hardware, not redesigning the layout.
Access happens over HTTP using standard verbs. You PUT an object to create it, GET it to read it, and DELETE it to remove it. This REST-style interface, popularized by S3, became the de facto standard. Rival clouds and open-source systems like MinIO and Ceph all speak “S3-compatible” APIs, which means code written for one provider often runs on another with a small config change. If you want the broader picture of how remote storage fits into everyday computing, a primer on what cloud storage is and how it works is a useful companion read.
How object storage differs from file and block storage
The fastest way to understand object storage is to compare it with the two older models. File storage organizes data in a hierarchy of folders and is what you see on a laptop or a shared network drive. Block storage chops data into fixed-size blocks and presents them as a raw disk, which databases and virtual machines love. Object storage sits alongside both, optimized for massive volumes of unstructured data.
| Feature | Object storage | File storage | Block storage |
|---|---|---|---|
| Structure | Flat namespace with keys | Folder hierarchy | Fixed-size blocks |
| Metadata | Rich and custom per object | Limited file attributes | Minimal |
| Access method | HTTP/REST API | SMB/NFS protocols | iSCSI, Fibre Channel |
| Scalability | Near-limitless | Limited by tree depth | Limited by volume size |
| Best for | Backups, media, data lakes | Shared documents | Databases, VMs |
| Editing in place | No, rewrite the whole object | Yes | Yes |
That last row matters more than people expect. You cannot open an object, change one byte, and save it. You rewrite the entire object. For a 50 MB photo that is fine. For a 500 GB video file, it is painful unless the tool supports multipart uploads, which split the file into chunks and stitch them together server-side.
Why the model scales so well
Two engineering choices explain the scale. The first is erasure coding, a technique that stores data as mathematical fragments spread across many disks. Instead of keeping three full copies of every object (triple replication), a system might store 12 fragments where any 8 can rebuild the data. That cuts storage overhead dramatically while keeping durability high. Large-scale systems have documented this approach publicly, and it is now standard practice.
The second is eventual consistency giving way to strong consistency. Early object stores sometimes returned stale data right after an update because replicas had not synced yet. In December 2020, AWS announced strong read-after-write consistency for all S3 operations, meaning a completed write is immediately visible to every subsequent read. That change removed a long-standing headache for developers building on top of the service.
Durability claims are worth reading carefully. Providers advertise figures like 99.999999999% durability, often called “11 nines,” which describes the probability that an object survives a given year without loss. These are vendor claims, not independently audited numbers, and they describe durability, not availability. A service can keep your data perfectly safe while being temporarily unreachable. Availability SLAs are separate and usually lower, often in the 99.9% to 99.99% range.
What object storage is used for
The workload list is long, but a few patterns dominate:
- Backup and archive. Cheap cold tiers store years of snapshots at a fraction of the cost of primary disks.
- Media assets. Photos, video, and audio for websites and apps live in buckets and get served through content delivery networks.
- Data lakes. Analysts dump raw logs, clickstreams, and sensor data into buckets, then query them with tools that read directly from storage.
- Static website hosting. HTML, CSS, and images can be served straight from a bucket without a web server.
- Machine learning. Training sets with millions of images or audio clips are stored as objects and streamed to GPUs during training.
The pattern behind all five is the same: write once, read many times, rarely edit. Object storage is superb at that. It is mediocre at random in-place edits and poor at low-latency transactional work. If you are running a relational database, keep it on block storage.
Practical tips and common mistakes
Having watched teams migrate to object storage, a few lessons come up again and again.
Do not treat buckets like folders. People build elaborate key prefixes that mimic a directory tree, then wonder why listing operations are slow. Prefixes help with organization and access policies, but they are not directories, and listing millions of keys under one prefix can be expensive.
Set lifecycle rules on day one. Objects accumulate silently. A rule that moves anything older than 90 days to a colder tier and deletes anything older than seven years can cut bills by half or more. Teams that skip this step often discover a five-figure monthly bill six months later.
Watch egress costs. Uploading is usually cheap or free. Downloading is where providers make money. A design that pulls the same object repeatedly from a bucket, instead of caching it at the edge, will burn budget fast.
Version everything important. Most providers offer object versioning, which keeps prior copies when an object is overwritten or deleted. It is the closest thing to an undo button in a system that otherwise has none.
Encrypt by default. Server-side encryption is now a checkbox on nearly every platform. Turn it on, and use customer-managed keys if compliance requires it.
Test restores, not backups alone. A backup you have never restored is a hope, not a plan. Run a restore drill quarterly.
For anyone weighing where to keep large media libraries, the tradeoffs between remote and on-device storage are worth studying in detail; this breakdown of cloud storage versus local storage key differences covers cost, speed, and access patterns side by side.
The standards and environment
Standards bodies have tried to formalize cloud storage interfaces. The Storage Networking Industry Association publishes the Cloud Data Management Interface (CDMI), a specification for managing data in cloud storage systems. The ISO/IEC 19086 series addresses cloud service-level agreements. In practice, though, the S3 API won the interoperability battle by default, and “S3-compatible” is the phrase buyers look for.
Open-source options keep the market honest. MinIO offers a lightweight S3-compatible server that runs on a single machine or a cluster. Ceph’s RADOS Gateway provides an S3-compatible front end on top of a distributed storage cluster. Both let organizations keep data on-premises while using tooling written for the cloud.
Terminology also drifts. Azure calls its service Blob Storage, which is the same model under a different name. “Blob,” “object,” and “cloud storage” get used interchangeably in marketing, which confuses buyers comparing quotes. When evaluating providers, compare the actual guarantees: durability, availability, request pricing, egress pricing, and consistency model. The label on the box tells you very little.
Cost comparisons deserve the same scrutiny. Storage pricing per gigabyte looks simple, but request charges, retrieval fees for cold tiers, and minimum storage durations can triple the effective price. A quick scan of current storage unit deals and local price comparisons is a reminder that physical storage pricing varies just as widely, and the same discipline applies: read the fine print, not the headline number.
Where object storage falls short
Honesty matters here. Object storage is not a universal replacement for every disk. It struggles with: (Amazon S3)
- Databases. Relational engines need block-level access and low-latency random writes.
- Frequent small edits. Rewriting a whole object to change one field is wasteful.
- POSIX file semantics. Applications that expect file locks, renaming, and directory operations will break or need a compatibility layer.
- Very low latency. A local SSD responds in microseconds; an object store round trip is measured in milliseconds.
Hybrid setups often make sense. Keep hot transactional data on block storage, push analytical and archival data to object storage, and use a caching layer for anything read repeatedly.
Conclusion
Object storage trades hierarchy for scale, and that trade has paid off enormously since S3 arrived in 2006. The flat namespace, rich metadata, and HTTP interface make it the default home for backups, media, data lakes, and training sets. It is not a drop-in replacement for every workload, and the pricing and durability claims deserve careful reading. But for write-once, read-many data at volume, nothing else comes close. Understand the model, set lifecycle rules early, watch egress costs, and it will quietly do its job for years.
Frequently asked questions
What is object storage in simple terms?
It is a way of storing data as individual items, called objects, in one flat space instead of nested folders. Each object carries its own data, descriptive metadata, and a unique key, and you retrieve it over the internet using that key.
Is object storage the same as cloud storage?
No. Cloud storage is a delivery model, meaning storage you rent over the internet. Object storage is an architecture, meaning how data is organized. Most cloud storage services use object storage underneath, but cloud storage can also offer file and block options.
Can I run a database on object storage?
Generally no, at least not a traditional relational database. Databases need block-level access and fast random writes, which object storage does not provide well. Some newer analytical engines read directly from object storage, but the database engine itself still runs on compute with block storage attached.
What does “11 nines of durability” mean?
It is a vendor claim that an object has a 99.999999999% chance of surviving a given year without loss. In practice it describes how the provider replicates or erasure-codes data across many devices and locations. It says nothing about uptime, which is covered separately by availability SLAs.
Why can’t I edit part of an object?
Because objects are immutable by design. The system identifies each object by its key and treats the whole thing as one unit. To change anything, you upload a new version that replaces the old one. Some tools simulate partial edits by splitting large files into chunks, but the underlying rule still applies.
How do I keep object storage costs under control?
Three habits do most of the work: set lifecycle rules to move or delete old data automatically, cache frequently read objects at the edge to avoid egress charges, and pick the right storage tier for each dataset. Review the bill monthly, because request and retrieval fees often outweigh the per-gigabyte price.
