There is a specific failure mode in S3 cost work: you enable the right setting, the console confirms it, and the bill does not move. Not because the setting is broken, but because it only governs objects that arrive after you flip it. The petabyte already sitting in the bucket carries on exactly as before.
Two of these are worth knowing in detail, because both have a second-order consequence that is worse than the original problem.
S3 Bucket Keys apply to new uploads only
With SSE-KMS encryption and no bucket key, S3 calls AWS KMS on every single request against an encrypted object. On a read-heavy bucket that is a KMS request charge per read, forever.
S3 Bucket Keys fix that, and the saving is usually large. But enabling a bucket key applies to new uploads only. Objects already in the bucket do not use it.
So the common sequence is: enable Bucket Keys on a bucket holding years of data, see the KMS line barely move, and conclude the feature does not work. It works. It just never applied to the 40 TB that was already there.
The fix, and the thing it breaks
Existing objects adopt the bucket key only when rewritten, via a CopyObject operation with the same object as both source and target. Copy each object over itself and it picks up the new setting.
Which introduces a problem most people find afterwards:
Copying an object resets its creation date, and S3 lifecycle Transition and Expiration Days are counted from object creation.
Rewriting the bucket restarts the lifecycle clock on every object you touch. Objects that were 300 days into a 365-day expiry are now at day zero. Objects due to transition to Glacier next month will now wait the full period again.
For a bucket with an aggressive expiry policy, that can cost more in retained storage than the KMS calls you were trying to eliminate. The right move is to work out the lifecycle impact per bucket before the rewrite, not after, and to consider whether it is worth doing at all on buckets whose objects will age out soon anyway.
Lifecycle rules written before September 2024 are grandfathered
This is the more consequential one, because the trigger is unrelated to the thing that changes.
Starting September 2024, the default S3 Lifecycle behaviour prevents objects smaller than 128 KB from being transitioned to any storage class. Before that date, the default let sub-128 KB objects through, but only into Glacier Flexible Retrieval and Glacier Deep Archive.
Configurations created before September 2024 retain the previous transition behaviour unless modified. So if your lifecycle rules predate that change and nobody has touched them, they are still running the old behaviour today, quietly archiving small objects that a newly-written rule would leave in Standard.
Any edit switches the whole configuration
Here is the trigger, in AWS's own framing: if you create, edit, or delete rules, the default transition behaviour for your configuration changes to the updated behaviour.
Not the rule you edited. The configuration. Add one unrelated expiry rule for a new prefix, and every existing rule in that bucket moves to the 128 KB floor with it. Delete a rule you no longer need, same result.
There is no warning in the console, no diff showing the behavioural change, and the rules themselves look identical afterwards. It is one of the cleanest examples I know of a change whose blast radius is much larger than the edit that caused it.
Why the floor is at 128 KB
AWS's stated reason is that a transition request is charged for each object, so for smaller objects the transition costs can outweigh the storage savings. That is true, and it is a sensible default. But it is worth understanding the actual numbers, because the floor is set far above where the storage maths flips.
There are two separate overheads:
- Glacier Flexible Retrieval and Deep Archive add 40 KB of metadata per archived object: 32 KB charged at the destination Glacier rate, and 8 KB charged at the S3 Standard rate. That 8 KB at Standard pricing is the expensive part and it is easy to miss.
- Glacier Instant Retrieval works differently. No 40 KB overhead, but a 128 KB minimum billable object size. Flexible Retrieval and Deep Archive are listed as NA in that column.
The storage-only break-even is around 10 KB
Working from published us-east-1 list rates per GB-month:
- S3 Standard, first 50 TB tier: $0.023
- Glacier Flexible Retrieval: $0.0036
- Glacier Deep Archive: $0.00099
A caveat on those three numbers, carried from my source notes: AWS's own pricing tables did not render across three separate fetch attempts, so these rates rest on secondary corroboration from three independent sources that agree exactly. Re-read the AWS pricing page directly before reusing them for anything that matters.
On that basis, and accounting for the 40 KB overhead, the storage-only break-even is 9.80 KB for Deep Archive and 15.42 KB for Flexible Retrieval. That is my arithmetic on the rates above, computed at KB = 1024 and GB = 1024^3, not a measured saving and not a figure AWS publishes.
Which puts the 128 KB floor roughly 8.3x to 13.1x above the point where archiving starts winning on storage alone. The gap is not an error on AWS's part. The transition request is a one-off per-object charge that the storage comparison ignores, and AWS is setting a default that is safe across all usage patterns. But the consequence is a real band, roughly 10 KB to 128 KB, where the old behaviour was saving you money and the new default will not.
To put a number on that band: 50 million objects at 64 KB each cost about $70.19/month in S3 Standard versus about $13.31/month in Deep Archive. That Deep Archive figure breaks down as $3.02 for the data, $1.51 for the 32 KB overhead at the Glacier rate, and $8.77 for the 8 KB overhead at the Standard rate. Again, my arithmetic, same caveat on the underlying rates.
Note what that breakdown shows: two-thirds of the Deep Archive cost is metadata overhead, not data. At 64 KB per object you are mostly paying to keep the index.
The minimum duration charge nobody budgets for
One more thing before you decide to archive a small-object bucket. Minimum storage duration is 180 days for Glacier Deep Archive and 90 days for Glacier Flexible Retrieval. Deleting or overwriting an object earlier is still charged for the remainder of that duration.
So a pipeline that archives objects and then deletes them a week later pays the full 180 days anyway. Archiving is not a cheap place to put things you are about to remove, and combining a Glacier transition with a short expiry rule in the same lifecycle configuration is usually a mistake.
How to check which behaviour you are actually on
You cannot tell from looking at the rules, and you cannot tell from the console. But it is directly queryable.
Call GetBucketLifecycleConfiguration and read the x-amz-transition-default-minimum-object-size response header:
varies_by_storage_classmeans the pre-September-2024 default. You are grandfathered, and one edit will end that.all_storage_classes_128Kmeans the current default. Already switched.
There is a related gap worth knowing: the same operation returns no LastModified or modification timestamp, on the configuration or on any individual rule. So there is no API answer to "when did someone last touch this bucket's lifecycle rules". The header tells you which side of the change you are on, and that is the only signal available.
What to do with this
- Before enabling Bucket Keys on an existing bucket, decide whether you will rewrite the existing objects, and model what resetting every creation date does to your lifecycle rules first.
- Audit the transition header across buckets. Anything returning
varies_by_storage_classis running behaviour that one unrelated edit will change. - For grandfathered buckets, find out where your object sizes actually sit. If the bulk is above 128 KB, the change is irrelevant and you can edit freely. If a lot sits in the 10 KB to 128 KB band, an edit has a real cost and should be planned rather than done in passing.
- Check minimum duration against your expiry rules. Archiving anything you plan to delete within 180 days is usually worse than leaving it in Standard.
The general lesson underneath both of these: in S3, a setting is a contract with future writes, not a statement about the bucket. Anything already stored keeps the terms it arrived under until something rewrites it, and the rewrite has its own price.
Sources
AWS statements are from AWS documentation. Break-even figures and monthly totals are my arithmetic, labelled as such above, and rest on per-GB rates that carry the corroboration caveat noted in the text.
- Reducing the cost of SSE-KMS with Amazon S3 Bucket Keys
- Configuring an S3 Bucket Key at the object level
- S3 Lifecycle configuration elements
- Transitioning objects: general considerations (128 KB default)
- Working with archived objects (40 KB overhead, minimum durations)
- Understanding and managing S3 storage classes
- GetBucketLifecycleConfiguration API reference
- Amazon S3 pricing (verify rates here directly)