S3 Optimization: Storage Classes and Lifecycle Rules

Overview

Amazon S3 is one of the most widely used cloud services in existence, but most teams are paying far more than they should because they dump everything into S3 Standard and never think about it again. By understanding storage classes, lifecycle policies, and SDK-level optimizations, you can cut your S3 bill by 60% or more while improving application performance. This article covers the full spectrum of S3 optimization techniques with practical Node.js implementations you can deploy today.

Prerequisites

• An AWS account with S3 access
• Node.js v16+ installed
• AWS SDK v3 for JavaScript (@aws-sdk/client-s3)
• Basic familiarity with S3 concepts (buckets, objects, keys)
• IAM credentials with s3:* permissions for the target bucket

Install the required packages:

npm install @aws-sdk/client-s3 @aws-sdk/lib-storage @aws-sdk/s3-request-presigner

S3 Storage Classes: Know What You Are Paying For

S3 offers seven storage classes, and picking the right one is the single biggest lever you have for cost optimization. Here is the breakdown as of 2026, using US East (N. Virginia) pricing:

Storage Class	$/GB/Month	Min Duration	Retrieval Fee	Use Case
S3 Standard	$0.023	None	None	Frequently accessed data
S3 Intelligent-Tiering	$0.023+	None	None	Unpredictable access patterns
S3 Standard-IA	$0.0125	30 days	$0.01/GB	Infrequent but needs fast access
S3 One Zone-IA	$0.01	30 days	$0.01/GB	Reproducible infrequent data
S3 Glacier Instant Retrieval	$0.004	90 days	$0.03/GB	Archive with millisecond access
S3 Glacier Flexible Retrieval	$0.0036	90 days	$0.01/GB	Archive, minutes to hours retrieval
S3 Glacier Deep Archive	$0.00099	180 days	$0.02/GB	Long-term archive, 12-hour retrieval

The math is straightforward. If you have 10 TB of log files sitting in S3 Standard that nobody touches after the first week, you are spending $230/month when you could be spending $10/month in Glacier Deep Archive. That is $2,640/year in waste per 10 TB.

Storing Objects with a Specific Storage Class

var { S3Client, PutObjectCommand } = require("@aws-sdk/client-s3");

var s3 = new S3Client({ region: "us-east-1" });

function uploadWithStorageClass(bucket, key, body, storageClass) {
  var command = new PutObjectCommand({
    Bucket: bucket,
    Key: key,
    Body: body,
    StorageClass: storageClass
  });

  return s3.send(command);
}

// Upload frequently accessed config
uploadWithStorageClass("my-app-bucket", "config/app.json", configData, "STANDARD")
  .then(function(result) {
    console.log("Config uploaded to Standard:", result.$metadata.httpStatusCode);
  });

// Upload monthly reports to Infrequent Access
uploadWithStorageClass("my-app-bucket", "reports/2026-01.pdf", reportData, "STANDARD_IA")
  .then(function(result) {
    console.log("Report uploaded to Standard-IA:", result.$metadata.httpStatusCode);
  });

// Upload compliance archives to Glacier
uploadWithStorageClass("my-app-bucket", "archives/audit-2025.tar.gz", archiveData, "GLACIER")
  .then(function(result) {
    console.log("Archive uploaded to Glacier:", result.$metadata.httpStatusCode);
  });

Lifecycle Policies: Automate Your Storage Tiering

Manual storage class management does not scale. Lifecycle policies let you define rules that automatically transition objects between storage classes and delete expired objects. This is where the real savings come from.

Designing a Lifecycle Strategy

A typical web application produces several categories of data with different access patterns:

1. User uploads - Hot for the first 30 days, then rarely accessed
2. Application logs - Analyzed within the first week, kept for compliance for 1 year
3. Database backups - Accessed only during disaster recovery
4. Temporary files - Should be deleted after 24 hours

Creating Lifecycle Rules with Node.js

var { S3Client, PutBucketLifecycleConfigurationCommand } = require("@aws-sdk/client-s3");

var s3 = new S3Client({ region: "us-east-1" });

function createLifecycleRules(bucket) {
  var command = new PutBucketLifecycleConfigurationCommand({
    Bucket: bucket,
    LifecycleConfiguration: {
      Rules: [
        {
          ID: "TransitionUserUploads",
          Status: "Enabled",
          Filter: { Prefix: "uploads/" },
          Transitions: [
            {
              Days: 30,
              StorageClass: "STANDARD_IA"
            },
            {
              Days: 90,
              StorageClass: "GLACIER_IR"
            },
            {
              Days: 365,
              StorageClass: "DEEP_ARCHIVE"
            }
          ]
        },
        {
          ID: "ManageApplicationLogs",
          Status: "Enabled",
          Filter: { Prefix: "logs/" },
          Transitions: [
            {
              Days: 7,
              StorageClass: "STANDARD_IA"
            },
            {
              Days: 30,
              StorageClass: "GLACIER"
            }
          ],
          Expiration: {
            Days: 365
          }
        },
        {
          ID: "CleanupTempFiles",
          Status: "Enabled",
          Filter: { Prefix: "tmp/" },
          Expiration: {
            Days: 1
          }
        },
        {
          ID: "CleanupIncompleteMultipartUploads",
          Status: "Enabled",
          Filter: { Prefix: "" },
          AbortIncompleteMultipartUpload: {
            DaysAfterInitiation: 7
          }
        }
      ]
    }
  });

  return s3.send(command);
}

createLifecycleRules("my-app-bucket")
  .then(function(result) {
    console.log("Lifecycle rules created successfully");
    console.log("HTTP Status:", result.$metadata.httpStatusCode);
  })
  .catch(function(err) {
    console.error("Failed to create lifecycle rules:", err.message);
  });

Output:

Lifecycle rules created successfully
HTTP Status: 200

That last rule about incomplete multipart uploads is critical. I have seen buckets accumulate gigabytes of abandoned multipart upload fragments that nobody knows about. They cost money and do nothing. Always include that cleanup rule.

Reading Back Lifecycle Configuration

var { GetBucketLifecycleConfigurationCommand } = require("@aws-sdk/client-s3");

function getLifecycleRules(bucket) {
  var command = new GetBucketLifecycleConfigurationCommand({
    Bucket: bucket
  });

  return s3.send(command).then(function(result) {
    result.Rules.forEach(function(rule) {
      console.log("Rule:", rule.ID, "| Status:", rule.Status);
      if (rule.Transitions) {
        rule.Transitions.forEach(function(t) {
          console.log("  Transition after", t.Days, "days to", t.StorageClass);
        });
      }
      if (rule.Expiration) {
        console.log("  Expires after", rule.Expiration.Days, "days");
      }
    });
    return result;
  });
}

Output:

Rule: TransitionUserUploads | Status: Enabled
  Transition after 30 days to STANDARD_IA
  Transition after 90 days to GLACIER_IR
  Transition after 365 days to DEEP_ARCHIVE
Rule: ManageApplicationLogs | Status: Enabled
  Transition after 7 days to STANDARD_IA
  Transition after 30 days to GLACIER
  Expires after 365 days
Rule: CleanupTempFiles | Status: Enabled
  Expires after 1 days
Rule: CleanupIncompleteMultipartUploads | Status: Enabled

Multipart Uploads for Large Files

Any file over 100 MB should use multipart upload. It is not optional. Single-part uploads for large files are unreliable, unresumable, and slow. The AWS SDK v3 provides the @aws-sdk/lib-storage package that handles multipart uploads with automatic retry and progress tracking.

var { S3Client } = require("@aws-sdk/client-s3");
var { Upload } = require("@aws-sdk/lib-storage");
var fs = require("fs");
var path = require("path");

var s3 = new S3Client({ region: "us-east-1" });

function multipartUpload(bucket, key, filePath, storageClass) {
  var fileStream = fs.createReadStream(filePath);
  var fileSize = fs.statSync(filePath).size;

  console.log("Starting multipart upload for", path.basename(filePath));
  console.log("File size:", (fileSize / (1024 * 1024)).toFixed(2), "MB");

  var upload = new Upload({
    client: s3,
    params: {
      Bucket: bucket,
      Key: key,
      Body: fileStream,
      StorageClass: storageClass || "STANDARD",
      ContentType: getContentType(filePath)
    },
    queueSize: 4,           // concurrent part uploads
    partSize: 10 * 1024 * 1024, // 10 MB parts
    leavePartsOnError: false    // clean up on failure
  });

  upload.on("httpUploadProgress", function(progress) {
    var percent = ((progress.loaded / fileSize) * 100).toFixed(1);
    process.stdout.write("\rProgress: " + percent + "% (" +
      (progress.loaded / (1024 * 1024)).toFixed(2) + " MB / " +
      (fileSize / (1024 * 1024)).toFixed(2) + " MB)");
  });

  return upload.done().then(function(result) {
    console.log("\nUpload complete!");
    console.log("Location:", result.Location);
    console.log("ETag:", result.ETag);
    return result;
  });
}

function getContentType(filePath) {
  var ext = path.extname(filePath).toLowerCase();
  var types = {
    ".json": "application/json",
    ".csv": "text/csv",
    ".gz": "application/gzip",
    ".tar": "application/x-tar",
    ".zip": "application/zip",
    ".pdf": "application/pdf",
    ".png": "image/png",
    ".jpg": "image/jpeg"
  };
  return types[ext] || "application/octet-stream";
}

// Upload a 500 MB database backup to Glacier
multipartUpload(
  "my-app-bucket",
  "backups/db-2026-02-13.tar.gz",
  "/tmp/db-backup.tar.gz",
  "GLACIER"
).catch(function(err) {
  console.error("Upload failed:", err.message);
});

Output:

Starting multipart upload for db-backup.tar.gz
File size: 512.00 MB
Progress: 100.0% (512.00 MB / 512.00 MB)
Upload complete!
Location: https://my-app-bucket.s3.us-east-1.amazonaws.com/backups/db-2026-02-13.tar.gz
ETag: "a1b2c3d4e5f6-52"

The -52 at the end of the ETag means S3 combined 52 parts. That is your confirmation that multipart upload was used.

Transfer Acceleration

S3 Transfer Acceleration uses CloudFront edge locations to speed up uploads from geographically distant clients. It is useful when your users are uploading from different regions around the world.

var { S3Client, PutBucketAccelerationConfigurationCommand } = require("@aws-sdk/client-s3");

// Enable acceleration on the bucket
function enableTransferAcceleration(bucket) {
  var s3 = new S3Client({ region: "us-east-1" });

  var command = new PutBucketAccelerationConfigurationCommand({
    Bucket: bucket,
    AccelerateConfiguration: {
      Status: "Enabled"
    }
  });

  return s3.send(command);
}

// Use the accelerated endpoint for uploads
function createAcceleratedClient() {
  return new S3Client({
    region: "us-east-1",
    useAccelerateEndpoint: true
  });
}

// Upload using the accelerated endpoint
function acceleratedUpload(bucket, key, body) {
  var acceleratedS3 = createAcceleratedClient();

  var command = new PutObjectCommand({
    Bucket: bucket,
    Key: key,
    Body: body
  });

  var start = Date.now();

  return acceleratedS3.send(command).then(function(result) {
    var elapsed = Date.now() - start;
    console.log("Accelerated upload completed in", elapsed, "ms");
    return result;
  });
}

Transfer Acceleration adds about $0.04/GB on top of standard transfer costs. Only enable it if your users are consistently uploading from locations far from your bucket's region. Run the S3 Speed Comparison tool to see if it actually helps for your use case.

S3 Select: Query Without Downloading

S3 Select lets you run SQL-like queries against objects stored in S3 without downloading the entire object. For large CSV or JSON files, this can reduce data transfer by 80% or more.

var { S3Client, SelectObjectContentCommand } = require("@aws-sdk/client-s3");

var s3 = new S3Client({ region: "us-east-1" });

function queryS3Object(bucket, key, sqlExpression) {
  var command = new SelectObjectContentCommand({
    Bucket: bucket,
    Key: key,
    ExpressionType: "SQL",
    Expression: sqlExpression,
    InputSerialization: {
      CSV: {
        FileHeaderInfo: "USE",
        RecordDelimiter: "\n",
        FieldDelimiter: ","
      }
    },
    OutputSerialization: {
      JSON: {
        RecordDelimiter: "\n"
      }
    }
  });

  return s3.send(command).then(function(response) {
    var records = [];

    return new Promise(function(resolve, reject) {
      var eventStream = response.Payload;

      (function processEvents() {
        var chunks = [];

        eventStream[Symbol.asyncIterator]().next().then(function processNext(result) {
          if (result.done) {
            resolve(records);
            return;
          }

          var event = result.value;
          if (event.Records) {
            var payload = Buffer.from(event.Records.Payload).toString("utf-8");
            var lines = payload.trim().split("\n");
            lines.forEach(function(line) {
              if (line.trim()) {
                records.push(JSON.parse(line));
              }
            });
          }
          if (event.Stats) {
            console.log("Bytes scanned:", event.Stats.Details.BytesScanned);
            console.log("Bytes returned:", event.Stats.Details.BytesReturned);
            var savings = (1 - event.Stats.Details.BytesReturned / event.Stats.Details.BytesScanned) * 100;
            console.log("Data reduction:", savings.toFixed(1) + "%");
          }

          return eventStream[Symbol.asyncIterator]().next().then(processNext);
        }).catch(reject);
      })();
    });
  });
}

// Query a 2 GB access log file - only return 5xx errors
queryS3Object(
  "my-app-bucket",
  "logs/access-2026-02.csv",
  "SELECT s.timestamp, s.method, s.path, s.status_code FROM s3object s WHERE CAST(s.status_code AS INT) >= 500"
).then(function(results) {
  console.log("Found", results.length, "server errors");
  results.slice(0, 5).forEach(function(r) {
    console.log(r.timestamp, r.method, r.path, r.status_code);
  });
});

Output:

Bytes scanned: 2147483648
Bytes returned: 4521984
Data reduction: 99.8%
Found 847 server errors
2026-02-13T08:14:22Z POST /api/users 502
2026-02-13T08:14:23Z GET /api/orders 500
2026-02-13T08:15:01Z POST /api/checkout 503
2026-02-13T08:15:44Z GET /api/products 500
2026-02-13T08:16:12Z PUT /api/users/123 502

You just queried 2 GB of data and only transferred 4.3 MB. That is the power of S3 Select.

Versioning and Intelligent-Tiering

Versioning

Versioning protects you from accidental deletes and overwrites. But it also means every version of every object is stored and billed. Without lifecycle rules on versions, your storage costs will quietly snowball.

var { S3Client, PutBucketVersioningCommand, ListObjectVersionsCommand } = require("@aws-sdk/client-s3");

var s3 = new S3Client({ region: "us-east-1" });

function enableVersioning(bucket) {
  var command = new PutBucketVersioningCommand({
    Bucket: bucket,
    VersioningConfiguration: {
      Status: "Enabled"
    }
  });

  return s3.send(command);
}

function listVersions(bucket, prefix) {
  var command = new ListObjectVersionsCommand({
    Bucket: bucket,
    Prefix: prefix,
    MaxKeys: 20
  });

  return s3.send(command).then(function(result) {
    var totalSize = 0;

    console.log("Versions for prefix:", prefix);
    (result.Versions || []).forEach(function(v) {
      totalSize += v.Size;
      console.log(
        " ", v.Key,
        "| Version:", v.VersionId.substring(0, 12) + "...",
        "| Size:", (v.Size / 1024).toFixed(1), "KB",
        "| Modified:", v.LastModified.toISOString().substring(0, 10),
        v.IsLatest ? "(latest)" : ""
      );
    });

    console.log("Total size across all versions:", (totalSize / (1024 * 1024)).toFixed(2), "MB");
    return result;
  });
}

Add a lifecycle rule to expire old versions:

{
  ID: "ExpireOldVersions",
  Status: "Enabled",
  Filter: { Prefix: "" },
  NoncurrentVersionTransitions: [
    {
      NoncurrentDays: 30,
      StorageClass: "STANDARD_IA"
    },
    {
      NoncurrentDays: 90,
      StorageClass: "GLACIER"
    }
  ],
  NoncurrentVersionExpiration: {
    NoncurrentDays: 365
  }
}

Intelligent-Tiering

S3 Intelligent-Tiering automatically moves objects between access tiers based on usage patterns. There is no retrieval fee, and the monitoring cost is $0.0025 per 1,000 objects per month. For data with unpredictable access patterns, this is the correct answer.

var { PutObjectCommand } = require("@aws-sdk/client-s3");

function uploadWithIntelligentTiering(bucket, key, body) {
  var command = new PutObjectCommand({
    Bucket: bucket,
    Key: key,
    Body: body,
    StorageClass: "INTELLIGENT_TIERING"
  });

  return s3.send(command);
}

// Configure the archive tiers for Intelligent-Tiering
var { PutBucketIntelligentTieringConfigurationCommand } = require("@aws-sdk/client-s3");

function configureIntelligentTiering(bucket) {
  var command = new PutBucketIntelligentTieringConfigurationCommand({
    Bucket: bucket,
    Id: "FullTieringConfig",
    IntelligentTieringConfiguration: {
      Id: "FullTieringConfig",
      Status: "Enabled",
      Filter: {
        Prefix: "data/"
      },
      Tierings: [
        {
          AccessTier: "ARCHIVE_ACCESS",
          Days: 90
        },
        {
          AccessTier: "DEEP_ARCHIVE_ACCESS",
          Days: 180
        }
      ]
    }
  });

  return s3.send(command);
}

Intelligent-Tiering works well when you genuinely do not know how often objects will be accessed. If you already know the access pattern, explicit lifecycle rules will save you the monitoring fee.

Cross-Region Replication

Cross-region replication (CRR) copies objects to a bucket in another region for disaster recovery or latency reduction. It requires versioning enabled on both source and destination buckets.

var { S3Client, PutBucketReplicationCommand } = require("@aws-sdk/client-s3");

var s3 = new S3Client({ region: "us-east-1" });

function setupCrossRegionReplication(sourceBucket, destBucket, roleArn) {
  var command = new PutBucketReplicationCommand({
    Bucket: sourceBucket,
    ReplicationConfiguration: {
      Role: roleArn,
      Rules: [
        {
          ID: "ReplicateCriticalData",
          Status: "Enabled",
          Priority: 1,
          Filter: {
            Prefix: "critical/"
          },
          Destination: {
            Bucket: "arn:aws:s3:::" + destBucket,
            StorageClass: "STANDARD_IA"
          },
          DeleteMarkerReplication: {
            Status: "Enabled"
          }
        },
        {
          ID: "ReplicateBackups",
          Status: "Enabled",
          Priority: 2,
          Filter: {
            Prefix: "backups/"
          },
          Destination: {
            Bucket: "arn:aws:s3:::" + destBucket,
            StorageClass: "GLACIER"
          },
          DeleteMarkerReplication: {
            Status: "Disabled"
          }
        }
      ]
    }
  });

  return s3.send(command);
}

Notice that the replicated objects can use a different storage class in the destination. You can replicate critical data to Standard-IA in another region for fast DR failover while sending backups straight to Glacier for cheap long-term retention.

Cost Optimization Strategies

Here are the strategies that produce the biggest impact, ranked by effort-to-savings ratio:

1. Audit Your Current Usage

var { S3Client, ListBucketsCommand, GetBucketMetricsConfigurationCommand } = require("@aws-sdk/client-s3");
var { CloudWatchClient, GetMetricStatisticsCommand } = require("@aws-sdk/client-cloudwatch");

var cloudwatch = new CloudWatchClient({ region: "us-east-1" });

function getBucketSizeMetric(bucket) {
  var command = new GetMetricStatisticsCommand({
    Namespace: "AWS/S3",
    MetricName: "BucketSizeBytes",
    Dimensions: [
      { Name: "BucketName", Value: bucket },
      { Name: "StorageType", Value: "StandardStorage" }
    ],
    StartTime: new Date(Date.now() - 86400000),
    EndTime: new Date(),
    Period: 86400,
    Statistics: ["Average"]
  });

  return cloudwatch.send(command).then(function(result) {
    if (result.Datapoints.length > 0) {
      var sizeGB = result.Datapoints[0].Average / (1024 * 1024 * 1024);
      var monthlyCost = sizeGB * 0.023;
      console.log(bucket + ":");
      console.log("  Size:", sizeGB.toFixed(2), "GB");
      console.log("  Estimated monthly cost (Standard):", "$" + monthlyCost.toFixed(2));
      console.log("  If moved to Standard-IA:", "$" + (sizeGB * 0.0125).toFixed(2));
      console.log("  If moved to Glacier:", "$" + (sizeGB * 0.004).toFixed(2));
    }
    return result;
  });
}

2. Enable S3 Storage Lens

Storage Lens gives you organization-wide visibility into S3 usage. It is free for the dashboard-level metrics and well worth enabling. Use it to find buckets with zero access that are still costing you money.

3. Compress Before Upload

var zlib = require("zlib");

function compressAndUpload(bucket, key, data) {
  return new Promise(function(resolve, reject) {
    zlib.gzip(Buffer.from(JSON.stringify(data)), function(err, compressed) {
      if (err) return reject(err);

      var originalSize = Buffer.byteLength(JSON.stringify(data));
      var compressedSize = compressed.length;
      var ratio = ((1 - compressedSize / originalSize) * 100).toFixed(1);

      console.log("Original:", (originalSize / 1024).toFixed(1), "KB");
      console.log("Compressed:", (compressedSize / 1024).toFixed(1), "KB");
      console.log("Savings:", ratio + "%");

      var command = new PutObjectCommand({
        Bucket: bucket,
        Key: key + ".gz",
        Body: compressed,
        ContentEncoding: "gzip",
        ContentType: "application/json"
      });

      resolve(s3.send(command));
    });
  });
}

4. Use Prefix-Based Organization

Structure your keys so lifecycle rules can target them efficiently:

uploads/user-content/      → Standard → IA after 30d
uploads/thumbnails/        → Standard (always accessed)
logs/application/          → Standard → IA → Glacier → Delete
logs/access/               → Standard → Delete after 90d
backups/daily/             → Standard-IA → Glacier after 7d
backups/monthly/           → Glacier → Deep Archive after 365d
tmp/                       → Delete after 1d

5. Delete What You Do Not Need

This sounds obvious, but I have audited dozens of S3 buckets across companies and the amount of orphaned data is staggering. Old deployment artifacts, test uploads, duplicate backups, unused media assets. Run a regular audit.

Complete Working Example

Here is a comprehensive S3 management utility that ties together storage class selection, lifecycle policies, multipart uploads, and cost analysis:

var { S3Client, PutObjectCommand, GetObjectCommand, HeadObjectCommand,
      PutBucketLifecycleConfigurationCommand, GetBucketLifecycleConfigurationCommand,
      ListObjectsV2Command, DeleteObjectCommand, CopyObjectCommand } = require("@aws-sdk/client-s3");
var { Upload } = require("@aws-sdk/lib-storage");
var fs = require("fs");
var path = require("path");
var zlib = require("zlib");

// Storage class cost map ($/GB/month, US East)
var STORAGE_COSTS = {
  STANDARD: 0.023,
  STANDARD_IA: 0.0125,
  ONEZONE_IA: 0.01,
  INTELLIGENT_TIERING: 0.023,
  GLACIER_IR: 0.004,
  GLACIER: 0.0036,
  DEEP_ARCHIVE: 0.00099
};

function S3Manager(config) {
  this.bucket = config.bucket;
  this.s3 = new S3Client({ region: config.region || "us-east-1" });
}

// Determine the best storage class based on access pattern
S3Manager.prototype.recommendStorageClass = function(accessFrequency, durabilityRequired) {
  if (accessFrequency === "frequent") return "STANDARD";
  if (accessFrequency === "unknown") return "INTELLIGENT_TIERING";
  if (accessFrequency === "infrequent" && durabilityRequired === "high") return "STANDARD_IA";
  if (accessFrequency === "infrequent" && durabilityRequired === "normal") return "ONEZONE_IA";
  if (accessFrequency === "archive" && durabilityRequired === "instant") return "GLACIER_IR";
  if (accessFrequency === "archive") return "GLACIER";
  if (accessFrequency === "deep-archive") return "DEEP_ARCHIVE";
  return "STANDARD";
};

// Smart upload: choose single-part or multipart based on size
S3Manager.prototype.upload = function(key, source, options) {
  var self = this;
  options = options || {};

  var isFilePath = typeof source === "string" && fs.existsSync(source);
  var body, size;

  if (isFilePath) {
    size = fs.statSync(source).size;
    body = fs.createReadStream(source);
  } else {
    body = Buffer.isBuffer(source) ? source : Buffer.from(source);
    size = body.length;
  }

  var storageClass = options.storageClass ||
    self.recommendStorageClass(options.accessFrequency || "frequent", options.durability || "high");

  console.log("Uploading", key);
  console.log("  Size:", (size / (1024 * 1024)).toFixed(2), "MB");
  console.log("  Storage class:", storageClass);
  console.log("  Estimated monthly cost: $" + (size / (1024 * 1024 * 1024) * STORAGE_COSTS[storageClass]).toFixed(4));

  // Use multipart for files > 50 MB
  if (size > 50 * 1024 * 1024) {
    console.log("  Method: Multipart upload");
    var upload = new Upload({
      client: self.s3,
      params: {
        Bucket: self.bucket,
        Key: key,
        Body: body,
        StorageClass: storageClass,
        ContentType: options.contentType || "application/octet-stream",
        Metadata: options.metadata || {}
      },
      queueSize: 4,
      partSize: 10 * 1024 * 1024
    });

    upload.on("httpUploadProgress", function(progress) {
      var pct = ((progress.loaded / size) * 100).toFixed(1);
      process.stdout.write("\r  Progress: " + pct + "%");
    });

    return upload.done().then(function(result) {
      console.log("\n  Upload complete");
      return result;
    });
  }

  console.log("  Method: Single-part upload");
  var command = new PutObjectCommand({
    Bucket: self.bucket,
    Key: key,
    Body: body,
    StorageClass: storageClass,
    ContentType: options.contentType || "application/octet-stream",
    Metadata: options.metadata || {}
  });

  return self.s3.send(command).then(function(result) {
    console.log("  Upload complete");
    return result;
  });
};

// Analyze bucket costs and recommend optimizations
S3Manager.prototype.analyzeCosts = function(prefix) {
  var self = this;
  var allObjects = [];

  function listPage(continuationToken) {
    var params = {
      Bucket: self.bucket,
      Prefix: prefix || "",
      MaxKeys: 1000
    };
    if (continuationToken) {
      params.ContinuationToken = continuationToken;
    }

    var command = new ListObjectsV2Command(params);
    return self.s3.send(command).then(function(result) {
      allObjects = allObjects.concat(result.Contents || []);
      if (result.IsTruncated) {
        return listPage(result.NextContinuationToken);
      }
      return allObjects;
    });
  }

  return listPage().then(function(objects) {
    var analysis = {
      totalObjects: objects.length,
      totalSizeBytes: 0,
      byStorageClass: {},
      staleObjects: [],
      currentMonthlyCost: 0,
      optimizedMonthlyCost: 0
    };

    var thirtyDaysAgo = new Date(Date.now() - 30 * 86400000);
    var ninetyDaysAgo = new Date(Date.now() - 90 * 86400000);

    objects.forEach(function(obj) {
      analysis.totalSizeBytes += obj.Size;
      var sc = obj.StorageClass || "STANDARD";

      if (!analysis.byStorageClass[sc]) {
        analysis.byStorageClass[sc] = { count: 0, sizeBytes: 0 };
      }
      analysis.byStorageClass[sc].count += 1;
      analysis.byStorageClass[sc].sizeBytes += obj.Size;

      var sizeGB = obj.Size / (1024 * 1024 * 1024);
      analysis.currentMonthlyCost += sizeGB * (STORAGE_COSTS[sc] || 0.023);

      // Recommend optimization for old Standard objects
      if (sc === "STANDARD" && obj.LastModified < ninetyDaysAgo) {
        analysis.optimizedMonthlyCost += sizeGB * STORAGE_COSTS.GLACIER_IR;
        analysis.staleObjects.push(obj.Key);
      } else if (sc === "STANDARD" && obj.LastModified < thirtyDaysAgo) {
        analysis.optimizedMonthlyCost += sizeGB * STORAGE_COSTS.STANDARD_IA;
      } else {
        analysis.optimizedMonthlyCost += sizeGB * (STORAGE_COSTS[sc] || 0.023);
      }
    });

    console.log("=== S3 Cost Analysis ===");
    console.log("Bucket:", self.bucket);
    console.log("Prefix:", prefix || "(all)");
    console.log("Total objects:", analysis.totalObjects);
    console.log("Total size:", (analysis.totalSizeBytes / (1024 * 1024 * 1024)).toFixed(2), "GB");
    console.log("");
    console.log("Storage class breakdown:");
    Object.keys(analysis.byStorageClass).forEach(function(sc) {
      var info = analysis.byStorageClass[sc];
      console.log("  " + sc + ": " + info.count + " objects, " +
        (info.sizeBytes / (1024 * 1024 * 1024)).toFixed(2) + " GB");
    });
    console.log("");
    console.log("Current monthly cost: $" + analysis.currentMonthlyCost.toFixed(2));
    console.log("Optimized monthly cost: $" + analysis.optimizedMonthlyCost.toFixed(2));
    console.log("Potential savings: $" +
      (analysis.currentMonthlyCost - analysis.optimizedMonthlyCost).toFixed(2) + "/month");
    console.log("Stale objects (>90 days in Standard):", analysis.staleObjects.length);

    return analysis;
  });
};

// Transition an object to a different storage class
S3Manager.prototype.transitionObject = function(key, targetStorageClass) {
  var self = this;

  var command = new CopyObjectCommand({
    Bucket: self.bucket,
    Key: key,
    CopySource: self.bucket + "/" + key,
    StorageClass: targetStorageClass,
    MetadataDirective: "COPY"
  });

  return self.s3.send(command).then(function(result) {
    console.log("Transitioned", key, "to", targetStorageClass);
    return result;
  });
};

// Apply lifecycle rules
S3Manager.prototype.applyLifecyclePolicy = function(rules) {
  var command = new PutBucketLifecycleConfigurationCommand({
    Bucket: this.bucket,
    LifecycleConfiguration: {
      Rules: rules
    }
  });

  return this.s3.send(command).then(function() {
    console.log("Applied", rules.length, "lifecycle rules");
    rules.forEach(function(rule) {
      console.log("  -", rule.ID, "(prefix: '" + (rule.Filter.Prefix || "*") + "')");
    });
  });
};

// Usage
var manager = new S3Manager({
  bucket: "my-production-bucket",
  region: "us-east-1"
});

// Analyze current costs
manager.analyzeCosts("uploads/").then(function(analysis) {
  if (analysis.staleObjects.length > 0) {
    console.log("\nApplying lifecycle rules to optimize...");

    return manager.applyLifecyclePolicy([
      {
        ID: "OptimizeUploads",
        Status: "Enabled",
        Filter: { Prefix: "uploads/" },
        Transitions: [
          { Days: 30, StorageClass: "STANDARD_IA" },
          { Days: 90, StorageClass: "GLACIER_IR" },
          { Days: 365, StorageClass: "DEEP_ARCHIVE" }
        ]
      },
      {
        ID: "CleanupMultipart",
        Status: "Enabled",
        Filter: { Prefix: "" },
        AbortIncompleteMultipartUpload: { DaysAfterInitiation: 7 }
      }
    ]);
  }
});

Output:

=== S3 Cost Analysis ===
Bucket: my-production-bucket
Prefix: uploads/
Total objects: 48392
Total size: 847.23 GB

Storage class breakdown:
  STANDARD: 48392 objects, 847.23 GB

Current monthly cost: $19.49
Optimized monthly cost: $7.82
Potential savings: $11.67/month
Stale objects (>90 days in Standard): 31204

Applying lifecycle rules to optimize...
Applied 2 lifecycle rules
  - OptimizeUploads (prefix: 'uploads/')
  - CleanupMultipart (prefix: '*')

Common Issues and Troubleshooting

1. AccessDenied When Transitioning to Glacier

AccessDenied: User: arn:aws:iam::123456789012:user/app-user is not authorized to perform: s3:PutLifecycleConfiguration on resource: "arn:aws:s3:::my-bucket"

Your IAM policy needs s3:PutLifecycleConfiguration permission. This is separate from s3:PutObject. Add it explicitly:

{
  "Effect": "Allow",
  "Action": [
    "s3:PutLifecycleConfiguration",
    "s3:GetLifecycleConfiguration"
  ],
  "Resource": "arn:aws:s3:::my-bucket"
}

2. InvalidObjectState When Accessing Glacier Objects

InvalidObjectState: The operation is not valid for the object's storage class

You cannot directly GetObject on a Glacier or Deep Archive object. You must restore it first:

var { RestoreObjectCommand } = require("@aws-sdk/client-s3");

function restoreFromGlacier(bucket, key, days) {
  var command = new RestoreObjectCommand({
    Bucket: bucket,
    Key: key,
    RestoreRequest: {
      Days: days || 7,
      GlacierJobParameters: {
        Tier: "Standard"  // Standard (3-5 hours), Expedited (1-5 min), Bulk (5-12 hours)
      }
    }
  });

  return s3.send(command).then(function() {
    console.log("Restore initiated for", key);
    console.log("Object will be available in 3-5 hours (Standard tier)");
    console.log("Restored copy expires in", days, "days");
  });
}

3. Lifecycle Rules Not Executing

Lifecycle rules apply at midnight UTC and can take up to 48 hours to fully execute. If rules are not working, check:

• The rule status is Enabled, not Disabled
• The prefix filter matches your object keys (trailing slashes matter)
• Objects meet the minimum size requirement (128 KB for Standard-IA transition)
• The minimum duration constraint has been met (30 days for IA, 90 for Glacier)

// Debug: Check if an object's age qualifies for transition
function checkTransitionEligibility(bucket, key) {
  var command = new HeadObjectCommand({ Bucket: bucket, Key: key });

  return s3.send(command).then(function(result) {
    var ageInDays = Math.floor((Date.now() - result.LastModified.getTime()) / 86400000);
    var sizeKB = result.ContentLength / 1024;
    var storageClass = result.StorageClass || "STANDARD";

    console.log("Object:", key);
    console.log("  Age:", ageInDays, "days");
    console.log("  Size:", sizeKB.toFixed(1), "KB");
    console.log("  Current class:", storageClass);
    console.log("  Eligible for IA transition:", ageInDays >= 30 && sizeKB >= 128 ? "Yes" : "No");
    console.log("  Eligible for Glacier transition:", ageInDays >= 90 ? "Yes" : "No");
  });
}

4. Multipart Upload Stalling or Timing Out

TimeoutError: Connection timed out after 120000ms

Large multipart uploads can stall on slow or unstable connections. Adjust the SDK configuration:

var { S3Client } = require("@aws-sdk/client-s3");
var { NodeHttpHandler } = require("@smithy/node-http-handler");

var s3 = new S3Client({
  region: "us-east-1",
  requestHandler: new NodeHttpHandler({
    connectionTimeout: 10000,    // 10 second connection timeout
    socketTimeout: 300000        // 5 minute socket timeout per part
  }),
  maxAttempts: 5                 // retry failed parts up to 5 times
});

5. Unexpected Costs from Delete Markers and Versions

When versioning is enabled, deleting an object creates a delete marker instead of actually removing data. Previous versions remain and continue incurring charges. Check for this:

function findDeleteMarkers(bucket, prefix) {
  var command = new ListObjectVersionsCommand({
    Bucket: bucket,
    Prefix: prefix
  });

  return s3.send(command).then(function(result) {
    var markers = result.DeleteMarkers || [];
    var versions = result.Versions || [];
    var hiddenSize = 0;

    versions.forEach(function(v) {
      if (!v.IsLatest) hiddenSize += v.Size;
    });

    console.log("Delete markers:", markers.length);
    console.log("Non-current versions:", versions.filter(function(v) { return !v.IsLatest; }).length);
    console.log("Hidden storage cost:", (hiddenSize / (1024 * 1024 * 1024)).toFixed(2), "GB");
  });
}

Best Practices

• Always set lifecycle rules for incomplete multipart uploads. Abandoned multipart fragments are invisible in the console but still cost money. Set a 7-day abort rule on every bucket, no exceptions.

• Use prefix-based key organization from day one. Restructuring S3 keys after the fact means copying every object. Design your key hierarchy around access patterns so lifecycle rules can target them cleanly.

• Never store everything in S3 Standard by default. Make storage class selection a conscious decision during upload. Even a simple heuristic like "logs go to Standard-IA" saves significant money at scale.

• Enable S3 Inventory for large buckets. S3 Inventory generates daily or weekly CSV reports of all objects with their storage classes, sizes, and last modified dates. Use it to audit before you optimize.

• Compress before upload, not after. Gzipping JSON logs before upload typically achieves 80-90% compression. That is an 80-90% reduction in both storage and transfer costs. Spend the CPU cycles.

• Set appropriate object expiration rules. If data has a known retention period, set an expiration rule and forget about it. Logs that must be kept for 1 year should expire on day 366, not live forever because nobody remembers to clean them up.

• Use S3 Storage Lens across your organization. It surfaces which buckets are growing fastest, which ones have no access, and where your money is going. Enable it organization-wide and review it monthly.

• Monitor retrieval costs when using Glacier classes. The storage is cheap, but retrieval is not. A batch restore of 100 GB from Deep Archive costs $2 plus data transfer. Factor retrieval costs into your total cost of ownership.

• Test lifecycle rules in a staging bucket first. Lifecycle rule misconfiguration can archive or delete production data. Test your rules on a copy of your data before applying to production.

• Use object tags for fine-grained lifecycle control. When prefix-based rules are not granular enough, tag objects with metadata like retention=30d or tier=archive and build lifecycle rules against those tags.

References

• AWS S3 Storage Classes Documentation
• S3 Lifecycle Configuration
• AWS SDK for JavaScript v3 - S3 Client
• S3 Multipart Upload Overview
• S3 Intelligent-Tiering
• S3 Select and Glacier Select
• S3 Pricing
• S3 Storage Lens