FinOps on The Cloud Optimist

Why Tagging your AWS Resources is a Must?

Tue, 24 Jun 2025 07:30:00 +0200

Introduction

The more services we deploy on AWS, the easier it is to get lost. At first, we know all the services we use by heart, the number of Lambdas or EC2 instances running. But little by little, it’s easy to feel overwhelmed, especially when different projects pile up. Quickly finding which resources belong to which project, or identifying those that weren’t properly cleaned up after a PoC, can quickly become a headache. And that’s without even mentioning cost visibility! Fortunately, there’s a simple yet incredibly powerful practice: tagging.

In this article, I want to show you that a good tagging strategy is crucial for your organization, the way you handle cost, and to secure your AWS account.

Ready? Tag along with me!

Why are tags useful?

Imagine labels on moving boxes. Without them, it’s impossible to know what’s inside or which room they belong to. Tags on AWS are the same! A tag is a piece of information (in key-value format) that you assign to your resources (EC2 instances, S3 buckets, RDS databases, etc.). A good tagging strategy allows you to, among other things:

Identify orphan resources: This is the classic scenario. A resource without a Project or Owner tag? There’s a good chance it has been forgotten and is consuming resources (and therefore money) for nothing. Listing untagged (or poorly tagged) resources is an essential first step in cleaning up.
Allocate costs: By tagging your resources with a project identifier, cost center, or team, you can then use AWS Cost Explorer to filter your expenses and understand precisely which projects consume the most. This is essential for internal chargebacks or simply for optimizing your budget.
Automate actions: Tags can serve as triggers for automation scripts (for example, backing up all EC2 instances with the tag Backup=Daily).
Manage access and security: The AWS IAM service can use tags to grant granular permissions.

In short, tagging is the foundation of good Cloud governance. For more details, I invite you to consult the tagging guide proposed by AWS.

Fortunately, we can associate multiple tags with the same resource. But this raises the question: how many tags are necessary?

This will depend on your company and each project, but overall, there are some tags that won’t hurt, no matter your situation:

Project: The name of the project linked to the resource. Usually, the GitHub repository name works well.
Environment: The desired environment (dev, uat, prod, …). Even if you have separate AWS accounts, this will help you identify if a resource got lost during deployment.
Owner: The owner of the resource. This could be a person, but more ideally a team (frontend, backend, security, …).

Depending on your use case, you’ll surely have other ideas for tags, but with the ones above, you’ll already have a good start! And in case of doubt, feel free to read the tagging best practices from AWS.

Let’s now look at some concrete examples of using tags in AWS.

AWS Cost Explorer: Tracking costs thanks to tags

One of the most concrete benefits of tagging is the visibility it provides over your expenses. AWS Cost Explorer is the tool of choice for this.

Once your resources are correctly tagged (for example, with the Project tag), you need to activate these tags for cost allocation in the AWS Billing and Cost Management console (Billing and Cost Management -> Cost Organization -> Cost Allocation Tags). Be aware that there might be a delay before the activated tags appear in Cost Explorer.

Once activated, you can:

Filter by tag: In Cost Explorer, you can filter your costs by the value of a specific tag.
Group by tag: You can also choose to group your expenses by tag. This will give you an overview of the cost distribution across different projects, environments, etc.
Create budgets based on tags: With AWS Budgets, you can set alert thresholds for costs associated with specific tags, helping you avoid nasty surprises.

Example of filtering by tag for my blog: the costs are minimal, am I a FinOps expert?

This ability to dissect your AWS bill by tags transforms cost management from an obscure chore into a transparent and controllable exercise. It’s a must for any organization concerned about its Cloud budget. You can now add FinOps to your LinkedIn bio!

AWS Resource Explorer: Keeping an eye on deployed resources

Now that we are convinced of the usefulness of tags, how do we list all the resources that use a specific tag? We will see this together, with one part in the AWS console, and another part in the terminal (for all the geeks reading this article!).

Visual exploration in the AWS console

If you are not yet familiar with AWS Resource Explorer, now is the time to discover it! This relatively recent service allows you to search and discover your AWS resources across all regions of your account, using a simple interface, much like a search engine.

The main advantage of Resource Explorer is its ability to give you a unified view. No more jumping from region to region. You activate indexing, and then you can search for your resources by name, ID, and of course… by tag!

It’s an excellent tool for:

Getting a quick overview.
Visually exploring resources associated with a specific tag.
Quickly identifying resources without having to code anything.

To use it, activate it in the desired regions (or all of them), let it index your resources, and then use the search bar with a syntax like tag.key:Project tag.value:Cloud*.

Example of a tag search for my blog: only three resources manage this blog!

A more technical approach with AWS CLI

For those who, like me, like to have control via the command line, or who need to automate these searches, the AWS CLI remains a key ally. More specifically, it’s the resourcegroupstaggingapi service that will interest us.

The key command is get-resources. Here is a typical example to list the ARNs of all resources having the tag Project with the value “Cloud Antoine Delia”:

1
2
3

aws resourcegroupstaggingapi get-resources \
    --tag-filters "Key=Project,Values=Cloud Antoine Delia" \
    | jq "[.ResourceTagMappingList[].ResourceARN]"

Which gives us:

1
2
3

[
  "arn:aws:s3:::antoiXXXXXXXXXXXXX"
]

Let’s break it down a bit:

aws resourcegroupstaggingapi get-resources: This is the API call; so far, so good.
--tag-filters "Key=Project,Values=Cloud Antoine Delia": This is where we specify our filter. We are looking for the tag Project with the value “Cloud Antoine Delia”. You can add multiple filters.
| jq "[.ResourceTagMappingList[].ResourceARN]": jq is a wonderful tool for manipulating JSON on the command line. Here, we use it to cleanly extract the list of ARNs of the found resources (you can do without it, but why make life difficult?).

Hey, wait a minute! In the console, you show us three resources, and now there’s only one! Where’s the catch?

Well spotted! You should know that when you make your API call, you are using a default region. However, if you have resources in various regions, you will need to specify it. Thus, if we add --region us-east-1 just before the jq pipe, we indeed get our two missing resources.

[
  "arn:aws:acm:us-east-1:6XXXXXXXXXXX0:certificate/f4ca3b13-XXXXXXXXXXXXX",
  "arn:aws:cloudfront::6XXXXXXXXXXX0:distribution/ERSXXXXXXXXXX"
]

Besides this detail that you must not forget, this command is extremely powerful because you can integrate it into scripts to:

Generate regular reports on resources by project.
Automatically detect resources that do not comply with your tagging policy.
Combine with other AWS CLI commands to perform actions on the listed resources.

For example, you could list all your resources of a certain type (e.g., all your EC2 instances) and check which ones are missing essential tags.

And if you’re wondering if AWS doesn’t already offer a service for this… It does! But we will talk about that in a future article (for the curious, I’m talking about AWS Config).

AWS IAM: Securing the use of your resources

Your resources are deployed in AWS, and now you want to give a team permission to manage all of that. But here’s the thing, in your AWS account, you also have critical resources that must absolutely not be compromised.

AWS IAM is here for you! Using a simple policy, you can specify that only resources with a certain tag can be modified by a user or group.

Let’s take the following example: you would like to allow a team to start or stop certain EC2 instances, but prevent them from accidentally stopping a critical EC2 instance! You just need to add the following policy to your users:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowStartStopEC2IfProjectCloud",
      "Effect": "Allow",
      "Action": [
        "ec2:StartInstances",
        "ec2:StopInstances"
      ],
      "Resource": "arn:aws:ec2:*:*:instance/*",
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/Project": "Cloud Antoine Delia"
        }
      }
    },
    {
      "Sid": "AllowDescribeToSeeInstances",
      "Effect": "Allow",
      "Action": "ec2:DescribeInstances",
      "Resource": "*"
    }
  ]
}

Thus, your users will be autonomous in using their resources, without having the possibility to impact other resources.

Conclusion

You’ve understood it: a rigorous tagging strategy isn’t an option; it’s a necessity for operating serenely on AWS. Whether on the organizational, financial, or security management level, AWS gives you the means to take full advantage of your tags.

So, a little piece of advice: if you haven’t already, define a clear tagging policy in your organization, apply it, and use these tools to regularly check that everything is in order. You’ll thank me later!

AWS Lambda: INIT Phase To Be Billed Soon - Should We Be Alarmed?

Tue, 13 May 2025 07:30:00 +0200

Introduction

Ah, Lambdas. Probably my favorite AWS service! Almost instantly, they allow you to create a small script or even a fully hosted API backend in the Cloud, all at a low cost thanks to the “pay-as-you-go” model. But is it too good to be true?

A few days ago, AWS announced a change in how the Lambda initialization phase is billed. Starting August 1, 2025, this phase will systematically be included in the billed duration calculation, for all Lambdas.

If you weren’t aware, until now, if you used Lambda functions packaged as ZIP archives with AWS-managed runtimes (like Python, Node.js, etc.), the duration of this INIT phase wasn’t billed. It was a small “gift” from AWS (quite nice of them). But all good things must come to an end, and we will now have to pay for this initialization time for our Lambdas.

So, will this make using Lambdas too expensive? And how can we minimize this initialization part as much as possible? Today, let’s explore these questions!

Understanding the Lambda Lifecycle

Before diving into billing, it’s a good opportunity to recall the Lambda lifecycle.

For simplicity, I won’t go into the details of Lambda Extensions and Lambda SnapStart.

It consists of three main phases:

INIT: This is the “cold start” phase. When a new instance of your function needs to be created to handle a request, Lambda prepares the environment. This phase lasts a maximum of 10 seconds.
INVOKE: This is where your code (the function handler) is executed to process the request.
SHUTDOWN: When the execution environment hasn’t been used for a certain period, the Lambda shuts down to free up resources. If a new request arrives, the Lambda will have to go through the INIT phase again.

During the INIT phase, your Lambda performs several actions:

Retrieves your code (from S3 for a ZIP, or ECR for a Docker image).
Configures the environment with the allocated memory, chosen runtime, etc.
Starts the runtime (Runtime INIT).
Executes the static code of your function (everything outside the handler, like initializing global variables or boto3) (Function INIT).

The key takeaway here is that the INIT phase only occurs during a cold start. If a request arrives while an execution environment is already “warm” (ready and reused), this phase is skipped, and execution goes directly to the INVOKE phase. This is called a “warm start,” which is much faster.

AWS doesn’t disclose its logic for transitioning a Lambda from ‘warm’ to ‘cold’.

The Billing Change in Detail

Currently, Lambda billing is based on two factors:

The number of requests.
The execution duration of your code, rounded up to the nearest millisecond (the cost of this duration depends on the memory allocated to the function).

Until August 1, 2025, for functions packaged as ZIPs with managed runtimes, the INIT phase duration was not counted in the “Billed Duration”. This could be seen in CloudWatch logs:

Before August 1, 2025
Note that Billed Duration is the rounded-up value
of Duration without considering Init Duration
REPORT RequestId: xxxxx   Duration: 250.06 ms   Billed Duration: 251 ms   Init Duration: 100.77 ms

After August 1, 2025
Billed Duration is now the rounded-up value
of Duration + Init Duration
REPORT RequestId: xxxxx   Duration: 250.06 ms   Billed Duration: 351 ms   Init Duration: 100.77 ms

As you can see, AWS will now take your Init Duration into account in addition to the Duration when calculating the Billed Duration.

What Impact on the Bill?

So, this all sounds interesting, but will this change break the bank?

Let’s take an example to see more clearly. Imagine a Python Lambda configured with 1024 MB of memory, deployed in the eu-west-1 (Ireland) region.

Assume the following:

The Lambda receives 10 million invocations per month.
The cold start rate is 1% (average provided by AWS), meaning 100,000 cold starts per month.
The average invocation duration (Duration) is 3 seconds.
The average initialization duration (Init Duration) is 1 second.
Cost per request: $0.20 per million requests.
Duration cost (x86): $0.0000166667 per GB-second.

Details of the calculation

Cost Calculation BEFORE August 1, 2025:

Request Cost: 10 million req * ($0.20 / 1 million req) = $2.00
Duration Cost:
- Warm starts (9.9 million): Billed duration = 3 seconds.
- Cold starts (100,000): Billed duration = 3 seconds (INIT not billed).
- Total billed duration (seconds): (9,900,000 * 3 s) + (100,000 * 3 s) = 29,700,000 seconds + 300,000 seconds = 30,000,000 seconds.
- Total GB-seconds: 30,000,000 seconds * (1024 MB / 1024 MB) = 30,000,000 GB-s.
- Duration Cost: 30,000,000 GB-s * $0.0000166667/GB-s = $500.00
Total Monthly Cost (Before): $2.00 + $500.00 = $502.00

Cost Calculation AFTER August 1, 2025:

Request Cost: $2.00 (unchanged)
Duration Cost:
- Warm starts (9.9 million): Billed duration = 3 seconds.
- Cold starts (100,000): Billed duration = 3 seconds (Duration) + 1 second (Init Duration) = 4 seconds.
- Total billed duration (seconds): (9,900,000 * 3 s) + (100,000 * 4 s) = 29,700,000 seconds + 400,000 seconds = 30,100,000 seconds.
- Total GB-seconds: 30,100,000 seconds * (1024 MB / 1024 MB) = 30,100,000 GB-s.
- Duration Cost: 30,100,000 GB-s * $0.0000166667/GB-s = $501.67
Total Monthly Cost (After): $2.00 + $501.67 = $503.67

In this specific scenario, the increase is only $1.67 per month. This is indeed minimal and confirms AWS’s communication. But be careful, as the actual impact will strongly depend on:

The memory allocated to your Lambdas (more memory = higher cost per ms).
The actual duration of your INIT phase.
Your cold start rate (can be higher if your traffic is irregular).

Therefore, it’s wise to check with your own numbers!

Below is a small script that will allow you to quickly test this on your side.

Python Script

def calculate_lambda_costs(
    total_invocations_per_month: int,
    cold_start_rate: float,
    average_invocation_duration_sec: float,
    average_init_duration_sec: float,
    allocated_memory_mb: float, # Changed GB to MB for consistency with common AWS console values
    cost_per_million_requests: float = 0.20,
    cost_per_gb_second: float = 0.0000166667,
):
    num_cold_starts = total_invocations_per_month * cold_start_rate
    num_warm_starts = total_invocations_per_month - num_cold_starts

    request_cost = (total_invocations_per_month / 1_000_000) * cost_per_million_requests

    allocated_memory_gb = allocated_memory_mb / 1024 # Convert MB to GB for calculation

    # Duration cost BEFORE
    billed_duration_warm_starts_sec_before = num_warm_starts * average_invocation_duration_sec
    # For cold starts, only invocation duration was billed
    billed_duration_cold_starts_sec_before = num_cold_starts * average_invocation_duration_sec

    total_billed_duration_sec_before = billed_duration_warm_starts_sec_before + billed_duration_cold_starts_sec_before
    total_gb_seconds_before = total_billed_duration_sec_before * allocated_memory_gb
    duration_cost_before = total_gb_seconds_before * cost_per_gb_second

    total_monthly_cost_before = request_cost + duration_cost_before

    # Duration cost AFTER
    billed_duration_warm_starts_sec_after = num_warm_starts * average_invocation_duration_sec
     # For cold starts, invocation + init duration will be billed
    billed_duration_cold_starts_sec_after = num_cold_starts * (average_invocation_duration_sec + average_init_duration_sec)

    total_billed_duration_sec_after = billed_duration_warm_starts_sec_after + billed_duration_cold_starts_sec_after
    total_gb_seconds_after = total_billed_duration_sec_after * allocated_memory_gb
    duration_cost_after = total_gb_seconds_after * cost_per_gb_second

    total_monthly_cost_after = request_cost + duration_cost_after

    return total_monthly_cost_before, total_monthly_cost_after


def display_results(cost_before, cost_after):
    cost_difference = cost_after - cost_before
    print(f"\nEstimated monthly cost BEFORE change: ${cost_before:.2f}")
    print(f"Estimated monthly cost AFTER change: ${cost_after:.2f}")
    print(f"Estimated monthly cost increase: ${cost_difference:.2f}")
    if cost_difference > 0 and cost_before > 0:
        percentage_increase = (cost_difference / cost_before) * 100
        print(f"Percentage increase: {percentage_increase:.2f}%")
    elif cost_difference > 0:
        print("Percentage increase: Infinite (original cost was zero)")


if __name__ == "__main__":
    print("AWS Lambda Cost Calculator (Before/After Init Duration Pricing Change)")
    print("Please enter the values for your scenario or press Enter to use default values.\n")
    default_invocations = 10_000_000
    default_cold_start_rate = 0.01  # 1%
    default_invocation_duration = 3.0  # seconds
    default_init_duration = 1.0  # seconds
    default_memory_mb = 1024.0 # 1024 MB

    try:
        invocations_str = input(f"Total invocations per month (default: {default_invocations:,}): ")
        total_invocations_input = int(invocations_str) if invocations_str else default_invocations

        cold_start_rate_str = input(f"Cold start rate (e.g., 0.01 for 1%, default: {default_cold_start_rate}): ")
        cold_start_rate_input = float(cold_start_rate_str) if cold_start_rate_str else default_cold_start_rate

        invocation_duration_str = input(f"Average invocation duration in seconds (default: {default_invocation_duration}): ")
        invocation_duration_input = float(invocation_duration_str) if invocation_duration_str else default_invocation_duration

        init_duration_str = input(f"Average initialization duration in seconds (default: {default_init_duration}): ")
        init_duration_input = float(init_duration_str) if init_duration_str else default_init_duration

        memory_mb_str = input(f"Memory allocated to Lambda in MB (e.g., 512, default: {default_memory_mb}): ")
        memory_mb_input = float(memory_mb_str) if memory_mb_str else default_memory_mb

        cost_before, cost_after = calculate_lambda_costs(
            total_invocations_per_month=total_invocations_input,
            cold_start_rate=cold_start_rate_input,
            average_invocation_duration_sec=invocation_duration_input,
            average_init_duration_sec=init_duration_input,
            allocated_memory_mb=memory_mb_input,
        )

        display_results(cost_before, cost_after)

    except ValueError:
        print("\nError: Please enter valid numbers.")
    except Exception as e:
        print(f"\nAn unexpected error occurred: {e}")

How to Monitor Your INIT Phase and Estimate the Impact?

A script is nice, but if you have a bunch of Lambdas to check, you might find the task exhausting.

Fortunately, AWS provides some tools to check this efficiently. As seen earlier, each Lambda will generate a REPORT log containing its Init Duration. We can easily aggregate this information using CloudWatch Logs Insights. And the icing on the cake, AWS even provides us with the right query.

filter @type = "REPORT" and @billedDuration < ceil(@duration + @initDuration)
| stats sum((@memorySize/1024/1000) * (@billedDuration/1000)) as BilledGBs,
        sum((@memorySize/1024/1000) * ((ceil(@duration + @initDuration) - @billedDuration)/1000)) as EstimatedInitGBs,
        (EstimatedInitGBs / (EstimatedInitGBs + BilledGBs)) * 100 as EstimatedInitCostPercent

Keep in mind that CloudWatch Logs Insights is billed at $0.005 per GB of data scanned (source). Adjust the query time range to limit costs.

Just run this query on your Lambda log groups (using the /aws/lambda prefix or specific function prefixes), and you’ll get a result like this:

BilledGBs	UnbilledInitGBs	Ratio
512.8007	86.6699	0.1446

This query gives you three key pieces of information:

BilledGBs: The total GB-seconds currently billed.
EstimatedInitGBs: The total GB-seconds consumed during the INIT phase that were not previously billed (this is an estimation based on the difference between current billed duration and potential future billed duration including INIT).
EstimatedInitCostPercent: The percentage that these previously unbilled INIT GB-seconds represent compared to the total consumed GB-seconds (current billed + estimated init). This approximates the potential percentage increase in your duration costs due to the change.

In our example, the cost share of the INIT phase represents approximately 14% of the total execution cost (current + potential init cost)! Depending on your current bill, this could be significant.

Understanding and Optimizing Your Lambda

Okay, so you know you might have to pay more. But aren’t there levers we can pull to reduce this increase in your bill?

Here are some tips for making the best use of this INIT phase and thus reducing your future costs.

Strategically Use the INIT Phase

Your first instinct might be to remove all the INIT code (code outside the handler) and put it in your handler. That way, no more INIT billing! But not only would you just be shifting the problem (because executing that code will still be billed), it would be even worse, because now even your ‘warm start’ Lambda executions would have to process this code!

Yes, remember that the code in the INIT phase is executed only during cold starts. Therefore, it’s the ideal place for expensive initialization operations that can be reused by subsequent (“warm start”) invocations. This is even recommended by AWS. Thus, the INIT phase is perfect for:

Importing heavy libraries or dependencies.
Establishing connections to other AWS services (S3, DynamoDB, etc.) via SDKs (boto3 for Python).
Creating database connection pools.
Fetching parameters or secrets from Systems Manager Parameter Store or Secrets Manager.

Maximize this initialization phase to reduce the execution time of Lambdas running in ‘warm start’.

Optimize Package Size

This is often the simplest lever, because sometimes we import all sorts of things into our Lambda. So, make sure to include only strictly necessary dependencies. Also, be sure to exclude anything related to your development environment (I can’t tell you how many times I’ve found a node_modules or tests folder in Lambdas…). Finally, don’t hesitate to consult AWS articles, as some directly tackle the subject of cold starts.

Lambda SnapStart

Available for Java, .NET, and Python runtimes, SnapStart is a very interesting feature for combating cold starts. When you enable it, Lambda takes a “snapshot” of the initialized execution environment after the first INIT phase. For subsequent cold starts, Lambda restores this snapshot instead of redoing the entire INIT phase, saving you significant time.

I imagine reading about this feature, you’re thinking “Wow, this is great, I’m going to enable this feature on all my Lambdas!”. Except there are a few subtleties to be aware of.

Firstly, SnapStart is currently only compatible with specific versions of certain languages (Python 3.12 minimum, NodeJS not supported…). The same goes for regions; only a limited number support this feature at the time of writing. Finally, this feature is not free and incurs its own costs. Make sure to check the potential cost if you want to enable it on your Lambdas.

Provisioned Concurrency

If your application has predictable traffic or if cold start latency is unacceptable, you can use “Provisioned Concurrency”. You ask Lambda to keep a certain number of execution environments pre-initialized (warm) at all times.

Advantage: Requests hitting these provisioned instances never experience a cold start. The INIT phase is done upfront, before the first request.

Disadvantage: You pay for the time these environments are provisioned, whether they receive requests or not. The INIT phase is also billed during this pre-initialization.

Here too, it’s up to you to see if it’s worth enabling this on some of your key Lambdas!

Conclusion

The upcoming change in billing for Lambda’s INIT phase on August 1, 2025, is primarily a standardization. For the majority of Lambdas (specifically those using managed runtimes and ZIP packaging that previously benefited from the free init), this means the duration of this phase will now be added to the billed duration during cold starts.

Even if the financial impact will likely be small for you, it’s an excellent opportunity to:

Understand your Lambda’s lifecycle and what happens during initialization.
Measure the INIT phase duration of your critical functions using CloudWatch tools.
Optimize this phase if necessary, by reducing package sizes, using SnapStart, or considering Provisioned Concurrency for specific use cases.

So, don’t wait until August! Familiarize yourself with CloudWatch Logs Insights now. This will allow you to identify which Lambdas to optimize first and avoid any nasty surprises on your AWS bill.