What it actually is
A Virtual Private Cloud is your own private IP address space inside an AWS Region, plus the software-defined routers, filters and gateways that decide which packets get in, out and across. The mental model the docs bury: a VPC does not "contain" your instances the way a building contains rooms. Every network interface you create is attached to a subnet, and every packet it sends is evaluated, in order, by a security group, a network ACL, and a route table. Nothing else decides connectivity. If you can name which of those three said no, you can fix any VPC problem.
Two facts make the rest easier. First, a subnet lives in exactly one Availability Zone, so "multi-AZ" always means "more than one subnet". Second, "public" and "private" are not properties of a subnet. A subnet is public because its route table sends 0.0.0.0/0 to an internet gateway, and an instance in it is reachable from the internet only if it also has a public IPv4 address (or an Elastic IP) and its security group allows the traffic. That is the whole definition.
The model
The primitives, from the outside in:
VPC: a CIDR block you choose. For IPv4 the allowed range is /16 (65,536 addresses) down to /28 (16 addresses). AWS recommends RFC 1918 ranges such as 10.0.0.0/16.
Subnet: a slice of the VPC CIDR pinned to one AZ. AWS reserves 5 IPv4 addresses in every subnet (network address, VPC router, Amazon DNS, one reserved for future use, and broadcast), so a /28 gives you 11 usable addresses, not 16.
Route table: a list of destination-to-target rules, associated with one or more subnets. The most specific route wins. Every VPC has a main route table that subnets use by default if you do not associate another one.
Internet gateway (IGW): the door to the public internet. One per VPC at a time. It performs the one-to-one mapping between an instance's private address and its public IPv4 address.
NAT gateway: lets instances in private subnets start outbound connections to the internet or to other networks, while refusing inbound connections that originate outside. A public NAT gateway sits in a public subnet, needs an Elastic IP, and its subnet must route to the IGW. A private NAT gateway has no Elastic IP and is for reaching other VPCs or on-premises networks through a transit gateway or virtual private gateway.
Security group: a stateful allow-list attached to a network interface. No deny rules. Return traffic is allowed automatically.
Network ACL: a stateless, numbered allow-and-deny list attached to a subnet. Because it is stateless, you must allow the ephemeral return ports yourself.
Endpoints: gateway endpoints (S3 and DynamoDB) are a route table target and cost nothing; interface endpoints (AWS PrivateLink) are network interfaces in your subnets and are billed.
What AWS operates: the router, the gateways, the mapping layer, the scaling of the NAT gateway. What you operate: the address plan, every route, every filter rule, and the decision about how many AZs to pay for. A VPC gives you no resilience you did not draw. One NAT gateway in one AZ is a single point of failure for every private subnet routed through it.
A recent change worth knowing: AWS added Regional NAT gateways. You create one with aws ec2 create-nat-gateway --vpc-id <vpc> --availability-mode regional, no subnet is needed, and it expands to the AZs where your workloads have network interfaces. The docs say expansion can take up to 60 minutes, during which new-AZ traffic is processed through existing AZs, and that converting from zonal to regional resets existing connections. It does not support private NAT. This article builds the classic zonal version because it is what you will find in most accounts today, and because the cost math below is the same shape for both.
When to use it, when not to
You do not choose whether to use a VPC. Almost every compute service you launch lands in one. The real decisions are what to put around it:
You need | Reach for | Why not the confusable alternative |
|---|---|---|
Instances in private subnets calling the internet | NAT gateway | An IGW alone only serves instances with public IPs |
Private subnets calling S3 or DynamoDB | Gateway endpoint | NAT would bill you per GB for traffic that can be free |
Private subnets calling most other AWS APIs | Interface endpoint (PrivateLink) | Cheaper than NAT only if traffic is high; billed hourly per AZ |
IPv6 outbound only | Egress-only internet gateway | NAT gateway works for IPv6 via DNS64/NAT64, but you rarely need it |
Connecting a handful of VPCs | Peering | Transit gateway is for many-to-many; it adds hourly attachment charges |
Debugging "why can't A reach B" | Reachability Analyzer | Flow logs only tell you what happened to real traffic that already ran |
When not to hand-build one: for a quick experiment, the default VPC exists in every Region and is fine. For production, define the VPC in CloudFormation or CDK so the address plan is reviewable. The boto3 below is a teaching device, not a deployment method.
What it costs
The VPC itself, subnets, route tables, security groups, network ACLs and internet gateways are free. You pay for specific components on the pricing page. Verified from aws.amazon.com/vpc/pricing:
NAT gateway: $0.045 per hour per gateway (the page's examples use US East (Ohio)) plus $0.045 per GB processed. A zonal NAT gateway left running for a 730-hour month is about $32.85 before a single byte moves. That is the line item that surprises people, and it is per AZ if you deploy one per AZ.
Public IPv4 address: $0.005 per hour, for in-use and idle addresses alike. Roughly $3.65 a month each. The NAT gateway's Elastic IP counts.
Gateway endpoints (S3, DynamoDB): no hourly or data processing charge.
Reachability Analyzer: $0.10 per analysis.
Data transfer: out to the internet is $0.09 per GB in the page's example. Cross-AZ traffic inside a Region is billed per GB in each direction; the VPC pricing page cites $0.01 per GB for peering traffic that crosses AZs. Note that traffic through a NAT gateway pays the NAT processing charge and then the transfer charge.
Interface endpoint hourly and per-GB rates are not on the VPC pricing page I fetched, so check the PrivateLink pricing page before you do the break-even math against NAT.
Free tier reality: the free tier does not cover NAT gateways. Public IPv4 addresses had a free-tier allowance on some EC2 usage in the past; do not count on it for a lab and assume everything below bills.
Cost of following along, assuming you finish in one hour: NAT gateway $0.045, two public IPv4 addresses (NAT and the test instance) about $0.01, three analyses $0.30, plus one t4g.nano instance for an hour, whose rate you should look up for your Region. Under half a dollar if you run the cleanup. Forget the NAT gateway for a month and it is about $33.
The limits that bite
Verified from the VPC quotas page (per Region unless noted):
VPCs per Region: 5, adjustable. Raising it also raises the internet gateway quota.
Subnets per VPC: 200. Route tables per VPC: 200 (the main one counts).
IPv4 CIDR blocks per VPC: 5, adjustable up to 50, primary and secondary both counted.
Routes per route table: 500 non-propagated (adjustable up to 1,000), and 100 propagated routes that cannot be raised. If you learn about that last one from a BGP session, advertise a default route instead.
Security groups per network interface: 5 (up to 16), and rules per security group: 60 inbound and 60 outbound. The page states that the two quotas multiplied cannot exceed 1,000, so raising one constrains the other.
NAT gateways per AZ: 5. Only the
pending,activeanddeletingstates count.Elastic IP addresses per Region: 5. This is the one a NAT-per-AZ design hits first.
Network interfaces per Region: 5,000, enforced per AZ.
DNS queries to the Route 53 Resolver: 1,024 packets per second per network interface, to the .2 address. Hard limit.
A limit that is not a quota: a VPC CIDR cannot be shrunk or replaced once things live in it, you can only add secondary blocks. Overlapping CIDRs cannot be peered or joined to the same transit gateway routing domain cleanly. Choose the address plan as if you will connect to every other network you own, because you will.
Build it
We build a two-tier VPC in two AZs: public subnets with a test instance, private subnets with a NAT gateway for outbound traffic, a free S3 gateway endpoint, and then use Reachability Analyzer to prove the path, break it, and fix it. The code follows the AWS documentation for each API; I did not execute it for this issue, so read the output of each step before moving on.
Prerequisites: Python 3.10+, boto3, AWS credentials for a sandbox account, and a Region with at least two AZs. The IAM actions below are the EC2 APIs this script calls. The Reachability Analyzer pages I reviewed do not list the exact IAM actions the service needs, so for the lab use a sandbox role with broad EC2 permissions and tighten it afterward against the service authorization reference.
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": [
"ec2:CreateVpc", "ec2:DeleteVpc", "ec2:ModifyVpcAttribute", "ec2:DescribeVpcs",
"ec2:CreateSubnet", "ec2:DeleteSubnet", "ec2:DescribeSubnets",
"ec2:CreateInternetGateway", "ec2:AttachInternetGateway",
"ec2:DetachInternetGateway", "ec2:DeleteInternetGateway",
"ec2:CreateRouteTable", "ec2:CreateRoute", "ec2:AssociateRouteTable",
"ec2:DeleteRouteTable", "ec2:DescribeRouteTables",
"ec2:CreateNatGateway", "ec2:DeleteNatGateway", "ec2:DescribeNatGateways",
"ec2:AllocateAddress", "ec2:ReleaseAddress",
"ec2:CreateVpcEndpoint", "ec2:DeleteVpcEndpoints",
"ec2:CreateSecurityGroup", "ec2:DeleteSecurityGroup",
"ec2:AuthorizeSecurityGroupIngress", "ec2:DescribeSecurityGroups",
"ec2:RunInstances", "ec2:TerminateInstances", "ec2:DescribeInstances",
"ec2:CreateNetworkInsightsPath", "ec2:StartNetworkInsightsAnalysis",
"ec2:DescribeNetworkInsightsAnalyses", "ec2:DeleteNetworkInsightsAnalysis",
"ec2:DeleteNetworkInsightsPath", "ec2:CreateTags",
"ssm:GetParameter"
],
"Resource": "*"
}]
}
Step 1: the network. Save as vpc_lab.py.
import boto3, time, json
REGION = "us-east-1"
ec2 = boto3.client("ec2", region_name=REGION)
ssm = boto3.client("ssm", region_name=REGION)
state = {}
def tag(rid, name):
ec2.create_tags(Resources=[rid], Tags=[{"Key": "Name", "Value": f"vpclab-{name}"},
{"Key": "lab", "Value": "sl141"}])
azs = [a["ZoneName"] for a in ec2.describe_availability_zones(
Filters=[{"Name": "state", "Values": ["available"]}])["AvailabilityZones"]][:2]
vpc = ec2.create_vpc(CidrBlock="10.20.0.0/16")["Vpc"]["VpcId"]
state["vpc"] = vpc
tag(vpc, "vpc")
ec2.modify_vpc_attribute(VpcId=vpc, EnableDnsHostnames={"Value": True})
subnets = {}
for i, az in enumerate(azs):
for tier, base in (("public", i), ("private", i + 10)):
sid = ec2.create_subnet(VpcId=vpc, AvailabilityZone=az,
CidrBlock=f"10.20.{base}.0/24")["Subnet"]["SubnetId"]
tag(sid, f"{tier}-{az}")
subnets[(tier, az)] = sid
state["subnets"] = {f"{t}:{a}": s for (t, a), s in subnets.items()}
igw = ec2.create_internet_gateway()["InternetGateway"]["InternetGatewayId"]
ec2.attach_internet_gateway(InternetGatewayId=igw, VpcId=vpc)
tag(igw, "igw")
state["igw"] = igw
# Public route table: 0.0.0.0/0 -> IGW. This route is what makes a subnet "public".
pub_rt = ec2.create_route_table(VpcId=vpc)["RouteTable"]["RouteTableId"]
tag(pub_rt, "public-rt")
ec2.create_route(RouteTableId=pub_rt, DestinationCidrBlock="0.0.0.0/0", GatewayId=igw)
for (tier, az), sid in subnets.items():
if tier == "public":
ec2.associate_route_table(RouteTableId=pub_rt, SubnetId=sid)
state["public_rt"] = pub_rt
Step 2: NAT gateway, private route tables and the free S3 endpoint. One NAT in the first AZ keeps the lab cheap; the comment says what production looks like.
eip = ec2.allocate_address(Domain="vpc")["AllocationId"]
state["eip"] = eip
nat = ec2.create_nat_gateway(SubnetId=subnets[("public", azs[0])],
AllocationId=eip)["NatGateway"]["NatGatewayId"]
state["nat"] = nat
ec2.get_waiter("nat_gateway_available").wait(NatGatewayIds=[nat])
# Production: one NAT per AZ, each private route table pointing at its own AZ's NAT.
priv_rt = ec2.create_route_table(VpcId=vpc)["RouteTable"]["RouteTableId"]
tag(priv_rt, "private-rt")
ec2.create_route(RouteTableId=priv_rt, DestinationCidrBlock="0.0.0.0/0", NatGatewayId=nat)
for (tier, az), sid in subnets.items():
if tier == "private":
ec2.associate_route_table(RouteTableId=priv_rt, SubnetId=sid)
state["private_rt"] = priv_rt
# Gateway endpoint: adds a prefix-list route to the private table. No hourly or per-GB charge.
ep = ec2.create_vpc_endpoint(VpcId=vpc, ServiceName=f"com.amazonaws.{REGION}.s3",
VpcEndpointType="Gateway",
RouteTableIds=[priv_rt])["VpcEndpoint"]["VpcEndpointId"]
state["s3_endpoint"] = ep
Step 3: a test instance in a public subnet, with a security group that starts closed.
sg = ec2.create_security_group(GroupName="vpclab-web", Description="sl141 lab", VpcId=vpc)["GroupId"]
state["sg"] = sg
ami = ssm.get_parameter(
Name="/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-arm64"
)["Parameter"]["Value"]
inst = ec2.run_instances(
ImageId=ami, InstanceType="t4g.nano", MinCount=1, MaxCount=1,
NetworkInterfaces=[{"DeviceIndex": 0, "SubnetId": subnets[("public", azs[0])],
"AssociatePublicIpAddress": True, "Groups": [sg]}],
TagSpecifications=[{"ResourceType": "instance",
"Tags": [{"Key": "lab", "Value": "sl141"}]}],
)["Instances"][0]["InstanceId"]
state["instance"] = inst
ec2.get_waiter("instance_running").wait(InstanceIds=[inst])
json.dump(state, open("vpc_lab_state.json", "w"), indent=2)
print(json.dumps(state, indent=2))
Step 4: trace the path with Reachability Analyzer. The source is the internet gateway, the destination is the instance, TCP 443. The security group has no inbound rules yet, so the analysis should say no, and say which component said it.
def analyze(label):
path = ec2.create_network_insights_path(
Source=state["igw"], Destination=state["instance"], Protocol="tcp",
FilterAtSource={"DestinationPortRange": {"FromPort": 443, "ToPort": 443}},
)["NetworkInsightsPath"]["NetworkInsightsPathId"]
aid = ec2.start_network_insights_analysis(
NetworkInsightsPathId=path)["NetworkInsightsAnalysis"]["NetworkInsightsAnalysisId"]
while True:
a = ec2.describe_network_insights_analyses(
NetworkInsightsAnalysisIds=[aid])["NetworkInsightsAnalyses"][0]
if a["Status"] != "running":
break
time.sleep(5)
print(label, a["Status"], "found:", a.get("NetworkPathFound"))
for e in a.get("Explanations", []):
print(" ", e.get("ExplanationCode"), e.get("Direction"), e.get("Port"))
return path, aid
before = analyze("closed SG")
Verification, part one: expect found: False with an explanation code ENI_SG_RULES_MISMATCH, which is what the AWS getting-started guide shows for a security group that does not allow the traffic. That is Reachability Analyzer naming the filter, with no packet sent.
Step 5: fix it, analyze again.
ec2.authorize_security_group_ingress(GroupId=state["sg"], IpPermissions=[{
"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443,
"IpRanges": [{"CidrIp": "0.0.0.0/0", "Description": "lab only"}]}])
after = analyze("open SG")
Verification, part two: expect found: True. The analysis response carries ForwardPathComponents (igw, then the network ACL, security group and so on, each with a sequence number) and ReturnPathComponents. Print them with json.dumps and read the hops. Then break it differently: delete the 0.0.0.0/0 route from the public route table with ec2.delete_route(RouteTableId=state["public_rt"], DestinationCidrBlock="0.0.0.0/0") and analyze again. The explanation will now point at the route table, not the security group. Same destination, different component named. That is the habit to build: when connectivity fails, stop guessing and ask which of the three filters said no.
Verification of the private tier: from the console or CLI, aws ec2 describe-route-tables --route-table-ids <private_rt> should show a 0.0.0.0/0 route to your NAT gateway and a route to the S3 prefix list through vpce-.... The most specific route wins, so S3 traffic takes the free endpoint and everything else takes the metered NAT.
When it breaks
Instance has a public IP but is unreachable: three suspects in order. Subnet route table has no 0.0.0.0/0 to the IGW; security group has no inbound rule; network ACL denies or forgets ephemeral return ports. Run Reachability Analyzer before guessing.
Private instance cannot reach the internet: its route table has no default route to a NAT gateway, the NAT gateway sits in a subnet whose route table lacks the IGW route, or the NAT gateway is in a different AZ that is having a bad day.
NatGatewayLimitExceeded/AddressLimitExceeded: you hit 5 NAT gateways per AZ or 5 Elastic IPs per Region. Request an increase in Service Quotas. Elastic IPs are in theec2service code, NAT gateways invpc.InvalidSubnet.ConflictorInvalidVpc.Range: CIDR overlaps an existing subnet, or is outside the /16 to /28 range.DependencyViolationon delete: something still depends on the object. Delete order matters; see cleanup.A surprise NAT bill: S3 or DynamoDB traffic going through NAT because no gateway endpoint is in the route table, or cross-AZ traffic hitting a single NAT. Compare the NAT's processed bytes in CloudWatch against what you expected.
Cleanup
Order matters because of dependencies. Run this in the same folder as the state file.
import json, time, boto3
state = json.load(open("vpc_lab_state.json"))
ec2 = boto3.client("ec2", region_name="us-east-1")
ec2.terminate_instances(InstanceIds=[state["instance"]])
ec2.get_waiter("instance_terminated").wait(InstanceIds=[state["instance"]])
ec2.delete_nat_gateway(NatGatewayId=state["nat"])
ec2.get_waiter("nat_gateway_deleted").wait(NatGatewayIds=[state["nat"]])
ec2.release_address(AllocationId=state["eip"])
ec2.delete_vpc_endpoints(VpcEndpointIds=[state["s3_endpoint"]])
time.sleep(20) # endpoint route removal is asynchronous
for rt in (state["public_rt"], state["private_rt"]):
for assoc in ec2.describe_route_tables(RouteTableIds=[rt])["RouteTables"][0]["Associations"]:
if not assoc["Main"]:
ec2.disassociate_route_table(AssociationId=assoc["RouteTableAssociationId"])
ec2.delete_route_table(RouteTableId=rt)
for sid in state["subnets"].values():
ec2.delete_subnet(SubnetId=sid)
ec2.detach_internet_gateway(InternetGatewayId=state["igw"], VpcId=state["vpc"])
ec2.delete_internet_gateway(InternetGatewayId=state["igw"])
ec2.delete_security_group(GroupId=state["sg"])
ec2.delete_vpc(VpcId=state["vpc"])
Also delete the Reachability Analyzer analyses and paths you created: aws ec2 delete-network-insights-analysis --network-insights-analysis-id <nia-...> first, then aws ec2 delete-network-insights-path --network-insights-path-id <nip-...>. The docs say undeleted analyses are removed automatically 120 days after creation, and a path cannot be deleted while it still has analyses.
Confirm the bill is zero: aws ec2 describe-nat-gateways --filter Name=state,Values=available,pending and aws ec2 describe-addresses should both be empty for the lab, and Cost Explorer filtered on the lab=sl141 tag should flatline. The two items that keep billing if you skip steps are the NAT gateway and any Elastic IP you did not release.
The one thing to remember
A VPC is three filters and a route table. Free to define, expensive in exactly two places: the NAT gateway you left running and the traffic you sent through it when a free endpoint would have done. Draw the route tables before you draw the boxes, and when something fails, ask the tool which component said no instead of reasoning from memory.
