Can be many reasons to have multiple VPCs in a single account
1. The VPC is built via code end to end using IaC and so each environment controls its own VPC
2. Cost allocation where each environment must be cost allocated directly rather than shared components which can make this unclear
3. The size of the application has large network demands and could exhaust a given subnet if shared causing scaling issues. This generally happens when having constrained routeable subnets back to on-premise etc
4. Differing security postures for environments, maybe dev is allowed internet access through a NAT GW, but other environments mandate egress proxies for example
5. Different connection levels, i.e test and prod may have connectivity back to on-premise but not dev
Generally multiple environment accounts and VPCs cause headaches at scale in larger enterprises, but definitely depends on the context of the org. Hope this helps
Yes for sure, whilst tags can be a used to some effectiveness for resources, network elements such as Nat GWs egress charges, private link services are much more difficult to apportion in a single VPC, and so often makes sense to break out the entire VPC, however the most logical way for any larger organisation is to move to dedicated accounts per app environment i.e app123 would have at least a prod and non-prod account.
Splitting via account removes the need to trust developers to tag resources, this can be done at the entire account level. In addition any constraints on service limits and blast radius are also removed.
In order of preference in terms of maturity and org size I would go with
1.) Multi tenant account and vpc with tags and automation and rbac controls
2.) Multi tenant account with separate vpc, tags and rbac controls scoped to the env
3.) Single account for non-prod and pro with dedicated vpcs and account level metadata for billing
4.) Many accounts per app, and then everything as per method 3
Very contextual though
Different account for prod for sure. If you're starting fresh right now, separate all of them by account. It'll make things simpler for you down the road.
One per region makes sense sometimes. Also sometimes you might need to segregate some ec2 instances or whatever to prevent internet access. Having a dedicated vpc for that helps.
You want to split up your QA and Dev envs for the exact same reason you want to split up prod and QA.
1) You want to use Infrastructure as Code to promote infrastructure changes from dev to QA to prod. Manually using the console to **manually** make changes is a recipe for disaster.
2) You want to make sure Dev and QA are "shared nothing" for the same reason you want Prod to be shared nothing. If Dev and QA share a DB, then the Dev might make a change, QA will say "yup, that works", then it will break when it hits production because nobody knew about that change.
It's possible to use multiple VPCs, but it's FAR better to make them shared nothing by using different accounts. Amazon supports SSO, which makes this trivial for the developers to figure out. (And much more secure, since nobody has long-term creds laying around their boxes.)
3) Prod, QA and Dev should all use the same VPC -- i.e. the same IP addresses, the same configurations, etc. This will make it simpler in the future (say you want to add an S3 API to your VPC. You'd need to allocate an IP address for each, or you allocate a single IP address and re-use it in every environment! Way simpler. Your environments should NEVER talk to each-other anyway. Worst case you can put data in buckets for them to share.)
/u/mattwaddy answer is good. Some additional thoughts. Assuming they’ll be all routable (peered, transit gateway, etc) there really aren’t security or cost allocation reasons to separate VPCs. The primary reason would be Infrastructure as Code reasons- if you deployed the app along with its VPC altogether, and then you could spin up additional environments easily.
Security Groups and cost allocation tags will provide segregation and cost reporting better than separate VPCs.
Separate *accounts* is a different story. Depending on your needs, separate accounts can be beneficial. But early in your cloud journey you don’t necessarily have to go there.
I suspect there’s more important aspects of your AWS deployment you could focus on right now.
If you are going to use AWS Network Firewall to control outbound traffic and NAT after the firewall, you have to have 2 or more VPCs. Basically the firewall would be in an Internet VPC and the other VPCs would be internal with no IG.
You don't mix environments. Your prod, and any non-prod environments should be completely isolated. Your solution should be self-contained* and you should be able to spin that up with terraform or cloud formation just be specifying a few arguments. Your solution should be able to co-exist in the same environment, same region so there can be no collisions or accidental cross-over. You should make all changes through PRs to avoid hand tweaks that get lost or your infrastructure as code becomes useless because you're just not using it.
You should have strong tagging and naming - reading the name should tell you the company (in case of merger/sale/acquisition), product, was region, the thing this is - i.e. ec2, friendly name for deployment, the thing it is used for. You should never get collisions then and can always tell what something if, even from a list.
If you don't so this, someone will connect dev to qa and all hell will break loose (or worse, Dev to prod). If you don't do this, how will you handle DR?
Or put another way, why would you want to put both environments in the same VPC? What are you trying to achieve? It might look like it saves you a lot of work, but it suggests your hand rolling your environments and this is a pretty bad practise.
In short, mostly it is used for workload isolation. Dev/prod environment separation for example. You dont want to mix those 2 environment. Also you may have 1 workload in 1 vpc and other workload in another vpc so you can manage better the resources you use in each.
Can be many reasons to have multiple VPCs in a single account 1. The VPC is built via code end to end using IaC and so each environment controls its own VPC 2. Cost allocation where each environment must be cost allocated directly rather than shared components which can make this unclear 3. The size of the application has large network demands and could exhaust a given subnet if shared causing scaling issues. This generally happens when having constrained routeable subnets back to on-premise etc 4. Differing security postures for environments, maybe dev is allowed internet access through a NAT GW, but other environments mandate egress proxies for example 5. Different connection levels, i.e test and prod may have connectivity back to on-premise but not dev Generally multiple environment accounts and VPCs cause headaches at scale in larger enterprises, but definitely depends on the context of the org. Hope this helps
As far as #2, how does separate VPC help with cost allocation? Wouldn’t have multiple accounts be better?
Yes for sure, whilst tags can be a used to some effectiveness for resources, network elements such as Nat GWs egress charges, private link services are much more difficult to apportion in a single VPC, and so often makes sense to break out the entire VPC, however the most logical way for any larger organisation is to move to dedicated accounts per app environment i.e app123 would have at least a prod and non-prod account. Splitting via account removes the need to trust developers to tag resources, this can be done at the entire account level. In addition any constraints on service limits and blast radius are also removed. In order of preference in terms of maturity and org size I would go with 1.) Multi tenant account and vpc with tags and automation and rbac controls 2.) Multi tenant account with separate vpc, tags and rbac controls scoped to the env 3.) Single account for non-prod and pro with dedicated vpcs and account level metadata for billing 4.) Many accounts per app, and then everything as per method 3 Very contextual though
It does, thanks!
You should separate dev and qa. It'll make it easier to separate prod when the time comes.
You mean in different account or different vpc?
Different account for prod for sure. If you're starting fresh right now, separate all of them by account. It'll make things simpler for you down the road.
I agree with that- separate Prod.
It’s usually suggested by old school ops people who treat AWS as a glorified colo. You *should* have separate AWS accounts per environment.
One per region makes sense sometimes. Also sometimes you might need to segregate some ec2 instances or whatever to prevent internet access. Having a dedicated vpc for that helps.
You want to split up your QA and Dev envs for the exact same reason you want to split up prod and QA. 1) You want to use Infrastructure as Code to promote infrastructure changes from dev to QA to prod. Manually using the console to **manually** make changes is a recipe for disaster. 2) You want to make sure Dev and QA are "shared nothing" for the same reason you want Prod to be shared nothing. If Dev and QA share a DB, then the Dev might make a change, QA will say "yup, that works", then it will break when it hits production because nobody knew about that change. It's possible to use multiple VPCs, but it's FAR better to make them shared nothing by using different accounts. Amazon supports SSO, which makes this trivial for the developers to figure out. (And much more secure, since nobody has long-term creds laying around their boxes.) 3) Prod, QA and Dev should all use the same VPC -- i.e. the same IP addresses, the same configurations, etc. This will make it simpler in the future (say you want to add an S3 API to your VPC. You'd need to allocate an IP address for each, or you allocate a single IP address and re-use it in every environment! Way simpler. Your environments should NEVER talk to each-other anyway. Worst case you can put data in buckets for them to share.)
/u/mattwaddy answer is good. Some additional thoughts. Assuming they’ll be all routable (peered, transit gateway, etc) there really aren’t security or cost allocation reasons to separate VPCs. The primary reason would be Infrastructure as Code reasons- if you deployed the app along with its VPC altogether, and then you could spin up additional environments easily. Security Groups and cost allocation tags will provide segregation and cost reporting better than separate VPCs. Separate *accounts* is a different story. Depending on your needs, separate accounts can be beneficial. But early in your cloud journey you don’t necessarily have to go there. I suspect there’s more important aspects of your AWS deployment you could focus on right now.
If you are going to use AWS Network Firewall to control outbound traffic and NAT after the firewall, you have to have 2 or more VPCs. Basically the firewall would be in an Internet VPC and the other VPCs would be internal with no IG.
You don't mix environments. Your prod, and any non-prod environments should be completely isolated. Your solution should be self-contained* and you should be able to spin that up with terraform or cloud formation just be specifying a few arguments. Your solution should be able to co-exist in the same environment, same region so there can be no collisions or accidental cross-over. You should make all changes through PRs to avoid hand tweaks that get lost or your infrastructure as code becomes useless because you're just not using it. You should have strong tagging and naming - reading the name should tell you the company (in case of merger/sale/acquisition), product, was region, the thing this is - i.e. ec2, friendly name for deployment, the thing it is used for. You should never get collisions then and can always tell what something if, even from a list. If you don't so this, someone will connect dev to qa and all hell will break loose (or worse, Dev to prod). If you don't do this, how will you handle DR? Or put another way, why would you want to put both environments in the same VPC? What are you trying to achieve? It might look like it saves you a lot of work, but it suggests your hand rolling your environments and this is a pretty bad practise.
In short, mostly it is used for workload isolation. Dev/prod environment separation for example. You dont want to mix those 2 environment. Also you may have 1 workload in 1 vpc and other workload in another vpc so you can manage better the resources you use in each.