Search
  • Jeff Loughridge

Outbound Internet Access via Web Proxy and AWS VPC Peering

AWS provides the mechanisms to create VPC designs that run the gamut of the complexity spectrum. You can deploy your application in a single VPC with only public subnets. Other applications may be better suited to a collection of VPCs with both public and private subnets. You can use VPCs as building blocks for large-scale distributed applications, a design decision that requires VPCs to communicate to one another. In this post, I’ll examine a VPC design pattern in which a VPC is dedicated for providing internet access using VPC peering with a web proxy.


Consider the scenario in which you maintain applications in separate VPCs. The applications require outbound access to the Internet and do not offer any Internet-facing services. When the number of VPCs is many, centralizing the egress point for all traffic to the Internet may help an organization meet specialized security requirements. The centralized point for policy enforcement can be implemented with a pair of security appliances in a VPC dedicated to providing egress to the Internet. In a networking chalk talk at re:Invent 2018, an AWS engineer referred to this design pattern as an outbound services VPC. The use of web proxies is one implementation of an outbound services VPC.


Why centralize outbound Internet access? From a security perspective, the company might want to perform URL filtering, IDS/IPS, or application-layer filtering. By centralizing outbound access, the company’s security team can manage policy in a smaller number of places. Security appliances are not needed in every application VPC, which has benefits in terms of cost and decoupling the application lifecycle from the outbound services VPC.


The applications are instantiated in other dedicated VPCs. Traffic from these VPCs must traverse the outbound services VPC to reach the Internet. Recall that transitive traffic in the VPC is prohibited: traffic that is sourced in one VPC and directed toward a second VPC must terminate in the second VPC. For this reason, you can’t rely on VPC peering alone for enabling applications VPCs to traverse the outbound services VPCs. The traffic would be dropped.


By adding a pair of proxy servers in the outbound services VPC, the limitation on the non-transitive nature of VPCs can be worked around. The servers in the application VPCs that require outbound access are configured to use the proxies in the outbound services VPC. This terminates the TCP connection in the outbound services VPC and creates a new TCP session for the proxy to Internet web server connection.


VPC peering is ideal for inter-VPC connectivity for a host of reasons.

  • VPC peering is highly available service provided as a native VPC component

  • VPC peering traffic uses the private AWS network backbone

  • VPC peering can be established between AWS regions

  • VPC peering traffic is encrypted when established between regions

  • Network throughput with VPC peering is higher than you’ll experience with AWS Managed VPN


Let’s examine a diagram of this design.



The diagram depicts EC2 servers in application VPCs. These instances must be configured to use the web proxies in the outbound services VPC. The outbound service VPC can be a simple as two web proxies running on EC2 instances. A more robust implementation would use a proxy farm fronted by an ELB.


Let’s check out how we can use a squid proxy in the outbound services VPC to proxy connections from an application VPC.


Here is the outbound VPC template.



AWSTemplateFormatVersion: 2010-09-09
Description: >-
    AWS CloudFormation template to create an Outbound VPC with a proxy for the App VPCs
Parameters:
  KeyName:
    Description: Name of an existing EC2 KeyPair to enable SSH access to the instances
    Type: String
  SourceIpCIDR:
    Description: CIDR Range allowed to SSH to Squid Proxy
    Type: String
    MinLength: '9'
    MaxLength: '18'
    Default: 0.0.0.0/0
    AllowedPattern: (\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})/(\d{1,2})
    ConstraintDescription: Must use a valid IP CIDR range using slash notation (e.g., x.x.x.x/y)
  AmazonLinuxAMI:
    Type: 'AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>'
    Default: '/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2'
Resources:
  OutboundVpc:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 172.16.0.0/24
      EnableDnsSupport: 'true'
      EnableDnsHostnames: 'true'
      Tags:
      - Key: Name
        Value: !Sub '${AWS::StackName}-internet-access'
  IGW:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: !Sub '${AWS::StackName}-internet-access'
  IgwAttachment:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref OutboundVpc
      InternetGatewayId: !Ref IGW
  OutboundPublicSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      Tags:
        - Key: Name
          Value: !Sub '${AWS::StackName}-output'
      VpcId: !Ref OutboundVpc
      AvailabilityZone: !Sub ${AWS::Region}a
      MapPublicIpOnLaunch: false
      CidrBlock: 172.16.0.0/24
  InternetRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId:
        !Ref OutboundVpc
      Tags:
      - Key: Name
        Value: !Sub '${AWS::StackName}-internet-access'
  InternetDefaultRoute:
    Type: AWS::EC2::Route
    DependsOn:
      - IGW
      - IgwAttachment
    Properties:
      RouteTableId: !Ref InternetRouteTable
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref IGW
  InternetRouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId:
        !Ref OutboundPublicSubnet
      RouteTableId:
        !Ref InternetRouteTable
  OutboundVpcSecurityGroup:
    Type: 'AWS::EC2::SecurityGroup'
    Description: SG to permit all traffic
    Properties:
      Tags:
        - Key: Name
          Value: !Sub '${AWS::StackName}-outbound'
      GroupName: !Sub '${AWS::StackName}-outbound'
      GroupDescription: Allow all traffic
      VpcId: !Ref OutboundVpc
      SecurityGroupIngress:
      - IpProtocol: tcp
        CidrIp: !Ref SourceIpCIDR
        FromPort: 22
        ToPort: 22
      - IpProtocol: tcp
        CidrIp: 172.17.0.0/16
        FromPort: 3128
        ToPort: 3128
  SquidEC2Instance:
    Type: 'AWS::EC2::Instance'
    Properties:
      Tags:
        - Key: Name
          Value: !Sub '${AWS::StackName}-squid'
      KeyName: !Ref KeyName
      ImageId: !Ref AmazonLinuxAMI
      InstanceType: t3.nano
      NetworkInterfaces:
        - AssociatePublicIpAddress: true
          DeviceIndex: 0
          PrivateIpAddress: 172.16.0.100
          GroupSet: [ !Ref OutboundVpcSecurityGroup ]
          SubnetId: !Ref OutboundPublicSubnet
      UserData:
        Fn::Base64: |
          #!/bin/bash -xe
          yum install -y squid
          systemctl start squid.service
          systemctl enable squid.service
Outputs:
  SquidProxyDNS:
    Description: DNS name for squid proxy
    Value: !GetAtt SquidEC2Instance.PublicDnsName
  InternetRouteTable:
    Description: Internet Route Table ID
    Value: !Ref InternetRouteTable
    Export:
      Name: !Sub ${AWS::StackName}-InternetRouteTable
  OutboundVpc:
    Description: VPC ID of Outbound VPC
    Value: !Ref OutboundVpc
    Export:
      Name: !Sub ${AWS::StackName}-OutboundVpc

The next template creates one application VPC with an EC2 instance for testing.


AWSTemplateFormatVersion: 2010-09-09
Description: >-
    AWS CloudFormation template to create an App VPC that uses a proxy in the Outbound VPC
    for HTTP/HTTPS Internet access.
Parameters:
  KeyName:
    Description: Name of an existing EC2 KeyPair to enable SSH access to the instances
    Type: String
  OutboundVpcStack:
    Description: Name of the Cloudformation stack used to create the Outbound VPC
    Type: String
  AppVpcCIDR:
    Description: CIDR Range for App VPC
    Type: String
    Default: 172.17.0.0/24
Mappings:
  UbuntuRegionMap:
    us-east-1:
      Ubuntu18AMI: ami-0ac019f4fcb7cb7e6
    us-west-1:
      Ubuntu18AMI: ami-063aa838bd7631e0b
Resources:
  AppVPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref AppVpcCIDR
      EnableDnsSupport: 'true'
      EnableDnsHostnames: 'true'
      Tags:
      - Key: Name
        Value: !Sub '${AWS::StackName}-app'
  AppPrivateRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref AppVPC
      Tags:
      - Key: Name
        Value: !Sub '${AWS::StackName}-app'
  AppPrivateSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      Tags:
        - Key: Name
          Value: !Sub '${AWS::StackName}-app'
      VpcId: !Ref AppVPC
      AvailabilityZone: !Sub ${AWS::Region}a
      CidrBlock: !Ref AppVpcCIDR
      MapPublicIpOnLaunch: false
  AppVpcSecurityGroup:
    Type: 'AWS::EC2::SecurityGroup'
    Description: SG to permit SSH for management
    Properties:
      Tags:
        - Key: Name
          Value: !Sub '${AWS::StackName}-app'
      GroupName: !Sub '${AWS::StackName}-app'
      GroupDescription: Allow traffic from Outbound Services VPC
      VpcId: !Ref AppVPC
      SecurityGroupIngress:
      - IpProtocol: tcp
        CidrIp: 172.16.0.0/24
        FromPort: 22
        ToPort: 22
  AppPrivateRouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId:
        !Ref AppPrivateSubnet
      RouteTableId:
        !Ref AppPrivateRouteTable
  AppEC2Instance:
    Type: 'AWS::EC2::Instance'
    Properties:
      Tags:
        - Key: Name
          Value: !Sub '${AWS::StackName}-app'
      KeyName: !Ref KeyName
      ImageId: !FindInMap [ UbuntuRegionMap, !Ref 'AWS::Region', Ubuntu18AMI ]
      InstanceType: t3.nano
      SecurityGroupIds:
        - !Ref AppVpcSecurityGroup
      SubnetId: !Ref AppPrivateSubnet
      UserData:
        Fn::Base64: |
          #!/bin/bash -xe
          cat << EOF > /etc/apt/apt.conf.d/95proxies
          Acquire::http::proxy "http://172.16.0.100:3128/";
          Acquire::https::proxy "https://172.16.0.100:3128/";
          EOF
          cat << EOF >> /etc/environment
          http_proxy="http://172.16.0.100:3128/"
          https_proxy="http://172.16.0.100:3128/"
          EOF
  VPCPeeringConnection:
    Type: AWS::EC2::VPCPeeringConnection
    Properties:
      VpcId: !Ref AppVPC
      PeerVpcId:
        Fn::ImportValue: !Sub '${OutboundVpcStack}-OutboundVpc'
  AppDefaultRoute:
    Type: AWS::EC2::Route
    Properties:
      RouteTableId: !Ref AppPrivateRouteTable
      DestinationCidrBlock: 0.0.0.0/0
      VpcPeeringConnectionId: !Ref VPCPeeringConnection
  OutboundVPCRoutetoAppVPC:
    Type: AWS::EC2::Route
    Properties:
      RouteTableId:
        Fn::ImportValue: !Sub ${OutboundVpcStack}-InternetRouteTable
      DestinationCidrBlock: !Ref AppVpcCIDR
      VpcPeeringConnectionId: !Ref VPCPeeringConnection

First launch the outbound services VPC stack.

aws cloudformation create-stack --stack-name outbound-vpc  --template-body file://outbound_vpc_proxy.yml --parameters  ParameterKey=KeyName,ParameterValue=YourSSHKeyHere --region us-east-1

Now launch a template for a single application VPC using the name of outbound services VPC stack so that the peer VPC ID can be imported.

aws cloudformation create-stack --stack-name app1 --template-body  file://app_vpc_proxy_access.yml --parameters  ParameterKey=OutboundVpcStack,ParameterValue=outbound-vpc --region us-east-1 ParameterKey=KeyName,ParameterValue=YourSSHKeyHere

To verify that the application VPC uses the proxy to reach Internet web servers, ssh into the squid proxy as ec2-user. Next ssh to the Ubuntu EC2 instance using the “ubuntu” username. Type ‘curl https://konekti.us’ and you will retrieve the contents of the Konekti landing page.


I’ll note several things about the templates.

  • AWS SSM can be used to locate the latest Amazon Linux AMI. See this blog post for more details.

  • For serving as a basic web proxy, no additional configuration for squid on Amazon Linux 2 is required.

  • Additional application VPCs can be deployed by overriding the default AppVpcCIDR parameter to use unique CIDR ranges.

The VPC design pattern described in this post is very helpful for companies that require strict security controls. If you have any questions, please contact Konekti using this form.

201 views

© 2020 by Konekti

  • LinkedIn
  • Twitter