AWS re:Invent 2017

I am not exactly Mr. Relevant on this topic–AWS re:Invent happened over a month ago in Las Vegas. I took a whole bunch of notes and spewed them into a summary as soon as I got back. I delivered a presentation to my coworkers and moved on. But as we roll into 2018, I am still thinking about what AWS is and what they have to offer businesses small and large. When I left the conference I was definitely high on what AWS had to offer. And a month later I still am. There are business problems to solve. There are conversations about what cloud native means. There are discussions about migrations and/or building greenfield. There are strategies that businesses are figuring out.

As a grizzled old network engineer, I find this world fascinating. I find these potential business transformation topics fascinating.

I’ve done a decent job of making myself uncomfortable in 2017. I changed jobs. I went to a devops conference. I focused on cloud and automation more. Now I went to a cloud conference. This was a different venue for me. There was a tangible difference in the feel of the conference. Much more talk aimed at developers and business outcomes rather than just focusing on widgets on a new platform.

It could be that because cloud technologies are newer to me, everything felt fresher and more forward-thinking than the other vendor conferences I have been to. Maybe I don’t have the battle scars, which enable some healthy skepticism yet, but I feel like diving into this universe of cloud technologies is reinvigorating my passion for the IT industry.

Another difference that struck me was the partner ecosystem. Walking around the Expo floor, I was plenty of the old guard companies–Cisco, NetApp, Palo Alto, etc. But I saw way more companies that I have never heard of before. These companies are cloud-focused, cloud-born and are solving cloud-problems and filling cloud-gaps for customers. I think this speaks to the different customers of cloud and the different pain points of cloud as compared to traditional data center operations.

I heard the phrase “undifferentiated heavy lifting” several times over the week-long conference. The first couple times it didn’t register, but I came to get it. What I came to realize is that AWS is just different than the traditional IT industry I have been a part of for the last 20 or so years. The effort is in a different place. For example, we may have a cool idea to implement some hot new technology in the data center. We spend months researching testing and then during some middle of the night maintenance window–we hope and pray that our solution is going to work. But usually, that infrastructure change is not noticable to anyone except us in IT operations. AWS does that month-long planning and deployment of your infrastructure for you. The part that you spent a lot of time and effort on that was several layers below adding value to the customer is the starting point for AWS. You build on top of the heavy lifting AWS has done for you.

Andy Jassey, CEO of AWS, talked about the culture of builders. The people that use AWS are building cool technology on top of AWS. The goal of AWS is to eliminate that “undifferentiated heavy lifting” from the workforce and enable to builders to focus on building.

During the keynote, Mark Okerstrom, the CEO of Expedia came on stage to talk about how they are utilizing AWS. He said, “AWS is not just a data center replacement, AWS has services that make companies better.” That quote resonated greatly with me. We are not talking about simply moving existing workloads to AWS. To take advantage of everyting a cloud solution offers, we need to understand those advantages. We need to not be afraid of scalability, of automating cloud infrastructure stacks, and of putting effort into monitoring security and performance. We need to think cloud native.

After a week in the desert, immersed in AWS’s version of the cloud, I really have come to believe AWS has differentiated itself. I am sure I will discover some cynicism about some of their services eventually, but right now I am buzzing from the possibilities.

They announced quite a few new and/or improved services over the course of the conference. Here are a few that really interested me:

  • DynamoDB Global Tables – This allows this AWS non-relational database to be deployed in a multi-master configuration across several regions. This results in synchronous replication between multiple read-write nodes across a geographically disparate area. This brings a great deal of application redundancy and performance capabilities in my opinion.

  • Sagemaker – I’ll quickly admit that I don’t have a deep knowlege of the machine learning space. The way this was explained to me though, is that the Sagemaker service enables quicker training and deployment of data models associated with machine learning. The possibility exists that an organization where machine learning was out of reach because of a manpower or knowledge deficiency now has an opportunity to accelerate a machine learning project.

  • Fargate – managing and delpoying containers without managing the underlying infrastructure. You don’t need to size the resources to allow your application to scale, Fargate manages that for you. (ECS is supported now, Kubernetes in the form of EKS coming later this year.)

  • Guard Duty – Security analysis by looking at events across your account(s) and looking for compromises or potential comporomises.

  • Privatelink – Allows customers to privately access SaaS services on Amazon’s backbone instead of over the internet. Some of the current partners include Cisco Stealthwatch, CA technologies – App experience, and Dynatrace.

Advertisements

Networking in AWS – VPC Edition

I’ve been spending some time attempting get a more in-depth knowledge around Amazon Web Services. It is dead simple to start spinning up compute instances and S3 buckets. As I dive deeper in, however, I have started to uncover some of the more complicated topics that a person or an organization would run into while beginning a “journey to the cloud.”

One of those areas is networking. Maybe I gravitate towards that area first since my background involves a heavy dose of traditional on-premise networking. I would imagine anyone in my shoes would have no problem grasping the concepts of VPCs, NAT gateways, and Internet Gateways. Still, there are a certain amount of steps involved in customizing a VPC.

A Virtual Private Cloud (VPC) is a networking space in an AWS footprint. They are unique to regions, meaning you can’t span VPCs between Virginia and Ireland for example. The use cases for building your own customized VPCs include some or all of the following:

  • The ability to pick your IP addressing scheme per data center (could be important in building VPNs from a local data center to a VPC, or VPC-to-VPC peering–larger topics for another time)
  • The ability to separate networking space for different business units such as HR, Finance, IT (And the ability to apply separate, granular firewall controls through security groups and ACLs)
  • The ability to separate dev/test/prod environments from one another

There is a default VPC, which contains default subnets, you can start deploying instances into this networking space right away. But you may want to take a more prescriptive approach to networking in AWS and build your own custom VPCs.

To create a new VPC

  • Go to AWS Services, under “Networking and Content Delivery,” choose VPC
  • Go to “Your VPCs,” click the big blue button that says “Create VPC”
    • Give the VPC a name (I called mine Bob VPC) and a CIDR block (A /16 is the largest block you can create, you will create subnets from this block)

CreateVPC1

Next, we need to create subnets within our VPC

  • Go to “Subnets,” click “Create Subnet”
  • Give the subnet a friendly name (I chose the format of my VPC name and the network address)
  • choose which VPC it goes in (my BobVPC)
  • you can specify the availability zone in which you want it to reside
  • specify the CIDR block for the subnet (I chose /24s for my example)

CreateSubnet

In the details pane, you can see how many IP addresses are left, VPC membership, network, and subnet-ID. AWS reserves the first three addresses in any network. I have created two new subnets, one for public instances and one for private instances.

SubnetPane_availableIP

By default, any subnet created in a custom VPC doesn’t assign a public IP to instances created within it. If you want instances in the subnet to automatically get a public IP, you can enable auto-assignment of those public IPs.

  • choose which subnet ( I chose my Bob 10.10.1.0/24 subnet to be my public-facing subnet)
  • from the dropdown on “Subnet Actions,” choose modify auto-assign IP settings
  • Check the box for auto-assign

Create_auto_assign_public

For the subnets with public IPs, we need to create a way for those instances to get to the internet. This is an Internet Gateway.

  • Go to Internet Gateway
  • click “Create Internet Gateway”
  • Give the IGW a friendly name ( I chose BobVPC IGW in this case)

Create_IGW

  • We then need to associate the IGW with the VPC
  • choose your new IGW and click “Attach to VPC,” select your VPC (Bob VPC)

AttachIGW_to_VPC

OK, we have a VPC created, we have a subnet that auto-assigns public IPs to instances within it, we have an internet gateway. We now need to create a route for that subnet to get to the Internet Gateway.

  • Go to “Route Tables”
  • Click “Create Route Table,” lets name it “BobVPC Public Route,” make sure it is associated with the correct VPC

Create_PublicRouteTable

  • Our route table is created, let’s create a route, choose our route table
  • Click on the routes tab and add a new route, by clicking edit
  • Click add another route, since this is default, we are going to do an all 0s destination (0.0.0.0/0)
  • Click in the target box, and it should show you the options, choose the IGW you created, Click Save

Add_IGW_Route

  • Lastly, we need to associate the subnets with this route table, click on the Subnet tab
  • Click Edit, then choose the subnets you wish to route to the internet, Click Save

RouteTables_AssociateSubnets

We can now create an EC2 instance. During the launch process, choose our new VPC and the subnet you want the instance to live in. Once that instance is launched into my “Bob VPC 10.10.1.0” subnet, it will get a public IP address and will be able to access the internet via our Internet Gateway.

EC2_Instance_VPCChoice

What if we don’t want our EC2 instance to get a public IP? We have our old friend NAT.

Much like we created an Internet Gateway for our public instance, we create a NAT Gateway for our private instance. The key here is that when creating this NAT Gateway, it needs to be associated with a public subnet.

  • Go to NAT Gateways, click “Create NAT Gateway”
  • Choose a subnet in which to place the NAT GW (This is where we need to choose our public subnet, “Bob VPC 10.10.1.0”

NatGW_Subnet

  • We can have AWS automatically assign our external IP, which is called an Elastic IP

NATGW_Assign_EIP

Next we’ll need to add a route to the NAT GW for your private subnets. You can either edit the default route table within your VPC, or create a new route table. I am going to create a new route table.

  • Go to “Route Tables”
  • Click “Create Route Table,” lets name it “Bob VPC NAT,” make sure it is associated with the correct VPC
  • Let’s add a default route, click on the Routes tab, choose Edit
  • Add another route, and choose all 0s again (0.0.0.0/0)
  • In the target, you should see the NAT gateway you created, choose that and click save

RouteTable_NAT_defaultRoute

  • Then we need to associate this route table with our private subnet, click on the Subnet tab
  • Click Edit, then choose your private subnet (in my example it is “Bob VPC NAT”)

NAT_GW_RouteTable_SubnetAssociation

Any new EC2 instances created with this subnet will have NAT access to the internet, but will not have a public IP.

Like many of the services within AWS, there is a low barrier to entry for getting started, but once you get past the surface, there is a world of dragons. Beware.

Here is the link to the VPC user guide:

http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Introduction.html