Reading...
Play button
Play button
Use LEFT and RIGHT arrow keys to navigate between flashcards;
Use UP and DOWN arrow keys to flip the card;
H to show hint;
A reads text to speech;
90 Cards in this Set
- Front
- Back
|
EC2
|
Elastic Cloud Compute - known as instances. Multipe services fall under this including Elastic Load Balancer, Auto Scaling, EBS Volumes. Multiple pricing models \Reserved instances - for known amount of usage you can purchase at a discoutn
On-demand - paid by the hour Spot instances - bid on unused EC2 instances for non-prod applications |
|
|
AMI
|
Amazon Machine Image - is a template for a pre-built software configuration - commonly used with Auto scaling etc
|
|
|
VPC
|
Virtual Private Cloud - launch AWS resources inside a virtual network, with private and public subnetsshould always be used.
Isolates a group of AWS resources, they become part of the same network. Can use VPC peering to share resouces but in the same region |
|
|
S3
|
Simple Storage Service, is object storage.
Can integrate with CloudFront for CDN, version objects with lifecycle policies. It is a simnple key-value store Objects stay within one region but are syned across Azs \Read after write |
|
|
S3 RRS
|
Reduced Redundancy Storage, for easily reproducable storage, provides 4 nines durability
|
|
|
Redshift
|
Datewarehouse Solution for petabyte scale
|
|
|
SWF
|
Fully managed "work flow" service. Gauranteed execution of workflow. Allows architect to implement distributed asynchronous applications as work flows (primarily an API)
|
|
|
SQS
|
Simple Queue Service - highly scalable message queue for applications.
Gaurantees delviery of at least 1 message but does not guarantee no duplicates Two types of polling, Long and Short, long allows the service to wait until a message is available before sending a response and will return all messages from all SQS services, short will return messages from just a subset of servers and increases API requests |
|
|
SWF Work Flow
|
Coordinates and manages the execution of activities that can be run asynchronously across multiple devices
Tasks interest with the workers that are a part of the work flow, Activity Task - tells the worker to perform a function, Decision Task - Tells the decider the state of the work flow execution Worker can be any component (EC2 instance or a person) Provides consistent execution and gaurantees order in which tasks are executed and that there are no duplicates, execution can last up to 1 year |
|
|
SNS
|
Simple Notification Service / coordinates and manages deliverz of messages to specific end points (SQS/Email/SMS/HTTPS/App).
Topic - what a message is sent to SNS sends messages to all subscribed to a specific topic |
|
|
Elastik Beanstalk
|
Deploy complete application environments automatically, used for less complex environments with no management requirements
|
|
|
Cloud Formation
|
Allows you to create and provision resources in a reusable template fashion
Can source control templates Everything has an API, this allows you to code everything, create and provision resources using templates. Uses JSON style syntax, build templates and script your environment |
|
|
IAM
|
Identity Access Management - web service for managing permissions, will apply to all regions, allows for granular control
|
|
|
Availability zone
|
A group of DCs in specific geographic region. AZ in a region have direct low latency link within the region but each AZ is isolated
|
|
|
Service availability within regions
|
some aws instances work globally (IAM) but others do not
|
|
|
ELB
|
Elastic Load Balancer - should often be used with auto-scaling, can distribute traffic to instances in different regions
|
|
|
Route 53
|
Internal and external DNS , can also register and transfer domains
Latency, GEO location, basic and failover routing for single or multi region |
|
|
EC2 Storage Options
|
Instance store volumes - ephemeral storage, block level attached to the underlying host
EBS Backed - network attached storage, persistance storage |
|
|
EBS
|
Elastic Block Store - network atached block level, can provision additional IO
Start at 1GiB up to max 16,384GiB (16TiB) \must be in same availability zone, only attached to one instance |
|
|
S3 Details
|
Designed for 11 nines durability, and 4 nines availability
Can encrypt at rest and in transit \Buck names are unique across all S3 regions, provides eventual consistency pay for each version, can use versioning and lifecycle for automated backup and archiving, can also move to Glacier |
|
|
Import / Export
|
Snail mail data to AWS, they import to any storage platform within one business day of receiving
|
|
|
RDS
|
Relational Database Service - fully managed by AWS, cannot access to underlying OS. Supports - MySQL, Postgres, Oracle, MS SQL, Aurora
|
|
|
Aurora
|
AWS Fork of MySQL - 5 times better performance than MySQL and a lower price point
|
|
|
ElastiCache
|
Fully managed in-memory cache engine - works with Memcached and Redis (but app needs to be built to work with these)
|
|
|
S3 Buckets
|
grouping of information that have sub-names, are similar to folders, every bucket must have a unique name across S3
Only 100 buckets in an account at a tmie, ownership cannot be changed All buckets are private by default |
|
|
EC2 - Classic
|
Deprecated version of EC2 - no longer available to new accounts created after December 2013
|
|
|
EC2 Instance Types
|
T2 "Burstable Performance Instances" \M2 Instances (nice balance)
C4 (Compute Optimised) R2 (Memory Optimised) G2 (GPU Optimized) I2 (Storage Optimized) EBS Optimised |
|
|
Lamda
|
.. |
|
|
CloudTrail
|
Security and compliance, monitors all actions against an AWS account / API
|
|
|
CloudWatch
|
Monitoring AWS services - Can help with:
Shutting down inactive instances Monitoring changes with the CloudTrail integration EC2 has basic out of the box monitoring For status checks for starting and stopping instances |
|
|
Directory Services
|
For connecting toexisting AD or creating a simple AWS AD
|
|
|
EMR
|
Elastic MapReduce, Hadoop clusetering tool to intergrate with Hadoop clusters. Uses EC2 instances, provides admin access to the OS level. Can be used with DynamoDB, Redshift, S3
Mappers split large data files for processing, Reducers take the result and combine it back into a data fie |
|
|
Recommendations
|
Using multi-avilability zones
Purcahse reserved instance in a DR zone as there is no gaurantee of capacity Route 53 for DNS failover Use elasticity when available use Elastic IP use SQS |
|
|
Cloud Front
|
Global CDN, origin can be S3 or an Elastic Load Balancer CNAME that distributes traffic across origin instances
Signed URL for private content with one-time use Can integrate with Route 53 for alternative CNAMS Deisgned for cachning, |
|
|
CloudWatch (AWS Config)
|
A service for detailed configuration information about the environment
|
|
|
S3 standard storage availability
|
99.999999999 durability and 99.99 availability, most expensive
|
|
|
S3 RRS
|
99.99% with reduced redundancy and cost
|
|
|
Glacier
|
Archival storage, hours to check in and check out data
|
|
|
DynamoDB
|
Fully managed NoSQL, similar to Mongo, HA and scaling built in. Use multi read replicas, cannot scale writes, increase size of instance and ElastiCache for caching DB sessions info
|
|
|
Shared Security Model
|
Customer is responsible for Guest OS, aplication, and configuration of AWS security groups firewall etc
|
|
|
AWS Security
|
Secure access using API endoints, HTTPS and SSL/TLS
VPC with built in firewalls IAM for unique users Multi factor auth Private subnets encryption perfect froward secrecy logs \AWF confid for asset identification Key management Cluod HSM Trusted Advisor with support |
|
|
More Security
|
OS has fitrewall with Iptables, FirewallID, TrendMicro integration with EC2
|
|
|
S3 Security
|
build in AES-256 for data at rest, decrypted as it is sent to the customer
|
|
|
EBS Security
|
Encryted on the EC2 instance and copied to EBS, snapshots are also encrypted
|
|
|
RDS Security
|
All DBs can be encrypted, this then covers backups and read replicas and SSL to connect
|
|
|
IAM Access Controls
|
ECS - permissions to revboot, start stop etc configurable to the instance ID
EBS - attach, delete and detach |
|
|
CloudWatch Security
|
API only via SSL endpoints, uses IAM for permissions, requests are signed HMAC-SHA1
|
|
|
AWS CSA PRO
|
.. |
|
|
Storage Gateway
|
Connects local data center software appliances to cloud storage at Amazon
Gateway cached volumes - create volumes and mount them via iSCSI on-prem, stores data in S3 and caches data on-prem Gateway storage volumes - stores data locally, gatway will snapshot data and store them in S3 |
|
|
EC2 Troubleshooting (1/3)
|
Connectivity Issues - Ports on the correct security group are not open, all ports are closed by default
Cannot attached EBS to EC2 - same availability zone, can snapshot and create a new volume in the right zone |
|
|
EC2 Troubleshooting (2/3)
|
Cannot launch instances - Reached your capacity limit
Cannot download updates - does not have a public IP or belong to the public subnet Application slows on T2 micro - it uses CPU credits and you app is using too many |
|
|
EC2 Troubleshooting (3/3)
|
Elastic IP detached when stopped - EC2 classic is likely the problem
AMI unavailable in other regions - only available in the created region, can copy but will get a new AMI ID Capacity error in placement group - stop and start all instances as they need to be placed close together |
|
|
Placement Group
|
Is a cluster of instances within the same AZ, have low latency 10gb network
|
|
|
VPC Troubleshooting (1/3)
|
New ECS does not get a Public IP - modify auto-assisng on the subnet
S2S VPN working but no resource access - need to add on-premise routes to the Virtual Private Gateway route table NAT instance configured but private Subnet cannot download packages - need to add 0.0.0.0/0 route to the i-xxxxx on the route table for private subnets |
|
|
VPC Troubleshooting (2/3)
|
Cannot create VPC connection bewween VPCs - only works in the same region
Traffic is not making it to the instances even through security grop rules are correct - check Network Access Control List that proper ports from the right sources are open Error when attaching multiple internet gateways to a VPC - only one internet gateway can be attached to a VPC |
|
|
VPC Troubleshooting (3/3)
|
Error when attaching multple Virtual Private Gateways to a VPC - only one Virtual Private Gateway is needed on a VPC
VPC Security Group does not have enough rules for the app - assign the EC2 instance to multipe security groups Cannot SSH or resources inside a private subnet - VPN is not setup or you have not connected to an EC2 instance within the VPC |
|
|
ELB Troubleshooting (1/2)
|
Load balancing between availability zone is not working - enable cross-zone load balancing
Instances are healthy but not reporting as healthy - ensuer the resource it is chking is available or modify to one that is \ELB is listening to port 80 but traffic is not making it to the instances - listeners are not the same as security groups, port 80 needs to be open on the SG that ELB is using |
|
|
ELB Troubleshooting (2/2)
|
Access logs on web servers show IP address of ELB not the source traffic - enable access logs to S3 \cannot add instances from a specific subnet - the subnet needs to be added to the ELB
|
|
|
Auto Scale Troubleshooting
|
Auto scale instance starts and stop frequently - change the threshold of the policy to be lower \auto scale does not occur - ensure max number of instances is not the same as the current number
|
|
|
EBS Volume Details
|
Meased in IOPS, 256KB blocks or smaller, any operations greater are split into 256 i.e. a 512KB operation counts as 2 IOPS
|
|
|
EBS Volume Types
|
General Purpose SSD - normally root volume, 3 IOPS/GB - 1GiB to 16TiB, for dev/test and small DB
Provisioned IOPS - mission criticap with sustained IOPS, 4GiB to 16TiB, provision via IOPS up tp 20,000IOPS Magnetic - low cost, archive data, 1GiB to 1024GiB |
|
|
EBS Snapshopts
|
They are incremental. When original snapshot is deleted data is still available,
take during non prod hours |
|
|
EC2-Classic
|
Instances are given a public IP and cname, also receives a private IP but is not part of a VPC, if it is shut down will lose ist private IP
|
|
|
Security Groups
|
Used as a firewall in front of an EC2 instance, an instance can belong to multiple security groups. Security groups can reference themselves as source traffic in firewall rules
|
|
|
Security Group details
|
up to 100 seucirty groups per VPC, each can hape up to 50 rules
up to 5 security groups per EC2 instance cannot create deny rules - by dafult everything is denied responses to inbound traffic are allowed regardless of outbound rules (stateful), same applies to outbound instances cannot communicate with each other in the same group unless ports are open with the exception of the default security group |
|
|
Security Group differences
|
VPC security groups - you can change the instances associated, EC2-Classic - once you launch an instance you cannot change the security group
|
|
|
Placement Group Troubleshooting (1/2)
|
Instances can be restarted and remain in the placement group
all instances should at first be started in a single request to make them physically as close as possible adding instances after creation may lead to insufficient capacity errors menaing all instances need to be stopped and started for move them all together |
|
|
Placement Group Troubleshooting (2/2)
|
Instances cannot be moved into a placement group, they need to be created in it
cannot be connected, insances must have 10Gb networking |
|
|
RDS Details
|
minimum 5GB with max 3TB disk capacity, SSD vs Provisioned IOPS
automatic AZ failover with synchronous replication automated Point in time bacups , DB engine must be transactional |
|
|
VPC Details
|
Define custom CIDR (ip address range) inside each subnet
configure route tables between subnets configure internet gateways and attach them to subnets |
|
|
Default VPC
|
allows the user easy access to a VPC without configuring it from scratch
has an internet gateway attached each instance added has a default private and public ID address (public IP addresses are routed/ attached to an ENI tht has a private IP address attached to an instance (NAT) |
|
|
VPC Peering
|
Creates a direct network route between one VPC and another, allowing sharing of resources between teo subnets
can be created between other AWS accounts but must be in the same region |
|
|
VPC Limits
|
5 VPCs per region, with 5 internet gateways (equal to your VPC limit as only one per VPC)
50 customer gateways per region 200 route tables per region with 50 entries per route table 5 elastic IP addresses 100 security groups with 50 rules per group security groups are on the VPC level |
|
|
VPC Networking (1/3)
|
each subnet must have a route table
by default all subnet traffic is allowed to each other avaiable subnet within your VPC which is called the local route you cannot modify the local route best practice, leave the defaul route and create a new route table when routes are needed for specific subnets |
|
|
VPC Networking (2/3)
|
Internet gateway provides NAT translation for instances with a public IP (public to private IP)
to allow acces from a VPC you must attach an internet gateway, ensure that the subnets route table points to the internet gateway, ensure that instances in the subnet have a pbulic IP address or Elastic IP address, and ensure that your network access control and security group rulea allow the relevant traffic to and from the instance |
|
|
VPC Networking (3/3)
|
Instances launched into a private subnet cannot communicate with the internet. Higher security but cannot download software and updates, to resolve must create a NAT instance
Nat instance must be created in a public subnet, and must be a part of the private subnets route table |
|
|
VPC Security (1/4)
|
Security Groups - operate at the isntance layer, supports allow rules only, is stateful so return traffic is allowed, evaluates all traffic before deciding to allow traffic
|
|
|
VPC Security (2/4)
|
Network ACL, operates at the network/subnet level, supports allow AND deny rules
stateless so return traffic must be allowed through an outbound rule Process rules in number order i.e. is it is denied at a low number and allowed at a high number it will be ignored applies at the network level so all instances inside the subnet i.e. one deny will block all traffic on the port denied to all instances |
|
|
VPC Security (3/4)
|
In the Network ACL the last rule is deny all by default
best practice to increment rule numbers by 10 |
|
|
VPN/VPG on VPC (1/2)
|
Virtual Private Network allows you to extend a subnet from one location to another, allows access to resources, all traffic is encrypted by default
VPG, virtual private gateway acts as the connector on the VPC. The VPG is connected to the VPC and the VPN is associated with teh customer gateway Customer Gateway, acts as the connector on the on-premise side, both a VPG and Customer gateway are required to establish a VPN connection |
|
|
VPN/VPG on VPC (2/2)
|
VPN has two routes to the VPG, only one VPG is attached to a VPC
Configure public IP on customer gateway Route table must include routes for the on-premise network used by the VPN and point them to the VPG |
|
|
Alternative VPN Options
|
Configure OpenVPN Instance (not site-to-site however)
lives in a public subnet, connect to the public IP using the OpenVPN client The client receives a new IP, only exists on the OpenVPN server but the OpenVPN server routes the traffic |
|
|
High Availability for OpenVPN
|
Prive the OpenVPN with an elastic IP, create a backup instance running in another AZ, use a script to check availability and attach / move IP
|
|
|
CloudFrot performance
|
Slow DNS can cause performance issues, as edge location is dependant upon this, longer caching periods increase performance
Query strings reduce cache "hits" as they are often unique |
|
|
Decoupled systems
|
Two AWS services for this are SWF (Simple Work Flow Service), SQS (Simple Queue Service)
|
|
|
EMR Details (1/2)
|
AWS EMR has pre-configured the Hadoop AMI for best performance based on instance size, single mapper configured to handle 128MB split files
to increase performance adjust the split, but may need more mappers split files are loaded into memory |
|
|
EMR Details (2/2)
|
Goal is to use as many mappers without running out of memory
|
|
|
EMR Cluster Architecutre (1/2)
|
load data into cluster (can use S3 as local file system to increase performance)
Analyse by creating scripts Store resuls in HDFS or S3 Read the results Master Node - provides informatio about the data (where it comes from and goers to) Core Node - processes and stores data on HDFS or S3 Task Node - Processes data send sends back to ther core node |
|
|
EMC Cluster Architecture (2/2)
|
HDFS stores the reduced data on the local file system - not persistant
Bootstrap - can instll additional software or change configuration before hadoop starts S3 is good for storage… |
|
|
Elasticity and Scalability
|
Prive the OpenVPN with an elastic IP, create a backup instance running in another AZ, use a script to check availability and attach / move IP
|
