Guest User

Untitled

a guest
Dec 16th, 2018
121
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. How to use the ArchiveTeam Warrior with Auto Scaling on AWS:
  2.  
  3. First, make your launch configuration:
  4.  
  5. 1. Go to https://console.aws.amazon.com/ec2/autoscaling/home#LaunchConfigurations:
  6. 2. Create Launch Config
  7. 3. Pick the Amazon Linux 2 AMI
  8. 4. Pick your instance type. I'm using t3.nano which is small and cheap but can occasionally spike up to high networking performance for a little bit. If you've just made your AWS account, you can get a t2.micro on the free tier, but make sure you understand how free tier works, easy to accidentally get some charges if you don't know what you're doing. Use a bigger instance type for more network bandwidth and faster CPU, which double each step you go up in a particular instance type family.
  9. 5. on the "Create Launch Configuration" page:
  10. a) give you launch configuration a name
  11. b) optionally, request spot instances. This will save some money, at the risk that you might get terminated randomly if EC2 wants the capacity back.
  12. c) under "Advanced Details", put in the User Data, changing the env values to suit you:
  13.  
  14. #!/bin/bash
  15. yum update -y
  16. amazon-linux-extras install -y docker
  17. systemctl enable docker
  18. systemctl start docker
  19. docker run --detach \
  20. --env DOWNLOADER="aws_warrior" \
  21. --env SELECTED_PROJECT="auto" \
  22. --env CONCURRENT_ITEMS="3" \
  23. --publish 80:8001 \
  24. --restart always \
  25. archiveteam/warrior-dockerfile
  26.  
  27. 6. Add Storage. The warrior will use up to 60 GB per concurrent item, but usually less. The OS needs a few GBs itself. Set your disk size appropriately. Using magnetic volumes (under Volume Type) will save a little money but have a slower disk.
  28. 7. Security group - get one with port 80 open to just your IP address
  29. 8. Set a key pair that you like, in case you want to SSH in to do anything manually, then finish the Launch Configuration.
  30.  
  31.  
  32.  
  33. Now for the Autoscaling group:
  34. 1. click "Create an Auto Scaling group using this launch configuration"
  35. 2. Give your group a name, pick how many instances you want, and select your public subnets. That'll be all of them if you haven't made others before.
  36. 3. skip the scaling policies and notifications
  37. 4. add some tags to help you keep track of the instances launched by autoscaling.
  38. 5. Finish creating the group.
  39.  
  40. Done! Now it'll launch some instances. If you want more, change the Desired Capacity. If you want to cycle your IPs, just terminate the instances and they'll get replaced. If you want to see what an instance is up to, you can go to it's URL and see.
RAW Paste Data