Cloud Instance Manager
Manage AWS Instances Without Console Access
Problem Statement
With the rapid adoption of cloud nowadays, the cloud operations team in a medium to large sized software company has faced the challenges of managing cloud resources for all internal development teams. They receive requests 24/7 from multiple teams (working on different projects) for access to cloud environments (mostly server access) for various reasons such as production environment debugging, testing, simulations, training, hosting static sites etc. Once the access was provisioned, the next challenge was to ensure that a shared responsibility model (such as the one offered by all major cloud service providers) is followed where the teams who are provisioned access and the cloud operations team share responsibility of managing the lifetime and billing of those resources.
Problem Analysis
With the fast pace growth in the size of teams in our company as well as the ongoing AWS hosted projects, we faced the same challenges where our cloud ops was getting an increasing short-termed request for AWS services especially EC2 for various demos and developments needs. Even with a small number of requests it was getting hard to manage the assignments, keeping track of the allocations, keys sharing, instance state i.e powering on/off coordination and dependency and keeping up with the billing of instances.
Make use of IAM and let user control there instance
One obvious solution to some of the above issues is to utilize the inbuilt AWS identity services where for every use case and requester, create custom IAM users, roles and policies and let them control their instance. However, this strategy has an overhead on both the cloud administrator and the developer’s end.
- Creating IAM policies and attaching them to users and roles gets cumbersome as the complexity of the policies increases. Also, this solution comes with the added risk of giving anyone more than required access.
- Each IAM entity has a lifecycle of its own and as the scale increases, it becomes a hassle to ensure that each is maintained and cleaned up properly.
- This solution was not cloud agnostic as using AWS specific services means that the solution cannot scale to other cloud providers.
Excluding above, still not all the original issues are addressed, like
SSH Key management
Sharing EC2 SSH Keys with the users is still an inconvenience for cloud administrators as they have to be shared separately using a secure medium. Users requested more seamless methods for receiving their PEM keys.
AWS Console Learning Curve
From users or developers perspective who got their accounts on AWS are not really aware of the AWS environment found the console to be cluttered and only wanted to manage their dedicated EC2 instance.
Bill Tracking
Even if someone fills the learning curve and gets used to the console even now there is no easy way to view customized billing information for each EC2 server on the AWS console, and therefore developers never had access or visibility to how much cost their individual resources were incurring to the total monthly bill.
Introducing Cloud Instance Manager
Cloud-instance-manager is a web based application that we have developed keeping an account of all the above issues, it simplifies the management of EC2 machines for all the stakeholders by providing,
- No IAM roles and policies
- No complex console
- Easy bill and cost tracking
- Effortless coordination
- Secure key sharing
Cloud Instance manager comes with two interfaces: an Admin and Developer interface.
Admin interface, as the name suggests, is a web console for administrators. The administrator can:
- View all the instances and their relevant information, for a cloud account.
- Create and delete users
- Assign instance to users
- Upload the ssh key for the instances
- View instances cost information
Developer Interface is a specifically designed minimal interface with suitable/selective functionality where a user can view only their assigned instances and have the options to
- View their assigned EC2 information (region, IP, state)
- Start and stop instances independently
- Keep track of their daily and monthly resource bills
- Download their EC2 access SSH Keys
Features & Flow
The general workflow is that any user needing an AWS instance will request the Admin. Admin will provision the instance in the cloud and make that user’s account in the cloud instance manager in case it’s a new user. And also assigns the instance to that user.
Once the instance is assigned to a user. User can then control and manage the instance
Apart from that admin can upload the ssh key for the instances and any user having that instance assigned can download it without the need of asking to share it.
These are only the starting functionality and it can be enhanced by putting all the basic and needed features that are lacking in AWS console.
Want to learn more go ahead and give it a try
Yes this is an open source python flask based dockerized application that comes with a minimal system requirements and easy installation.
This is just the base line and there are many more features planned to overcome and augment cloud console blind spots. Some upcoming features includes:
- Ability to manage multiple accounts at the same time.
- Support of multi cloud, Azure being the next.
- A simple yet useful feature would be to send billing alerts to a user if their instance reaches a certain cost threshold.
Give it a try and feel free to suggest and even contribute towards more features and share your thoughts in comments.
