Creating a useful API for AWS Operations with the AWS SDK Go
I think that all of the DevOps know the Boto library in Python. Boto3 is a really wonderful library for AWS operations. But nowadays GoLang is a popular language like Python in the DevOps community. Sometimes you can need only one binary executable file instead of tons of python scripts. AWS SDK for Go is really powerful. It includes many AWS operations, especially on EC2 and S3. The SDK removes the complexity of coding directly against a web service interface also includes helpful utilities on top of the AWS APIs that add additional capabilities and functionality. For example response paginator for easily iterate over large sets of paginated API results or S3 download and upload manager to transfer large objects. There is an official web site to use the SDK.
https://aws.amazon.com/sdk-for-go/
You can find Source Code and detailed API reference, also examples. But unfortunately, the examples are not enough. The important missing is these examples are not from real life or real needs. So I coded some useful examples. For example; searching instances, starting or stopping or terminating instances, allocating or releasing elastic IPs, creating the snapshot, creating or deleting tags, etc…
You can find my GitHub repo address at end of the page for these and more examples, but now I will explain more detail about the main structure so that you can understand examples and code more action methods. We can start with the “ec2” package.
The main package URL is “github.com/aws/aws-sdk-go/aws” you have to add it and then you must add service packages that you want to use it.
import (
“github.com/aws/aws-sdk-go/service/ec2”
“github.com/aws/aws-sdk-go/aws”
)
At first, you must log in to the AWS. This action is called creating a session. As you know, you need credentials. You can give the credentials to the session package in 3 different ways.
“Environment” Option:
You should set two or three environments on your current shell your script is working on currently.
export AWS_ACCESS_KEY_ID=XXXXXXXXXXX
export AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXX
export AWS_REGION=us-east-1 // Not required
Go Code:
sess, err := session.NewSession(&aws.Config{
Region: aws.String(“us-west-2”)},
)
“SharedConfigEnable” Option:
If you use “awscli” tool, you know it well.
By default, NewSession will only load credentials from the shared credentials file (~/.aws/credentials). By default, NewSession will only load credentials from the shared credentials file (~/.aws/credentials).
You must create credentials file in “.aws” directory in your HOME directory.
aws_access_key_id = XXXXXXXXXX
aws_secret_access_key = XXXXXXXXXXXXXXXXXXXXX
region = us-east-1 // Not required
Go Code:
sess := session.Must(session.NewSessionWithOptions(session.Options{
SharedConfigState: session.SharedConfigEnable,
}))
“NewStaticCredentials” Option:
This option is not popular, because we don’t want to add keys in our code. Probably you will put these codes to Git repos like GitHub, so this is very dangerous. But nonetheless, you want, you can use like below:
sess, err := session.NewSession(&aws.Config{
Region: aws.String(“us-west-2”),
Credentials: credentials.NewStaticCredentials(“AKID”, “SECRET_KEY”, “TOKEN”),
})
After creating a session, we need an instance from this session.
ec2svc := ec2.New(sess)
Every AWS action has an Input object (struct) and output object (struct). You can find all input and output objects from this URL. “https://docs.aws.amazon.com/sdk-for-go/api/"
For example; you want to stop an instance which you know instance id. So you need an INPUT object. All INPUT objects name start with action name and end with “Input” word. ( ….Input )
In this action, “StopInstancesInput” is our Input Name. Here is the ec2 service API page.
https://docs.aws.amazon.com/sdk-for-go/api/service/ec2/
When you search the “StopInstancesInput” on this page, you can find the object.
type StopInstancesInput struct {
// Checks whether you have the required permissions for the action, without
// actually making the request, and provides an error response. If you have
// the required permissions, the error response is DryRunOperation. Otherwise,
// it is UnauthorizedOperation.
DryRun *bool `locationName:”dryRun” type:”boolean”`// Forces the instances to stop. The instances do not have an opportunity to
// flush file system caches or file system metadata. If you use this option,
// you must perform file system check and repair procedures. This option is
// not recommended for Windows instances.
//
// Default: false
Force *bool `locationName:”force” type:”boolean”`// Hibernates the instance if the instance was enabled for hibernation at launch.
// If the instance cannot hibernate successfully, a normal shutdown occurs.
// For more information, see Hibernate Your Instance (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Hibernate.html)
// in the Amazon Elastic Compute Cloud User Guide.
//
// Default: false
Hibernate *bool `type:”boolean”`// The IDs of the instances.
//
// InstanceIds is a required field
InstanceIds []*string `locationName:”InstanceId” locationNameList:”InstanceId” type:”list” required:”true”`
// contains filtered or unexported fields
}
There are only 4 variables available. Two of them has default values, so it is enough to set just two of them.
input := &ec2.StartInstancesInput{
InstanceIds: []*string{
aws.String(“i-0h7f12625779808a9”),
},
DryRun: aws.Bool(false),
}
Then call the action function:
result, err := ec2svc.StartInstances(input)
if err != nil {
data := map[string]string{“Error”: err.Error()}
jsonData, _ := json.Marshal(data)
fmt.Println(string(jsonData))
}
If “err” variable is nil, your result variable type will be “StopInstancesOutput”.
type StopInstancesOutput struct {
// Information about the stopped instances.
StoppingInstances []*InstanceStateChange `locationName:”instancesSet” locationNameList:”item” type:”list”`
// contains filtered or unexported fields
}
If you are writing a function for stopping the instance, here it is:
func test2(ec2svc *ec2.EC2, input *ec2.StopInstancesInput) (*ec2.StopInstancesOutput, error) {
result, err := svc.StopInstances(input)if err != nil {
return nil, err
}
return result, nil
}
I tried to explain the general structure easily. But you can find many examples on my GitHub repo below.
