Java + Cronjob to Read Files From S3
Cron jobs are normally used to schedule commands at a specific time. You can utilize them for tasks similar running backups, monitoring the status of the system, or running system maintenance tasks.
Cron jobs are a helpful utility for system administrators. And when you lot are administering a organization in the cloud, cron jobs are still very useful – you lot still have to do a lot of administrative tasks on your systems.
1 style of running cron jobs in the deject is to use a function as a service (FaaS), similar Lambda in the AWS ecosystem.
Functions execute when they are triggered to practise so, and they run code in the cloud without the demand to provision or maintain any infrastructure. Also functions tin can be configured to run at a certain time or with certain periodicity, like traditional cron jobs.
In this web log mail, I will use the AWS ecosystem to show you a physical example on how to create a cron job using a part in the cloud.
Amazon CloudWatch events
In guild to utilise a Lambda function as a cron task, nosotros demand to empathise Amazon CloudWatch events.
Amazon CloudWatch events are sent when there are changes in the AWS resources. These events can trigger an AWS Lambda function. When your AWS resources change country, they automatically transport CloudWatch events to the event stream.
Therefore, you tin create a rule that triggers a specific Lambda function when something happens. For example, you lot tin automatically invoke a Lambda function when there is a change in an AutoScaling group.
In improver, CloudWatch events can invoke a Lambda function to execute on a regular schedule. And in this way y'all tin can have, for example, a Lambda office that turns off all your testing and evolution EC2 instances after 6pm and another 1 that turns them on subsequently 8am.
Setting up the demo
I want to bear witness you an case of a Lambda role that can perform actions on your EC2 instances. I volition be using AWS SAM to define my Lambda office equally infrastructure as lawmaking.
If you want to effort this demo out, y'all need to have an AWS account and one or more EC2 instances configured in your AWS account. These are the ones that nosotros are going to manipulate from the Lambda functions. EC2 instances are the AWS version of virtual machines in the cloud.
Yous can effort the demo on AWS Cloud9 IDE (a browser based IDE), as AWS SAM is already configured in that IDE. If you want to know how to use AWS Cloud9 IDE to operate Lambda functions y'all can cheque out this video.
In this example, we are going to start and terminate EC2 instances using 2 unlike AWS Lambdas that get triggered at a given time. We are starting the instances at 8am everyday and turning them off at 6pm when the day is over.
For that we are going to apply a CloudWatch result to trigger the Lambda at the right time and also the AWS SDK to perform the operations in the instances.
The finalised code for this example is available in this GitHub repository. To go this code working in AWS Cloud9 IDE, you need to configure your GitHub account in the IDE to exist able to clone the project and then clone it inside the IDE.
When y'all have that ready, only run this command inside the cloned directory:
$ sam deploy --guided When running that control you will get a set of questions that you need to answer in club to configure this project to run successfully.
The outset thing you need to ascertain is a name for your projection. Then yous'll set the region where it is getting deployed - pick the same where your EC2 instances are. Next we need to give the deploy script a list of the instances that nosotros want to dispense. And then we are done – it will deploy the project to our AWS account.
Defining the AWS Lambda function
The kickoff thing I want to prove yous is how nosotros define a AWS Lambda function that gets triggered in a specific fourth dimension using AWS SAM. This definition volition exist in the file called "template.yml".
This is how a function looks. Let's look at the of import lines:
The first line is the proper name of the function, in this case "StartInstanceFunction".
So nosotros have the "Backdrop" definition. The showtime holding is the "Handler". Here we will specify the module (file) where the code that needs to execute is and and so the method inside that module.
And then we accept the "CodeUri", which is the path that shows you where to discover that file. In this instance, our code will be inside a directory chosen "cron" in a file called "handler.js" and in a method called "startInstance".
Later that we have the "Runtime" definition. I will exist using NodeJS version 12, but yous tin can utilise Python, Java, Go, C#, or whatsoever makes you happy. Lambda supports multiple runtimes out of the box and you can bring your own runtime if you lot want to.
And so nosotros accept the "Surroundings" definition that nosotros will use to define the one ecology variable. This variable will allow usa to transport to the code dynamically different instances ids, depending on the configuration when we deploy.
Later that nosotros take a section called "Policies" which is where we define the permissions that this particular Lambda function will have.
Information technology is important to know that all Lambda functions are created without any permissions. That means that they cannot do annihilation on whatsoever other AWS resources.
In gild for this Lambda function to start an EC2 instance, it needs permissions to do that item action on that particular AWS resource. In this particular policy we are granting permissions to start ALL EC2 instances in this AWS business relationship. ALL is represented with the "*" in the resource section.
If you have this piece of code running in production, I recommend that you limit the resources to exactly the ones that you desire this Lambda to be able to get-go.
And finally, the terminal department is the "Events" section. Here we will ascertain how this Lambda function will get triggered. This function will get triggered with a scheduled CloudWatch event that triggers the Lambda everyday at 8 in the morning. Basically at 8 every solar day it will turn on all the EC2 instances that you lot specify.
There are many rules to form this cron expression: for case, to say that you would like this to run simply Mon through Friday, write cron(0 eight ? * Mon-Friday *). Yous tin observe more info in the documentation site of CloudWatch events here: https://docs.aws.amazon.com/lambda/latest/dg/services-cloudwatchevents-expressions.html.
Coding the AWS Lambda function
At present that we have divers the Lambda part nosotros need to provide some code to information technology. In the folder "cron", in the file "handler.js", nosotros demand to add the method chosen "startInstance" that looks like this:
This method will get chosen when the function is triggered every day at eight am. Information technology will become the list of EC2 instances from an environmental variable that we passed all the instances ids during deployment time. Then it volition create an array of them.
When it has that, information technology will call the AWS SDK and ship the array of instances id as a parameter. And if there is any error information technology will log it and complete. Immediately after this Lambda finishes execution, you can go to your EC2 console and see how your instances plough on.
The function to turn off the EC2 instances is very similar with a few differences. You can find the code for that role in this link and cheque it out.
Running the cron job
To run this cron job, there is not much left to exercise. Subsequently the two functions are deployed in your AWS account, in the same region as your instances, they will execute and exercise what they were programmed to do.
Now you need to await until 8am or 6pm to see if they work. Or if you desire to test it out right at present, change the event fourth dimension in the Lambda definition to a time that works for you. Brand sure that the instance is on if you are planning to turn them off or the other style around, and so yous can see the changes.
At present look and see what happens in the EC2 console. Right after the fourth dimension you lot ready up, yous will see the example go off or on and then do the opposite at the other time y'all setup. This will go forever until you remove the Lambda functions.
Cleaning up your AWS account
Afterwards completing this demo I recommend you that you turn off (or remove the instance you created to test) and remove the Lambda functions you just created.
Removing the lambda functions is every bit piece of cake as going into your CloudFormation service in your AWS management console and removing the stack of resources that AWS SAM created.
Also don't forget to terminate and remove the EC2 instances if you created them for this demo.
To conclude
AWS Lambda functions are a very useful tool to perform all kinds of tasks in your AWS account. You lot tin can basically get notifications of any changes in the AWS resources through CloudWatch events and and so you can access well-nigh all the services using the AWS SDK. So yous can perform all kinds of maintenance tasks and automatic tasks over your infrastructure.
Thanks for reading.
I'g Marcia Villalba, Programmer Advocate for AWS and the host of a youtube channel called FooBar where I have over 250 video tutorials on Serverless, AWS and software engineer practices.
- Twitter: https://twitter.com/mavi888uy
- Youtube: https://youtube.com/foobar_codes
Larn to lawmaking for free. freeCodeCamp's open source curriculum has helped more than 40,000 people get jobs as developers. Get started
Source: https://www.freecodecamp.org/news/using-lambda-functions-as-cronjobs/
0 Response to "Java + Cronjob to Read Files From S3"
Enviar um comentário