How to build cron jobs with Netlify functions

Burak Tarım's photo
Burak Tarım
Posted 2022-11-06 · Updated 2022-11-11 · 4 min

Context

Heavybit is a developer tools investor that invests in companies like Mux, Sanity, Contentful, Stripe, LaunchDarkly, tailscale, and more.

To promote the different jobs of their portfolio companies, they wanted to display those on their website.

As a result, we leveraged Scheduled Functions from Netlify to create a cron job to regularly update their Jobs page with the latest vacancies.

By automating this task, we also saved time to the Heavybit content team who no longer needed to manually add the jobs.

Tech stack

The Heavybit website is built with the following tech stack:

Considering we are working in a serverless environment, we could not just create cron jobs. As an alternative, we've used Netlify's Scheduled Functions.

How to build cron jobs with Netlify functions?

Netlify functions are files we put under the netlify/functions folder in our project.

After we deploy them, we can access these functions just like an API endpoint.

We are going to follow these three steps to build our cron job:

  1. Create and test the function file.
  2. Add logic to fetch and save the data.
  3. Schedule the function.

Step 1: Create and test the function file

Creating the function file is pretty easy, all you have to do is create a file under netlify/functions folder in your project root.

You can use Javascript, Typescript, or Go.

Netlify Functions folder

After that, we can paste this "Hello World" example code to try it with Postman:

import { Handler } from "@netlify/functions";
const handler: Handler = async (event, context) => {
return {
statusCode: 200,
body: JSON.stringify({ message: "Hello World" }),
};
};
export { handler };
Postman to try Netlify Functions

It works!

We successfully created and tested our first Netlify function. Let's move on to the next step.

Step 2: Add logic to fetch and save the data

Heavybit invested in multiple companies, and they are all using different platforms to publish job vacancies.

We implemented our logic to fetch and save the data, however, the function was taking a lot of time to execute.

We also hit the execution time limit after we ran the function.

TimeoutError on Netlify Functions

As a remedy, we used background functions with higher execution times, which perfectly fit this use case.

How can we turn our function into a background function?

All we had to is add -background suffix to the file name and implement a few changes in the code (e.g. changing function type and remove response).

Background Netlify Functions

After running our function again, we successfully fetched and saved data to Sanity.

There it is!

We created our Netlify function to fetch jobs and save them to Sanity.

Now it's time to do the last step: schedule our function to keep the data up to date.

Step 3: Schedule the function

To schedule our function, we need to specify it as a scheduled function in our netlify.toml configuration file.

We can schedule it as @yearly, @monthly, @hourly, and so on.

[functions."jobs-background"]
schedule = "@hourly"

Once you deploy it, you can see your function under the functions tab as scheduled.

Scheduled Netlify Function

Final words

Leveraging Netlify Functions helped Heavybit saved hours of work monthly and avoid keeping outdated jobs on their website, ensuring fresh and relevant content to their website visitors.

If your team is spending time on manual tasks and needs some help with automation, don't hesitate to get in touch and we'll assess how we can help.


Recent articles

What is a frontend developer?

When you visit any website, what you are greeted by as a user is the website's frontend. The frontend of a website is essentially anything the user can see and interact with. Text,...
Omar Benseddik's photo
Omar Benseddik
2022-10-26 · 8 min