No More Zapier Timeouts: Transcribe Large Audio Files with Deepgram and Zapier’s Webhooks

Deepgram recently became available in Zapier, making it a breeze to transcribe audio without writing any code. Check out this docs article, “Zapier and Deepgram” to get started creating your first zap with our new Zapier integration.

While creating a basic Zap with Zapier and Deepgram is a straightforward process, some users will hit a wall when they find out that Zapier has a thirty second timeout on all actions that run within its workflows. Deepgram is very fast at transcribing large files, but you might find that files larger than 200MB sometimes take longer than the 30 seconds allowed by Zapier.

The good news is that there is a workaround to this problem, although it isn’t a no-code solution. You will have to create a basic server to handle an HTTP request. But don’t worry - this tutorial will walk you through how to build the entire workflow.

Overview

We’ll be using these tools and integrations to build our workflow:

• Deepgram

• Zapier

• Node Express

• A deployment service of your choice to deploy the Node server (I used Fly.io)

• Amazon S3

• CloudConvert

• Dropbox

Here is a high-level overview of what we’ll build. It’s important to understand that this workflow actually requires the building of two separate Zaps.

Step One: Build the Server

We will start by building a server, since we will need this for both the Zaps we are going to create in Zapier. We’ll build a Node Express server, but you can easily do this in your language of choice.

Step Two: Create Zap #1

We will create a Zap to transcribe an audio file with Deepgram. The audio file will come from an S3 bucket, and then we’ll use CloudConvert to create a signed URL of the audio file. The audio file will be sent to Deepgram along with a callback URL (for the Node server we built) where we want the transcription response to be sent.

Step Three: Create Zap #2

We will create a Zap that uses Zapier webhooks to watch for data sent to a Zapier webhook URL. That webhook URL will receive the transcript sent from our Node server. This Zap will trigger once the transcription shows up, and then the Zap will create a text file of the transcription in Dropbox.

Walkthrough

Step One: Build the Server

Deepgram’s callback feature lets you provide a callback URL for where you would like the transcription to be sent once it is ready. This is a helpful feature when transcribing larger files that might take longer to transcribe (which means this is a good work-around for dealing with Zapier’s timeout restriction).

If you would like to understand the callback feature in more depth, I recommend you read this article in the Deepgram docs.

Node Express Server

Here is the starting code for a basic Node server. (The example project can be found in this github repo.) This code example is missing logic within the /hook endpoint and the sendTranscriptiontoZapier function that we will add soon.

const express = require("express");
const app = express();
const axios = require("axios");
app.use(express.json({ limit: "2MB" })); // need to set this when sending larger files in express


// Route handler for webhook
app.post("/hook", (req, res) => {});


// Function to send transcription data to Zapier
function sendTranscriptionToZapier(transcription) {}


const PORT = 3000;
app.listen(PORT, () => {
 console.log(`Webhook server is running on port ${PORT}`);
});

Note: we include Axios because we will eventually be using that library to post the transcription data to the Zapier webhook URL.

POST /hook Endpoint

Inside the /hook endpoint, we need to add some logic to check if the transcript has arrived. If it has arrived, we send it to Zapier. 

app.post("/hook", (req, res) => {
 let transcription = "";


 const transcript =
   req.body?.results?.channels?.[0]?.alternatives?.[0]?.transcript;


 if (transcript) {
   transcription = req.body;
   sendTranscriptionToZapier(transcription);
 } else {
   transcription = "No transcription available.";
 }
});

sendTranscriptiontoZapier function

We don’t have the webhook URL from Zapier yet, but we can write the logic to make the post request (later on we’ll put in the correct webhook URL). Use the command `npm install axios` to add Axios, which is a library that makes it easier to write the POST request.

function sendTranscriptionToZapier(transcription) {
 axios
   .post("WEBHOOK_URL_HERE", transcription)
   .then((response) => {
     console.log("Successfully sent transcription data to Zapier");
   })
   .catch((error) => {
     console.error(
       "Error sending transcription data to Zapier:",
       error.message
     );
   });
}

Deploy the server

You will need to deploy the server. You can do this using your service of choice. I used Fly.io. They provide documentation for deploying Javascript apps, but I found this YouTube video and this article by Hayk Simonyan to be all I needed. You may also want to look at the configuration files in my github project to see how I configured my project for Fly.io.

Step Two: Create Zap #1

Now we will create the first Zap. This Zap will trigger when we add an audio file to a specific folder in an S3 bucket. Let’s take it step by step through the Zapier form.

Trigger: New File in Folder in Dropbox

In the dashboard in your Zapier account, click on the “Create Zap” button in the left nav. This will set up a template with a trigger and an action.

For the “App & event”, choose Amazon S3 as the app and “New or Updated File” as the event. Then connect your AWS account by providing your access key and secret access key.