Ever since switching to orgmode as my GTD system, I've been trying to (slowly) optimize my capture system for things coming to me from different places. One of the things that I was not satisfied with was capturing and processing inputs from Slack.

Slack messages can easily get lost in the fast-paced stream of conversations. Manually copying and pasting messages introduces too much friction, and also removes some context (backlinks), unless a link to the original message is also manually copied. There are ways to save messages in Slack itself, but it just makes it another Inbox to check, and I'm trying to reduce that.

I started by using the saved messages feature to save messages in a single place, and later either shifting them to my org inbox manually, or directly working on them and completing them. Then, I shifted to using the Slack Todoist integration to save slack messages to Todoist, and pulling them from Todoist into my org Inbox.

I've now found a better mechanism that allows me to seamlessly capture Slack message with it's context into orgmode. Here's a demo:

Demo breakdown

This method uses userscripts to add a custom button to the hover menu that comes up for any message in slack, and triggers org protocol capture with the message details when the button is clicked. The message details include the sender, message text, and a direct link back to the message. I've set up this protocol handler to ask me to enter the heading of the capured item, but it can be as easily set up to directly capture the message without user input.

Setup and Implementation

Prerequisites

1. Slack running in a browser (instead of a desktop app)
2. Browser extention for userscripts (Tampermonkey, Violentmonkey, Greasemonkey, etc)
3. Emacs with orgmode installed
4. Org protocol setup

I didn't find a good way to inject userscripts into the Slack destop app, so for now, this method requires using Slack in a browser. It also works when Slack is installed using the "Install Page as App" feature of the browser.

Update: I got a comment that using this in the Slack app is also possible, though I've not evaluated this yet. You can check this comment for details.

Setting up Org Protocol Capture

Setting up org protocol capture involves two steps: configuring your OS to use Emacs to open org-protocol:// links, and configuring Emacs to save the captured data as you want.

For OS setup, please check the guides for your OS here: https://orgmode.org/worg/org-contrib/org-protocol.html#orge00964c

On Emacs side, this is the minimal config required:

(server-start)
(setq-default org-agenda-files '("~/org"))
(setq-default my-org-inbox
    (expand-file-name "inbox.org" "~/org"))
(setq-default org-capture-templates
      '(    
    ("i" "Inbox" entry (file my-org-inbox)
         "* %?\n%i\n%U"
         :kill-buffer t)
    ("l" "Inbox with link" entry (file my-org-inbox)
         "* %?\n%i\n%a\n%U"
         :kill-buffer t)))
(setq-default org-protocol-default-template-key "l")
(require 'org-protocol)

An optional enhancement that can be seen in the demo is: open a new emacs frame to capture this message, then close it automatically after the capture has been done. For this, I use the config snippet from prot's excellent post: https://protesilaos.com/codelog/2024-09-19-emacs-command-popup-frame-emacsclient/

To run emacsclient with the popup frame parameter, I use:

emacsclient --create-frame -F \
    '((prot-window-popup-frame . t) (name . "org-protocol-capture") (width . 80))' \
    -- %u

Emacsclient's args can be set in the desktop entry file (linux) or org-protocol.app file (OSX) when setting up org-protocol.

The userscript

For the userscript, I searched for an existing published userscript for Slack that adds a button, then tweaked it a bit to configure the button according to my needs.

I've published the userscript here: https://greasyfork.org/en/scripts/521908-slack-org-protocol-capture

From this, I learned about an interesting API called MutationObserver that seems very useful for userscripts.

Notes

Using emacs-slack

Another simple approach for this can be to use the emacs-slack package to directly use Slack from Emacs. I tried this, and it does not work very well for me because:

1. My org limits Slack session authentication to 1 week, and authenticating in emacs-slack is a little cumbersome.
2. We also use Slack huddles, and it does not work well with emacs-slack.

Possible Improvements

1. Make org capture template configurable
2. Add tooltip to the button
3. Pass even more metadata like message timestamp, channel name, emojis, etc.
4. Maybe some keyboard shortcuts to trigger the capture?

Limitations / Drawbacks

Since this is dependent on the HTML structure of the slack web app, it can be fragile and can break easily if there are changes in the app.

This userscript uses the MutationObserver API to observe DOM changes in the whole slack workspace. So, it has some impact on performance. However, I've been using this daily for the last several months and I have not noticed any issues.

Here's how the overall flow will look like:

We want this roundup post triggered on some schedule (maybe daily), so the Cloud Scheduler is required to send an event to a Google Pub/Sub topic that triggers our cloud function, which queries the slack API to get channels details, filter recently created, and post it back to a slack channel. Secret Manager is used to securely store slack's bot token and signing secret.

Note that the credentials shown in any screenshots below are not valid.

Create the slack app

The first step will be to create the slack app. Go to https://api.slack.com and click on "Create an app". Choose "From scratch" in the first dialog; enter an app name and choose a workspace for your app in the second dialog. In the next screen, copy the "Signing Secret" from the "App Credentials" section and save it for later use.

Next, go to the "OAuth and Permissions" tab from the left sidebar, and scroll down to "Scopes" -> "Bot Token Scopes". Here, add the scopes:

• channels:read: required to query public channels and find their creation times
• chat:write: required to write to a channel (where the bot is invited)

Next, scroll up on the same screen and click "Install to Workspace" to install to your workspace. Click "Allow" in the next screen to allow the installation. Next, copy the "Bot User OAuth Token" from the "OAuth Tokens for Your Workspace" section on the same page and save it for later use.

💡Keep track of the Bot User OAuth Token and Signing Secret you copied above.

Post to a Slack channel from a Google Cloud Function

Next, we will try to use the credentials copied above to enable a Google Cloud Function to send a message to a Slack channel.

Google Cloud Basic Setup

We will use gcloud cli for the following sections, so install and initialize the Google Cloud CLI if not done yet. If you already have gcloud cli, run gcloud components update to update it to the latest version.

Create a new project for this if required, or choose an existing project, set it as default, and export the project id as a shell environment for using later. Also export the region you want to use.

export PROJECT_ID=slackbot-project
export REGION=us-central1

gcloud config set project ${PROJECT_ID}

You will have to enable billing for this project to be able to use some of the functionality we require.

You may also have to enable the Secret Manager, Cloud Functions, Cloud Build, Artifact Registry, and Logging APIs if this is the first time you're using Functions in this project. Note that some services like Secret Manager need billing to be setup before they can be enabled.

gcloud services enable --project slackbot-project \
        secretmanager.googleapis.com \
        cloudfunctions.googleapis.com \
        cloudbuild.googleapis.com \
        artifactregistry.googleapis.com \
        logging.googleapis.com

Create a service account

By default, Cloud Functions uses a default service account as its identity for function execution. These default service accounts have the Editor role, which allows them broad access to many Google Cloud services. Of course, this is not recommended for production, so we will create a new service account for this and grant it the minimum permissions that it requires.

SA_NAME=channelbot-sa
SA_EMAIL=${SA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com

gcloud iam service-accounts create ${SA_NAME} \
    --description="Service Account for ChannelBot slackbot" \
    --display-name="ChannelBot SlackBot SA"

Store secrets and give permissions to service account

First, we need to store the secrets in Secret Manager.

printf $SLACK_BOT_TOKEN | gcloud secrets create \
    channelbot-slack-bot-token --data-file=- \
    --project=${PROJECT_ID} \
    --replication-policy=user-managed \
    --locations=${REGION}

printf $SLACK_SIGNING_SECRET | gcloud secrets create \
    channelbot-slack-signing-secret --data-file=- \
    --project=${PROJECT_ID} \
    --replication-policy=user-managed \
    --locations=${REGION}

And give our service account the roles/secretmanager.secretAccessor role on these secrets.

gcloud secrets add-iam-policy-binding \
    projects/${PROJECT_ID}/secrets/channelbot-slack-bot-token \
    --member serviceAccount:${SA_EMAIL} \
    --role roles/secretmanager.secretAccessor

gcloud secrets add-iam-policy-binding \
    projects/${PROJECT_ID}/secrets/channelbot-slack-signing-secret \
    --member serviceAccount:${SA_EMAIL} \
    --role roles/secretmanager.secretAccessor

Create and deploy the function

Here's a simple HTTP function that sends a message to slack on any HTTP call:

import functions_framework
from slack_bolt import App

# process_before_response must be True when running on FaaS
app = App(process_before_response=True)

print('Function has started')

@functions_framework.http
def send_to_slack(request):
    print('send_to_slack triggered')
    channel = '#general'
    text = 'Hello from Google Cloud Functions!'
    app.client.chat_postMessage(channel=channel, text=text)
    return 'Sent to slack!'

functions-framework
slack_bolt

Assuming main.py and requirements.txt are present in src-v1 folder, deploy using:

gcloud beta functions deploy channelbot-send-to-slack \
    --gen2 \
    --runtime python310 \
    --project=${PROJECT_ID} \
    --service-account=${SA_EMAIL} \
    --source ./src-v1 \
    --entry-point send_to_slack \
    --trigger-http \
    --allow-unauthenticated \
    --region ${REGION} \
    --memory=128MiB \
    --min-instances=0 \
    --max-instances=1 \
    --set-secrets 'SLACK_BOT_TOKEN=channelbot-slack-bot-token:latest,SLACK_SIGNING_SECRET=channelbot-slack-signing-secret:latest' \
    --timeout 60s

💡We're using --allow-unauthenticated here just to test it out. It will be removed in later sections.

Test it out

Once the deployment is complete, we can view the function logs using:

gcloud beta functions logs read channelbot-send-to-slack \
	--project ${PROJECT_ID} --gen2

If everything was successful above, once of the recent log statements should say: Function has started.

Next, add the bot to the #general channel using /invite @ChannelBot in the general channel on your slack workspace.

Next, find the service endpoint using:

gcloud functions describe channelbot-send-to-slack \
    --project ${PROJECT_ID} \
    --gen2 \
    --region ${REGION} \
    --format "value(serviceConfig.uri)"

This will give a URL like https://channelbot-send-to-slack-ga6Ofi9to0-uc.a.run.app.

To trigger the channel post, just do curl ${SERVICE_URL}. This should result in a test message from ChannelBot to the #general channel.

Trigger via Google Pub/Sub

Now, instead of an unauthenticated HTTP trigger, we would like to trigger this via Google Pub/Sub. We would also like to pass the channel name and the message to post in the event.

Google Pub/Sub basics

Pub/Sub enables you to create systems of event producers and consumers, called publishers and subscribers. Publishers communicate with subscribers asynchronously by broadcasting events. Some core concepts:

• Topic. A named resource to which messages are sent by publishers.
• Subscription. A named resource representing the stream of messages from a single, specific topic, to be delivered to the subscribing application.
• Message. The combination of data and (optional) attributes that a publisher sends to a topic and is eventually delivered to subscribers.
• Publisher. An application that creates and sends messages to a single or multiple topics.

In this section, we will create a topic, create a subscription for our cloud function to listen to messages to that topic, and produce messages manually to that topic using gcloud cli. The message will contain the channel name and message to post, and the cloud function will post that message to the specified slack channel.

Create pub/sub topic

First, we need to create a topic.

export PUBSUB_TOPIC=channelbot-pubsub
gcloud pubsub topics create ${PUBSUB_TOPIC} \
    --project ${PROJECT_ID}

Grant permissions to the service account

Next, we need to give the roles/pubsub.editor role to the service account we're using for the function execution so that it can create a subscription to this pub/sub topic.

gcloud pubsub topics add-iam-policy-binding ${PUBSUB_TOPIC} \
    --project ${PROJECT_ID} \
    --member serviceAccount:${SA_EMAIL} \
    --role roles/pubsub.editor

Update the function code

Here's the main.py we'll need to listen to pub/sub events, extract channel and text, and sent it to slack:

import base64
import json
import functions_framework
from slack_bolt import App

# process_before_response must be True when running on FaaS
app = App(process_before_response=True)

print('Function has started')

# Triggered from a message on a Cloud Pub/Sub topic.
@functions_framework.cloud_event
def pubsub_handler(cloud_event):
    try:
        data = base64.b64decode(
            cloud_event.data["message"]["data"]).decode()
        print("Received from pub/sub: %s" % data)
        event_data = json.loads(data)
        channel = event_data["channel"]
        text = event_data["text"]
        app.client.chat_postMessage(channel=channel, text=text)
    except Exception as E:
        print("Error decoding message: %s" % E)

Before deploying, we also need to enable the Eventarc API in this project.

gcloud services enable --project ${PROJECT_ID} \
    eventarc.googleapis.com

Deploy and Test

Now, there's a slightly modified version of the deploy command to deploy this:

gcloud beta functions deploy channelbot-send-to-slack \
    --gen2 \
    --runtime python310 \
    --project ${PROJECT_ID} \
    --service-account ${SA_EMAIL} \
    --source ./src-v2 \
    --entry-point pubsub_handler \
    --trigger-topic ${PUBSUB_TOPIC} \
    --region ${REGION} \
    --memory 128MiB \
    --min-instances 0 \
    --max-instances 1 \
    --set-secrets 'SLACK_BOT_TOKEN=channelbot-slack-bot-token:latest,SLACK_SIGNING_SECRET=channelbot-slack-signing-secret:latest' \
    --timeout 60s

The main changes are:

• Changed entry-point to the new function pubsub_handler
• Replaced --trigger-http with --trigger-topic
• Removed --allow-unauthenticated

Before sending a pub/sub message, we will also need to give the roles/run.invoker role to our service account to be able to trigger our newly deployed function.

gcloud run services add-iam-policy-binding channelbot-send-to-slack \
    --project ${PROJECT_ID} \
    --region ${REGION} \
    --member=serviceAccount:${SA_EMAIL} \
    --role=roles/run.invoker

To test this out, we can send a pub/sub message using gcloud cli:

gcloud pubsub topics publish ${PUBSUB_TOPIC} \
    --project ${PROJECT_ID} \
    --message '{"channel": "#general", "text": "Hello from Cloud Pub/Sub!"}'

Post new channels roundup using cloud scheduler

Manually post recently created channels

Now that we have gained the capability to trigger a message from pub/sub to slack, we can add some logic to fetch the recently created channels from slack and post it as a message on this trigger.

Here's the modified main.py to do this:

import base64
import json
import time
import functions_framework
from slack_bolt import App

# process_before_response must be True when running on FaaS
app = App(process_before_response=True)

print('Function has started')

# Triggered from a message on a Cloud Pub/Sub topic.
@functions_framework.cloud_event
def pubsub_handler(cloud_event):
    try:
        data = base64.b64decode(
            cloud_event.data["message"]["data"]).decode()
        print("Received from pub/sub: %s" % data)
        event_data = json.loads(data)
        max_days = event_data["max_days"] # Max age of channels
        channel = event_data["channel"]
        recent_channels = get_recent_channels(app, max_days)
        if len(recent_channels) > 0:
            blocks, text = format_channels(recent_channels, max_days)
            app.client.chat_postMessage(channel=channel, text=text,
                                        blocks=blocks)
        else:
            print("No recent channels")
    except Exception as E:
        print("Error decoding message: %s" % E)


def get_recent_channels(app, max_days):
    max_age_s = max_days * 24 * 60 * 60
    result = app.client.conversations_list()
    all = result["channels"]
    now = time.time()
    return [ c for c in all if (now - c["created"] <= max_age_s) ]

def format_channels(channels, max_days):
    text = ("%s channels created in the last %s day(s):" %
            (len(channels), max_days))
    blocks = [{
        "type": "header",
        "text": {
            "type": "plain_text",
            "text": text
        }
    }]
    summary = ""
    for c in channels:
        summary += "\n*<#%s>*: %s" % (c["id"], c["purpose"]["value"])
    blocks.append({
        "type": "section",
        "text": {
            "type": "mrkdwn",
            "text": summary
        }
    })
    return blocks, text

After deploying this with the same command above (just change --source ./src-v2 to --source ./src-v3), we can send a pub/sub event to trigger it:

gcloud pubsub topics publish ${PUBSUB_TOPIC} \
    --project ${PROJECT_ID} \
    --message '{"channel": "#general", "max_days": 7}'

And it posts a message like this:

Create schedule

Next, we want to periodically schedule this message. For this, we will configure a cron job in Google Cloud Scheduler to send a Pub/Sub event with the required parameters periodically.

Before we create a schedule, we will have to enable the Cloud Scheduler API

gcloud services enable --project ${PROJECT_ID} \
    cloudscheduler.googleapis.com

To schedule the Pub/Sub trigger at 1600 hours UTC time every day:

gcloud scheduler jobs create pubsub channelbot-job \
    --project ${PROJECT_ID} \
    --location ${REGION} \
    --schedule "0 16 * * *" \
    --time-zone "UTC" \
    --topic ${PUBSUB_TOPIC} \
    --message-body '{"channel": "#general", "max_days": 1}'

After this, a Pub/Sub event should be fired to the channelbot-pubsub topic every day, which should result in a slack message to #general with a list of channels created in the last day.

Closing Thoughts

Full code samples for this can be found in this github repo. I've also included a Makefile with targets split into sections associated with the different steps in this post.

I also plan to follow this up with a part 2 where we will use slack's slash commands to allow the end-user of this bot to setup the channel and frequency of posting of the recent channels list, and even configure multiple schedules. Please comment below if this is something you will be interested in.