Here's how the overall flow will look like:

We want this roundup post triggered on some schedule (maybe daily), so the Cloud Scheduler is required to send an event to a Google Pub/Sub topic that triggers our cloud function, which queries the slack API to get channels details, filter recently created, and post it back to a slack channel. Secret Manager is used to securely store slack's bot token and signing secret.

Note that the credentials shown in any screenshots below are not valid.

Create the slack app

The first step will be to create the slack app. Go to https://api.slack.com and click on "Create an app". Choose "From scratch" in the first dialog; enter an app name and choose a workspace for your app in the second dialog. In the next screen, copy the "Signing Secret" from the "App Credentials" section and save it for later use.

Next, go to the "OAuth and Permissions" tab from the left sidebar, and scroll down to "Scopes" -> "Bot Token Scopes". Here, add the scopes:

• channels:read: required to query public channels and find their creation times
• chat:write: required to write to a channel (where the bot is invited)

Next, scroll up on the same screen and click "Install to Workspace" to install to your workspace. Click "Allow" in the next screen to allow the installation. Next, copy the "Bot User OAuth Token" from the "OAuth Tokens for Your Workspace" section on the same page and save it for later use.

💡Keep track of the Bot User OAuth Token and Signing Secret you copied above.

Post to a Slack channel from a Google Cloud Function

Next, we will try to use the credentials copied above to enable a Google Cloud Function to send a message to a Slack channel.

Google Cloud Basic Setup

We will use gcloud cli for the following sections, so install and initialize the Google Cloud CLI if not done yet. If you already have gcloud cli, run gcloud components update to update it to the latest version.

Create a new project for this if required, or choose an existing project, set it as default, and export the project id as a shell environment for using later. Also export the region you want to use.

export PROJECT_ID=slackbot-project
export REGION=us-central1

gcloud config set project ${PROJECT_ID}

You will have to enable billing for this project to be able to use some of the functionality we require.

You may also have to enable the Secret Manager, Cloud Functions, Cloud Build, Artifact Registry, and Logging APIs if this is the first time you're using Functions in this project. Note that some services like Secret Manager need billing to be setup before they can be enabled.

gcloud services enable --project slackbot-project \
        secretmanager.googleapis.com \
        cloudfunctions.googleapis.com \
        cloudbuild.googleapis.com \
        artifactregistry.googleapis.com \
        logging.googleapis.com

Create a service account

By default, Cloud Functions uses a default service account as its identity for function execution. These default service accounts have the Editor role, which allows them broad access to many Google Cloud services. Of course, this is not recommended for production, so we will create a new service account for this and grant it the minimum permissions that it requires.

SA_NAME=channelbot-sa
SA_EMAIL=${SA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com

gcloud iam service-accounts create ${SA_NAME} \
    --description="Service Account for ChannelBot slackbot" \
    --display-name="ChannelBot SlackBot SA"

Store secrets and give permissions to service account

First, we need to store the secrets in Secret Manager.

printf $SLACK_BOT_TOKEN | gcloud secrets create \
    channelbot-slack-bot-token --data-file=- \
    --project=${PROJECT_ID} \
    --replication-policy=user-managed \
    --locations=${REGION}

printf $SLACK_SIGNING_SECRET | gcloud secrets create \
    channelbot-slack-signing-secret --data-file=- \
    --project=${PROJECT_ID} \
    --replication-policy=user-managed \
    --locations=${REGION}

And give our service account the roles/secretmanager.secretAccessor role on these secrets.

gcloud secrets add-iam-policy-binding \
    projects/${PROJECT_ID}/secrets/channelbot-slack-bot-token \
    --member serviceAccount:${SA_EMAIL} \
    --role roles/secretmanager.secretAccessor

gcloud secrets add-iam-policy-binding \
    projects/${PROJECT_ID}/secrets/channelbot-slack-signing-secret \
    --member serviceAccount:${SA_EMAIL} \
    --role roles/secretmanager.secretAccessor

Create and deploy the function

Here's a simple HTTP function that sends a message to slack on any HTTP call:

import functions_framework
from slack_bolt import App

# process_before_response must be True when running on FaaS
app = App(process_before_response=True)

print('Function has started')

@functions_framework.http
def send_to_slack(request):
    print('send_to_slack triggered')
    channel = '#general'
    text = 'Hello from Google Cloud Functions!'
    app.client.chat_postMessage(channel=channel, text=text)
    return 'Sent to slack!'

functions-framework
slack_bolt

Assuming main.py and requirements.txt are present in src-v1 folder, deploy using:

gcloud beta functions deploy channelbot-send-to-slack \
    --gen2 \
    --runtime python310 \
    --project=${PROJECT_ID} \
    --service-account=${SA_EMAIL} \
    --source ./src-v1 \
    --entry-point send_to_slack \
    --trigger-http \
    --allow-unauthenticated \
    --region ${REGION} \
    --memory=128MiB \
    --min-instances=0 \
    --max-instances=1 \
    --set-secrets 'SLACK_BOT_TOKEN=channelbot-slack-bot-token:latest,SLACK_SIGNING_SECRET=channelbot-slack-signing-secret:latest' \
    --timeout 60s

💡We're using --allow-unauthenticated here just to test it out. It will be removed in later sections.

Test it out

Once the deployment is complete, we can view the function logs using:

gcloud beta functions logs read channelbot-send-to-slack \
	--project ${PROJECT_ID} --gen2

If everything was successful above, once of the recent log statements should say: Function has started.

Next, add the bot to the #general channel using /invite @ChannelBot in the general channel on your slack workspace.

Next, find the service endpoint using:

gcloud functions describe channelbot-send-to-slack \
    --project ${PROJECT_ID} \
    --gen2 \
    --region ${REGION} \
    --format "value(serviceConfig.uri)"

This will give a URL like https://channelbot-send-to-slack-ga6Ofi9to0-uc.a.run.app.

To trigger the channel post, just do curl ${SERVICE_URL}. This should result in a test message from ChannelBot to the #general channel.

Trigger via Google Pub/Sub

Now, instead of an unauthenticated HTTP trigger, we would like to trigger this via Google Pub/Sub. We would also like to pass the channel name and the message to post in the event.

Google Pub/Sub basics

Pub/Sub enables you to create systems of event producers and consumers, called publishers and subscribers. Publishers communicate with subscribers asynchronously by broadcasting events. Some core concepts:

• Topic. A named resource to which messages are sent by publishers.
• Subscription. A named resource representing the stream of messages from a single, specific topic, to be delivered to the subscribing application.
• Message. The combination of data and (optional) attributes that a publisher sends to a topic and is eventually delivered to subscribers.
• Publisher. An application that creates and sends messages to a single or multiple topics.

In this section, we will create a topic, create a subscription for our cloud function to listen to messages to that topic, and produce messages manually to that topic using gcloud cli. The message will contain the channel name and message to post, and the cloud function will post that message to the specified slack channel.

Create pub/sub topic

First, we need to create a topic.

export PUBSUB_TOPIC=channelbot-pubsub
gcloud pubsub topics create ${PUBSUB_TOPIC} \
    --project ${PROJECT_ID}

Grant permissions to the service account

Next, we need to give the roles/pubsub.editor role to the service account we're using for the function execution so that it can create a subscription to this pub/sub topic.

gcloud pubsub topics add-iam-policy-binding ${PUBSUB_TOPIC} \
    --project ${PROJECT_ID} \
    --member serviceAccount:${SA_EMAIL} \
    --role roles/pubsub.editor

Update the function code

Here's the main.py we'll need to listen to pub/sub events, extract channel and text, and sent it to slack:

import base64
import json
import functions_framework
from slack_bolt import App

# process_before_response must be True when running on FaaS
app = App(process_before_response=True)

print('Function has started')

# Triggered from a message on a Cloud Pub/Sub topic.
@functions_framework.cloud_event
def pubsub_handler(cloud_event):
    try:
        data = base64.b64decode(
            cloud_event.data["message"]["data"]).decode()
        print("Received from pub/sub: %s" % data)
        event_data = json.loads(data)
        channel = event_data["channel"]
        text = event_data["text"]
        app.client.chat_postMessage(channel=channel, text=text)
    except Exception as E:
        print("Error decoding message: %s" % E)

Before deploying, we also need to enable the Eventarc API in this project.

gcloud services enable --project ${PROJECT_ID} \
    eventarc.googleapis.com

Deploy and Test

Now, there's a slightly modified version of the deploy command to deploy this:

gcloud beta functions deploy channelbot-send-to-slack \
    --gen2 \
    --runtime python310 \
    --project ${PROJECT_ID} \
    --service-account ${SA_EMAIL} \
    --source ./src-v2 \
    --entry-point pubsub_handler \
    --trigger-topic ${PUBSUB_TOPIC} \
    --region ${REGION} \
    --memory 128MiB \
    --min-instances 0 \
    --max-instances 1 \
    --set-secrets 'SLACK_BOT_TOKEN=channelbot-slack-bot-token:latest,SLACK_SIGNING_SECRET=channelbot-slack-signing-secret:latest' \
    --timeout 60s

The main changes are:

• Changed entry-point to the new function pubsub_handler
• Replaced --trigger-http with --trigger-topic
• Removed --allow-unauthenticated

Before sending a pub/sub message, we will also need to give the roles/run.invoker role to our service account to be able to trigger our newly deployed function.

gcloud run services add-iam-policy-binding channelbot-send-to-slack \
    --project ${PROJECT_ID} \
    --region ${REGION} \
    --member=serviceAccount:${SA_EMAIL} \
    --role=roles/run.invoker

To test this out, we can send a pub/sub message using gcloud cli:

gcloud pubsub topics publish ${PUBSUB_TOPIC} \
    --project ${PROJECT_ID} \
    --message '{"channel": "#general", "text": "Hello from Cloud Pub/Sub!"}'

Post new channels roundup using cloud scheduler

Manually post recently created channels

Now that we have gained the capability to trigger a message from pub/sub to slack, we can add some logic to fetch the recently created channels from slack and post it as a message on this trigger.

Here's the modified main.py to do this:

import base64
import json
import time
import functions_framework
from slack_bolt import App

# process_before_response must be True when running on FaaS
app = App(process_before_response=True)

print('Function has started')

# Triggered from a message on a Cloud Pub/Sub topic.
@functions_framework.cloud_event
def pubsub_handler(cloud_event):
    try:
        data = base64.b64decode(
            cloud_event.data["message"]["data"]).decode()
        print("Received from pub/sub: %s" % data)
        event_data = json.loads(data)
        max_days = event_data["max_days"] # Max age of channels
        channel = event_data["channel"]
        recent_channels = get_recent_channels(app, max_days)
        if len(recent_channels) > 0:
            blocks, text = format_channels(recent_channels, max_days)
            app.client.chat_postMessage(channel=channel, text=text,
                                        blocks=blocks)
        else:
            print("No recent channels")
    except Exception as E:
        print("Error decoding message: %s" % E)


def get_recent_channels(app, max_days):
    max_age_s = max_days * 24 * 60 * 60
    result = app.client.conversations_list()
    all = result["channels"]
    now = time.time()
    return [ c for c in all if (now - c["created"] <= max_age_s) ]

def format_channels(channels, max_days):
    text = ("%s channels created in the last %s day(s):" %
            (len(channels), max_days))
    blocks = [{
        "type": "header",
        "text": {
            "type": "plain_text",
            "text": text
        }
    }]
    summary = ""
    for c in channels:
        summary += "\n*<#%s>*: %s" % (c["id"], c["purpose"]["value"])
    blocks.append({
        "type": "section",
        "text": {
            "type": "mrkdwn",
            "text": summary
        }
    })
    return blocks, text

After deploying this with the same command above (just change --source ./src-v2 to --source ./src-v3), we can send a pub/sub event to trigger it:

gcloud pubsub topics publish ${PUBSUB_TOPIC} \
    --project ${PROJECT_ID} \
    --message '{"channel": "#general", "max_days": 7}'

And it posts a message like this:

Create schedule

Next, we want to periodically schedule this message. For this, we will configure a cron job in Google Cloud Scheduler to send a Pub/Sub event with the required parameters periodically.

Before we create a schedule, we will have to enable the Cloud Scheduler API

gcloud services enable --project ${PROJECT_ID} \
    cloudscheduler.googleapis.com

To schedule the Pub/Sub trigger at 1600 hours UTC time every day:

gcloud scheduler jobs create pubsub channelbot-job \
    --project ${PROJECT_ID} \
    --location ${REGION} \
    --schedule "0 16 * * *" \
    --time-zone "UTC" \
    --topic ${PUBSUB_TOPIC} \
    --message-body '{"channel": "#general", "max_days": 1}'

After this, a Pub/Sub event should be fired to the channelbot-pubsub topic every day, which should result in a slack message to #general with a list of channels created in the last day.

Closing Thoughts

Full code samples for this can be found in this github repo. I've also included a Makefile with targets split into sections associated with the different steps in this post.

I also plan to follow this up with a part 2 where we will use slack's slash commands to allow the end-user of this bot to setup the channel and frequency of posting of the recent channels list, and even configure multiple schedules. Please comment below if this is something you will be interested in.

Dialyzer is a static analysis tool for Erlang that identifies software discrepancies, such as definite type errors, code that has become dead or unreachable because of programming errors, and unnecessary tests, in single Erlang modules or entire (sets of) applications.

Dialyzer is integrated with rebar3 (a build tool for Erlang), and its default output looks like this:

This is a good starting point, but it's not very useful in some cases:

1. If you have lots of warnings, this output covers several screens, and it becomes difficult to parse through everything.
2. If you run this in some sort of continuous integration (like Jenkins), then the console output is not very friendly.

One way to improve this is to generate an HTML report which can be published/emailed/opened in the browser.

So, I build a rebar3 plugin that generates a nicely formatted color HTML report from the dialyzer output. The plugin can be found on hex.pm, or on github.

Usage

Make sure you're using rebar3 version 3.15 or later.

1. Add the plugin to your rebar.config:

{plugins, [
    %% from hex
    {rebar3_dialyzer_html, "0.2.0"}
    
    %% or, latest from git
    {rebar3_dialyzer_html, {git, "https://github.com/srijan/rebar3_dialyzer_html.git", {branch, "main"}}}
]}.

2. Select raw format for the dialyzer warnings file generated by rebar3 (this is a new flag available from rebar 3.15):

{dialyzer, [
    {output_format, raw}
]}.

3. Run the dialyzer_html rebar3 command:

$ rebar3 dialyzer_html          
===> Generating Dialyzer HTML Report
===> HTML Report written to _build/default/dialyzer_report.html

Here's how the report looks:

How I built the plugin

rebar3 dialyzer

The rebar3 built-in dialyzer plugin does the following:

1. Runs dialyzer with the options configured in rebar.config
2. Converts the output to ANSI color format and write it to console (it has a custom function for this formatting)
3. Converts the output to basic format (using built-in dialyzer:format/2) and write it to a dialyzer_warnings file.

I wanted to find out the easiest way to get a nicely formatted HTML report, ideally without forking the rebar3 project itself.

The first thing I needed was a way to save the raw (machine parse-able) dialyzer output to the warnings file instead of the default formatted output. For this, I submitted a new feature to the rebar3 project, and it introduces a new config to enable this. So, this plugin needs rebar3 version 3.15 or later.

Plugin vs Escript

Next, to actually parse and output the HTML file, I would need to run some Erlang code. There are two options I considered:

1. Escript called from Makefile/wrapper
   This option works okay, but we cannot re-use any rebar3 internal function or State. I wanted to use rebar3's own custom function for formatting the dialyzer warnings, so decided to not go with this option.
   
2. Custom rebar3 plugin
   Doing it this way makes it easy for anyone to use, and I can re-use things already implemented in rebar3 itself. So, I decided to use this option.
   

HTML output

Now, in the custom rebar3 plugin, I needed to convert the ANSI color-coded output given by rebar_dialyzer_format:format_warnings/2 into something for HTML.

I thought of the following options:

1. rebar3 uses the cf library to convert tagged strings to ANSI color codes. I can use something like dependency injection to replace the cf module with my own module so that the tagged strings are directly converted to HTML without even going to the intermediate ANSI color-coded format.
   
   This method seemed very hacky, so I decided not to pursue it. But, if rebar3 makes the dialyzer format interface configurable, I can reevaluate this approach.
   
2. Convert by writing a library in Erlang for ANSI code to HTML tags conversion.
   There is a library called ansi_to_html in elixir - but didn't want to add a huge dependency like that.
   But writing a new Erlang library to do this can be a future optimization.
   
3. Convert using a JS library after page load. I found a javascript library called ansi_up which can convert ANSI codes to HTML color tags, or it can add CSS classes that can be styled as required.
   

I opted for approach #3 because it was the easiest. I also grouped the warnings by app name so that all warnings for a single app are in one place, and the report includes the number of warnings per app.

Also, if the JS library could not be loaded (for example due to no internet, or any security headers), then it will still show the basic formatted output using dialyzer:format/2.

Future Improvements

1. I want to remove the dependency on Javascript, and want to write/use a pure Erlang library that can convert the ANSI codes to HTML.
2. Ideally, rebar3 itself can separate the dialyzer warning parsing and formatting into different functions, and make it possible to override the formatting function so that any plugin can pass its own formatting function into the dialyzer plugin.
3. This can even run git commands in the shell to figure out if any lines changed in the most recent commit involve a warning, and maybe highlight them especially in the report. This can be useful CI reports on pull requests.
4. Maybe make the format plug-able so the report can be saved in any JSON / XML or any custom format.

Let me know in the comments below, or on twitter/github if you have any suggestions for this plugin.

At work, we use the multi-app project pattern to organize our codebase. This lets us track everything in a single repository but still keep things isolated.

For isolation, we wanted to restrict apps to only be able to call the public interfaces of other apps (similar to facade pattern). However, since everything in Erlang is in a global namespace, nothing prevents code in one app to call the (exported) functions from another app.

Next best solution—detect the above scenario and raise warnings during code review/CI.

Xref to the rescue:

Xref is a cross reference tool that can be used for finding dependencies between functions, modules, applications and releases.

Xref includes some predefined analysis patterns that perform some common tasks like searching for undefined functions, deprecated function calls, unused exported functions, etc.

How it works: when xref server is started and some modules/applications/releases are added for analysis, it builds a Call Graph: a directed graph data structure containing the calls between functions, modules, applications or releases. It also creates an Inter Call Graph which holds information about indirect calls (chain of calls). It exposes a very powerful query language, which can be used to extract any information we want from the above graph data structures.

To demonstrate this, I created a sample multi-app repository: library_sample. There are some cross-app function calls in this code that we want to detect.

This repo is supposed to represent the functionality of a physical Library. It has four apps: library, library_api, library_catalog, and library_inventory. library_catalog has metadata about the books in the library, library_inventory has information about the availability of books, return dates, etc., library_api has HTTP handlers which call the above, and library is the main app which brings it all together.

Let’s say we want that library_api can call library_catalog and library_inventory functions, but catalog and inventory cannot call each other directly.

First, we clone the repo and run rebar3 shell:

$ git clone https://github.com/srijan/library_sample
Cloning into 'library_sample'...
remote: Enumerating objects: 29, done.
remote: Counting objects: 100% (29/29), done.
remote: Compressing objects: 100% (19/19), done.
remote: Total 29 (delta 3), reused 29 (delta 3), pack-reused 0
Unpacking objects: 100% (29/29), 910.62 KiB | 2.53 MiB/s, done.

$ cd library_sample

$ ./rebar3 shell
===> Verifying dependencies...
===> Analyzing applications...
===> Compiling library_inventory
===> Compiling library_catalog
===> Compiling library
===> Compiling library_api
Erlang/OTP 23 [erts-11.1.7] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe]

Eshell V11.1.7  (abort with ^G)
1>

Then, we start xref and add our build directory for analysis:

1> xref:start(s).
{ok,<0.185.0>}

2> xref:add_directory(s, "_build/default/lib", [{recurse, true}]).
{ok,[library_api,library_app,library_catalog,
     library_inventory,library_sample_app,library_sample_sup,
     library_sup]}

Using xref:q/2 for querying the constructed call graph:

3> xref:q(s, "E | library_inventory || library_catalog").
{ok,[]}

4> xref:q(s, "E | library_catalog || library_inventory").
{ok,[{{library_catalog,get_by_id,1},
      {library_inventory,get_available_copies,1}}]}

This means that there are no direct calls from the library_inventory application to the library_catalog application. But, there is a direct call from library_catalog:get_by_id/1 to library_inventory:get_available_copies/1.

The query E | library_catalog || library_inventory can be read as:

• E = All Call Graph Edges
• | = The subset of calls from any of the vertices. So | library_catalog creates a subset which contains calls from the library_catalog app.
• || = The subset of calls to any of the vertices. So, || library_inventory further creates a subset of the previous subset which contains calls to the library_inventory app.

To get both direct and indirect calls, closure E has to be used:

5> xref:q(s, "closure E | library_catalog || library_inventory").
{ok,[{{library_catalog,get_by_id,1},
      {library_inventory,get_all,0}},
     {{library_catalog,get_by_id,1},
      {library_inventory,get_available_copies,1}}]}

This tells us that there is an indirect direct call from library_catalog:get_by_id/1 to library_inventory:get_all/0.

The query language is very powerful, and there are more interesting examples in the xref user’s guide.

But this only runs the required queries manually in Erlang shell. We want to be able to run it in continuous integration. Luckily, rebar3 comes with a way to specify custom xref queries to run when running ./rebar3 xref, and to raise an error if they don’t match against the expected value defined.

Here’s the xref section from my rebar.config:

{xref_queries, [
                {"closure E | library_catalog || library_inventory", []},
                {"closure E | library_inventory || library_catalog", []}
               ]}.

This performs the two queries I want and matches them against the the target value of []. Sample output:

$ ./rebar3 xref
===> Verifying dependencies...
===> Analyzing applications...
===> Compiling library_inventory
===> Compiling library_catalog
===> Compiling library
===> Compiling library_api
===> Running cross reference analysis...
===> Query closure E | library_catalog || library_inventory
 answer []
 did not match [{{library_catalog,get_by_id,1},{library_inventory,get_all,0}},
                {{library_catalog,get_by_id,1},
                 {library_inventory,get_available_copies,1}}]

So, now this is ready for automation.