Enrich Streams with User Data

The purpose of this guide is to help you show how you can append user or entity data to each of your incoming events.

To append user properties and entity data to each of your events, you will utilize a custom Access Key. Access Keys are authentication tokens that can be used to grant permission to read or write data to Keen. They can be specified to automatically enrich events with entity data.

Each Access Key can be customized with a few capabilities and options:

1. Permission to read or write (or both)
2. The type of query and any specific filters to be applied on each analysis. Keen will see this and automatically apply your defined filters on all queries executed from this Access Key.
3. Properties you’d like to “Autofill” on every event streamed. This allows you to append user properties or entity data to every incoming event.

By defining the Access Key’s autofill property, you can bake a particular user, customer, or company’s entity data into each event. For example:

"autofill": {
  "user_id": 232130982,
  "name": "Some Name"
}

These properties will show up in every single event written with that key.

Enriching Incoming Webhook Data Streams

Autofill can also be used to enrich your incoming webhook data stream via Keen’s Data Enrichment Add-Ons. If you’re streaming SendGrid email data, long URL strings which exist in all Click events can be enriched and parsed to become more useful using the Keen URL Enrichment. The SendGrid data model also includes in each event the IP address, so wherever a user opens the email you can maximize the power of the Access Keys and use autofill to enrich those Opened events with IP to Geo Data.

Enrich all of your events: Keen has five data enrichment tools for even more cool analyses.

(Note: Because it’s a webhook, if a property is missing we currently have no way of notifying you if events fail. As always, test your integration.)

Generate a Key with an Autofill Definition

Our UI tool for managing Access Keys gives you a simple point-and-click method for generating keys, as well as an easy way to see all of the keys you have created.

As always, these Keys can be provisioned and customized via our lovingly-crafted API programmatically.

Below is an example Access Key definition. You’ll see the autofill property being used to include some entity data + the IP to Geo and User Agent Data Enrichments:

{
  "name": "Access Key for Acme Corp with Data Enrichments",
  "is_active": true,
  "permitted": ["writes"],
  "options": {
    "writes": {
      "autofill": {
        "customer": {
          "id": "93iskds39kd93id",
          "name": "Acme Corp."
        },
        "keen": {
          "addons": [ // Keen Addons to be used
            {
              "name": "keen:ip_to_geo", // IP to Geo parser add-on
              "input": {
                "ip": "ip_address" // Make sure the "ip_address" field exists in each event sent
              },
              "output" : "geo"
            },
            {
              "name": "keen:ua_parser", // User Agent parser add-on
              "input": {
                "ua_string": "user_agent" // Make sure the “user_agent” field exists in each event sent
              },
              "output": "tech"
            }
          ]
        }
      }
    }
  }
// continue rest of Access Key definition
}

Read more about Access Key customization.

Access Keys Best Practices

Access Keys are a way of protecting and securing your data. By defining custom permissions for different users, you can lean on Keen’s security features to serve data and customer-facing analytics instead of building your own tools from scratch.

In cases where you are preparing this data to present analytics for your customers, it is general best practice to create an Access Key for each customer as a part of your customer’s on-boarding flow into your product. In some cases, you may also want to consider creating a separate Project for each of your customers.

By providing each of your customers with their own separate project, each of your customers’ data is contained in their own dedicated data silo. Since each project receives it’s own set of API Keys, it adds an additional form of security that prevents one customer’s data from being written to and contaminating another customer’s data store. Read more about when you might want to: create a separate project for each of your customers.