You may need to access data from a source not already supported by Lucid. In order to do this, you have two choices:

Fetch data using EditorClient.oauthXhr and add it to the document by creating data sources and collections in your extension as described here.
Create a data connector which can access and manage data in response to requests from Lucid's servers, and add the data connector to your extension.

For simple integrations, the first approach may be entirely sufficient. However, if you want your data to update when changes are made to it in Lucid documents, or if you want updates that happen in the data's source to be automatically reflected in Lucid documents, then you'll need to create a data connector.

A data connector is a collection of callback endpoints that translate between external representations of data and Lucid's representation. Data connectors are bidirectional, allowing for data updates to flow from external sources into Lucid documents and from Lucid documents back to their external sources. A data connector must provide at least one URL at which it can respond to requests made by Lucid's servers.

When Lucid makes a request to your data connector, it will come in the form of a data action. Data actions are triggered either explicitly by code in your extension, or automatically based on user interactions with Lucid documents. Each data action has a name, a callback url suffix, an OAuth token for accessing the data on behalf of the requesting user, and other information that is relevant to fulfilling the user's request.

To add a data connector to your extension you must:

Declare your data connector in your manifest.json.
Implement your data connector.
Expose a URL Lucid's servers can use to make requests to your data connector.

Declare your data connector

To declare a data connector, include an array of dataConnectors in your manifest.json. Each data connector must contain the following:

Field	Description	Example
name	Then name you will use to refer to this data connector in your extension	DemoDataConnector
oauthProviderName	The name of an OAuth provider defined in your `mainfest.json`	asana
callbackBaseUrl	The base url Lucid will send callback events to	https://www.example.com/
dataActions	A mapping of the data actions supported by this data connector to the url suffix that should be appended to the callbackBaseUrl when making requests for that type of data action.	{"Import" : "import"}

Any time you add or update declarations for any data connectors, you will need to package and upload your manifest, then install the extension for yourself again before your data connector can be used by your extension.

Implement your data connector

As an example, let's start with an "Import" data action which instructs the data connector what data to import onto a document.

To trigger an "Import" data action from your extension, call performDataAction on the EditorClient in lucid-extension-sdk:

const client = new EditorClient();
client.performDataAction({
    dataConnectorName: 'DemoDataConnector',
    actionName: 'Import',
    actionData: {'requestedItems': ['id-1', 'id-2']},
    asynchronous: true,
});

If your manifest.json defines callbackBaseUrl = https://www.example.com/ and dataActions = {"Import" : "import"}, Lucid will make a POST request to https://www.example.com/import with a data action as the request body. The data action will include {'requestedItems': ['id-1', 'id-2']} in its data, and because the action was called with asynchronous: true it will also include a token that can be used to POST the requested data back to the Lucid. Additionally, the data action will include an OAuth access token that can be used to access the data on behalf of the user who triggered the data action. The access token can be found in the body of the data action under action.context.userCredential as well as in the Authorization header of the request.

For your convenience, the lucid-extension-sdk provides helpful wrappers for implementing your data connector that will handle request signature validation, data action routing, and more. The data connector itself can be defined using the DataConnector class. Request handling for data actions can be added by calling either defineAsynchronousAction, or defineAction on the Data connector, and providing the implementation for data actions with a specified name:

new DataConnector(new DataConnectorClient(cryptoDependencies)).defineAsynchronousAction('Import', async (action) => {
    const client = action.client; // <- an authorized client for sending data back to the document
    const actionName = action.name; // <- "Import"
    const actionData = action.data; // <- {'requestedItems': ['id-1', 'id-2']}
    const userCredential = action.context.userCredential; // <- the OAuth access token for the user who triggered the data action
});

To add a new collection to the document containing the requested items, you must first define what the data will look like using a schema. For this example, you will use data of the following type:

type MyItemType = {id: string; name: string; age: number; isSingle: boolean};

You can define the schema for your data by calling declareSchema and specifying the types for each of the fields in your data:

const myCollectionSchema = declareSchema({
    primaryKey: ['id'],
    fields: {
        'id': {type: ScalarFieldTypeEnum.STRING},
        'name': {type: ScalarFieldTypeEnum.STRING},
        'age': {type: ScalarFieldTypeEnum.NUMBER},
        'isSingle': {type: ScalarFieldTypeEnum.BOOLEAN},
    },
});

// Infer the TS type:
type MyItemType = ItemType<typeof myCollectionSchema.example>;

let dataItem: MyItemType = {id: 'id-1', name: 'John', age: 30, isSingle: true};

To fulfill the import request, retrieve the data that was requested, then use the authorized client provided to post the data back to the document:

const makeDataConnector(
    client: DataConnectorClient,
) => {
    return new DataConnector(client).defineAsynchronousAction('Import', async (action) => {
        const itemsToAdd: MyItemType[] = [
            {id: 'id-1', name: 'John Doe', age: 30, isSingle: true},
            {id: 'id-2', name: 'Jane Doe', age: 31, isSingle: true},
        ];

        action.client.update({
            dataSourceName: 'Demo Data Source',
            collections: {
                'My Collection': {
                    schema: {
                        fields: myCollectionSchema.array,
                        primaryKey: myCollectionSchema.primaryKey.elements,
                    },
                    patch: {
                        items: myCollectionSchema.fromItems(itemsToAdd),
                    },
                },
            },
        });
    });
}

Field labels

When fields are displayed in the card details panels, or in other visualizations, the field name will be used as the label for the field. You can specify the labels that will be used by providing a custom label as part of the schema while adding a collection. This is done by adding a collection to a data source that has been imported. Include the fields from the source you want to display, specify which field is the primary key, then add field labels for any field names you want to override:

source.addCollection('track-tickets', {
    fields: [
        {name: 'id', type: ScalarFieldTypeEnum.STRING},
        {name: 'description', type: ScalarFieldTypeEnum.STRING},
        {name: 'assigned', type: [ScalarFieldTypeEnum.STRING, ScalarFieldTypeEnum.NULL]},
        {name: 'state', type: [ScalarFieldTypeEnum.STRING, ScalarFieldTypeEnum.NULL]},
    ],
    primaryKey: ['id'],
    fieldLabels: {
        'id': 'Ticket',
        'description': 'Description',
        'assigned': 'Assigned',
        'state': 'State',
    },
});

Expose a URL for your data connector

The final step is to use your data connector to handle incoming requests by calling dataConnector.runAction:

const response = await dataConnector.runAction(requestUrl, requestHeaders, requestBodyAsJson);

As an example, this code sets up a simple development server using node express that will accept and respond to requests made to your data connector:

import {DataConnectorClient} from 'lucid-extension-sdk';
import {makeDataConnector} from './dataconnector';
import * as express from 'express';
import * as crypto from 'crypto';

const client = new DataConnectorClient({Buffer, crypto});
const dataConnector = makeDataConnector(client);
dataConnector.runDebugServer({express});

node ./serve.js

You will need to expose the URL you are using to accept requests publicly so that Lucid's servers can send it requests.

Examples

For a more thorough example, you can look at the source code for Lucid's Asana Cards extension here.
You can find the extension here if you want to try it out.