OSDI

The Open Supporter Data Interface (OSDI) effort seeks to define an API and data structures for interoperability among products in the progressive cause-based, campaign and non-profit marketplace. The existence of a common API will reduce customer costs related to moving data between different systems, lower integration costs and enhance the ability of innovators to create products for the marketplace.

OSDI membership is made up of progressive vendors and organizations as well as invited non-partisan and mainstream industry vendors.

More Information about OSDI can be found at: opensupporter.org

The Github source for these documents can be found at: https://github.com/opensupporter/osdi-docs

If you are looking at those sources now and want to see the prettier github pages version, look here: https://opensupporter.github.io/osdi-docs

Experiment with our prototype server: http://api.opensupporter.org

Sections

API Overview and Structure

OSDI used a combination of approaches to provide flexible reading of data, simple operations for simple scenarios, and general purpose CRUD access.

Version

This document represents OSDI version 1.2.0

Working with OSDI in Real Life

HAL Browser

OSDI Servers SHOULD expose the HAL Browser to provide a consistent interface for developers, scripters, digital, tech and data staff to work with.

In the course of writing scripts, reports, applications and other utilities that integrate via OSDI, examining and inspecting the different resources available on a server is a significant component of time spent. By having a consistent interface to work in, customers can further decrease their costs.

Ask your OSDI vendor for the URL to their HAL Browser.

Simple Code Examples

Python - Get People
Ruby - Get People

jQuery Plugin

An OSDI jQuery plugin has been created for use with OSDI’s non-authenticated POST feature. The plugin facilitates a javascript implementation of OSDI, allowing for HTML forms to POST data into an OSDI system using AJAX in a user’s browser, all without much coding.

Learn more about the plugin and download a copy here.

REST + HAL

Generally, OSDI follows traditional RESTful practices for accessing resources and collections of resources as well as creating, editing, updating, and deleting resources.

OSDI also implements the JSON+HAL spec hypermedia standard, providing links to associated collections and resources. JSON+HAL specifies a simple way to include these links in API output. The combination of linking and a specification allows generic clients to be written and, indeed, many languages have HAL clients. Linking itself makes it easier to both reason about and write clients for an API.

In addition to providing links for associated collections and resources, JSON+HAL specifies a way to embed the actual associated resources (or collections) in the same API response as the links. For example, a request for a collection of Person resources will return a link for each Person resource, and may also return the actual Person resources to which the links point. This allows OSDI providers and users to reduce the number of server round-trips needed to retrieve a set of data, at the expense of working with larger (possibly much larger) response sizes.

Embedding is optional for associated resources or collections. However, any resources or collections that are embedded must also be accessible by link. OSDI clients may check to see if a related resource is embedded in the response, and if not present, should fall back to getting the resource via its link.

By default, server responses should expand first level instances unless otherwise specified. For example, in a response for a collection of resources, those resources should be embedded.

Finally, OSDI implements the OData query language for filtering collections.

Back to top…

Versioning

OSDI uses Semantic Versioning. In practice, this means:

  • Breaking changes will use a new major version number (eg: 2.0)
  • New features will use a minor version number (eg: 1.1)
  • Incremental Bug fixes may use a sub-minor version number (1.1.1)

Current Version

When accessing a server, a client can determine the OSDI version by examining the osdi_version attribute in the API Entry Point (AEP).

Back to top…

Helpers

OSDI also allows a client to perform a number of operations at once that in a traditionally RESTful API would take multiple requests through the use of helpers. For example, helpers can be used to create a new Person resource and register that this new person also signed a petition at the same time, something that with REST would require two operations (first creating the person, then associating them with the petition).

Back to top…

API Entry Point and linking

All access through OSDI starts at the API Entry Point (AEP). The AEP is a resource that acts like a directory of the types of resources available on a server. It also includes capability information like the maximum query pagesize and links to helper endpoints.

Your service provider can tell you what the AEP URL is for your account.

You can explore the AEP with a user-friendly interface by visiting our prototype endpoint.

Back to top…

Curies

You may have noticed that most links are prefaced with a name space “osdi” and that in the _links section there is a key labeled “curies.” The link section defines links to relationships between objects and curies define those relationships. You will find documentation on the particular relationship by using the templated curie link. For example, given the following links section:

"_links": {
    "curies": [{ "name": "osdi", "href": "http://api.opensupporter.org/docs/v1/{rel}", "templated": true }],
    "self": {
        "href": "http://api.opensupporter.org/api/v1/answers/46"
    },
    "osdi:question": {
        "href": "http://api.opensupporter.org/api/v1/questions"
    }
}

In order to fetch documentation on the question relationship, you would visit the following url: http://api.opensupporter.org/docs/v1/question

Any links not prefaced with a curie name space are defined here.

Vendors who add their own vendor-specific relationships must defined their own curie and preface their relationships with their own curie namespace. For example,

"_links": {
    "curies": [
        { "name": "osdi", "href": "http://api.opensupporter.org/docs/v1/{rel}", "templated": true },
        { "name": "fb", "href": "http://facebook.com/docs/v1/{rel}", "templated": true }
    ],
    "self": {
        "href": "http://api.opensupporter.org/api/v1/question_answers/46"
    },
    "osdi:question": {
        "href": "http://api.opensupporter.org/api/v1/questions"
    },
    "fb:profile": {
        href: "http://facebook.com/profiles/1234"
    }
}

Back to top…

Collections and Navigation

When retrieving collections, the response representation will include some common attributes.

Name Type Description
total_pages integer The number of pages applicable to this collection.
total_records integer The total number of resources matching this collection.
page integer The page number of this response.

Collection responses may include additional links for navigation to previous and next pages.

Name Description
next The link for the next page of results.
previous The link for the previous page of results.

The parameters per_page and page control pagination.

  • ?per_page specifies how many results to return per page.
  • ?page specifies the starting page to start with.

Back to top…

Resource Expansion

Consult the documentation from your vendor or implementer to determine if expansion is supported.

The server may support including related resources in responses. For example, a client might wish to have a response containing people also contain their event attendance resources. In this case, the attendance resources would be contained within an _embedded collection within each person object.

To request that the server included these related resources, the $expand query parameter is used, and the value would be the related OSDI collection, with its prefix ‘osdi:attendances’

GET https://osdi-sample-system.org/api/v1/people?$expand=osdi:attendances

The $expand parameter can contain a comma separated list of resources to include.

Certain OSDI resources have Related Objects, which are inline objects as opposed to a linked related resource collection link.

Some servers may choose not to automatically return all inline related objects, for example if collecting the needed information for the object is expensive or time consuming. In this case the server will omit those objects, unless the client includes the $expand query parameter containing the comma separated additional objects to return, specified by their attribute name (without an osdi: prefix).

GET https://osdi-sample-system.org/api/v1/people?$expand=divisions

Back to top…

Filtering Collections

When retrieving collections, a client may request that the server filter the results according to a query. OSDI makes use of a subset of the OData query language to accomplish this. The filter string is the value of the ‘filter’ query parameter.

See OData Filter Query for more information.

General information can be found at odata.org.

Conventions

  • String literals are enclosed in single quotes, eg: 'Jon'
  • Integers are not quoted, eg: 5
  • The whole query string is not enclosed in any quotes
  • Object properties are referenced using /, not ., e.g. birthdate/month

Operators

OSDI supports the following OData operators:

Name Description Example
eq Exact match first_name eq ‘John’
ne Not Equal exact match first_name ne ‘John’
gt Greater than birthdate/month gt 1980
ge Greater or equal than created gt ‘2013-11-17T18:27:35-05’
lt Less than birthdate/year lt 1980
le Less or equal than created le ‘2013-11-17T18:27:35-05’
or Logical OR first_name eq ‘John’ or first_name eq ‘Jon’
and Logical AND first_name eq ‘John’ and last_name eq ‘Doe’

OSDI defines the following OPTIONAL extension operators:

Name Description Example
like Case insensitive match first_name like ‘john’ # returns John or john
re Matches a regular expression first_name regexp ‘/[Rr]ob/’ # Returns robert, Robert, rob, roberto

Functions

OSDI defines the following OPTIONAL extension functions:

Name Description Example
near Returns entries near a location within a radius gender eq ‘Female’ and near(‘10011’, ‘5 miles’)

Virtual Field Names

There are some resource fields that should be searchable, but are not directly addressable using the filter syntax. This is the case for array fields and fields on array items. To allow querying of these properties, we expose Virtual Field Names at the filter level.

OSDI implementations should add special case query handlers for these filter options, where the parent resource should be returned if any of the array items match the condition.

Resource Field Virtual Field
Donation recipient.display_name recipient_display_name
Donation recipient.legal_name recipient_legal_name
Message targets.href target_href
Outreach targets.given_name target_given_name
Outreach targets.family_name target_family_name
Outreach targets.ocdid target_ocdid
Petition targets.name target_name
Person email_addresses.address email_address
Person phone_numbers.number phone_number
Person postal_addresses.postal_code postal_code
Person postal_addresses.region region

Examples

Find all males in a given ZIP code: GET /api/v1/people?filter=gender eq 'Male' and address.postal_code eq '10011'

Find new signups on or since a date and time (Eastern Time) GET /api/v1/people?filter=created ge '2013-11-17T18:27:35-05'

Find all people associated with a given email address: GET /api/v1/people?filter=email_address eq 'jane@example.com'

Back to top…

Encryption

Providers should support secure HTTPS connections using TLS 1.0 and above, and reject non-secure HTTP connections.

If necessary, providers may support non-secure HTTP connections in addition or instead.

Back to top…

Authentication

Clients and providers may use a variety of mechanisms to authenticate and authorize operations. The specification does not currently require supporting a specific method. However, there are many choices which can work with this specification.

  • Cookie-Based Authentication
  • HTTP Basic
  • HTTP Digest
  • Token-Based Authentication
  • OAuth and OAuth 2.0
  • OpenID

Future versions of this specification may officially support one or more of these methods, or provide standard ways of implementing various methods, or may in other ways be more specific about security and authentication.

Not all interactions with the API will require authentication, and some behaviors might differ based on whether your request includes authentication or not. For example, when used in a javascript application, where authentication secrets can’t be kept from users, a non-authenticated method may be used, and the server response may be truncated so as to not leak data to unauthenticated users. Some specific non-authenticated behaviors included in this specification are outlined on each resource page.

OSDI Token Based Authentication

While OSDI does not currently mandate implementation of token-based authentication, for those that do implement this method of authentication the following standard should be followed.

For header-based token authentication, the header should be named OSDI-API-Token (case sensitive), as in this example:

OSDI-API-Token: [your token here]

For URL query string-based token authentication, the query parameter should be named osdi-api-token (case insensitive), as in this example:

https://api.opensupporter.org/api/v1/?osdi-api-token=[your token here]

Back to top…

Mime Types

When sending requests or responses, the preferred mime type is application/json.

Servers and clients are strongly encouraged to be liberal in accepting entities with a missing or incorrect mime type.

Back to top…

Error Handling

If the attempt to access, update, or create the a resource or collection fails, the server shall return the appropriate HTTP error code representing the failure.

Within the response body, the server shall include descriptive information on the nature of the failure.

Back to top…

Flexibility and Server Behavior

Not all systems that implement OSDI will implement all aspects of the specification.

There are no required fields in OSDI, and many relations are left up to each individual system and server.

Some servers may support some or all of the different resource collections. For example, a peer to peer donation system might support Donations and People but not events. In order to find out what resources are available and what URIs to use to access them, do a GET on the AEP URL.

Some servers may support certain helpers and not others. The AEP and associated resources also includes links to the helper endpoints available.

Similarly, matching behavior will be determined by each implementing system. For example, some systems may match people based on email address or other information.

Deviations from RESTful Behavior

This section outlines areas where the expectations of the OSDI customer community differ from boilerplate RESTful behavior and specify best practice expected behavior.

Merging Objects (hashes) and Arrays on update

Numerous resources in OSDI contain embedded objects (hashes) and arrays. For example, on Person, Birthday and Custom Fields are Objects (hashes) and Postal Addresses is an array.

When dealing with updates to these elements of resources, servers should merge rather than replace these elements. This is more consistent with expected user behavior. This relates to helpers and POST’s to collections.

For example, sending a person_signup_helper, or a POST on the person collection with only 1 address should not delete any existing addresses. Similarly, including only 1 custom field, should not delete all existing custom fields on a resource.

The behavior for PUT is currently unspecified and up to server behavior. Contact your vendor for more details.

Deleting the full contents of an Object (hash) or Array

In order to cause the deletion of the contents of an Object (hash) or Array, a request should include that element, but set its value to null

Back to top…

Common Elements

All OSDI resources share a set of common fields for consistency. These are listed below.

Common Fields

A set of common fields that appear on all resources is included first, for reference.

Name Type Description
identifiers strings[] A unique string array of identifiers in the format [system name]:[id]. See the general concepts document for more information about identifiers.
created_date datetime A read-only property representing the date and time the resource was created on the local system.
modified_date datetime A read-only property representing the date and time the resource was last modified on the local system.

Control Headers

An “osdi:control” JSON object may contain common OSDI control headers which can be used on an OSDI POST, PUT, PATCH, Helper or other function calls to modify server behavior.

Name Type Description
return_response boolean Defaults to true, if specified as false, the operation does not need to return the resource representation in the response

Return Response

see issue 325

When mapping OSDI to certain systems, an extra proprietary API call may be necessary to return the resource representation when doing an OSDI POST, PUT, PATCH, Helper or other function. An OSDI client that does not need the representation can set this as false so a proxy or middleware server can avoid this extra call.

While OSDI clients may be uninterested in the attribute vvalues, the clients may be interested in navigating links to related collections. Therefore, instead of the representation, the proxy may return the self link, related links, and identifiers, or an empty response.

Request
POST https://osdi-sample-system.org/api/v1/people/

Header:
OSDI-API-Token:[your api key here]

{
   "osdi:control": {
      "return_response": false  /* default is true if not specified */
   },
    "identifiers": [
        "foreign_system:1"
    ],
    "family_name": "Edwin",
    "given_name": "Labadie",
    "additional_name": "Marques",
    "origin_system": "OpenSupporter",
Response
200 OK

Content-Type: application/hal+json
Cache-Control: max-age=0, private, must-revalidate

{
    "identifiers": [
        "osdi_sample_system:d91b4b2e-ae0e-4cd3-9ed7-d0ec501b0bc3"
        ],
    "_links": {
        "self": {
            "href": "https://osdi-sample-system.org/api/v1/people/d91b4b2e-ae0e-4cd3-9ed7-d0ec501b0bc3"
        },
        "osdi:attendances": {
            "href": "https://osdi-sample-system.org/api/v1/people/d91b4b2e-ae0e-4cd3-9ed7-d0ec501b0bc3/attendances"
        },
        "osdi:donations": {
            "href": "https://osdi-sample-system.org/api/v1/people/d91b4b2e-ae0e-4cd3-9ed7-d0ec501b0bc3/donations"
            }
        }
    }
}

Back to top…

Notational Conventions

In this specification, when defining models, the following notational conventions are used.

Convention Description
Type[] An array of objects of type ‘type’
Type[]* A reference to a collection of resources of type ‘type’
Type* A reference to a single resource of type ‘type’
string A string
datetime A date and time representation. In JSON this is a string. The contents of this attribute shall be ISO 8601
Object A complex attribute represented by a JSON object
decimal A number in decimal notation such as 12.15. Used for currency.
flexenum One of a list of values, or another value. For example, for party_identification on people, if the person is a Democrat they should be marked as “Democratic” with that exact spelling and casing, but if they are not one of the defined types then you can use another value instead, such as “Working Families”.

In the description of string types, sometimes the specification will list a set of acceptable values such as

Name Type Description
gender string one of “Male”, “Female”, “Other”

In these cases, the string value should conform to one of the choices unless specified otherwise.

Back to top…

References

ID Title URL
RFC5646 Tags for Identifying Languages https://tools.ietf.org/html/rfc5646

Authors and Leadership

  • Leo Aguayo, Organizer
  • Tim Anderegg, New Organizing Institute (NOI)
  • Topper Bowers, Independent
  • Beth Becker, Indigo Strategies
  • Gilbert Chan, Organizer
  • Josh Cohen, Washington United For Marriage (Editor)
  • Jeff Crigler, Catalist
  • Gustavo Costa, The Action Network
  • Michael Eskin, Blue State Digital
  • Jascha Franklin-Hodge, Blue State Digital
  • Abraham Godong, FasterCampaigns
  • Tim Gutowski, Trilogy Interactive
  • Harlan Hill, Indigo Strategies
  • Tim Holahan, BroadStripes
  • Ben Krokower, FasterCampaigns
  • Eli Lee, The Quad
  • Dave Leichtman, Microsoft Corporation
  • Marc Love, Independent
  • Walter Ludwig, Indigo Strategies
  • Drew Miller, NGP VAN
  • Joe McLaughlin, Citizen Action of New York
  • Mark Paquette, TheDataBank
  • Charles Parsons, Salsa Labs
  • Rich Ranallo, Revolution Messaging
  • Jason Rosenbaum, The Action Network
  • Shaie Sachs, NGP VAN
  • Ben Stein, Mobile Commons
  • Ben Stroud, Targetsmart Communications
  • Ray Suelzer, UFCW International Union
  • Nate Thames, ActBlue
  • Jim Pugh, ShareProgress
  • Sylvia Rolle, Washington United for Marriage
  • Chris Thomas, Sierra Club
  • Brian Vallelunga, Trilogy Interactive (Editor)
  • Sandra Wechsler, The Quad
  • Nathan Woodhull, ControlShift Labs
  • Ryan Zarkesh
  • Misha Zhurkin, Catalist
  • Kayley Whalen, Trans United
  • Hayden Mora, Trans United
  • Sonya Reynolds, Independent

Additional Acknowledgments

  • Reed Probus, Web, Logo & Graphic Design
  • Nathan Tabak, Whitepaper writing and editing
  • Anthony Whittaker, Evangelism and Booth Duty
  • Scott Wooledge, V1 Logo

Leadership

See our governance committee members and executive officers.

Back to top…

Contributing and Contact

Anyone is welcome to contribute by filing GitHub issues. To join our committees for specification discussion, please contact us at http://opensupporter.org or via email at info@opensupporter.org.

To build and view these documents locally, use Jekyll, as configured via the Gemfile in this repo:

  • cd /path/to/your/checkout/of/this/repo
  • bundle install
  • bundle exec jekyll serve

Back to top…