One of the most difficult questions when creating a new product is how to ensure that the product has a chance to attract a large user base. The good news is that the big players in the technology market such as Facebook, Google, and Twitter let developers create applications that can access the existing user data through publicly available APIs. This means, for example, that every Twitter user is a prospective customer for a well-developed Twitter application.

One of the recent and still underutilized arrivals to the public API space is the Gmail API, which provides RESTful access to a user’s Inbox messages, Inbox configuration, message labels, and the ability to draft and send messages on a user’s behalf. With this toolkit, it would be possible to implement a full web-client that replicates all of Gmail functionality anew. This is not the purpose of the API, however, and the more interesting use cases should come from innovative new features:

  • creative Inbox visualizations;
  • helping people reduce email clutter (consider trying Unroll.me if you haven’t already);
  • providing personalized analytics tools (who do you communicate with the most?);
  • extracting important event details;
  • putting reminders directly into an Inbox;
  • and many others.

In this article, I will walk you through the steps required to start building your own application on top of the Gmail API. I will explain how these steps work when developing a Web application built on top of a Java-based web server (with parts of the code snippets taken from the official API documentation). Keep in mind that I am using this example just for convenience — the API works similarly for other programming languages (C++, Python, JavaScript) as well as for desktop and mobile applications.

Accessing the Gmail API

The high-level process for accessing the Gmail API is depicted in the following figure.

Gmail API Architecture

Given that Gmail contains sensitive user data, the initial steps in the process aim to ensure that a particular user’s data are accessed exclusively with the necessary permissions. Hence, a developer needs to register a project with Google (steps 1 and 2), and the user must authorize the registered project or application to access the Gmail account (steps 4 and 5). Once this is done, the desired Gmail API methods can be invoked, while passing the required credentials with the requests (step 8). To use the API functionalities from Java, the API client needs to be included on the application’s classpath.

Step 1: Create a Gmail API Project

Creating a new Gmail API project amounts to logging into the Google Developer Console using an existing Google account, choosing the option to create a new project, and selecting Gmail API as one of the APIs used by the project. At this point, you project will be uniquely identified and Google’s monitoring systems will make sure to relate your application’s API accesses to the project. To handle project identification programmatically, the Developer Console lets you create a file with project information. You will load this file into a GoogleClientSecrets object at runtime and pass the object when invoking the API methods. Note that the project credentials will be tied to a specific domain so you will need to get a different file for each domain you intend to support.

Step 2: Authorize User Access for the Gmail API Application

To let the user authorize access to their Gmail account, each Gmail API application needs to follow the OAuth 2.0 protocol. For a web server, you will need to implement two servlets (for each servlet I provide a brief code snippet).

The first servlet, an implementation of AbstractAuthorizationCodeServlet, needs to prepare an authorization request and send it to the Google’s authorization servers. The authorization request contains the project information stored in the GoogleClientSecrets object, the type of access required (gmail.readonly if the intent is to only read messages), and a callback URI that is served by the servlet that handles the authorization server’s response.

// Serving path /authorizeuser
public class ExampleMailFetcher extends
  AbstractAuthorizationCodeServlet {
...
  @Override
  protected String getRedirectUri(HttpServletRequest req)
      throws ServletException, IOException {
    GenericUrl url = new GenericUrl(req.getRequestURL().toString());
    url.setRawPath("/authorizecallback");
    return url.build();
  }

...

  @Override
  protected AuthorizationCodeFlow initializeFlow() throws IOException {

    // Load the file generated in Google Developer Console
    GoogleClientSecrets clientSecrets =
      GoogleClientSecrets.load(jsonFactory, new FileReader(CLIENT_SECRET_PATH));

    return new GoogleAuthorizationCodeFlow.Builder(
      new NetHttpTransport(), JacksonFactory.getDefaultInstance(), clientSecrets,
      Arrays.asList("https://www.googleapis.com/auth/gmail.readonly"))
      .setAccessType("offline").build();
  }
...
}

Following the AbstractAuthorizationCodeServlet‘s request, the user is redirected to a Google page in which she can login and grant the application the required access permissions. Once the permissions have been granted, the second servlet, an implementation of AbstractAuthorizationCodeCallbackServlet serving the callback URI, receives an access token for that user to be used when performing the API invocations. Conveniently enough, the token is automatically extracted from the HTTP request parameters. Typically, the callback servlet is supposed to store the token for other (business-logic) servlets to use. For simplicity, our callback servlet does not store the token. Instead, the servlet implementation simply returns an HTTP response containing promotional messages among the last 10 active threads.

// Serving path /authorizecallback
public class ExampleMailFetcherCallback extends AbstractAuthorizationCodeCallbackServlet {
...
  @Override
  protected void onSuccess(HttpServletRequest req, HttpServletResponse resp,
    Credential credential)
      throws ServletException, IOException {

    // Creates a new Gmail API client.
    Gmail service = new Gmail.Builder(new NetHttpTransport(),
      new JacksonFactory(), credential)
      .setApplicationName(“Example Fetcher”).build();

    // Retrieve a page of Threads.
    ListThreadsResponse threadsResponse = service.users().threads().list(USERNAME)
      .setMaxResults(Long.valueOf(10)).execute();
    List threads = threadsResponse.getThreads();

    // For each listed Thread, fetch the full thread by ID.
    for (Thread thread : threads) {
      Thread detailedThread = service.users().threads()
        .get(USERNAME, thread.getId()).execute();

      // Extracts the labels from the first message to see if the thread is
      // promotional.
      List
Step 3: Access the Appropriate Gmail API Methods

To invoke the API, it is necessary to create an instance of the Gmail API client. Using the client, it is possible to access the user’s message threads, individual messages, Inbox labels (for example, there is a label associated to the Promotions tab), Gmail history, draft messages, and message attachments. In Java, all these resources are available by executing appropriate commands on clientObject.users().resourceType() object (where clientObject and resourceType are placeholders).

Once an API client is created, the code in the above snippet invokes a method that lists threads for a particular user (service.users().threads().list(USERNAME)), and then iteratively invokes a method to get individual threads (service.users().threads().get(USERNAME, thread.getId())). These invocations are passed to the Gmail API servers as HTTP GET and HTTP POST requests (for example, GET https://www.googleapis.com/gmail/v1/users/userId/threads/list retrieves the list of threads). The returned objects are data structures with a number of complex fields. For example, each thread has a set of messages, while each message has an associated body, set of labels, unique thread and message identifiers, etc. Our callback servlet takes the labels of a first message in each thread, and compares the labels to a designated Promotions label. And voila, in a couple of relatively simple steps you have a functional Gmail app.

By now, I hope I achieved my goal and made you curious to find out more — please refer the official Gmail API documentation for all the technical details and examples that do not fit in a blog post. If you really plan to proceed with building an application, I would also like to highlight several other important considerations.

Gmail API Performance Considerations

Serving requests is expensive in terms of bandwidth and computation so the Gmail API enforces limits related to the rate and overall number of requests. Each application has one billion quota units per day, and for each user of your app there is a limit of 25 units per second. To make the math a bit more complex, each type of request is worth a particular number of units – getting a thread costs 10 units, listing the Inbox labels costs 1 unit, while sending a new message on the user’s behalf costs 20 units. To illustrate what this entails in practical terms, I will analyze a couple of simple use cases.

  • Retrieving the last 10 message threads, excluding attachments, costs 110 quota units. This means that the operation will not be faster than 5 seconds.
  • Retrieving the full Inbox, assuming 1,000 threads and 100 attachments, costs 10,600 quota units. This means that you would be able to serve less than 100,000 users daily.

The conclusion that can be drawn is that the limits are likely acceptable in many cases, but the creators of very successful applications will have to pay extra to increase the “freebee” limits.

Gmail API Complexity

The sensitivity of user data you can access through the API also means that there is an added level of legal and technical complexity.

  • As you sign up to use the API, you will need to acknowledge a set of terms and conditions. While legalese is not the easiest language to read, you should spend time reading the documentation and understand how you are supposed to use the user data.
  • Unless you are developing a transient application, the data you retrieve from the API will preferably be stored in a secure manner. Hence, you should consider employing some sort of an encrypted storage (one example is encrypted Datastore that is part of Google’s technology offering).
  • Your application needs to be capable of relating the user data with a particular user (especially if the user is to access the data at a later time). Hence, your application should have user authentication, while the stored Gmail data should be keyed by the user ID used in your application. If this is not done carefully, you can easily end up in a ruinous scenario in which one user is reading another user’s Inbox messages.
  • The basic tokens you get from the authorization server have a short lifetime. To perform longer transactions, your application will need to use special refresh tokens. These tokens have a longer lifetime and are repeatedly exchanged for new short-lifetime tokens whenever an old one expires.

Thank you for reading the article and don’t hesitate to contact me with comments and questions.