Azure Feature Wish List

Exploring Windows Azure services.

updated: 2009-02-25 (v1.2)

This page contains features and other 'wish list' items I've collected while working with Microsoft's Azure Storage. You might notice that most of the requests I list here relate to 'application protocol' level items - HTTP. That's because I am currently most interested in these issues and I have not found too many others covering the same space. I know there are lots of important database-related features that Azure Storage can include, too. I leave the work of documenting those feature requests to others.

The Azure team maintains serveral active forums for anyone who wishes to join. There is also an official blog. You'll find lots of the items on this page covered in these other sources. However, I had a hard time keeping track of progress on several of these items. So I started this page to try to keep a easy-to-find linkable list of items related to Azure Storage.

Of course, this list is in no way official or comprehensive. It's not meant to be a 'sink' for Azure Storage feature requests. If you like some of the stuff here - cool. If you think some of it is just lame, I accept that. If you think I've missed some important items, you're not be alone. I encourage you to publish your own feature lists for Azure Storage to consider. Feel free to post your comments on my Azure Feedback page.

Wish List

Normalize Request Signing 2009-02-25

Currently Azure Storage (Table, Blob, and Queue) uses a request-signing pattern. However, all collections do not share the same rules. The Azure Storage Services documentation lists two rules for accessing storage:

Blob and Queue Storage (Shared Key Authentication)

StringToSign = VERB + "\n" +
  Content-MD5 + "\n" +
  Content-Type + "\n" +
  Date + "\n" +
  CanonicalizedHeaders + 

Table Storage (Shared Key Authentication)

StringToSign = VERB + "\n" + 
  Content-MD5 + "\n" + 
  Content-Type + "\n" +
  Date + "\n" +

There are also different rules on how to handle the Date portion of the request signature.

Request signatures for Blob, Table, and Queue collections should be the same. Preferably, Blob and Queue collections should adopt the (simpler) Table signature rules.

In addition, using request-signing slows processing on both the client and the server. It also reduces the usability of the service since standard HTTP clients (common browsers, cUrl, wGet, WFetch) cannot form valid requests. This means standard HTTP clients must use a proxy service (custom-coded middleware) to access even public content. It would be better if Azure storage employed known public authentication schemes (HTTP Basic, HTTP Digest, etc.) and a shared authorization scheme (see elsewhere in this document).

Improve Resource Addressing by Removing URI 'Cruft' 2009-01-13

Currently, Azure Storage uses the following URI patterns to address resources:

  1. /customers()
  2. /customers()?$filter=(PartitionKe%20eq%20'preferred')
  3. /customers(PartitionKey="preferred",RowKey="c1234")

The first example returns all the Entities in the Customer table. The second example returns all the Entities in the Customers table assigned to the preferred partition. The last example returns a single Entity in the Customers collection that matches both the PartitionKey and RowKey values.

This URI pattern has too much 'technology leaking' and is unecessarily 'crufty.' Instead, the following URI patterns are simpler, more intuitive and more direct.

  1. /customers/
  2. /customers/preferred/
  3. /customers/preferred/c1234

Making this change will improve the usability of Azure Storage and increase the likelihood of adoption from a wide range of User-Agents and platforms. It will also hide the details of the underlying implementation and make it less brittle and less likely to 'break' as the underlying technology changes over time.

NOTEIt is also possible to create Entities that have no PartitionKey, but this is not advisable.

Support Permission-style Access Security based on HTTP Method + URI 2009-01-07

Currently Azure Storage a single user account that has full access to all operations (Create, Read, Update, Delete) for all Azure Storage objects (Tables and Entities). Azure Storage should also provide support for multiple user accounts with granular access to the same Azure Storage objects. In keeping with Azure Storage's support for REST-like interaction, it is proposed that access security for Azure Storage be based on permissions (and not roles) and that the permissions be mapped directly to HTTP methods: POST(Create), GET(Read), PUT(Update), and DELETE(Delete).

Further, it is proposed that these permissions be applied directly to the URIs that are used to request Azure Storage objects. For example http://[account-name] [POST, GET] is a security rule that allows only Read and Create rights for, in this case Authorities. This pattern can be expanded by using a templating notation (i.e. regular expressions). For example, the following rule defines read-only access for the selected /table: http://[account-name]*) [GET].

Finally, these security access rules (URI+HTTP-Method-list) can be associated with one or more user accounts to complete the access control features of Azure Storage. Also, it is possible to associate one or more security access rules with a role|group and then associate one or more user accounts with that group. In this way, Azure Storage can implement support for 'role-based' security.

NOTE: It is not recommended that access security be based on values that do not appear in the URI (i.e. cookies, custom headers, etc.) since this can break intermediaries (caching, security proxies, etc.) and could result in the exact same URI returning different data to the user-agent based on the contents of these 'hidden' values. For the same reasons, it is not recommended that access security be based on the contents of Entity objects (i.e. Kind) since this is data 'hidden' in the body of the response and not available via the URI itself.

Provide Support for HTTP Basic and Digest Authentication 2009-01-07

Currently Azure Storage supports a custom request-signing authetication pattern. In addition, Azure Storage should offer user-agents the option of using the Basic and/or Digest authentication in line with HTTP Authentication. While the Digest authentication algorithm is more secure than the Basic algorithm, both forms are essential for supporting automated user-agent interaction with the Azure Storage data servers.

Provide Support for Paging Indicators as Custom HTTP Headers 2009-01-07

Currently Azure Storage implements "paging" using custom "continuation" HTTP headers to provide hints for additional Entities available on the server. This offers a simple "forward-only" paging pattern, but falls short of full support for paging large amounts of data. Azure Storage needs a complete paging solution.

A good example of a pattern already 'vetted' by the community is the Feed Paging and Archiving standard developed for the Atom MIME-type. Since Azure Storage already supports Atom, using the above-mentioned RFC standard seems a good target.

For MIME-types other than Atom, it is suggested that a set of custom HTTP Headers be employed to cover the same values. Suggested names are: x-paging-first, x-paging-previous, x-paging-next, x-paging-last. In line with RFC5005 (see above), the values should be links, not scalar values. This allows the greatest flexibility for future changes to the way paging is used and/or computed (i.e. x-paging-next:

Another possible solution would be to consider the HTTP Header Linking draft from Mark Nottingham (i.e. link:<>; rel="first"). While this draft is currently dormant, it has potential for wider acceptance over adopting custom HTTP headers.

Support Cache-Control:no-cache and/or Pragma:no-cache HTTP Header 2009-01-07
Azure Storage should support the use of Cache-Control:no-cache (HTTP/1.1) and/or Pragma:no-cache (HTTP/1.0). This allows the user agent to indicate to the server that a fresh copy of the resource should be sent regardless of the expiration or validation caching in force. Specifically, in cases where user agents send Azure Storage the Cache-control:no-cache or Pragma:no-cache headers, any If-None-Match header should be ignored. This allows user agents to explicitly request a fresh copy of the resource and update any intermediary caches in the path between the user agent and the server.
Support Last-Modified. If-Modified-Since, and If-Unmodified-Since Headers 2009-01-07
Currently, Azure Storage supports conditional GET and conditional PUT via the ETag, If-None-Match (for conditional GET), and If-Match (for conditional PUT). In addition to ETags, Azure Storage should also support Last-Modified, If-Modified-Since (for conditional GET), and If-Unmodified-Since (for conditional PUT). Adding support for these HTTP Headers will allow clients to use the *-Modified Headers instead of ETag headers. Supporting this feature can reduce charges to the Azure Storage account-holder.
Support multiple media-types for the same Entity 2009-01-07
Currently, with the exception of BLOB support, Azure Table Storage only supports Atom and JSON Internet media types when requesting Table and Entity objects. That means clients that wish to consume other MIME-types (HTML, CSV, Plain Text, PDF, etc.) must rely on a 'transformation proxy' to convert the data to the desired format. Azure Storage should support server-driven content negotation using standard MIME-types via the Accept and Content-Type HTTP headers. That means the URI for the resource (/table/entity-id) should not change depending on the MIME-type (i.e. /atom/table/entity-id is not allowed). Text-based media types that are desirable include HTML, and CSV. Additional types that would improve the user-agent experience include PDF and SVG.
Support Content-MD5 Header 2009-01-07

2009-01-30 I confirmed that Azure table storing *does* support the content-MD5 header. I missed this in my initial review of the feature set.

Supporting the Content-MD5 header allows the user-agent to send an MD5 hash of the entity body with the POST/PUT request. The server can then (optionally) perform an MD5 hash and compare the results with the hash supplied by the user-agent. If they do not match, the server can reject the request and send an error response code. This is especially helpful for large and/or binary entity bodies sent from user-agents. Supporting this feature can reduce charges to the Azure Storage account-holder.
Support HTTP HEAD method 2009-01-07

Using the HEAD method will allow user-agents to check for the existence of a resource (Entity) before making the actual request. This call can also be used to cut down on traffic and bandwidth since making a HEAD call w/ the server-supplied ETag will allow the user-agent to determine if a new copy of the Entity exists on the server. This can improve scalability and runtime performance. Supporting this feature can reduce charges to the Azure Storage account-holder.

Return Location Header for POST, PUT, DELETE (2008-11-25)
When doing as POST, PUT or DELETE return the Location HTTP Header that that points to the updated resource (for POST and PUT) or the resource list (for DELETE). This value allows user-agents to perform a GET using the value in the Location header in order to be sure to get the most recent version of the resource. This is especially important when returning HTTP Status Code 201 (for POST) or 204 (for DELETE) [see below]. This can improve performance and scalability by lowering traffic between user-agent and server. Supporting this feature can reduce charges to the Azure Storage account-holder.
Do not return a body on POST or PUT responses 2009-01-07

Instead of returning the resulting Entity from a POST or PUT, return the appropriate HTTP Status Code with a Location HTTP Header that points to the resulting Entity. For POST, return 201 (Created). For PUT, return 204 (No Content). This is especially helpful when handling large binary objects as it cuts down on possibly needless traffic and bandwidth. Supporting this feature can reduce charges to the Azure Storage account-holder.

Return ETags and support HTTP Status code 304 for Entity Lists/Queries 2009-01-07
Currently Azure Storage returns an ETag for GETs on a single Entity, but not on GETs that return more than oneEntity. This means that user-agents cannot easily cache query results (or even simple, non-filtered lists). It also means that when a user-agent repeats the same GET for a list, it has no ETag to return which means Azure Storage cannot return a 304 (Not Modified). This increases traffic and bandwidth to Azure Storage servers. Supporting this feature can reduce charges to the Azure Storage account-holder.
Support Accept-Encoding and Content-Encoding Headers 2009-01-07
Azure Storage should support GZip and Deflate encoding when generating responses. When user agents send the Accept-Encoding HTTP header, Azure Storage servers should evaulate the header and, when appropriate, encode the results and return the Content-Encoding HTTP Header as part of the response. Encoding reduces bandwidth for both server and user-agent. All major browsers and most HTTP libraries automatically support GZip and Deflate. This can result in better response times from the server. Supporting this feature can reduce charges to the Azure Storage account-holder.