SDS Feature Wish List

updated: 2009-02-01 (v1.2)

by Mike Amundsen

This page contains features and other 'wish list' items I've collected while working with Microsoft's SDS. You might notice that most of the requests I list here relate to 'application protocol' level items - HTTP. That's because I am currently most interested in these issues and I have not found too many others covering the same space. I know there are lots of important database-related features that SDS can include, too. I leave the work of documenting those feature requests to others.

The SDS team maintains an active forum for anyone who wishes to join. They also have an SDS Team blog. You'll find lots of the items on this page covered in these other sources. However, I had a hard time keeping track of progress on several of these items. So I started this page to try to keep a easy-to-find linkable list of items related to SDS.

Of course, this list is in no way official or comprehensive. It's not meant to be a 'sink' for SDS feature requests. If you like some of the stuff here - cool. If you think some of it is just lame, I accept that. If you think I've missed some important items, you're not be alone. I encourage you to publish your own feature lists for SDS to consider. Feel free to post your comments on my SDS Feedback page.

Wish List

Support Permission-style Access Security based on HTTP Method + URI 2008-12-05

Currently SDS a single user account that has full access to all operations (Create, Read, Update, Delete) for all SDS objects (Authorities[no delete], Containers, Entities). SDS should also provide support for multiple user accounts with granular access to the same SDS objects. In keeping with SDS's support for REST-like interaction, it is proposed that access security for SDS be based on permissions (and not roles) and that the permissions be mapped directly to HTTP methods: POST(Create), GET(Read), PUT(Update), and DELETE(Delete).

Further, it is proposed that these permissions be applied directly to the URIs that are used to request SDS objects. For example http://data.database.windows.net [POST, GET] is a security rule that allows only Read and Create rights for, in this case Authorities. This pattern can be expanded by using a templating notation (i.e. regular expressions). For example, the following rule defines read-only access for the selected authority/container: http://my-authority.data.database.windows.net/my-container/(.*) [GET].

Finally, these security access rules (URI+HTTP-Method-list) can be associated with one or more user accounts to complete the access control features of SDS. Also, it is possible to associate one or more security access rules with a role|group and then associate one or more user accounts with that group. In this way, SDS can implement support for 'role-based' security.

NOTE It is not recommended that access security be based on values that do not appear in the URI (i.e. cookies, custom headers, etc.) since this can break intermediaries (caching, security proxies, etc.) and could result in the exact same URI returning different data to the user-agent based on the contents of these 'hidden' values. For the same reasons, it is not recommended that access security be based on the contents of Entity objects (i.e. Kind) since this is data 'hidden' in the body of the response and not available via the URI itself.

Provide Support for HTTP Digest Authentication 2008-12-05

Currently SDS supports the Basic form of HTTP Authentication. In addition, SDS should offer user-agents the option of using the Digest form. Both forms are essential for supporting automated user-agent interaction with the SDS data servers and the Digest authentication algorithm is more secure than the Basic algorithm.

Provide Support for Paging Indicators as Custom HTTP Headers (2008-12-02)

Currently SDS returns a maximum of 500 entities per query regardless of the number of entities that match the query criteria. Going forward it is important to provide support for paging control. Ideally this should be done as message metadata, not as part of the query itself. One possible exception would be to use the Take(n) query function currently supported by SDS as part of the paging control metadata. A good example of a pattern already 'vetted' by the community is the Feed Paging and Archiving standard developed for the Atom MIME-type. Since SDS seems committed to supporting Atom at some point in the future, using the above-mentioned RFC standard seems a good target.

For MIME-types other than Atom, it is suggested that a set of custom HTTP Headers be employed to cover the same values. Suggested names are: x-paging-first, x-paging-previous, x-paging-next, x-paging-last. In line with RFC5005 (see above), the values should be links, not scalar values. This allows the greatest flexibility for future changes to the way paging is used and/or computed (i.e. x-paging-next: http://example.com/container/entity-25)

Another possible solution would be to consider the HTTP Header Linking draft from Mark Nottingham (i.e. link:<http://example.com/container/>; rel="first"). While this draft is currently dormant, it has potential for wider acceptance over adopting custom HTTP headers.

Support Cache-Control:no-cache and/or Pragma:no-cache HTTP Header (2008-11-30)
SDS should support the use of Cache-Control:no-cache (HTTP/1.1) and/or Pragma:no-cache (HTTP/1.0). This allows the user agent to indicate to the server that a fresh copy of the resource should be sent regardless of the expiration or validation caching in force. Specifically, in cases where user agents send SDS the Cache-control:no-cache or Pragma:no-cache headers, any If-None-Match header should be ignored. This allows user agents to explicitly request a fresh copy of the resource and update any intermediary caches in the path between the user agent and the server.
Publish the Schema referenced in SDS EntitySet and Entity documents (2008-11-30)
Currently, SDS responses (EntitySet and Entity documents) contain several references to custom schema (http://www.w3.org/2001/XMLSchema, http://www.w3.org/2001/XMLSchema-instance, and http://schemas.microsoft.com/sitka/2008/03/ ). The "sitka" schema link currently returns 404. It would be better if this link pointed to the actual "sitka" schema defintion used by SDS. This could be either a simple XSD/DTD document or, like many references (see the "XMLSchema" example), an HTML document that includes details on how to find the most current implementation of the schema in question.
Register the SDS custom MIME-Type w/ IANA (2008-11-30)
It's good that SDS uses a custom MIME-type (application/x-ssds+xml) to identify EntitySet and Entity documents. It would be better if this custom MIME-type was properly registered with the IANA. This would make it easier for others to find and build in support for the custom MIME-type.
Support Last-Modified. If-Modified-Since, and If-Unmodified-Since Headers (2008-11-29)
Currently, SDS supports conditional GET and conditional PUT via the ETag, If-None-Match (for conditional GET), and If-Match (for conditional PUT). In addition to ETags, SDS should also support Last-Modified, If-Modified-Since (for conditional GET), and If-Unmodified-Since (for conditional PUT). Adding support for these HTTP Headers will allow clients to use the *-Modified Headers instead of ETag headers. Supporting this feature can reduce charges to the SDS account-holder.
Support multiple media-types for the same Entity (2008-11-29)
Currently, with the exception of BLOB support, SDS only supports POX (plain old XML) responses when requesting Authority, Container, and Entity objects. That means clients that wish to consume other MIME-types (Atom, JSON, etc.) must rely on a 'transformation proxy' to convert the data to the desired format. SDS should support server-driven content negotation using standard MIME-types via the Accept and Content-Type HTTP headers. That means the URI for the resource (/container/entity-id) should not change depending on the MIME-type (i.e. /atom/container/entity-id is not allowed). Text-based media types that are desirable include Atom, JSON, HTML, and CSV. Additional types that would improve the user-agent experience include PDF and SVG.
Support Content-MD5 Header (2008-11-29)
Supporting the Content-MD5 header allows the user-agent to send an MD5 hash of the entity body with the POST/PUT request. The server can then (optionally) perform an MD5 hash and compare the results with the hash supplied by the user-agent. If they do not match, the server can reject the request and send an error response code. This is especially helpful for large and/or binary entity bodies sent from user-agents. Supporting this feature can reduce charges to the SDS account-holder.
Support HTTP HEAD method (2008-11-29)

2008-12-03 The SDS team confirmed that, as of the Public CTP release (2008-11), SDS supports the HEAD method for all requests for 'single' objects (i.e. /v1/[container]) but not requests for 'lists' of objects (i.e. /v1/[container]?=). I'm still holding out for HEAD support for lists, too.

Using the HEAD method will allow user-agents to check for the existence of a resource (Entity) before making the actual request. This call can also be used to cut down on traffic and bandwidth since making a HEAD call w/ the server-supplied ETag will allow the user-agent to determine if a new copy of the Entity exists on the server. This can improve scalability and runtime performance. Supporting this feature can reduce charges to the SDS account-holder.

Return Location Header for POST, PUT, DELETE (2008-11-25)
When doing as POST, PUT or DELETE return the Location HTTP Header that that points to the updated resource (for POST and PUT) or the resource list (for DELETE). This value allows user-agents to perform a GET using the value in the Location header in order to be sure to get the most recent version of the resource. This is especially important when returning HTTP Status Code 201 (for POST) or 204 (for DELETE) [see below]. This can improve performance and scalability by lowering traffic between user-agent and server. Supporting this feature can reduce charges to the SDS account-holder.
Do not return a body on POST or PUT responses (2008-11-25)

2008-12-05 I confirmed that the entity body is not returned by SDS on POST or PUT. This request should not have been included in the list. The only remaining item is the need to return the Location header (see above).

Instead of returning the resulting Entity from a POST or PUT, return the appropriate HTTP Status Code with a Location HTTP Header that points to the resulting Entity. For POST, return 201 (Created). For PUT, return 204 (No Content). This is especially helpful when handling large binary objects as it cuts down on possibly needless traffic and bandwidth. Supporting this feature can reduce charges to the SDS account-holder.

Return ETags and support HTTP Status code 304 for Entity Lists/Queries (2008-11-25)
Currently SDS returns an ETag for GETs on a single Entity, but not on GETs that return more than oneEntity. This means that user-agents cannot easily cache query results (or even simple, non-filtered lists). It also means that when a user-agent repeats the same GET for a list, it has no ETag to return which means SDS cannot return a 304 (Not Modified). This increases traffic and bandwidth to SDS servers. Supporting this feature can reduce charges to the SDS account-holder.
Support Accept-Encoding and Content-Encoding Headers (2008-11-25)
SDS should support GZip and Deflate encoding when generating responses. When user agents send the Accept-Encoding HTTP header, SDS servers should evaulate the header and, when appropriate, encode the results and return the Content-Encoding HTTP Header as part of the response. Encoding reduces bandwidth for both server and user-agent. All major browsers and most HTTP libraries automatically support GZip and Deflate. This can result in better response times from the server. Supporting this feature can reduce charges to the SDS account-holder.