Continuing his web services experiments, Norm Walsh has apparently been defeated by the complexity of WSDL. Can't say that I blame him not giving up; the fewer moving parts is one of the reasons I like REST so much, there's less chance for APIs to get in the way.
I agree with Norm that some kind of web services description language is required and also that it can be less complicated that WSDL. I've been hoping that this is something that the REST community could bootstrap.
There's been work in this area already, e.g. Paul Prescod's Web Resource Description
Language and Mark Baker's RDF Forms. There was also this related thread on rest-discuss recently.
It'd be nice to see some of these proposals picked up and be refined and implemented. There seems to be agreement that there's an need for further work and specification in this area. Would be another nail in the coffin for the WS-* stack too.
While the final schedule isn't up yet, I've received an email to note that one of my XTech proposals has been accepted.
The paper is titled "Connecting Social Content Services using FOAF, RDF and REST", and here's the abstract I submitted:
A growing number of "social content" applications such as flickr, del.icio.us, audioscrobbler, and AllConsuming are making open web services part of their core offering to end users. These interfaces allow users to query, share, and manipulate the data managed on their behalf by these social content applications.
Web service interfaces make such sites more attractive to end users by removing the danger of data lock-in, while simultaneously providing the users with tools that allow them to gain the most value from their data. This translates into direct benefits for the service itself as the end users extend the reach and visibility of the application by publishing the content on their own websites and 'blogs.
Based on a brief review of the common features of a selection of these sites, this paper suggests some best practice guidelines that developers can follow when creating new service interfaces for similar applications.
Recognising that it is the end user that is the pivotal component in the success of such applications, these best practices will be used to propose a simple mechanism for connecting together social content based sites with the aim of provide richer, autonomous data exchange.
This architecture will focus on the use of FOAF descriptions of users as a service intermediary, and RESTful web services, exchanging RDF and RSS data, as the means of data exchange.
The other proposal as about my recent work to add RSS1.0/DC/PRISM based RSS feeds to the IngentaConnect website. Rather than a paper, I'll probably write up the main points in a separate blog posting.
Anyway, looking forward to the conference already, see you all there!
Starting here Norm Walsh is blogging the development of his "Where in the World?" (WITW) web service. The laudable goal being to explore web services architecture and design issues by actually building something, and seeking feedback along the way. Excellent stuff!
Elsewhere David Megginson is prompting similar discussion by posing a number of REST design questions.
I love this kind of public design discussion, so I can't help pitching to add my comments. Rather than leave them in Norm's comment form I thought I'd write them up here, and use this new fangled trackback thing, like this: nwn-whereintheworld.
Taking a step back, we can see that WITW has two different kinds of resources: people and landmarks. Actually there may be three: people, locations and landmarks. But I'm not clear whether Norm is expecting to allow someone to store a series of "usual" locations with one of them being current.
So for now lets assume there are just the two resources. A person who has some attributes, including a location, and a landmark.
Following the spirit of a RESTful design both types of resources should have URIs. WITW provides URIs for people (here's mine), but it doesn't provide a URI for each landmark. This would be my first alternation to the interface. I'd probably add a /landmarks "container" URI, and then /landmarks/stonehenge, etc for each landmark as it gets added.
The reason for doing this is I can then manipulate a landmark independently to a given user. E.g. POST to update the information (add comments, associate photos, etc), GET to discover its location, who configured it, and who is nearby.
Let me elaborate on what I mean by "container". There's probably a more general RESTful term, but its terminology I've been using in my internal documentation.
A container is is simply a resource that contains others. So /landmarks means "the set of all landmarks". Typically a GET on this would return a list of landmarks or a query form; the URL may be parameterised to search for landmarks. I'd make /witw/is be the container for people. Providing a hook for searching for users, listing all users, etc.
Generally I've designed things so that a POST to a container URI would allow me to create a member of that set (e.g. new landmark); i.e. modify that set. The response from that would be a 303 See Other, indicating the newly minted URI for the newly created landmark. This means that the server can retain control over the id allocation, but the client doesn't need to do anything other than follow the links.
Using 303s to allow a server to assign an identifier for a newly created object is something I've commonly seen and used. I also return 303s after a POST to a given URI, to redirect the browser/client to the results.
I notice that WITW doesn't do this which probably isn't a big issue. However I think it would be better to POST to a the person resource, i.e. /witw/is/ldodds to update my location rather than POST to /witw/ami/now. At the moment I have to POST to that URL to change my location and it returns a document which is a subset of what my URI returns. Norm could remove one template and output generator from the application by simply returning a 303 to the users URI.
As a final comment I just spotted that the stylesheet that transforms the XML results into HTML for my browser annotates the document with more information than is available in the source XML, namely links to MapQuest. multimap, etc. It also spots the userid attribute of landmark and resolve it to a link to the relevent WITW user document.
This information should definitely be pushed down into the XML. It took me a while to realise that this was an important aspect to a RESTful design; "hypermedia as the engine of application state" is a pithy phrase but requires some unpacking. In essence the client application shouldn't need to know how to generate additional URLs for your (or other) web services based on data in the response. The URLs should be provided, albeit in typed links/attributes, so that it can navigate to obtain additional behaviour. When designing a web site it's a bad idea to present an end user with a "dead end", this just applies the same criteria to machines.
It's also got some neat side effects. Because your clients are then encouraged to just consume the URIs its provided rather than generate them, you can silently change your URL structure without having to co-ordinate with them; a GET is a GET.
Anyway, that's all I could think of while munching my sandwiches this lunchtime. Looking forward to seeing WITW evolve further.
Liam Quin has been thinking about XML 2.0 and has posted an article to Advogato titled "Where Should XML Go?".
Quin is obviously trying to reach a wider community than just the hardcode XML users, noting in his diary that: Where would you go (or post) to ask people why they're not using XML? There are lots of good reasons not to use XML, and lots of good reasons to use it, so I'm particularly interested in people who would like to go with XML but who feel they can't.
Advogato seems like a good starting place to me. Of course there's an XML-DEV thread starting on the topic already, so the usual suspects will be weighing in very shortly.
I'm not sure what my most requested improvement to the core specification(s) would be. When asked about this before I've often responded that I'd be happy to see the work on packaging resume. Especially as there's work continuing in the area that could be standardised, such as the Open Office format and Rick Jelliffe's DZIP2.
I loved Jelliffe's From Wiki to XML via SGML article demonstrating how to use SGML SHORTREFs to parse Wiki markup as SGML interesting, and thats made me wonder whether that might be an SGML feature worth unearthing. Not likely to be a popular suggestion though! And of course one can simply use an SGML parser when one needs the extra power.
But the syntax could certainly be friendlier, and I wonder whether that might address some users dislike of XML, the format; the can still use XML tools to process their config files, Wiki markup, CSV documents, etc.
At some point I'm intending to write a little series of tutorials for Sparql which, coupled with Twinkle, will hopefully prove useful to people getting to grips with the language.
For the moment though I thought I'd post a few sample queries to get people started. The examples include some that refer to my own FOAF description, and a few from rdfdata.org. All of these can be simply cut-and-pasted into the Twinkle query box to run them.
Caveat: the specification isn't finished, so syntax may change in the future. I'll try and keep these current though.
List all of the names, weblogs and encrypted emails of people defined in http://www.ldodds.com/ldodds-knows.rdf
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
SELECT ?name, ?weblog, ?sha1
FROM <http://www.ldodds.com/ldodds-knows.rdf>
WHERE
(?x foaf:name ?name)
(?x foaf:weblog ?weblog)
(?x foaf:mbox_sha1sum ?sha1)
That doesn't list everyone in the document though, as not all people in my "knows" description have weblogs. We can amend the query to become the equivalent of an SQL OUTER JOIN to retrieve everyone with a name, encrypted email, but optionally a foaf:weblog property:
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
SELECT ?name, ?weblog, ?sha1
FROM <http://www.ldodds.com/ldodds-knows.rdf>
WHERE
(?x foaf:name ?name)
[(?x foaf:weblog ?weblog)]
(?x foaf:mbox_sha1sum ?sha1)
'List the titles and publication dates of documents written by someone with the name "Leigh Dodds"
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
SELECT ?title, ?date
FROM <http://www.ldodds.com/ldodds-documents.rdf>
WHERE
(?x dc:title ?title)
(?x dc:created ?date)
(?x foaf:maker ?me)
(?me foaf:name "Leigh Dodds")
List the state and city for all Australian airports
PREFIX air: <http://www.daml.ri.cmu.edu/ont/AirportCodes.daml#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
SELECT ?state, ?city
FROM <http://www.daml.ri.cmu.edu/ont/AirportCodes.daml>
WHERE
(?x air:country "Australia")
(?x air:city ?city)
(?x air:state ?state)
List the name, mbox of all contributors to the 2003 Dublin Core conference, along with the title of the paper the (co-)authored
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
SELECT ?name, ?mbox, ?title
FROM <http://www.siderean.com/dc2003/dc2003_presentations.rdf>
FROM <http://www.siderean.com/dc2003/dc2003_agents.rdf>
WHERE
(?person foaf:publication ?doc)
(?doc dc:title ?title)
(?person foaf:mbox ?mbox)
(?person foaf:name ?name)
Notice that this one queries two sources, relying on the RDF graph to be automatically merged.
Find all the European princesses and the date they got married
PREFIX ged: <http://www.daml.org/2001/01/gedcom/gedcom#>
SELECT ?name, ?marriedOn
FROM <http://www.daml.org/2001/01/gedcom/royal92.daml>
WHERE
(?royal ged:title "Princess")
(?royal ged:name ?name)
(?royal ged:spouseIn ?family)
(?family ged:marriage ?marriage)
(?marriage ged:date ?marriedOn)
List the names, symbols, atomic weights and numbers of all the Noble Gases
PREFIX table: <http://www.daml.org/2003/01/periodictable/PeriodicTable#>
SELECT ?name, ?symbol, ?weight, ?number
FROM <http://www.daml.org/2003/01/periodictable/PeriodicTable.owl>
WHERE
(?element table:group ?group)
(?group table:name "Noble gas")
(?element table:name ?name)
(?element table:symbol ?symbol)
(?element table:atomicWeight ?weight)
(?element table:atomicNumber ?number)
I've just uploaded another iteration of my Java Sparql query tool, Twinkle.
Read the original announcement for background; read the list of changes; or just download Twinkle 0.2.
The release basically adds some usability niceties such as accelerator keys, tool tips, status bar, etc. Errors processing queries should now also now be visible in the GUI status bar. I also fixed up several bugs that meant that DESCRIBE, ASK, and CONSTRUCT query results weren't displaying correctly, or weren't appearing in the correct output format.
I've also added a new results format option that will dump SELECT style Sparql queries into a JTable, making the tool look much more like a conventional database query application.
As ever please feel free to send in bug reports or feature suggestions. Especially the bug reports!
Lauren Wood recently has recently been seeking input from people successfully using a Wiki in a corporate environment. I've been meaning to write up my own experiences in this area for a while, as I've spent the last few years nurturing a burgeoning Wiki culture at Ingenta. Lauren's request was the spur I needed to start to set down my thoughts.
Read on for some personal notes on the viral introduction of a Wiki into a corporate environment.
We've had Wiki environments running at Ingenta for about three years now. The original Wiki site was one I started for my own personal use as tool to support the R&D role which that my focus back then. I had a need to be able to quickly capture bookmarks and commentary on a number of wide ranging topics, as well as write up research proposals, notes, and experiments. I also needed to share these with my colleagues in the research team and the rest of the engineering department. I could have just put up a website, but wanted something quick to evolve. As I'd already used several public Wikis I decided to install a Wiki engine and give it a whirl.
After I'd had the tool up and running for a while, I got interest from the engineering team who wanted a similar environment for writing project documentation. So, starting from a clean install, we added a second Wiki environment, and got the engineering team contributing to it. They had the usual initial fun creating personal homepages, but then got down to work on documenting various tools, components and procedures.
At this point I got interested to see how far it could go. Like many ".com" companies, Ingenta had grown rapidly and was finding itself without an adequate content/knowledge sharing infrastructure. We'd had repeated plans to build a corporate intranet but (I suspect) like many projects of this nature, feature creep and analysis paralysis crept in to the extent that a simple document repository/intranet concept turned into an online employee database, internal workflows, document management system, the whole shebang. Needless to say that grand vision never did quite see the light of day.
Before joining Ingenta I'd worked for Praxis, a local IT company that had been bought out by Deloitte Consulting. Praxis had a intranet which consisted of a core set of pages (corporate information, project links, process docs, etc) which was richly supplemented by personal homepages owned and constructed by staff members. Deloitte had a completely centralised intranet, that had very little staff contribution (beyond under-utilised forums) plus a bespoke infrastructure (Lotus Notes).
I knew which of those environments I'd personally found most useful, and so wanted to see a similar de-centralised approach to organising internal information.
So I decided to try pushing the Wiki as a de facto intranet environment.
With the engineering department slowly building a useful set of technical content, the next goal was to get some of the other departments involved.
The first lesson I learned here is that users new to a Wiki environment expect it to be more like a website: they wanted one of their own. So, to the existing research and engineering Wiki's we inevitably added several others: IT support, QA, and one for capturing "intelligence" (competitor and industry activities, etc), etc.
After leaving these to bed themselves in for a while it quickly became obvious that only the engineering Wiki was growing organically. The others had had a brief flurry of usage but quickly became stagnant. I believe the reasons for this were several:
User Base -- Wiki's are nothing without constant nurturing. For that you need a decent population of users. The engineering department was larger than the others and so it was easier for them to reach a critical mass which kept the content moving.
Relevance -- The engineering Wiki was directly supporting the team as it went about it's business. Component documentation proved useful as teams grew and shrank; Detailed release documentation became an essential of the configuration management process; change requests were quickly written up and circulated amongst the team; best practice documentation was similarly shared. The other teams were often creating content purely for other users, who weren't directly contributing back to the Wiki, so they had gone fallow.
Good Citizens -- A number of my colleages had really got bitten by the Wiki bug, and were becoming good WikiCitizens. As the content base grew, more thought was being put into organising and reorganising the Wiki to keep the content fresh and (hopefully) easily navigable. While it's easy for one person to setup a Wiki, it's essential for the user community to take ownership for their own content and, perhaps most importantly, for other people's content.
Around this time I became manager of the engineering department (the poor fools have never recovered :) and pushed more of our processes into the Wiki: all change requests, specifications, and project write-ups were done in the Wiki rather than in lengthy Word documents. Test scripts for QA'ing the site went through a similar treatment.
Using "inter wiki" linking features/macros I tied the Wiki into our Bugzilla installation, then into features on the test and public services. Our shared corporate drive had been descending into a mess for some time, and as it was available as HTTP over the internal network, I created an "Intranet" page on the Wiki and linked to every useful resource I could find on it.
Basically I wanted to reinforce the relevance of the wiki to the engineering team, whilst using the Wiki environment as an organizing principle to start to knit together disparate internal resources, many of which were previously localised to individual departments.
With the move of specifications and the change request processes into the Wiki environment, the product management and QA teams began to use the Wiki. This increased the size of the community but kept the relevance-factor: the Wiki was becoming an increasingly important communication tool between the departments. In the face of a number of staff "reorganizations" this was a vital step.
From that point on the Wiki has really taken on a life of it's own. Moving from a technology focused tool, it's now being used by many more of the internal departments. For example the Sales teams, after some initial experimentation with a separate enviroment, are now using the Wiki to capture sales prospects, and most interestingly co-ordinate work on Request For Proposals.
Responding to an RFP involves a need for high-bandwidth communication between a number of cross-functional teams to ensure that all aspects of the proposal are correctly dealt with. Flurries of emails and shared, rapidly evolving, documentation is the norm. This work is now being co-ordinated through the Wiki environment. The initial RFP is published to the Wiki and the response team can co-ordinate around the shared pages, with everyone easily able to see the latest updates and revisions; change tracking and diff'ing are both essential Wiki feature's. What's more, many more people can now contribute ideas and suggestions.
It's still early days, but it's fascinating to see how a Wiki environment can facilitate sharing and contribution in this way. In fact, in my view, where you have many different teams within an organization, ensuring a good degree of information sharing requires extra work -- you have to step outside of the task at hand. Whereas, with the majority of teams using a Wiki, you'd have to do more work to stop information being shared. I know I'm not the only one keeping and interested eye on the RecentChanges page on our Wiki to see what's happening elsewhere in the company.
The Wiki concept has been so successful at Ingenta that it's started to expand outside of the organization.
While we're still some way from a public Wiki to provide documentation to support our services, we now have several private Wiki's running as a means to share documentation with clients. Essentially the project managers and technical teams can share information directly with their counter-parts at our clients. It's certainly come a long way from a simple research notebook.
At some point I'll try and synthesise our experiences over the past three years down into some practical recommendations for others wanting to follow a similar tack. But I think this posting has gotten long enough for now.
I've just uploaded beta-2 of my Java API to MusicBrainz RDF web service.
The API is Creative Commons licensed and is built around the Jena 2 Semantic Web toolkit.
The API provides raw access to the RDF returned from the service, but also a simple JavaBean layer for developers wanting a simpler interface to the data. You can read the Javadoc and view the changes since the last beta; these mainly consist of some bug fixes and support for a few new properties (including Amazon ASINs).
The API doesn't aim to mimic everything in the C/C++ API, e.g. track id calculation or submission, it's merely a read-only version suitable for embedding in Java applications.
I've included a trivial demo in this release: a simple command-line application that reads in a list of album names, looks them up in the service and aggregates the basic metadata into a new RDF document which is dumped to the console.