In the spirit of release early and often, I've just uploaded the first snapshot of Twinkle, a query tool for Sparql.
Twinkle is a simple Java interface that wraps the ARQ query library that Andy Seaborne is currently building as an add-on to Jena. ARQ is still under development and is not yet a supported Jena module, but I wanted to start playing with Sparql in something other than a command-line environment. Twinkle 0.1 is built on ARQ 0.9.2, which you grab from CVS or just try the web demo.
Twinkle was inspired by Elliotte Harold's XQuisitor which provides a simple GUI interface for playing with XQuery. XQuisitor wraps Saxon 7 which have XQuery support. I've shameless copied Harold's interface as well as cribbing from the code; my Swing is a bit rusty.
In this first release the tool is just functional incorporating loading and saving of queries and results, basic text editing, alongside the basic ARQ/Jena functionality: parsing of Turtle, N3, NTriples, and RDF/XML, and generation of query results as plain text and in the Sparql Variable Binding Results XML Format. ARQ also supports other query languages, but I've not exposed these through the GUI.
Later releases will add better error handling (it's pretty much non-existent at the moment), a separate thread for running queries so they can be cancelled, and various other UI improvements, e.g. showing results in a JTable.
Twinkle is published under the Creative Commons Attribution-ShareALike Licence.
To run the tool simply download it, and run: java -jar twinkle.jar from the directory you've unpacked the distribution into.
Interested to hear feedback, especially bug reports. Hope others find it useful as they're playing with Sparql.
Tim Bray posted some notes on Private Syndication, referring to this ZDNet piece by David Berlind.
I'm inclined to agree that this kind of syndication is as yet a largely untapped application agree and that it's one with a great deal of possibilities. I'd love to have a feed of my bank balance, credit card statements, etc. Might help me curb my spending :)
The kind of private syndication Bray and Berlind are talking about is the opposite end of the spectrum from the public feeds that most of us are consuming. It's important not to ignore the space in-between though: between per-user and mass-audience feeds there's a lot of other possibilities. E.g feeds tailored to a particular community or market. There's an uneasy relationship between advertising/marketing here.
For example a publisher may want to have one feed for content subscribers, Amazon may want separate feeds for regular purchasers, etc. The content of these feeds needn't be entirely marketing oriented though, there's scope for "premium" content feeds, e.g. pushing out entire articles or other relevant updates. In my own application area, it would be useful for publishers to be able to produce RSS feeds tailored to subscribers/non-subscribers. A subscriber feed may have the entire content, or direct links to it. A non-subscriber feed may have limited content and links to purchasing options instead.
Whatever these "tailored" feeds contain, and whether they're tailored for a restricted audience or individual user, the key to their success is going to be authentication support in aggregators. SSL, HTTP Auth, etc are all pre-requisites. And not only that: web based aggregators such as Bloglines (my own favourite) will have to ensure that these feeds are not shared with the rest of the user base. I've heard several stories of private RSS feeds being accidentally shared with the entire Bloglines community; as I understand it, they automatically add any feed to their global directory.
In fact a move to tailored feeds may take away some of the supposed value of RSS aggregators such as Bloglines. There won't be much they can share between users. There's been a lot written about the network overheads of RSS, this can only get worse with more tailored feeds.
Even without tailored feeds, support for authentication and non-shareable feeds would be a useful feature. At the moment I publish several private feeds internally to our company which are being rarely used as many of my colleagues are using Bloglines, or they are mobile and their desktop aggregator doesn't support HTTP Authorisation.
Bray closes his posting my stating his belief that Atom is best suited to producing "content-critical, all-business" feeds. It's a bold statement and I'd like to hear more about this: what exactly makes Atom better suited to carrying personalised/tailored content than any of the other RSS flavour? Kellan Elliott-McCrea has raised one issue already.
In his article, Berlind suggests that a delivery company might produce an RSS feed for every package they ship. I wonder whether, instead of requiring each company to produce fine grained feeds for all of their actions, whether it might be easier for credit card companies to act as the point of co-ordination. Actions relating to a purchase made on a card, e.g. dispatched, delivered, warranty expired, could be sent as a notification to the card company, who could then produce a secure tailored feed which aggregates all the relevant activities.
There's already some degree of communication between the companies (the actual transaction) so really this would only require a standard interface to exchange data suitable for packaging into a feed. I can definitely see a role for the Atom API there, but I'm not clear on the unique benefits of the Atom format.
Seems like the library community is getting interested in folksonomies and how they can be used to supplement data coming from OPACS and other structured metadata.
From The Shifted Librarian: I think controlled vocabularies and folksonomies can co-exist peacefully and even complement each other. And as librarians, let's start making use of them to complement what we're already doing.
Sounds like it'll be an interesting meeting. Take the red pill.
So any signs that "tag spam" has started yet?
"Tag Spam" will be (or is) the practice of associating unrelated content (pr0n links, adverts, viagra pill adverts, press releases) with well-known tags with the purpose of encouraging click-throughs to said content.
Seems like a natural extension of referrer, comment, and trackback spam, and I'm curious to know whether it's happening yet.
Actually doing it seems very easy given the open API's of sites like del.icio.us. And given Technorati's support of the rel="tag" attribute, I can create tag spam by just setting up a blog, posting a bunch of random entries and then pinging their site. Therefore seems just as hard to manage as the other types of attacks.
Time to start engineering in trust metrics I think.
As one of my new years resolutions was to not sit on ideas until I get time to implement them (I never do), so I've created an "Ideas" category that I can use to write these down. Here's my first one:
I'd like an application that would build be a personal timeline of my web activity.
Actually, I'm also interested in viewing other people's activities, but lets get on with describing what I mean.
I maintain a blog, a set of del.icio.us links, a reading list (at All Consuming) and a flickr photo collection. Each of these sites produce an RSS feed with timestamps attached to the activity I generate there: link descriptions, postings, and photos.
If I were organized enough I'd probably maintain a diary too to track travel and events. These would also obviously have dates too.
What I'd like to see is an application that would hoover up content from any RSS feed I generate as I go about my business on the 'net, and then format the data as a horizontal timeline.
The user interface I'm imagining is big black horizontal timeline that I can scroll along, or zoom in and out of. At a high level the timeline might just show me activity summarised by tags ("Look like in January 2005 I wrote a lot on the Semantic Web, tagged a bunch of stuff under SocialSoftware"). At a more detailed level, attached to the timeline will be actual photo thumbnails, blog entry titles, links, etc.
The seeds for this data (i.e how to find the relevant RSS feeds) can found via my FOAF description. So ideally the application would be configured by pointing it at my FOAF file. As a piece of social software the application would be configured by a collection of FOAF files or RDF blog roll, so I can see an aggregate timeline, e.g. for a community of interest.
The user interface for the aggregate view could be done in several ways. The simplest being to aggregate all activities on a single line. The other to have several parallel lines, with links between them to indicate, e.g. co-depiction in photos, shared creative output, etc.
A related, but simpler request, would be a tool that I would draw me a zoomable, navigable timeline (as SVG?) which I could throw data at. It'd be interesting to generate timelines for historical figures (e.g. Samuel Pepys, Charles Darwin) that could be used to navigate their writings and activities.
I think this'd be an interesting way to explore and present data about people and communities. Does such a beast exist?
Stefano says The future of the semantic web is LSI. While I agree that LSI is definitely a cool technology, and is an interesting alternative to Bayesian techniques I don't agree that it's the future of the semantic web. The semantic web isn't about text analysis. It's not about data mining a document corpus. But those are possible applications of the technology.
There are vast amounts of high quality data being produced with semantic web technologies all the time. No need to apply LSI there.
Danny has already deconstructed this piece so I won't comment a great deal further beyond saying that despite the title the article has nothing to say about what works or doesn't work about ontologies, it merely describes some of the issues a search engine vendor faces when indexing text.
The article also glosses over a lot of interesting activity elsewhere. For example Norvig says that:
Essentially what we're doing here is using the power of masses of untrained people who you aren't paying to do all your work for you, as opposed to trying to get trained people to use a well-defined formalism and write text in that formalism and let's just use the stuff that's already out there. I'm all for this idea of harvesting this "unskilled labor" and trying to put it to use using statistical techniques over masses of large data and filtering through that yourself, rather than trying to closely define it on your own.
What about all the buzz about folksonomies? What's that if its not harnessing "unskilled labor" to generate structured metadata? Just because its a few simple tags doesn't mean it's not metadata, just because it's categorization without using a formal ontology doesn't mean it's not generating useful machine-readable metadata. And should we ignore it because it's free-form and user-generated? Technorati apparently don't think so.
Over that metadata I can start making additional statements, drawing together related tags, drill down to extract rich metadata about the photos, etc. This is what the semantic web is about: building a machine-readable infrastructure over and above what we already have.
But while the new technorati service is undoubtedly cool, before one gets too excited over it read through this paper on folksonomies which compares tagging practices in flickr and del.icio.us. The different media and communities of the two sites lead to some quite different results. The technorati service can provide some good illustrations of that and other issues raised in the paper.
We don't have to throw everything out and start again. We don't have to restrict ourselves to mining what's out there, and we don't have to wait for "ontology astronauts" to deliver us a set of fixed ontologies before we can being doing useful work. While I agree with Shirky that comparisons of controlled vocabularies and folksonomies should account for economic factors, I disagree with the implication that it's an either/or choice.
What about a folksonomy created by a designated community of experts, e.g. researchers in the field of interest? It's not a controlled vocabulary in the usual sense, has the economic benefits of a folksonomy, but needn't suffer from some of its problems. What about applying an editorial layer on top of a folksonomy to draw together related tags, etc? It'd address some of the issues outlined in the folksonomy review paper I referenced above. There's plenty of room in between the two extremes of controlled vocabularies and user-generated tagging.
It's a requirement of the key technologies underlying the semantic web that one can create vocabularies in just such distributed fashion, and relate them together when we need to.
Boot-strapping machine readable data from existing sources. Late binding of data to application schemas.
That's the future and benefit of the semantic web IMO.
Mark Birbeck posted to rdfweb-dev on Friday to announce an XForms-based FOAF creator.
To use the tool you'll need to install the latest version (and patches) of Forms Player.
As Birbeck notes in the announcement the demo does nicely show-case some of the XForms features, notably extracting information for the form labels from the demo, being able to remotely fetch remove XML files, etc. In other words XForms is XML aware so creating and manipulating XML is a doddle. Overall it's a very nice demo.
This isn't the only FOAF-a-Matic style XForms implementation. Charles McCathieNevile gave a talk at XML Europe 2004 in which he described his own uses of XForms to generate RDF. The forms are available online. I played with an XForms version of the FOAF-a-Matic last year also, using it to demo the technology for a talk. There's also a simple FOAF editor in the Mozquito DENG examples.
XForm's tight integration with XML means that it doesn't play well with the syntactic variability of RDF. To make a complete FOAF editor with XForms one would have to use a syntactic profile to make the technologies mesh well. Coupled with a server-side XML repository it'd be simple to create a service to host FOAF profiles.
However while I don't consider the syntax issue a big problem, the main reason I've not pursued this idea further is that there's no seamless way to deploy the forms:
Formsplayer is IE only and requires an object tag to invoke the plugin. It also wouldn't deal with forms that XSmiles happily displayed. However to be fair I should retest this in the latest version.
XSmiles will view the XHTML/XForms file natively, but isn't exactly an ideal target environment. I don't want the user to have to install a specific application, otherwise one might as well create a desktop tool. McCathieNevile's forms don't work with the version I have installed here, so it seems as if there are still some issues with creating cross-player forms.
Mozaquito DENG requires Flash, which at least gives it portability, but one has to reference the form via a webservice instead of linking to it directly in the browser. This means you can't simply link a user to the form and expect it to fire up in formsplayer if thats what they have installed.
Chiba removes the browser issues by moving the forms processing to the server-side, but again had issues playing forms that worked in XSmiles.
The other toolkits I looked were either payware or didn't conform to the final Recommendation.
Perhaps I'm being unfair: has anyone got examples of non-trivial XForms that work seamlessly from a user perspective in multiple players? Presumably some Javascript trickery could be used to avoid a "click here if you're using formsplayer/Mozquito" scenario?
It seems a shame that XForms -- a simple to learn, and genuinely useful technology is being hampered by portability and deployment issues. I'd love to hear of reports to the contrary.
A quick follow up to my Views in Triple Stores posting.
Andrew Newman describes how my requirements can be addressed within Kowari. He also noted (privately) that views are crucial for lots of use cases, pointing me at this thread on kowari-general.
Richard Cyganiak was less keen on the idea. He's right that too wide a depth would pull in too much data, but you get the same problem with a "SELECT * FROM..." query. Just don't do it!
I'm also don't understand Cyganiak's comment on my suggestion to use namespaces as a means to filter predicates: Namespaces are just that, namespaces. Don’t overload them with access control. I'm not sure how he's using the term "access control" here, that's not what I was suggesting. And namespaces are intended to partition vocabularies, so not using them as a way to ignore data that's not of interest seems bizarre to me.
His note that filtering by data isn't addressed is fair, but I didn't include that as I'd counted that as part of the initial query to finding the initial node(s) of interest, not the construction of the "view".
Must admit I've not looked at CONSTRUCT (or indeed Sparql at all yet) so maybe that does address some of what I'm interested in (and I have a real-world application in mind: this is a real use case).
One additional filtering criteria, which would rely on the use of named graphs,
would be to filter by origin. i.e. "only return me data from these trusted sources."
I signed up for a flickr account shortly after it launched. Back then I was interested in browsing through the service to look at possibilities for data import/export (as FOAF) and so never actually played with the photo management and sharing features (duh).
Prompted by Christopher Schmidt's recent Flickr and RDF posting which gave a glowing report of flickr's facilities, API, and importantly the wallet-friendly price, I had a proper play with it last night. Like Christopher I've been interested in a photo annotation and management application for a while, but have been too lazy to actually do more than think about.
So I spent my entire months guest account bandwidth on uploading some photos and can see that I'm already very impressed. I've not even dug into the API yet, just the fact that the uploading was so easy, the tagging straight-forward, that it automatically extracted the photo datas (and camera model) from the images, and that there's plenty of ways to slice up and annotate the data are enough to get me hooked.
I don't often pay for software/services, but this one is slick enough to get my shilling. My only grumble was that at 8-9pm GMT the site had noticeably slowed down, hopefully they're going to be deal with that before the final launch.
It did also slightly creep me out that a photo of my wife and kid were already someone's "favourite", so I suspect I'll be making use of the privacy feature for some material.
(This posting in my grand tradition of discovering high-profile, "everyone's talking about it", absolutely essential websites 6 months after ones else)
I started this essay last week on the train between Bath and Oxford, but then got side-tracked with writing up my XTech 2005 submissions and fighting off a nasty stomach bug before finishing it.
Poking into my aggregator this morning I was interested to see this exchange between Bill de hÓra and Danny Ayers. Bill seems to have having similar thoughts about the humble list:
The thing about lists is that they have a lot of untapped value. I believe a lot of information gets left behind when all you have to work with are <ol>, <ul> and <li>. In terms of moving up the semantic and social software food chain, lists + metadata are a natural next step. Arguably an Amazon wishlist or a list of people on LinkedIn have more value if they're decoupled from the sites themselves. Passing them around and sharing them might be cool.
Which echoes some of my points below. Any read on for ramblings about the list pattern, bookmark management, and thoughts on how applying simple conventions to list structures can help bootstrap some interesting applications.
I've been obsessed by lists lately.
This may have been triggered in part by the impending return to work after an extended Christmas holiday. Or maybe it was the endless "List Programmes" on the telly: Top Ten This, Best Ever That, The People's Choice of the Other. It's probably largely due to some early spring cleaning and re-organization of my bookmarks and blog subscriptions that I'vebeen undertaking. And being abstraction prone(*) I can only look at sequences of URLs and lists of subscriptions for so long without musing on the underlying patterns.
Lists are all around us, and they're typed either by their context or convention.
A list of placenames and times on one screen at the train station describes arrivals, next to it resides the departure list; A printed series of food items and prices in a restaurant is a menu, at the checkout it's a receipt; my aggregator presents me with a list of reading material; my IDE, a change history; the "Edit" menu a list of text manipulation functionality; and so on ad infinitum.
Whether or not we're neurologically well adapted to list processing is irrelevant: culturally we've certainly embraced the basic "titled list/list-item" pattern for organizing many kinds of information, and so there's a positive feedback loop in place that encourages the publication of new information in a similar form.
It's this positive feedback, and the simplicity of the abstraction that's at least partly behind the success of RSS. In fact at the syntactic level, the list/list-item pattern is an architectural form that's present in many different vocabularies: RSS, OPML (nested lists), HTML (ordered and unordered lists), to name but a few. The RDF Sequence and Bag collection types merely represent this pattern at a higher level of abstraction. The excellent list processing features of Python allow these abstractions to be manipulated with ease.
The most flexible way to maintain a list of URLs on the web is to use del.icio.us. Ease of use, maintenance, and a uncluttered (if sparse) interface are all important factors that contribute towards its success. The RSS views and API are other essential ingredients.
The easiest (as opposed to most flexible) way to maintain a simple list is actually a Wiki page. IMO, del.icio.us has the edge over a Wiki environment for maintaining lists of URLs because of the tailored pop-up interface presented by its bookmarklets, but also because it automatically timestamps each items when added. The extra metadata is important.
Because list maintenance is so easy with del.icio.us I've extended my use of it's tagging function beyond simply classifying web pages, I've begun to use it to organize other pieces of information: articles I want to read later, articles I've written, a list of services that I'm registered with, and a list of the data exports. I've organized these under a "Me" tag, with separate Me/ToRead, Me/Articles and Me/Data sub-categories.
del.icio.us automatically provides me with RSS feeds of this data so I can include them into several different kinds of application. My publications can be attached to my CV, my data automatically harvested (e.g. via a Scutter), and my reading list and service homepages added directly to my Firefox personal toolbar as Live Bookmarks.
To digress slightly, it's this last facility that has finally allowed me to nail my ongoing difficulties with bookmark management. I've always used bookmarks in two ways: as a list of items to come back to for later reading, and a list of items for future reference. The differences are subtle: in the first instance I simply need a list, in the latter I need some level of classification to make the reference material easy to find at a later date. Using del.icio.us I can not only "bless" one tag as my ToRead list, but I can also easily access that list from any location, and still classify the items for later reference by assigning them multiple tags. Difficult to manage with a plain browser bookmarking facility.
This is where we loop back into the list typing -- by context or convention -- that I mentioned earlier. My "Me" tags are a convention that I've adopted for my own benefit. But it doesn't have to stop there. Michael Lenczner recently suggested using a "commentlog" tag to record one's blog mostings (read more). I'd been toying with a similar idea, adding a Me/MailingList/XML-DEV type tag structure to record my mailing list and ultimately news group postings, so it's interesting to see others also exploring this avenue. I've always felt that keeping track of one's correspondence is important.
Without much effort it's easy to think of other "well-known" tags that a user (and by extension a community) could adopt to make it easier to manage certain kinds of information and facilitate a number of interesting applications. Or indeed how some social content applications could be simply expressed as lists of URLs in del.icio.us.
The following table contains the results of a quick brain-storming session, listing some suggested del.icio.us tags, what they might contain, and the functionality they could provide to the end user and at an "aggregate" level, i.e. analysing across the del.icio.us community:
| Tag | Convention | User Function | Aggregate Function |
|---|---|---|---|
| Friends | List of blogs run by friends | Simple to transform into FOAF, and arguable even easier to maintain and include in a blog than XFN | Simple social network analysis |
| Subscriptions | List of RSS Feeds | Simple to transform into OPML or an RDF Blogroll for importing into an aggregator; allows easy migration between aggregators | Subscription and network analysis; e.g. how many subscriptions to a particular blog? Whats the history of subscription numbers? |
| WishList | List of product links | Cross-vendor "wish list" or gift suggestions | Product popularity, broken down by community |
| Gigs | List of links to gigs or events one has attended or is attending | Record gigs and/or events one has attended, including short reviews | Where are your friends going? Where have they been? |
| Location | List of links to URL indentifying geo location. Could be created by automatically logging GPS co-ordinates at regular intervals. del.icio.us will automatically add timestamps | Real-world annotations | Community travel, geolocation |
One interesting property of these kind of tags is that they're easy to locate automatically, because they use "well-known" names. This is especially true when you have a mechanism that can associate your FOAF self-description to your del.icio.us account. (Aside: There are some other possibilities in that space too).
Many of the above examples would perhaps be better served by tailored interfaces that would collect and publish additional properties over and above the basic URL+title+comment+timestamp structure that del.icio.us uses. But I believe there are several advantages in using del.icio.us, in however limited a form, when exploring these kinds of application:
However for really serious use I believe we need something more: a service like
del.icio.us that allows storage of arbitrary properties against web resources. And it seems obvious to me that such a system would be based around an RDF data store. Jo Walsh has already created an RDF version of the del.icio.us API, and this tutorial on creating a social content engine in RDF provides some additional thoughts.
I wouldn't necessarily go so far as to argue that the interface or API itself needs to be be RDF-only, just that RDF provides the desirable level of flexibility upon which such a service can be constructed.
As an open source application, this social content engine could be used to build any number of applications, but its real power, like del.icio.us would being provided as an open web service: one that could be used by other tailored services as a basic metadata storage engine.
It seems obvious to me that there's going to be a real need for this kind of service; it seems to be a natural evolution of online hosting arrangements, rather than just empty disk space or a huge inbox, I get a structured metadata store into which I can drop my content, and to which I can grant varied degrees of access for various services. See Dare Obasanjo's essay "Social Software is the Platform of the Future" for more in this space.
(*) being "abstraction prone" is similar to being accident prone, except instead of falling down, you tend to be drawn upwards.
Views are an important feature of relational databases, providing a way to abstract over complex queries, subset data to just the minium required for a given task, as well as providing a point around which a schema can be refactored without having to (immediately) change the applications that use it.
So far I'm not aware that there's any similar construct in RDF Triple Stores, and I'm curious why.
The scenario I'm imagining is this:
I've obtained a reference to a resource in an RDF graph, perhaps via a query. From this node in the graph I want to pull out a subset of the data accessible by navigating from this resource.
I can see two possible ways to create that subset, and these may be used in conjunction.
The first is to apply a "window" on the graph, and only extract the data that's within a certain distance of my origin. The MusicBrainz API has a similar notion of query depth; see the ASCII art under "Select & Get Documentation" section in in the docs, and the subsequent section for more information.
The second subset may be created by filtering out the classes and properties extracted from the database based on their namespaces. For example I might have a triple store containing a mixture of public/private data, with the latter in a separate namespace and I want to pull out just the public aspects for returning from a web service.
I see the combination of these subsetting techniques, along with a declarative mechanism to state how and when they should be applied as broadly analagous to a relational view.
Has anyone implemented anything like this, or aware of triple stores (or more likely APIs) that offer this facility?
Does anyone else see the utility of such a mechanism?
(Answers via email or trackback please as commenting is disabled 'cos of spam)
Via Catalogablog I've just learnt that Konfabulator is available for windows. Looked interesting, so I installed it.
I'm in love.
Looking forward to seeing this del.icio.us based widget.
Whilst upgrading my Audioscrobbler WinAmp plugin today to be conformant to the latest submission protocol (my RESTful redesign didn't gain any traction unfortunately) I discovered that they're been busy on their web services interface.
Alongside the original RSS 1.0 "recent tracks" feed, there is now an RSS 2.0 feed, a plain text version, and a FOAF export of your profile. They've been careful to add holdsAccount properties to associate you with your profile, link to your RSS feed, and even include a list of your friends. Nicely done.
I've a few nits though:
foaf:depiction is non-standard. I don't have an avatar image set up, so apparently this is a picture of me. It would be better to omit the property entirely in this case. However because some people don't use photos of themselves as their avatar image (perfectly reasonable) you even get odd assertions when there is a photo. Check out Matt Biddulph. I think what we're missing is an additional 
avatarImage property that could be associated with an OnlineAccount.rdf:seeAlso to the RSS 1.0 version of "Recent Tracks" could be typed to identify it as an rss:channelBeyond that, it's all good! I do hope they get round to fixing up with RSS 1.0 feeds soon though -- these are all broken because of their use of a placeholder URI instead of the MusicBrainz URL for a given track.
Very nice to see a service exposing data in this way.