This post is written in the agony of realization. The painful realization that months and months of work may possibly have been a wasted effort on my part – as well as on the part of others.
- The Past
- The Promised Land
- The Arrival
- The Breakdown
- The Present
- The Future
- Addendum (I)
- Addendum (II)
And so the story begins…
1. The Past
1.1 The Promised Land
I do not recall where the name first came up. And it wasn’t immediately clear to me what the significance of the name was. Regardless of how or why, I somehow found myself on a mysterious web page for a new startup company touting in vague terms a social web service like no other.
It was Twine.
I remember seeing a good looking front page, linking to a beta candidate signup form. Scanning the description of the service, I found several buzzwords that had been on my radar for years. Including RDF, RDFS, OWL, SPARQL and friends. All good stuff from the realm of Knowledge Representation (KR). The ethereal plane from where truth, or at least probabalistic truth, could be found. A place where all knowledge is broken down into statements, also known as triples, forming (directed acyclic) graphs of interconnected nodes in a truly compositional way.
This (KR) was the kind of stuff I had studied on my own while specializing my education. As with programming language concepts, data structure concepts were just as fascinating. KR data comes with it’s own semantics and a so-called inference engine takes the higher facts (logic) with the lower facts and uses these to form new facts. One example is transitivity; say there two facts in the knowledge base: a contains b and b contains c (well actually there are more facts, like the fact that contains is a transitive relation), then it is possible to infer that a contains c. This is a very useful fact, it can be useful in many different contexts. Having the semantics infrastructure able to make this kind of inference for programmers is a huge win and enables one to reason much more abstractly about things. Of course with great expressive power comes great computational cost and so there are downsides to this as well but as with so many other things, there is a computational gradient that one can adjust to choose the expressiveness to fit ones patience and wallet (in terms of hardware).
Needless to say, it took only seconds to sign up, I was hooked. This service was truly breaking new ground, bringing together a KR-driven web service to the masses. Such vision and audacity deserves a pad on the shoulder. Being tired of small incremental steps that infest thinking everywhere, Twine was treading new ground, giving a big “FU” to conventional and short-sighted thinking, driven by superficial gadgetry and purely social aspects and pointless time wasting games (no, I shall not name that nameless service, of which I too, for a week no less, was a member).
1.2 The Arrival
Time went and the day arrived where I got that glorious mail. I signed up to the beta service as an enthusiastic and somewhat skittish beta tester. It took me some time to find my way around. The UX is and was somewhat messy. The concepts behind the service though – not so.
Twine was (yes, I say was, because it has since then been crippled in various ways, although also extended) defined as a fusion of concepts – a new paradigm
– a social network of users
– a semantic bookmarking service
The first part is quite obvious. There is a community of users using this service, these users are able to send eachother messages. That makes up for the very basic aspects of having a social network, anything less than that and you can’t call it social. There’s more to it than that, but first I need to address the second point.
The second part is the novel aspect of Twine. It is the bookmarking service. With Twine, one does not just bookmark pages, one bookmarks types of pages. Twine has two different kinds of bookmarklets – a useless one (called the basic bookmarklet) and the useful one (called the advanced bookmarklet); alright, that is not fair, the basic one is useful, but it is meaningless to use it in the face of the advanced bookmarklet, which is really a necessity for “power” users that bookmark a lot.
So what does it mean to recognize type of a page? It means for example that if you find yourself on the page of YouTube, looking at some fine video of a nice car. Then when you press the bookmarklet on your browser links panel, and remember a bookmarklet is just a piece of code that executes in the context of the running page, this bookmarklet will recognize that it is on a YouTube page and look for patterns. Since all web pages are essentially templates where data is put into, the bookmarklet can just look for the template markup and extract the relevant pieces of data from it. This practice is also called screen scraping and I truly believe this to be a new and novel aspect of Twine. Never before had this concept been explored for the masses. Now other recognized pages include pages such as Amazon, where Twine is able to recognize different kinds of products, like books, their prices and such.
This is your bookmarks on steroids!
Anyway, I quickly started creating twines and bookmarking items. Twining can be quite addictive. Once you get past the point of no return you start to, if you’re anything like me (which in all likelihood and fairness, you’re probably not), mass-twine and systematically organize all your bookmarks and interests.
A great aspect of this is the involvement with other users. This is where people come together over interests. Each user has his own interest feed as it’s called. This is really only just a page on Twine where you see a chronological lising of new items in all the twines you follow. And remember, a Twine is really just a group of items with properties. In the interest feed one can also see what comments have been made to certain items. And one can also create searches for twines, items, members, comments and such. This all helps facilitate engaging with the community around shared interests.
1.3 The Breakdown
At the beginning there was an essential page on Twine. This was the Explore New Twines page. A very nice looking grid of new and often very interesting twines (especially due to the number of KR interested people on the service). There is still pages for top users and top twines though. The Explore New Twines page has since been removed due to spam abuse issues and a failure to control, monitor and manage the influx of new twines being displayed on this page.
So before I continue, I should restate that
– a twine is a group of items
– an item is some arbitrary data of some arbitrary type
Now items can be injected into the system either via the bookmarklets or via manual addition from within Twine. This is fair and square and works well. There is also the third way of injection which I know many users enjoy but which I’ve never really fancied, but it deserves mentioning. This is by posting an e-mail. All twines have a post-to e-mail address that can be used. Since the inception this mechanism has been extended so as to be able to deal with possible spam issues.
Item types range from simple notes, “bookmarks”, videos, books, persons, etc. The set of item types is not open, it is defined by the Twine designers and it has also narrowed down over time. Each item type has an associated template for the bookmarklet, so that if it encounters Amazon it will be able to fill in the book data, for example.
As a side note, Twine is also said to employ natural language parsing to search for keywords and tags on the pages it bookmarks,. This is another very nice feature which has great potential.
I will note that this personal story will not be a complete account for all features of Twine now and then. That is not the point.
During the months of beta testing there were massive problems around performance and stability. A users feedback twine was created to gather user feedback on bugs and feature requests. All in the spirit of twine, the service was eating its own dogfood.
Although privately I felt that the service was not really 100% ready for prime time when it launched, it did anyway and it was still compelling and useful enough for masses to start using.
There were problems though. A pretty big feature was the item properties feature which allowed one to attach new properties to items and fill them out. It may seem like a pretty mundane feature but it is still a quite important one. This feature was stripped completely. It might not have been a big deal had it not been for its symbolic effect. But the service promised it was to return at a later point in full glory. So far this has not happened.
The symbolic value here is that, as a Semantic Web service (now being pedantic with casing), Twine showed a glimpse of a future where users could create their own templates, their own item types and put arbitrary data into twine via the bookmarklet or via APIs. These APIs were promised to arrive at some point but no date was given and that was a wise decision as these have still not arrived. It is still possible to submit data via HTTP POST as the forms on the service does but it is not really a dedicared developer API – or one would be hard pressed to define it as such.
As months passed Twine mutated. And when I say mutated I mean it changed shape, it didn’t just grow in terms of features, it also shrunk. Some features were added, then removed again. New bugs occurred. Others were fixed. One got the feeling that it was really getting nowhere. But still, the development sprints came in succession and little nuggets of usefulness were thrown out to the hungry masses. And for an malnutritioned user, such nuggets prove quite effective.
There was also some, not to say a lot of, controversy between some users and staff. Who is to blame for what I will not comment on but in any event the situation could have been handled slightly more professionally.
2. The Present
Today the status is that I and others have added months and months worth of high-quality content to Twine – mostly in the form of well-structured and interlinked twines with technical, educational, funny, cultural, etc. bookmarks, documents, video, music, etc.
This has truly been, now speaking just for myself, a monumental undertaking.
The current status of Twine though, is that it is stagnating. Features keep getting removed. Few features are added. If something is broken it remains broken for months and months on end. If something is added, it is likely that it will be removed again later due to some issue or simply the designers wanting to rethink the feature. Such was the fate of ratings.
Maybe if Twine added advertisements on the site it would get the adequate funding to make the necessary investments. It is quite clear by now that the development team is understaffed and inadequate to make the significant progress that is needed to grow.
At this point, sadly, I have finally, and believe me this, I have been stretched beyond anyone’s definition of reasonable, reached the limits of my patience. The elasticity of my patience has been the vision of the future, inspired by the enthusiastic CEO, Nova Spivack. A very passionate and used-to-be engaging user (and owner).
It is not that Twine owes anything to anyone (except perhaps investors) but as user who’s heavily invested in the success of the service, one expects a certain dividend in terms of an equal investment on the other end; after all, if not Twine were to fight for success (or survival as it were), then whom?
Radar Networks, the company that builds Twine appears to have lost its way. Unable to fullfil the dream laid out by its founder. But it is not so hard to dream of the future (your mileage may vary), it is far harder to put those dreams into practice.
3. The Future
Dreams of a giant global graph and the global brain sound nice but ideas have to be exercised and put into practice. It sadens me that Twine no longer appears to be that place.
The options are
– Stay, watch it all fall further apart with morbid fascination
– Leave, look for other, greener pastures
This is a hard choice. The captain does not appear to be in control. The ship is headed straight for an iceberg. Unless a dramatic change happens it is a lost cause. Some would say it has already collided with the iceberg and it’s now only a matter of time. A last ditch desperate effort may be able to salvage the pieces, patch up the ship and sail to a port where it can be redone and brought back on track towards that vision.
The sad thing is that there is really nothing in Twine today that really requires KR technology. It can all easily be built with plain old relational data and yet the scalability problems (in terms of speed) are so great that it appears overwhelming and maybe even impossible to use the features that KR enables in the first place.
I wonder what would have happened if Twine would have started out as a more regular web service with the intent to migrate to a KR backend in time. That might have been the safer strategy. In any event it’s all irrelevant at this point and one of the reasons for joining it in the first place was the KR backend and the possibilities inherent in that choice.
I have looked into building a data migration tool for Twine. So far this looks more or less viable but will, ironically enough, require some screen scraping techniques to complete, simply because the data exposed by Twine is incomplete. One can use the news feeds to get a list of a users twines and from these harvest the Twine and Item data but the item data is incomplete so screen scraping will be required to finish that task. Also, it is necessary to look into the TOS of the service in case one wants to migrate to some other service.
I’d like to migrate my “investment” to some other service. Right now Thumbtack looks promising but it’s really in it’s infancy – although on the UX front they have done a lot of things right (iff still far from a great UX),
I also look to Google to bring this vision closer to reality. In particular because Google is in a unique (well, not quite unique) position to make a scalable solution. In particular one where an open-source community can submit new entity templates to the service. Even if closed, with the resources Google has, I’m sure they could build a lot of templates fast! I really do not understand how this compelling idea has not been more aggressively pursued by Microsoft and Google. Let’s see how Thumbtack fares and if Google comes out with something similar.
The killer feature here would be browser integration. And both Google and Microsoft are in the unique (well, almost unique) position to actually build in support for such services.
I secretly pray that someone will snatch up Twine, if not for the technology, then for the content, some of which is quite good. Of course most of the content can be found on the Web elsewhere, but it has been organized and assembled by human hand. Somebody ought to throw these guys and gals a lifeline.
Time will tell.
4. Addendum (I)
The reader might ask, so what else did you do, besides just posting content on Twine, content which Twine does not benefit from yet, since it hasn’t got advertisements? Well, let me list a few things I did
- Joined the beta feedback twine and engaged heavily with it
- Posted countless ideas to my Ideas twine
- Created a last.fm group for Twine users
- Created a linkedin group for Twine users
- Created the Users Empire twine
- Created the Users Auditorium twine
- Created the Users Etiquette twine
- Created Twinia twine, a twine about possible ways to integrate Twine with other servives
- Joined (by invitation) the private Power Users twine, then exited again due to fights
- Joined (by invitation) the private The Twine Lounge twine, for core power users, then exited again due to lack of engagement
- Created videos showcasing problems and possible fixes for Twine
I did get some Twine bumper stickers by mail, so no complaints here.
The twine lounge is, to me, a nursery. It is not about listening to and talking to users. It is about giving a false sense of listening. After all, the team is more or less unable to fulfill the vision and so they are not interested in having the feedback made public and also not in hearing it. This also explains why there is only one nurse – the creator of the twine. No CEO, no developers, only frustrated users.
A bleak assessment – but the truth of the matter.
5. Addendum (II)
Screen scraping is a concept that dates far back. Far earlier than current efforts by Microsoft or Radar Networks. I learned about the technique in the early RDF/XML days, where the community were exploring techniques to translate data into knowledge bases. Now Microsoft Live Labs actually has an ongoing project in this space called Entity Extraction, it is probably not tied to KR but it is about finding data on the Web. Microsoft also has another project called Thumbtack which is remarkably similar to Twine, right down to the visual theming (for shame!) But it does not at this time have the same social aspects and feel.
As for search engines employing domain-specific and targeted screen scraping we really don’t know a whole lot about what they are doing, but we do know they are going to have to become smarter to compete for the worlds advertisement money. Google’s future relies on it and I think it’s fair to say that Microsoft wants to crush Google and so they have to beat them at their own game.