Building TransforMap I

This post has been extracted from a too long forum answer that came up while preparing the TransforMap landing page and is displayed here for reference.

It prepends another draft from earlier this year that comprehends the narrative and will be delivered here soon. The whole is then to be concluded by a focus on federated knowledge engineering.

The parts are:

I Web Applications + Data Syndication
II Federated Tempospatial Data + Decoupled GIS
III Communication Infrastructure for Organized Networks

For those arriving from there, first some disambiguations.

  • allmende.io is my personal contribution to the Commons. To provide free and open communication infrastructure for self-organized initiatives.
  • Allmende were the pastures in villages where work and crop would have been shared equally. The word is the German translation of Commons.
  • These infrastructures have been provided to bootstrap 14mmm.org and transformap.co.

Now imagine the following to be written over there at GitHub:


When I see issues like Central user Database, User Authentication and Local User Database a strange feeling arrives regarding federated identity. Why do we see tendencies here to recentralize an aspect of an application?

Talking about the Unhosted, Offline First and noBackend approaches by remoteStorage.js and hood.ie. Yet again brewing their own soup. New buzzword: Front-End First.

Following discussions like Sync to non-commercial storage makes me wonder the importance of independent storage is not as prevailent as I believed.

A secure scuttlebutt , eventually routed by NDN and Level Replicate on the other side implement protocols close the work of the *ouchDB crowd. (Referred to as the ouch replication protocol).

I see DocPad's role here as the reference implementation for decoupled human interfaces. A way in helping to keep the dependency graph small for each component.


Why is it now that I can't replicate, say, from LevelDB running in IndexedDB, not even leveraging any other local PouchDB, to CouchDB or remoteStorage? Why can't we have data portability as simple as copying files?

For talking to the user's Dropbox account, you can use Dropbox.js, and there is a similar client-side library for GoogleDrive.

PouchDB acts both as an in-browser database engine, and a client for replicating data from and to a remote CouchDB instance with CORS enabled, which may be the user's own CouchDB server. Likewise, sockethub-client makes it easier to communicate with a Sockethub API which may be on the user's own server, and hoodie.js does the same for Hoodie's API.

All these libraries have a one-to-one relationship with a specific backend API. For PouchDB this is not such a limiting restriction, because there are quite a few software products that expose a CouchDB-compatible API. But as far as I know, only Sockethub itself exposes a sockethub-compatible API, and only Hoodie itself exposes a Hoodie-compatible API. That means that basing your app on them may allow the user to choose where they want their backend hosted, but in practice they do not have a free choice out of more than one backend software implementation.

Of course, Dropbox-js and the GoogleDrive JS library are even worse in that sense, because they only allow you to sync with per-user backend storage at one specific storage provider, so that's even more restrictive. That's why it is interesting to look at client-side libraries that try to be as generous as possible in the per-user backend they are compatible with.

via https://unhosted.org/practice/32/Client-side-libraries-for-per-user-backend.html

I like the definition of a webwrite spec, something similiar to writing remoteStorage or ouch (which they sometimes even just call JSON-over-HTTP), but closer to Hydra-style Hypermedia APIs.

But still, what is lacking to let APIs actually talk to each other, explain themselves and make the desired content flow? I'm yielding at the Federated Wiki's API now, which uses verbs like fork in a distributed manner (GitHub would dream of) that allows us to use the term federated.

GitHub is not the decentralized vision git itself is providing. From a wiki point of view (single point of view every point of view ) federation leads to a new way of http://wiki-allmende.rhcloud.com/view/wiki-version-control/view/wik-dvcs/view/semanticize-wik/view/socialize-wik/view/wik-architecture/view/wik-transport/view/wik-storage within a Chorus of Voices.

Another candidate to federate with is now also, next to TiddlyWikis (with Timelines) and any other Wiki, forkdb being used in wikidb for simple howto wikis.

Now, why can't I just fork everything into a federated wiki worksheet (IPython style) right now? The technology seems to be there.


But what we're lacking are the social protocols.

The whole point of a standardized protocol is to eliminate the need for centralized, proprietary protocols and solutions. On the Web we create common standards and protocols for that. We usually only let them compete before they become final specs to see what works best, and then continue with a single one that people agreed upon. (This is an over-simplified view of reality of course, but the general idea.)

via http://community.remotestorage.io/t/comparison-with-other-products/201/9.

Loomio aims at facilitating these conversations, and we're also trying to design these kinds of rhizomatic social activities in JSON-LD, but it remains hard. When do we know the competition is over? When do we start to cooperate on a truly global scale?


Appearantly building this TransforMap, whatever it may become, can be challenging. But I know for sure it can only succeed if we agree on a computational ethics that is at least semi-aware of its implications. More on this later.