Write locally, deploy globally (or: synchronizing Joomla sites)

Once in a while, I am somewhere where even my Verizon USB dongle can’t help me get connected: in n airplane, in a remote location, in a nuclear bunker… In those cases, I would love to be able to work on my site, but how do I do that?

Enters Blogg-X, a nifty little Java app that allows you to edit content locally, and then synchronize it globally to a Joomla site. It even allows you to synchronize to multiple sites (or at least it allows you to write for multiple sites). See: that’s what open source is all about! Problem solved!

Well, wait a second. First of all, Blogg-X is a Java application: when I change my editor in Joomla, how does the Java app know? Well, it doesn’t, and its text editor is proprietary and only vaguely related to Joomla. So if you use any extension, its results won’t show up in your content, and you will have a hard time reconciling changes.

Who cares? I still thought it was worthwhile. But then, the tragedy of open source happened: it doesn’t work. I downloaded the current version (2.13), but when I tried to install it, it told me that the zip file that was included in it was corrupt. Probably just a matter of padding or such, but I didn’t want to go through the trouble of figuring it out.

Next I tried the earlier version, 2.12. That one downloads and installs fine (hooray!), but unfortunately, the code doesn’t know how to login to a current version of Joomla. Sigh! So you are stuck with a new version that doesn’t work (on Linux, I am sure it works fine on Windows) or an old version that you’d have to hack.

And that’s of course when I started looking at the forum on the Blogg-X site. And found out that (a) the problem with the zip file was universal, and (b) that the first complaints had been logged months ago, but nobody had done anything about it.

Since I had a similar experience with a different component, zOOm Media Gallery – discontinued, it seems, since no new development has been released in a year, and the site is quiet to death – I was worried that Blogg-X might be one of those things that don’t quite work for you.

Time for an alternative. I started thinking about it for a second, looked at all the pluses and minuses of the issue, and realized that Blogg-X was created in a completely wrong architecture.

Why wouldn’t you want to write your content in Joomla? The same Joomla that you are using for the main site? You install it on your local machine, write content to your heart’s content, and then you upload it to your site. Easy as pie.

So, how do we do this technically?

Well, there are two problems to solve. The first one is the easier one: how do we upload and download content. The second one is harder: how do we deal with synchronization issues like concurrent access.

For the first one, we have two options: we either synchronize the site, or the content. Synchronizing the site means taking all the relevant files from the web server directory, all the relevant tables from the database, and copy them over. That’s what I did with this site, which existed under a different incarnation (diva.homelinux.org). Synchronizing content includes only the content items themselves, and maybe the images and categories/sections.

The advantage of synchronizing the site is that everything is going to be identical on your sites (local and Internet) once synchronization is done. That’s wonderful, and far superior to content item synchronization, since it includes installed components, non-content data (like calendars, user accounts, etc.). The downside is that it’s a lot of data that needs to be synchronized (in the case of diva, that was almost 1GB, since it included tons of pictures).

Let’s assume we want to go the route of complete synchronization. A good way to work around the 1GB issue is to synchronize only what’s necessary. A good idea is to keep a copy of the site as it was at the last synchronization on the server – either as the actual site, or as a series of metadata concepts. This needs to be done for both the web server directory and the SQL data, and we are golden.

The first step in such a site would be an initial sync: both sites would have to be tagged and brought down, and a merge performed. This can be safely avoided if one of the sites is the master, in which case all that needs to happen is a copy of the master into the local replica. This could use a version of com_jpack.

Once that’s done and both sites run off the same copy, both will change, and both will retain a copy of the latest backup. When the sites need to be merged, a tool/component will compare the local repository and databases with the last copy (not the remote). This will happen on the remote, as well. Now both changes are sent each other’s way, and the merge can begin.

Merge is straightforward for files: if a file has been modified only in one place, replicate the change to the other place. If the file has been modified in two places, ask which one is more current.

The database is a little of a challenge, mainly because it uses unique keys as identifiers. These keys are typically autoincrements, which is a really bad idea in synhronization – both servers are bound to create the same successive ID, and if you create content on both copies at the same time, then you are going to confuse everyone.

Still, no reason to assume that just because something is hard to do, it shouldn’t be done – even if the solution in the end has some rough edges.