The Guardian Lite android app

July 12th, 2010

Guardian Lite, an unofficial android reader for the Guardian newspaper, is now available on the Android Market (search for ‘Guardian’).

The application aims to encourage wide ranging browsing through the Guardian’s extensive collection of sections, tags and contributors as well as providing the fastest possible syncing for off line viewing.

Features

Online viewing

Articles are available for viewing immediately if a network connection is available. No pre syncing is required to begin browsing content.

Top stories

The current main stories from the network front of the Guardian website.

Contributors and tag links

Navigate through the Guardian’s current content with the tag and contributor links attached to each article. Discover relevant blogs and contributors for each section.

Favourite tags

Mark sections, tags and contributors for quick reference and offline syncing.
The favourites screen contains a custom news feed based on your favourites collection.

Offline syncing

Download your favourite article sets in the background for offline viewing.
Considerable attention has been given to making the syncing process as efficient as possible.

Syncing speeds of up to 100 articles per minute should be possible over a Wifi connection.

The application aggressively caches articles viewed while online for up to 24 hours. These articles can still be browsed when no network connection is available.

Guardian Lite is designed to run on any android 1.5 or higher device.

How does it work

Guardian Lite makes use of the Guardian’s excellent RSS feeds and Content API. The article content is sourced from the RSS feeds, while the Content API provides information about available sections and tags.

Information from these two data sources is collated in a Google appengine hosted proxy. The proxy formats and compresses the content for efficient delivery to user’s handsets over limited bandwidth connections.

To enable fast performance on low end devices, the application uses flat files and vanilla Java serialization for local storage.

Thanks to

Many thanks to those who assisted with device testing, including @gdb_, @robertomango, @bruntonspall, @brunns and Alan Lord.

New Zealand Government feeds

March 17th, 2010

NZ Govt feeds is an unofficial, experimental aggregation of New Zealand Government news items.

The list of source feeds is drawn from Mike Riverdale’s collection of Government feeds.

The site software was originally developed to drive the wellynews local news feed.

The system provides a number of outputs which might be of interest to developers:

- Content can be tagged.
Users may add tags to individual resources. These tagging votes are combined with automatic tagging to arrange news items into different output feeds.

- Every tag has an RSS feed.
For example, the Auckland tag page has it’s own RSS feed.

- Tags and publishers can be combined.
ie. Health and Wellington related items, or everything the Electricity Commission has said about Auckland.

- All feeds are also available in JSON format
See the API instructions for details.

The site still contains some references to it’s original Wellington related usage; these will be removed if this turns out to be of use to people.

Any enquires should be directed to enquires (at) wellington.gen.nz.

Multihoming a PHP application with Apache2::PerlSections

November 15th, 2009

Squadlist is a simple Mysql / PHP application which is used by a number of UK rowing clubs. Each instance of the application is hosted on a separate Apache virtual host and uses a separate Mysql database.

Unmodified, each installation requires a separate copy of the webapp and a separate Apache virtual host configuration file. Maintaining separate copies in this way would be complicated. Generally, I don’t do complicated, so needed a way to reduce the duplication.

Extracting the database configuration into the Apache virtual host

The application contains a service class which holds the database connection creditials for the given instance:

The first problem is that this file needs to be localised for each instance, requiring a separate copy of the application to be installed and maintained for each new instance.

We’re going to fix that by defining each instances unique database connection settings in the apache environment for each given vhost

Then in PHP, we can assess the apache environment to retrieve the database credentials specific to this instance.

Because our PHP app is completely stateless, this is a safe approach and we’re now down to one copy of the application files.

Automatically generate the apache vhosts

Next problem is maintaining the apache vhosts.

Adding a new install requires a new vhost file to be hand crafted and installed. Upgrading the system requires the DocumentRoot to be updated in each file.

Enter Apache2::PerlSections which lets you execute Perl code in your apache config files. By placing the credentials for each instance into a mysql database, we can use a Perl DBI script to automatically generate the required vhosts on apache startup.

‘instances’ is the Mysql table containing our credentials. The connection details for this instance are in the root owned apache conf file so this does not expose the access details for the instance databases.

Note how $PerlConfig lets you print conf directly into Apache. To reiterate to those who haven’t seen this approach before – that Perl code is in the Apache conf files and is actually been executed inside Apache on startup to generate the vhosts.

Adding a new instance of the application now consists of adding a row to ‘instances’, setting up the DNS and restarting apache.

Sceptics may well ask what drugs was I on I wrote this? I’m happy to answer; it was Oseltamivir

Wellynews tips – custom RSS feeds

November 15th, 2009

A number of people take the main wellynews RSS feed at http://wellington.gen.nz/rss. However, this is just one of many feeds which the system automatically creates.

Tag feeds

All tags have there own RSS feed. You just need to substitute the tagname into the url:

ie. http://wellington.gen.nz/transport/rss

This feed contains news items tagged as transport, or any of the transport sub tags (such as airport and parking). It also includes all news items published by organisations tagged with transport (ie. Snapper card).

A full list of available tags is shown here.

Combinations

If you want an RSS feed about, say, zombie outbreaks but only on the waterfront then you can use a combination feed.

ie. http://wellington.gen.nz/zombies+waterfront/rss

Note two tag names separated by a plus sign in the url. Interestingly enough, zombies and waterfront is really a valid combination and will return content.

Publisher combinations

If you’re looking for what a given publisher has said about a given tag, you can extract that using a publisher combination.

For example, mentions of Newtown made by Wellington City Council.

ie. http://wellington.gen.nz/wellington-city-council+newtown/rss

A list of available publishers is shown here.

This url scheme is inspired by the excellent implementation on the Guardian newspaper’s site.

Barcamp slides – Supporting the masses while holding down a day job

October 31st, 2009

Barcamp is an ‘un-conference’ event where all attendees are encouraged to give a presentation on something which might be of interest to their peers.

I spoke about some of the technical and non-technical things which have helped keep the support issues for Squadlist to a minimum.

The slides are available here:
Barcamp talk slides

Mystery Object

September 27th, 2009

Todays mystery object is the dangly bit which attaches a bicycle derailleur to the frame. This particular example was bent in the work bike shed nearly a year ago and proved extremely differcult to replace (been off a second hand French bicycle).

What I’ve learnt:

- They’re called ‘hangers’.
Like an electrical fuse they’re designed to break first in a collision to prevent damage to the frame. However, every manufacture seems to use many different variations, making them differcult to identify and replace.

- An after market has sprung up
After market supplies have moved in to supply replacements; manufactured in small runs.
ie. derailleurhanger.com

St John Street Cycles were able to confirm that my example is actually a ‘Number 6‘ and had one in stock.

On a Boat

July 29th, 2009

Photos from last weeks’ experimental canal boat trip on the Shropshire Union Canal.

The boat was hired from Mary Gary narrow boats, who were very helpful.

The Beeton Castle pub, at Beeston Locks rates a mention for good, no nonsense meals.

The Peak District

June 14th, 2009

Looking south towards Edale, during last weekend’s hike around Kinder Scout in the Peak District.

WCN hosted content

June 8th, 2009

Wellington Community Net (WCN) is a free web hosting service for Wellington community groups. WCN has been funded by the City Council for a number of years now, but that funding is now under review.

WCN hosts 600 odd community groups, meaning that alot of the content that Search Wellington indexes is probably hosted on WCN. I wanted to find out how much.

All WCN sites are physically hosted on the same machine. Taking a dump of all valid websites from Search Wellington, and filtering out the who’s DNS resolves to the WCN server, should identify WCN hosted sites.

The Search Wellington API can then be used to tag each of these sites with a hidden ‘wcnhosted’ tag. The following Perl script checks the DNS for the uses an API call to apply a tag to wcn hosted sites.

All of the news items and feeds published by these sites are automatically pulled into this tag, giving:

http://wellington.gen.nz/wcnhosted

Which shows that approximately 340 of 9000 odd indexed, news items originated from WCN.

This is a low measurement, as Search Wellington chooses not to list all groups on WCN and only includes news items with a permanent url, but I think shows that the loss of WCN would leave quite a significant hole in the local webspace.

Work Permit to Tier 1 (General) experience

February 13th, 2009

My Work Permit was due to expire on the 31st of March 2009. This 4 year work permit had been granted back in 2004 and at the time would have seen me through to permanent residency.

However in the run up to the 2005 election, in order to look tough on immigration, the Government retrospectively changed the permentant residency requirement to 5 years.

Work permit holders already in the system were left with a short fall between there leave to remain deadline and the permanent residence requirement.

After 5 and a half years in the country, I was facing the very real risk of been sent back home.

My employer was not in accredited to grant new Tier 2 work permits, so an extension was out of the question.

Furthermore, the local labour market had changed, meaning a work permit sponsorship was probably no longer justified in my sector.

Therefore, my only hope of extension was to switch in into Tier 1; previously the Highly Skilled category. Changes to the salary requirements made this possible.

The salary requirements for the old Highly Skilled scheme were very high and would have ruled my out. The Tier 1 scale is considerably lower.

My Tier 1 application was posted on the 3rd of February by Special Delivery next day. Payment was by personal cheque.

My main concern with this application was that having recently graduated, my degree certificate had not been yet been issued.

The documents requested from my university to confirm my degree did not turned up in a rather unofficial looking format,

which may not have been to the letter of the guidance instructions. One of my supporting bank statements was also a duplicate.

After posting the application, I became aware that I’d incorrectly completed the visa number field; I’d entered the VAF number.

The form was marked with a ’see enclosed passport’ note however.   I also discovered I’d made an 82 pence

error in calculating my earnings. I had alot to stress about.

3 February.

Package posed from Euston post office in central London. The package immediately showed as accepted on the Royal Mail track and trace website.

4 February.

The package mysteriously dropped off the track and trace service and stayed that way. There were snow storms across Britain this week, which could have resulted in delays.

A google search revealed that this could be standard behavior for Home Office mail. It’s not tracked because due to the volume of Home Office mail, it’s not been signed for.

Whatever the reason, this caused a considerable amount of stress. A lost passport at this stage of the game would be somewhat differcult to recover from.

9 February, Monday morning

750 Pounds deducted from cheque account; considerable relief as this suggests that the documents and passport made it to the Home Office.

13 February

The reference letter arrived, dated 11th of February.

“This letter confirms reciept of your application and that payment has cleared.”

27 February

The Tier 1 processing time page jumps ahead 2 weeks.

From 23 Feb -> received from 19 Jan to 2 Mar -> received from 17 Feb

6 March

The processing time page promptly jumps back 2 weeks!

27 March

My degree certificate had by now arrived. Posting a second application would be a way to get this new evidence before the Home Office and correct the

VAF number and salary error.

Once an application is rejected, you cannot submit new evidence in an appeal; a new application is required. However, due to the rule changes and my leave expiring, any second application would need to be posted before the 31st; after that date, I would no longer be allegeable to apply.

Therefore, a second application was posted at lunchtime; before the outcome of the first application was received.

4pm – phoned the Home Office.

Package dispatched in the post.

They don’t tell you the result or the content.

30th

Package is in pigeon hole at at the office.

Passport on top and open at the new visa – result!

One day before my leave was due to expire.

I asked my bank to stop the cheque on the second application, which was then returned about a week later.

Conclusion.

Paying by cheque does indeed seem to delay your application by 5 days.