Saturday, February 19, 2011

Capturing the moment

(now that I'm over the giddy part about being off-grid and on-line, time to make my actual point) 

It's easy to get spoiled living in place where a 1.5Mbps connection is the bargain-basement line and become utterly unable to appreciate how utterly life-altering it is to have a simple, reliable 200K of data connection.  Then you have a moment like the one captured in the screenshot below from Peter's computer (if it wasn't dark when this happened, I'd need to have one of Peter's expression that was just utter joyous disbelief)


After a day of permanently fastening cables, heatedly discussing the best pricing schemes [given our current reliance on mobile], and tromping around town to different locations looking for a backhaul link Peter and I sat down to take a look an audio issue with his laptops.  As it turns out, his hardware doesn't work out of the box on linux, so we had to search some forums and install some packages to fix the issue.

Just for fun, I thought I'd try to stream grooveshark while we were waiting for some stuff to install  (external speakers worked while internal ones didn't).  A minute or so of buffering later, we intersected the satisfaction of diagnosing and fixing a computer for free using online resources with the realisation of new technical possibilities (streaming) and the fully dominant music delivered by those possibilities. You may use the soundtrack below for recreating the moment in the comfot of your own home:


Joe Pug - Hymn 101 from Sam Molleur on Vimeo.

It's dark, there's no power...

...and I'm on the internet.  Ok, maybe this is childish, but being on wifi when there's no power for miles is friggin' cool.  :)

More toys revealed soon.

Wednesday, February 16, 2011

Always an Angle: Njabini Online

As I mentioned in an earlier post, our site in Njabini is a prime location for creating a truly cooperative community network.  The nagging problem is bandwidth.  Originally we had thought that Orange would come in with a middle mile solution for us through the Digital Villages initiative, but it seems that building backhaul is not in the DV model, even when the business case might exist.  Upon receiving the GPS coordinates of the Njabini site Orange promptly said, "sorry, can't help you".

After hearing the bad news, I nearly resigned myself to abandoning the Njabini site (which probably would have been prudent from a time management perspective), but I really REALLY hate being beaten and the FK / Njabini community is incredibly excited about lighting up the town with internet.

Without the promise of an "unlimited" connection from Orange, I was initially hard-pressed to find a reasonable business model.  Mobile connectivity is gosh-darn expensive to resell, and the idea of paying by the bit is rather unpalatable (though realistic as a service provider).

Then I looked at the rates... Information asymmetry is a dangerous thing. If you don't have easy access to a price list (which most people don't know they do), it's easy to fall victim to the design of the mobile operator's billing scheme.  In Kenya most operators sell bulk mobile internet connectivity in "bundles", which vary in price/MB based on how much you purchase at once.  For instance, Safaricom charges 3.33KSH/MB if you buy 100KSH at a time and .5KSH/MB if you buy 9,999KSH at a time.  The real scam, however, is that not buying bundles doesn't keep you off the internet, it just costs you 8KSH/MB.  Bundles also expire in 30-90 days, depending on the bundle, so one can't buy in bulk for the future very easily.

For most internet consumers, the idea of purchasing or consuming 9,999KSH (20GB of data) in 90 days is nothing short of ridiculous. Much like all things here, people tend to buy only what they need when they need it and pay a serious premium for the flexibility.  The rub is that they often think they're paying less. One of my team members, who lives in Nairobi and uses the internet daily (albeit rarely from a phone), was convinced until five minutes ago when we looked at the rates that data on mobile (with no bundle) is cheaper than using a USB modem.  Our partner in Njabini believed the same.  In a way they may be right because web pages designed for mobile devices are often much smaller than the complete version of the same pages, but the rate/MB is much, much higher.
  Using Fabfi, a single provider with a strong connection can buy the biggest bundles to get a good rate, passing on those savings to wifi consumers who can purchase connectivity flexibly at lower rates.  Local caching and services can then further increase that margin.  A real HSPA connection would even be fast enough to do this at a reasonable service level.

Of course, Njabini doesn't have a reasonable HSPA connection (heck, they only got grid power three years ago), but even sharing a crappy mobile connection with a couple of people is better than browsing all by yourself, so away I hacked.

I'm not proud of the aesthetics here, but I assure you the mounting is temporary...

The first linksys connects to the Safaricom 3g dongle and shares the connection to our headnode, which runs a local webcache and all the usual mumbo-jumbo.  While we're only getting 256k from the dongle, point-to-point speeds inside the network are impressive.

Over five hops (Ethernet -> 5Ghz -> Ethernet -> 2.4Ghz -> Ethernet) and 2.5km, we're pulling nearly 30Mbps real (in marketing bitrates that's about 75Mbps) to the Flying Kites house, where users connect to a wifi cloud.


Since FK is off the grid, we're using 60watts of solar and 14Ah of battery to keep the node up.  With two ubiquiti devices, this setup runs 24h in the current weather, and I expect to have enough margin to keep it up during waking hours in the rainy season.  Notably, users can still browse when all the power in town is out (a daily occurrence).  There's something very satisfying about checking your email in a dark room on the side of a mountain at midnight... 

We also have a wifi cloud in town and wired computers at a cyber cafe run by Peter, our local partner. 

This weekend the fabfi team will be in Njabini running a training for Peter and FK.  Wicked exciting!

NOW OMG PLEASE WILL SOMEBODY GET US SOME REASONABLE BANDWIDTH?  (I'll make my own ptp up to 10km...)

Sunday, February 6, 2011

Mobile, Mobile, Mobile, Harumph

So everybody says mobile is the next internet in Africa, and indeed Nairobi is chock full of teeny little stores that look like the one at right.  Despite their number, they all seem to do reasonably good business, at least in 20-somethings ogling, fondling asking questions about the latest offerings.  While having a shiny new phone may be "cool", it is absolutely not the solution for internet expansion in the next 5 years.

Even in the city, with 100% signal strength in the middle of the night, my brand new Orange 3G+ modem does about 256k.  Respectable, and certainly as good as the other options, but not mind-blowing.

Out in the rural areas, most places with mobile service have data speeds that will make you cry.  Do you know what a roundtrip time is?  If you do, then you'll know that 9 SECONDS is not even sorta-kinda-almost-close to acceptable.  (disclaimer: that was the worst I saw.  Average was closer to 700ms, and measured on a safaricom stick).  The facts are that long-range mobile wireless is hard to do well, a cell tower is only as good as it's backhaul, and the latest capacity numbers posted by wireless hardware companies are not much more than bench-test values.  In the real world, mobile eats it.

Nonetheless, when you've got nothing else, you might as well plug it into your fabfi.  Which Is why I'm working on this:



That's an Orange 3G+ modem in the back of a linksys.  Right now it connects, but needs to be hotplugged right before you try to do so for it to work (read: I don't have the modem commands worked out yet).  Nonetheless, progress...

Thursday, February 3, 2011

Welcome Mt. View (for real this time)

I don't want to speak too soon, but as I write this John and Nick are diligently repairing the power supply problems we've been having at Mt. View over the last week.  In all likelihood JoinAfrica should be back up for keepsies in Mt. View by the end of the day today.

If all goes well, we should be opening the network to general enrollment starting Saturday.  Sweet

Wednesday, February 2, 2011

Off the Grid in Njabini, repairs, re-engineering and all the other stuff we did last week.

This weekend was yet another learning experience for team fabfi.  Thanks to a little prodding from Paul, our most recent enabler, we made the trek out to a little agricultural town called Njabini, and we're undoubtedly better for it.

Njabini is officially in the middle of nowhere.  No internet (to speak of), regular power outages, nothing but Chinese-brand electronics (and not many of them).  If the path to development is clean water, clean power, email and Skype (and it is), then Njabini has a long way to go, but what it lacks in infrastructure it makes up for in community and cute kids.  



With the best growing climate in Kenya, Njabini is a reasonably prosperous rural town (as in more people have jobs than not, even if they don't pay much).  It also has a great, albeit somewhat competing, group of community leaders -- head of the agricultural association, head of the water board, the mayor, the director of the local orphanage -- all with a meaningful interest in finding ways to improve their little town. So why do we care?



First of all, Njabini is primed for implementing the vision of free-to-fee that I've always envisioned -- build a sustainable backbone of user/subscribers and use that backbone to support free access for surrounding residents.  Thanks to the vocal support and networking from Flying Kites Global (operator of the local orphanage) we have a whole bunch of people interested being that backbone as soon as we have the bandwidth.

Second, Njabini is HARD.  If you didn't bring it with you, you probably can't get it, and Murphy's law will apply in spades to everything you do bring (so bring two of everything).

Following from reason number two, one of the most important objectives for team Nairobi is getting them to be prepared for the unexpected.  Since we've begun deploying "node in a box", I've insisted that the team do all the installing themselves.  Making mistakes where it counts is the fastest way to learning how not to make mistakes, and during our first trip to Njabini, Nick got a hefty dose of walking the tightrope without a net. 

As it turned out, our nifty CPU power supply hack had a .5A current limitation that was only labelled on some of the supplies and went unnoticed into the first four nodes.  Even though we tried charging from a discharged battery in the lab, the city power was clean enough to let most of the supplies survive and the failures we did see were easily attributed to other circumstances.  In the field, however, we drove the batteries to the limit powering our gear while we wired up the first node, and hooking them up to the supplies on questionable mains promptly fried two supplies in a row before we realised the issue.  In any normal deploy, we'd just grab a battery charger from a local hardware store and be done, but the best Njabini had to offer were some tiny 1A universal power bricks, which Nick cleverly disassembled  and wired together like so:



This nearly worked out, except that wishful thinking got ahead of hard science and only two of the 1A bricks were used to power four devices and a switch, leaving the system a few watts short of equilibrium -- a problem that wasn't noticed until the next afternoon when the batteries cut out midway through peaking of a 2.5km link to the Flying Kites orphanage.  In the end, we were able to prove a link to FK from the center of town was possible, but weren't able to work out a permanent install before returning to Nairobi for some emergency maintenance.

While Nick and I were in Njabini on friday, Tom and John were in Mt. View installing a new node.  All was well and good on the new install except that the long-haul to Lavington disappeared, and given that the feed elements were still prototypes we fully expected that something had simply fallen off the reflector on the Lavington end. It took until Tuesday to get an engineer to let us into the Lavington tower where we found nothing amiss, and the old feed still firmly in place.  Meanwhile another power supply failure had occurred on the remote end of the link leaving Nick to scramble to find temporary power for testing that the link was A-OK. 

While waiting we replaced the old feed on the tower for good measure and got the link back up with no trouble, but were left with a pile of Mr. Fix-it for this week.  While this was all a huge pain in the tukkus, it did drive one point home for team Nairobi, and that's the value of testing and contingency planning.  Last night Nick and John spent some hours building and itemising the contents of a failsafe field kit containing all the equipment required to adapt to almost any unforseen circumstance.  We'll see how it works today when they go on a blitzkreig of node repair without me.  And let's hope that this time the fixes stay fixed for good.