Google legacy services for Email, Contacts, Calendar Data represents a significant footprint even excluding new Android platforms. It dawned on me that every one of these Android devices now processes voice data in the cloud for any text input, possibly a capability only Google’s scale can provide.  Perhaps this is why the voice recognition has not been integrated into competing platforms.    The power of the Google cloud becomes even clearer when you look at what kinds of data processing applications are becoming tethered to services like Google Docs.   The Google Refine tool linked here is one I think has applications beyond the average spreadsheet of data.

To put a keynote on what powers the Google cloud I might focus on the storage technology.  Michael Rubin does a great job of supporting some of my own widely held opinions around linux storage technology in his presentation below.   If you are like myself, optimizing layers of RAID, Crypto, LVM, iSCSI and clustering file systems every decade to support your own infrastructure you will probably enjoy the lessons learned during Google’s multi-year upgrade from Ext2 to Ext4.

www.youtube.com/watch?v=Wp5Ehw7ByuU

http://www.geek.com/articles/chips/apples-ios-4-hardware-encryption-has-been-cracked-20110525/

It seems reasonable to assume that both Apple and Blackberry will close these back doors soon.  For Blackberry it requires v6 software only available on the newest generation of RIM phones.  Some of the security issues may be linked to the legacy requirement of tethering these sorts of devices to a computer to perform operating system upgrades.  This is something else we expect to go away soon.

For myself, security on my superphone is primarily to protect my identity and information related to business and clients.  There is a flip side to the coin where it seems beneficial for some law enforcement agencies to have access to phones for investigative purposes.   Hopefully all the captured bad guys are on iOS or Blackberry.   I know at least two “good guys” on Android.   Peter Bishop from Fringe and Jack Bauer on 24 .  In the non-fiction arena the US military is also getting into Android app development.

May 13th Update: Still no Google Contacts (except for small addressbooks) on 11.04

More here: https://bugs.launchpad.net/ubuntu/+source/evolution-data-server/+bug/755043

Last week the Ubuntu Beta 1 was released.  The first beta is typically when I jump on board.   This allows me to see the new functionality in its intended state, and to file and contribute bugs related to my specific use cases.  Between my various business and personal devices, there is plenty of opportunity to test things out at this stage without much risk.  Any big issues are included in the release notes which are a good place to start for anyone considering the jump ahead of the official release.

This post might be a useful reference in the coming weeks for others with similar hardware and software configurations, notably some work peers eager to make the same migration on similar hardware.  Please check that the linked bugs are still not resolved in the distribution packages before executing anything.

For my upgrade, I use the console based tools. It is just the way I grew up on Debian and is a more robust process allowing you to see more output from the dpkg process.

#do-release-upgrade -d

Unity on FGLRX

Black Boxing Compiz – If you upgrade an fglrx based 10.10 system, you will arrive at an unusable Unity desktop with black boxes covering windows and flickering and moving around with mouse movements.   The workaround is to add the daily unity PPA to your apt sources and upgrade before the final reboot following the upgrade.  Here are the root commands:

 add-apt-repository ppa:unity/daily

 apt-get update && apt-get upgrade

At the time of this post, a fix was just released in unity 3.8.4. You can query your version using dpkg following the upgrade to see the workaround is probably no longer necessary.

 dpkg -l | grep unity

I have now switched back by removing the ppa from my Other Software sources using the Settings option from the Update Manager.

APT Software Sources

Some people that remain on the daily version may find themselves with a newer compiz or unity version than the apt archive.  This would be shown in apt using the following command.

apt-show-versions | grep newer

If this is the case, you can roll back once you have removed the apt source using the following command, or similar for a specific package.

apt-get install unity/natty

Evolution with Google Contacts

If you read through this, you will see that I have not completely fixed this problem, though for people with fewer contacts, it may work just fine.

Broken Google Contacts – Unfortunately Google integration is not working yet in Beta 1.  For myself, with all my contact and various calendar data in the cloud for use across my various devices including my Android phone, this is a pretty big inconvenience.   As of right now there is a patch committed for review however it has not been released.  To fix this problem we apply the patch ourselves.  To give ourselves access to package source code in order to rebuild the evolution package with the patch, we need to enable Source Code checkbox within the Ubuntu Software tab of the Settings dialog from the Update Manager.

APT Source Code Repositories

With the source code enabled we can download the evolution package source and apply the necessary patch, which in this case is modification of a single line of code as shown in the link above. I usually do these activities from within my /usr/src directory to keep the litany of temporary source files contained to a logical location.

apt-get source evolution-data-server

The patch can be applied by modifying the appropriate line of code using vi, in my case using the following command.  For a single small change like this, it is easier than downloading the diff and applying it appropriately using the patch command. 

vi /usr/src/evolution-data-server-2.32.2/addressbook/backends/google/e-book-backend-google.c

The patch requires changing this line

gdata_query_set_updated_min (query, ((long) &updated.tv_sec / 60));

to

gdata_query_set_updated_min (query, updated.tv_sec);

Next we grab the build dependencies

apt-get build-dep evolution-data-server

Then we build the package (from the /usr/src/evolution-data-server-2.32.2 directory)

dpkg-buildpackage -rfakeroot -uc -b

Finally we install the built packages assuming no other .debs are in the same location as the newly built debs (from the /usr/src directory)

dpkg -i *.deb

Looks like there is more to this.. it did not work completely and fails after loading about 500 contacts..   I’ve posted my results to the bug and will wait to see where things go from here.

Bug Reports:

Large Google Addressbook not syncing on Ubuntu 11.04

Secondary Evolution Contact Addressbooks do not Migrate Properly on Ubuntu 11.04

Rogers is a great ISP. Let me tell you why before we make some observations about the current Network Time issue.

Here in Toronto, there are a variety of communications companies offering Internet, Mobile, Print Media and Television services.   Personally, the most critical service is my residential high speed Internet service required for a variety of services not excluding web, email, vpn and dns.  In Toronto, Rogers consistently provides a reliable top-tier service that includes reasonable upstream bandwidth (>=1Mbit),  DHCP assigned IPs that rarely change (suitable for external cloud access and DNS),  authenticated outbound SMTP and relay of non-rogers domain emails (required for email hosting and SPF policy since they block outbound SMTP and do not provide IN-ADDR records for external domains),  excellent availability and uptime (possibly location specific) and none of the MTU issues that plague DSL and more recent FiOS-branded solutions that require a local PPTP client or PPTP enabled modem in order to connect.   In recent years they have made substantial improvements to customer service that today includes quick connections to live people.     The Extreme service was introduced with a market leading 4Mbps (downstream), and it is now 15Mbps (downstream), and has been upgraded multiple times (6Mbps, 8Mbps,10Mbps,12Mbps) without significant changes to my billing.   Finally the bundling of this with my wife’s magazines, our home phone, home television and a business (family) mobile plan with 6GB per device and applied bundle discount make it difficult to compete with, consistently.   One of my Bell executive friends has me on his short list… but a non-PPTP service does not yet exist.

On the mobile network, coverage and bandwidth is second to none in the Greater Toronto Area. It is worth noting that Toronto is also home to Rogers Communications headquarters (downtown) and business campus (Brampton).   Transfer speeds are awesome, to the point where I don’t really look for public Wifi as it tends to be less responsive.  Check out the screenshot below taken from my car, where podcasts automatically download daily for listening pleasure on the ride home. FYI, the Category 10 HSDPA radio on the Nexus One (my phone) has a maximum of 7.2 MBits/s (921KB/s)  which means I am hitting the approximate maximum application speed after network overhead.

Nexus One Downloading on Rogers

So what is up with Network Time on the Rogers Wireless network? Rogers Communications is a business built from layering new services on their existing network infrastructure.  How can they allow a bunch of broken mobile devices to dictate poor policy related to Network Time Protocol?   Here is what I have learned observing some other Rogers devices around the office.

• Rogers NTP servers appear to be broadcasting both GMT -5 EST and GMT -4 EDT.  Android phones and some iPhone experience a flip-flop between current time and off-by-one hour if they remain synced to network time.    If you have a phone status screen that displays the time and timezone being forwarded by Rogers NTP, you may see either of the following two screens depending on which signal you are receiving at that moment. The current NTP timezone is displayed in the greyed-out “Select Time Zone” field when Android phones (Android 2.3.3 in my case) are set to sync to the network.

Broken Rogers NTP on GMT -5 EST

Correct Rogers NTP on GMT -4 EDT

• Blackberry devices appear to not understand Daylight Savings time properly, despite this problem being communicated as resolved.   All the blackberry devices I have seen show the incorrect 5 hour (GMT -5) adjustment required prior to 2007.    None of them seem to be picking up the GMT -4 broadcast like iPhones and Android devices.  On the Blackberry 9000 (Bold), the incorrect network time (Seen as 10:29 below) combined with the incorrect Eastern Standard Zone information (-5) results in correct time being displayed (Seen as 11:30 below) to the user .

RIM 9000 Showing Incorrect Network Time and Zone Offset

RIM 9000 Showing Current v5 OS

Bold 9000 Shows Correct Time and Incorrect Offset

Similar observations were made on Blackberry 8900 (v5 OS) and 8300 (V4 OS) devices also running on Rogers.   There were some slight differences that may actually just be OS version-specific software implementation differences.   On the 8900, the network time displays as if it was GMT -4 (correct actual time) even though the zone offset is displayed as GMT -5 (which is not correct) and the resulting device display showed a correct time.  The following screen shots of this device show this correct network time and an incorrect timezone offset as -5, yet the display clock is still correct.   In this case it seems that perhaps policy or software bugs are showing strange results, that sill give the user the correct time.  The v4 8300 also shows these same results.  It seems unlikely that these RIM devices use a different NTP server, but it is conceivable that they are somehow tethered to their backend BES/BIS system which in turn is connected to the upstream NTP server insulating them from the flip-flop.

Blackberry 8900 Bad Zone Offset But Somehow Fine

Blackberry 8900 Showing v5 OS

At a distance, it looks like an NTP + BIS policy hack to make all Blackberry devices show the correct time even though the timezone information is incorrect.  There are possibly some unmanageable dependencies between large Rogers customers with possibly out-of-date BES servers that may be making proper implementation of NTP difficult.     Perhaps it is just a way to keep influential young Boomers, likely Rogers’ executives who use Blackberries, seeing the correct time whilst upsetting all the Android and iPhone customers in the background.

Enter the iPhone.  Equally troubled by DST issues, the Apple product has its own layer of software problems that are emphasized by the Network Time issues.  Below are some screenshots from my wife’s phone, which due to poor iOS software is flip-flopping between 1 hour behind and 2 hours behind because it actually has a DST bug that puts it an hour behind before it gets the wrong network time, putting it two hours behind at times.   Check out these screenshots beside the properly synced PVR.

Apple iPhone 3G Two Hours Off

iPhone 3G OS 4.2.1

The advantages of OTA (Over-The-Air) updates are very clear in the context of this particular issue.  Android phones, if they had an issue, could be addressed en-mass provided the proper support from the phone manufacturer to the carrier (still some partnership issues here).  iPhones need to be plugged in to a computer where your Apple account is registered, and then new 90MB+ iTunes must be downloaded and installed on that computer, and then you can receive an update directly from Apple, possibly customized for your carrier.   Blackberry users have a similarly tethered upgrade process, requiring local software, except that they have to go *find* specific carrier+model firmware, and there are enough clicks and accept dialogs along the way that most people never bother or try.

So we have hardware companies (RIM, Apple) trying to build phone software with varying levels of success.  My point here is that Rogers is not entirely to blame, but they should be able to manage these shortcomings more effectively and shift responsibility to the customer for devices that do not support OTA udpates.

In any case it appears some bad hardware paired with some bad policy decisions regarding management of mobile network time is having a dramatic effect on the customer experience.   Why do Bell and other providers not have the same problems?   There are many unanswered questions.   My observations suggest there is room for improvement.

Let’s speak to the impact.   The simple workaround is to just set your clock manually for the next month.

Potentially many subscribers were tricked on Sunday when they glanced at their phone and saw the adjusted time, only to show up an hour late for a lunch or afternoon appointment.    Can you imagine all those folks that had weddings or other large events booked that feel completely frustrated by this? What about emergency services, or enterprise support personnel that have alarms and reminders set to initiate various activities?   How many people are misled to the point that there is a significant financial or emotional impact, perhaps motivating them to consider calling their lawyer?

Bottom line, it is absolutely ridiculous.  Unacceptable product management.  Not well communicated to the users.

My imagination can envision the war room full of Rogers engineers scratching their heads in desperation while their technical superiors gaze at their top-of-the-line Android phones waiting to fire them all once the problem is resolved.    Perhaps it is a small room with only technical product and network team leaders, like the scene from Apollo-13 where they attempt to solve the carbon dioxide scrubbing problem presented when their “space hardware” ends up in an unexpected situation shown below.

At Rogers’ while technical leads scramble, their boss’s boss is unaware of the issue that will never manifest on his proudly gripped Blackberry.   Perhaps it will appear as a line item on the executive balanced scorecard, likely after April 3rd, when the problem has gone away.

Presumably it is a case of deja-vu, where they forgot to address the problem last year after it went away when the old and new devices fell back into sync in the first Sunday in April.  My expectation is that is what will happen this year.   Hopefully some new young technically inclined executives will better prepare themselves next year.  It has only been 4 years since DST changed.  I wonder how many DST projects have been completed to date within the Rogers organization, yet these issues still persist.

I find this all hard to believe, so I am waiting for some Rogers resource to explain to the world how this is really a very complex and difficult problem to solve, but only on Roger’s Mobile network.

Update: No official word on the root cause, but the DST issues were resolved on March 16th, 2011.   There is probably a litany of messed up calendar appointments, as timezone information behind calendar events is typically stored with the event, creating a ripple effect going forwards.  This would be most prominent on re-occurring appointments, or events synchronized between multiple invites where one or more updated or accepted an event during the short term DST issue.

Anonymous Reaction to Unmasking – This first article revolved around an interview on the specific process used to circumvent HBGary security practices leading to the posted internal emails.  The techniques were very simple and well known, but were ultimately catalyzed by an embarrassing interaction with Nokia’s Chief Advisor in Enterprise Risk & Security, Jussi Jaakonaho.   Jussi is LinkedIn with at least three intelligent people I have met personally, so without having met him, I will give him the credibility benefit of having had his first experience in this type of identity fraud and expect that he is probably more inclined to actually act according to what he already knows to be the best-practice.

For myself, the first article provided recognition that complacency around security practices is likely a reality for most organizations.  Where I have influence, I am also inclined to seek reassurance that access management  and software security update policies are being consistently applied across the enterprise, and to further inspect related verification criteria used by those responsible.

HBGary Technology and Products – This second article is the best abstract of the 44,000 HBGary emails I have read to date.   In addition to enlightening us on just how sophisticated the commercial malware space is today, there was significant information that re-enforces the commercial perspective that revenue potential trumps reporting known software vulnerabilities to the security community.  Specifically, there was overwhelming evidence that HBGary had a wide variety of  tested “Juicy Fruit” 0-Day exploits available to circumvent most combinations of enterprise security software running on Microsoft Windows operating systems.

In conclusion, after reading the second article I breathed a sigh of relief that my personal dependency on Windows OS platforms for security infrastructure ended well before HBGary began selling “Juicy Fruit” for $60K to the public and private enterprise.   The informed infrastructure architect may be having a second, hard, look at linux, BDS and solaris kernels over the former for any new projects this year.

On a related note, some colleagues of mine suggested that there is a good one time opportunity for Egypt while they are offline in this madness. All major businesses that may have been planning outage time for future updates and difficult upgrades to live services have an opportunity to push forward changes without fear of additional impact to current users.

Why not use this unscheduled downtime to upgrade all of Egypt’s major autonomous systems to IPv6? With only a day or two remaining until we completely run out of IPv4 addresses, it might present an opportunity to not only make some hard and fast changes but also create some media support for political players that are looking to create a message that might be well received by a society that has demonstrated an obvious dependency on its’ Internet service providers.

The reality is that there are many moving pieces, higher priority objectives and much larger issues that need to be resolved in Egypt.

For those of us not directly focused on Egypt, there is the impending end to IPv4. I predict that the immediate impact will be as great and as benign as Y2K. Enterprise decision makers that consume media headlines like as ARPAgeddon or IPcalypse will move quickly to tighten up their edge devices including web servers, routers and firewalls. In the new world of SSL-only transactions and federated services, network address translation at the ISP level will likely not be a viable option even with the smartest application layer gateway.

For the most part we can predict how the non-IT enterprise will react:

1. Initially there will be current reserves of IPv4 to keep the status quo (after all, running out != all used up)
2. New IPv6-only websites will appear creating awareness of incompatibility
3. Everybody will upgrade their edge devices and services (existing NAT/tunnelling allows internal IPv4 LANs to remain unchanged)

There will probably be an opportunity for networking consultants to pick and choose opportunities to upgrade and verify correct IPv6 functionality. It will peak now as forward and risk averse decision makers avoid unnecessary liability and then again a second time when the first mainstream, IPv6-only, web sites offer services online. Any organization that aims to reduce the complexity, effort and risk associated with firewalls, routers and web services has probably disabled IPv6 IPv4 even though most network devices have full support for IPv6 today.

Some of the challenges lie with larger enterprises that issue Public IPv4 addresses for internal services, such as VPN encryption domains.   Just writing that last sentence has me thinking of challenges ahead for some enterprise clients I am familiar with. If you are a small-to-medium sized business in the greater Toronto area and looking for assistance,  I can help route your request to some skilled consultants.   If you are enterprise, and are wondering about the impact to your federated services, we might be able to help you directly.

Here are some countdown links to keep your finger on the trigger, from one of my regular podcast sources Buzz out Loud where I originally came to realize how close this deadline is.

http://twitter.com/IPv4Countdown#

http://www.facebook.com/pages/IPv4-Countdown/162683847102050

After having implemented this type of Identity Federation technology in the enterprise, I do feel a bit late to the game, but realize this has more to do with setting priorities (hobby vs. family vs. work) then anything else.  It certainly has felt like a rewarding use of two hours and is actually going to eliminate time previously spent managing invalid user registrations.

Once the setup was complete, the site was tested using my Google credentials (xxx@gmail.com) to access the site.  My Google account has not been used for many things other than to keep my various Calendars and Contacts neatly synced in the cloud between my various computers, tablets and my Android phone.  For now, my Google account has become my new global identity credential.  The authentication process was similar to the federation implementations I am familiar with, however I suspect it may appear complicated to most people until it is a bit more ubiquitous on the Internet.  Here are the basic steps involved.

1. Enter the universal OpenID address for Google’s Identity Provider (a sort of global username for Google users)
   • You can usually just pop in your website or email here and it automatically redirects.  Today Yahoo works this way.. Google requires a URL, which is just configuration I need to correct.
2. Authenticate with Google (not necessary if you have already logged into Google recently)
3. Allow Google to share some of your personal information (your email address in this case)
4. Verify some new ianbmacdonald.com local profile settings (Blog alias to use for posting is one of these)
5. Use the site as any other newly registered user

Here is the screenshot version of this experience.

What are the obvious (or not) benefits?

For me (the Service Provider):

1. Email Addresses are verified by somebody else
   • I don’t have to worry about people registering with invalid email addresses
   • I don’t have to worry about old users not updating their email address
   • Way less email spam to me, the sysop / administrator
2. Brute force / Authentication attacks are no longer directed at my site
3. Administrative control over how “fresh” recent sessions need to be. (I am not a bank, so you’ll notice yesterdays login is still valid)

For the user (the Blogger):

1. No need to create a new user or share an overused password with yet another web site
2. Control over what information I share between my identity provider and service provider (Depends on your Identity Provider .. Google Yes)
3. Global review and revoke capability for my access to all my service providers (Depends on your Identity Provider .. Google Yes)
4. Opportunities for my service providers to improve my experience as they extend their services (Google users check out this google demo)

For the Identity Provider (i.e. Google):

1. More information on how users are consuming services on the Internet
2. Standard interfaces that drive collaboration and support for new B2B and B2C service federations
3. .. well just ask your trusted advisor, or perhaps the folks at N8 Identity ..

There are [many] more aspects that translate to quantifiable benefits however those listed here are the most relevant to what I am enabling here with federation on my web site.

I believe this Google demonstration might be somewhat insightful to web site strategist, as it showcases the benefits that additional OAuth extensions bring to OpenID.  It extends the information exchange beyond simple user attributes and authorizations to include complete service and information bundles.

A few subtleties I believe are not obvious until you start using a federated identity

This portion of the post will become quickly dated, but at the time of writing there are a few features related to my OpenID federation that I did not fully visualize the benefit of until I started to use them.

1. The ability to use the Google administrative portal to revoke my credentials on any site I have accessed using them
   • Now I can periodically review where I have been, and revoke access to unused and one-time use sites
   • I also understand where I actually have used my Google account vs. a local username or other federated ID in cases where I might have forgotten
2. The ability to use my personal web site URL to replace the awkward and hard to remember OpenID address (https://www.google.com/accounts/o8/id)
   • Now I use www.ianbmacdonald.com as my username, and my openid wordpress plugin redirects to google on my behalf .. pretty easy to remember
3. You can choose the email address to use with your Identity -> Simply register at Google with whatever email you want
   • You can have a federated identity for work and personal sites, separate and provided by Google (or at least until your company sets up an Identity Provider)

Your mileage may vary with other Identity Providers.  I plan to test a few, and might expect that other providers like Facebook might opt-you-in to sharing other personal information, and other’s like LinkedIn may eventually provide B2B use case benefits such as the ability to share attributes like a professional relationship or useful and relevant contact information such as your office phone number and job title.  The bottom line is that people are used to punching in their email address as their username which gives providers like Google and Yahoo a clear edge in usability.

OpenID installation using Debian and WordPress

Step 1: apt-get install wordpress-openid

Step 2: Apache2 Restart

Step 3: Enable OpenID and XRDS plugins in WordPress Admin Panel

.. and just like I have here,  you can now create a post using your federated identity.

Given the benefits, and time this is now going to save me personally, I regret not turning this on a couple of years ago when I first learned of the OpenID plugin for WordPress.

I will finish this post with a trailing thought, and an unanswered question for a future post.   If people will naturally continue to have multiple credentials, for example, one personal (for retail purchases), one public (for total anonymousness), and one professional identity (for business interactions),  the ability to switch between identities that Google now offers should become even more useful.  With multiple federated Google identities, can you simply switch Google identities in one browser window using the aforementioned feature and have your identity in another tab be dynamically switched upon browser refresh in another window?  I would expect so since the plugin and ability to do this have been around for some time now.

Doing anything here will void your warranty, and will prevent OTA (Over-The-Air) upgrade directly from Google to Android version 2.2. This upgrade is a significant modification.

In addition to giving you a bunch of new and pre-2.2 features, the upgrade process provides root access which is required to properly backup your phones applications and data. Without root access you so not retain some aspects of the system configuration and data, as well as “Market Links”. This last point, Market Links, describes your phone’s ability to determine if an application has an update in the market. Restoring applications backed-up on a non-rooted device will have no ability to self-update until they are manually downloaded from the Android Market.

I will re-iterate… root your phone before you go installing applications. If you don’t, restoring them onto your upgraded phone sucks, as noted above. aTrackDog solves some of this problem.. but most people will prefer to just “do it right”.

Locked Bootloader, Locked Phone and Locked SIM

When we talk about unlocking the Nexus One, we mean unlocking the bootloader. This is just like a “BIOS Setting” that allows you to load and boot other images onto your Nexus One. It is a fully supported capability of the phone, not some hack or magic set of unpublished keystrokes.

SIM-Locked means something different that does not apply to the Nexus One. This is when your phone is locked, via the SIM card and device settings, to a specific carrier. Phones that are SIM locked generally require a separate process that “unlocks” them from a carrier after which they can operate with “unlocked” SIM cards. Notably to use an HSPDA modem like the Rogers Rocket Stick you need an unlocked SIM card. I have always used Rogers and never had a SIM-locked phone, or a locked SIM card.. so I am no expert in this area. The bottom line here is that none of this applies to the Nexus One, which is shipped unlocked from a carrier perspective.

Nexus One Upgrade Primer

This was my first Android phone, so I had *NO* idea how the device was laid out internally. A little blurb that summed it all up seemed hard to find.. but I was able to string together the pieces after some time on the net. Understanding the basic device landscape was all I needed to have confidence in my upgrade process. I recommend reading this article.

The Nexus One comes shipped with two images (system and recovery) loaded into partitions on the onboard flash, and a bootloader (separate boot partition) that can boot up either of these images. The bootloader is like the BIOS in a computer and allows you to control some features, and optionally enable a manual selection of which image to load (system or recovery).

The system image is the one that boots by default when you power up your phone normally, and the basic google applications (messaging, browser, contacts, home page launcher, maps, mail client, gallery, etc.) are loaded into partitions alongside the system image in the onboard flash. More on this here.

The recovery image can be accessed by manually telling the bootloader to load it, and basically allows you to do some system tasks, such as execute a complete backup (aka Nandroid Backup), or wipe the device, in addition to a number of other advanced things.

ADB (Android Developer Bridge) and Fastboot are two tools that let you do things to your phone. Fastboot is used for moving images (system and recovery) to/from the phone as well as some basic configuration setting (unlocking the bootloader). ADB allows for more advanced commands and phone interaction with the development SDK.

Fastboot is enabled from the bootloader, and ADB is enabled in the phones settings. Both these tools/commands/modes have client software that runs on a computer connected by USB cable. You can launch an “adb shell” from most third party recovery ROMS.

Fastboot is used to *unlock* the bootloader as show below. ADB can be used to remove stock Google applications; I have used it remove my stock Navigation application so that the BRUT Navigation is default for voice commands. As of this writing, the BRUT mod is the only way I know of to get working navigation in Canada.

The Bootloader/Fastboot and Bootloader/Recovery modes are two alternate ways to start your phone, and can be accessed using the following buttons from a powered-off state:

• Fastboot: Hold down the trackball while pressing the power button.
• Recovery: Hold the volume-down button while pressing the power button.

In each case the volume buttons navigate the menus, and the power button acts as the selection key.

The Steps in the Upgrade Process (You will VOID your WARRANTY)

1. Unlock the Bootloader
2. Root the phone
3. Upgrade the Recovery Image
4. Perform a Nandroid Backup of the Stock System
5. Backup apps in userspace that you want to keep after the upgrade (MyBackup)
6. Wipe the phone using the Recovery Image
7. Upgrade the System Image
8. Repartition the Flash Card for Apps2SD (Optionally move data to a Class 6 chip)
9. Tweak settings, enjoy the extra RAM, USB tethering, improved performance
10. Install applications to your heart is content

My advice, if this is your first Android phone, is to wake up Saturday morning with a free 4-8 hours. it only takes an hour or so to *upgrade* .. but you will spend the rest of the day playing. A the end of this article I also give you a list of applications to experiment with.. YMMV.

The Actual Process

1. Unlock the Bootloader

I learned what I know for this step following instructions here.

• Download and extract the fastboot tool. As far as I know this has never changed versions. It is a very basic tool and there are windows, linux and mac versions in the .zip file.
• Jump into fastboot mode using the procedure described above (power on holding the trackball down)
• Execute the command ‘fastboot-windows oem unlock’ or ‘./fastboot-mac oem unlock’ or ‘./fastboot-linux oem unlock’. As root user (need USB superuser) on my Ubuntu system I executed the linux version: #./fastboot-linux oem unlock

2. Root the Device

This method for root access is called “Superboot” because it requires no modification to the running applications and configuration. You can actually safely enable the “himem” version of this modification which is a major improvement as noted in my last post. I learned this process reading this post. Get the radio version/build right, or you may end up bricking your phone.

• Download the Superboot zip file above and extract to a directory. Make sure you have the correct one that matches your shipped image. Go to “Settings->About Phone” and scroll down to the “Build Number” to figure out what version you have. I downloaded and installed the EPE54B version with himem for my phone.
• Put your device in bootloader mode – Turn off the phone then press and hold the trackball to enter the bootloader
• Open a terminal window to the directory containing the files, and type ‘chmod +x install-superboot-linux.sh’ followed by ‘./install-superboot-linux.sh’

As an aside, here is some info on the different radio builds.

3. Upgrade the Recovery Image

Originally I used fastboot to load my first recovery image. This is simply because I did not know about the Clockwork ROM Manager tool. Now I use that to load images.. its stupid-simple and impossible to mess up, so I will describe this process here.

• Download the Clockwork ROM Manager tool from the Android Market
• Install the Clockwork or RA Recovery Image

The original process I followed is here. Its just a few more steps with the same end goal in mind.

4. Perform a Nandroid Backup of the Stock System

• Boot into recovery: Power-on holding the volume-down button.
• Select Recovery from the menu. (Volume moves the highlighted item, and power button selects).
• Select Nandroid to make a copy of your phone and all images and data on your SD card

5. Backup apps in userspace that you want to keep after the system firmware upgrade

• Download MyBackup from the Market
• Execute a backup of applications you want to keep. No harm in doing them all as you can selectively restore any of them and/or their data

6. Wipe the phone and Upgrade the System Image

Originally I used the RA recovery image to wipe my phone, and loaded the new system image via fastboot off of the SD card. Clockwork ROM Manager does this all for you now.. pay a few bucks to these guys and you are off to the races.

• Install the Cyanogen 5.0.6 of newer via the ClockWork app

The original process I followed is here and used a Recovery Image installed from here.

The official Cyanogen Nexus upgrade docs are here.

Here is the cyanogen mod wiki.

7. Repartition the Flash Card for Apps2SD

I actually copied my SDcard image to a larger new 8GB class 6 chip at this step.. you can skip these optional steps. There is no loss of data if the process executes without error.

If you don’t need to upgrade your flash, you can do this now through the upgraded system tools provided by Cyanogen.

• Pop your flash card into your laptop
• Boot up a GParted live CD
• (Optionally) Copy your FAT32 SD partition to empty space on another disk
• (Optionally) Copy your FAT32 SD partition back onto a nice new Class 6 chip
• Resize your partitions, creating a 1GB ext4 primary partition at the end of the disk. I noted there are always a few empty megs before the FAT32 partition on all chips I have seen, and I always leave these intact
• Pop your flash back in the phone and reboot

Reboot your phone and enjoy. Tweak settings, enjoy the extra RAM, USB tethering, improved performance… Nothing else out there compares to Nexus-One + Cyanogen.. TODAY.

8. Install Applications until your heart is Content

• Brut Navigation
• Advanced Task Killer
• Live Scores
• OpenVPN Settings
• OI File Manager
• Scan2PDF
• SetCPU
• CamCard
• Barcode Scanner
• ConnectBot
• Locale + Plugins
• Fring
• K-9 Mail
• Google Sky Map
• miTorTV
• Transdriod
• SpeedTest
• Shazam
• PicSay
• Remote RDP and VNC
• Klaxon
• Astrid
• dPod
• Astro
• Documents To Go
• QuickOffice
• AndroRadio
• Soo many more…