Smart backup for your GMail account

May 28th, 2011 4 comments

There are a number of solutions out there that allow you to back up your Google mail, but all of them have some significant shortcomings. That’s pretty surprising given the popularity of GMail. BaGoMa aims to address all (or most) of them.

  1. BaGoMa is a command line application, so you can easily automate the backup. A GUI might be added in the future if there’s enough of a demand for it.
  2. It runs on all major operating systems: Linux, Windows, Mac OS X, basically anything that Python runs on. For Windows users the zip file even contains a stand-alone executable that has the Python interpreter and libraries built-in, so you don’t need to install anything else.
  3. It is open sourced, and contributions from the community are welcomed and encouraged. That also means you can read the source code  yourself to make sure your password and emails remain private.
  4. It has no arbitrary limitations on the tag/label names you can use in Google. Some other solutions out there expect you to only use ASCII. That means non-English users are out of luck. It even auto-detects the main types of folders in use so that it can skip folders such as “Spam” and “Trash”.
  5. Most importantly, it is tuned to work specifically with Google mail. There are a lot of generic applications that are able to backup an IMAP account. GMail however is not a true IMAP account. For example, if you apply 5 tags/labels to an email message, it shows up in 5 different IMAP folders. From the perspective of a regular IMAP application those look like 5 distinct messages, so they get backed up 5 times. It’s not clear what would happen when you attempt to restore your mail from such a backup. Will you end up with 5 identical copies of the message, each with a single tag/label, or will you end up with a single message that has 5 tags? BaGoMa does the right thing. In this case that means the message only gets downloaded (and backed up) once, and when restored all the original tags are applied to the same message.

The use of BaGoMa is extremely simple. To back up your account all you need is:

bagoma --email=your_user_id@gmail.com

It works seamlessly with Google Apps for Business, so if you’re hosting your business email with Google you can use your business email address instead of YourUserId@gmail.com to back up your business account. Please note that if you don’t specify a backup directory, BaGoMa will use a directory that matches your email address. Email addresses are case insensitive, but on operating systems with case-sensitive file systems you need to be consistent in how you write your email address, or specify the backup directory.

To restore your email you would run:

bagoma --email=your_user_id@gmail.com --action=restore

So what are you waiting for? Give it a try and stop worrying about forgetting your password and getting locked out of your account, or losing important emails if Google ever has an E-Mail outage.

BaGoMa homepage: http://sourceforge.net/p/bagoma/

BaGoMa downloads: http://sourceforge.net/projects/bagoma/files/

BaGoMa documentation: http://bagoma.sourceforge.net/

Categories: Uncategorized Tags: , , , ,

The Android IPC system

January 3rd, 2011 5 comments

The information below comes from a number of sources, including my own experiments with the Android IPC and some disparate internet sources.

The overall architecture of the Android IPC system is shown in the diagram below. It consists of four major blocks; one in kernel space, and the other three in use space. The dashed lines represent the logical RPC calls. The solid lines represent the actual data flow.

  • BinderDriver: This is the core of IPC system. It passes data between a ServiceProvider(s) and a ServiceUser(s). This kernel component is provided by Android.
  • ServiceProvider: Provides some kind of service. It parses the received RPC data from the BinderDriver and does the real work. Application developers will either make use of existing service providers (such as the Camera or AudioFlinger), or in some cases will write their own.
  • ServiceManager: This is a special singleton ServiceProvider that provides service manager services for other service providers. This component is provided by Android.
  • ServiceUser: This is the client. It remote calls the ServiceProvider by generating an RPC and sending it to the BinderDriver. Application developers typically write their own ServiceUser as part of their application.

Here is a typical flow of events for a fictitious MultServiceProvider (a service provider that multiplies two numbers for a client) and a MultServiceUser client which doesn’t know how to do multiplication (maybe because the numbers are quaternions) and needs to use the MultServiceProvider:

  1. ServiceManager runs first (at power-up) and registers a special node (node O) with the BinderDriver.
  2. The MultServiceProvider gets an IServiceManager proxy object for the special node O by calling the global “defaultServiceManager()” function.
  3. The MultServiceProvider then calls defaultServiceManager()->addService(“Multiplier”, new MultServiceProvider()) to add itself as a service provider and then waits in an infinite loop for someone to request its services. The addService RPC call is routed to the ServiceManager through the BinderDriver.
  4. The BinderDriver notices that the RPC is for the ServiceManager to add a new service, so besides routing the RPC to the ServiceManager it generates another node (let’s call it node M), for the new MultServiceProvider.
  5. The ServiceManager reads the data from the BinderDriver and processes the IServiceManager::addService RPC call.
  6. The MultServiceUser client process gets an IServiceManager proxy object for the special node O (again by using defaultServiceManager()).
  7. The client does an IServiceManager::getService(“Multiplier”) RPC call to get the MultServiceProvider. This call is routed to the ServiceManager through the BinderDriver.
  8. The ServiceManager reads the RPC data from the BinderDriver, processes the IServiceManager::getService request and returns back the node representing the MultServiceProvider.
  9. MultServiceUser calls MultServiceProvider::multiply(a, b). This call is routed to  the MultServiceProvider by the BinderDriver.
  10. The MultServiceProvider handles the MultServiceProvider::multiply RPC call and sends the product of the 2 numbers in a reply to the BinderDriver.
  11. The BinderDriver routes the reply back to the client.
  12. The client reads the data from the BinderDriver which contains the result of “a * b”.

In a future post I hope to discuss the whole architecture in more detail, with concrete code examples for how to use IBinder, IInterface, BBinder, BpInterface, BnInterface, etc… to create a ServiceProvider and a ServiceUser all in native C++ code on Android.

Categories: Uncategorized Tags:

Windows proxy control script

December 2nd, 2010 No comments

Here’s a simple script that lets you toggle on or off your internet proxy in Windows. Save it to a file called Proxy.vbs on your desktop and double-click the file to execute the script.

As shown below, the script will not change the proxy server address, but will simply turn it ON or OFF. If you want to also modify the proxy server and exception list every time you turn it on, update lines 37 and 38, and uncomment lines 40-41.

Parts of this script come from a comment someone posted on a web site, but I no longer have the link to provide proper attribution.

Const HKCU=&H80000001 'HKEY_CURRENT_USER
Const HKLM=&H80000002 'HKEY_LOCAL_MACHINE

Const REG_SZ		= 1
Const REG_EXPAND_SZ	= 2
Const REG_BINARY	= 3
Const REG_DWORD		= 4
Const REG_MULTI_SZ	= 7
Const HKCU_IE_PROXY	= "Software\Microsoft\Windows\CurrentVersion\Internet Settings"

Set oReg=GetObject("winmgmts:!root/default:StdRegProv")

Main

Sub Main()
	buttons = vbQuestion + vbYesNoCancel + vbDefaultButton1

	If GetValue(HKCU,HKCU_IE_PROXY,"ProxyEnable",REG_DWORD) = 1 AND _
	Len(GetValue(HKCU,HKCU_IE_PROXY,"ProxyServer",REG_SZ)) > 0 Then
		' If Proxy is set then default to turning it off
		buttons = buttons + vbDefaultButton2
	End If

	choice = MsgBox("Enable the internet proxy?", buttons, "Proxy setting")
	If choice = vbYes Then
		SetProxy True
	ElseIf choice = vbNo Then
		SetProxy False
	End If
End Sub

Sub SetProxy(enabled)
	If enabled = False Then
		CreateValue HKCU,HKCU_IE_PROXY,"ProxyEnable",0,REG_DWORD
		MsgBox "Proxy Disabled", vbInformation, "Proxy OFF"
	Else
		strProxyServer = "MyProxySvr:80"
		strProxyOveride = "*.domain.com;*.domain2.com;*domain3.com"
	
		'CreateValue HKCU,HKCU_IE_PROXY,"ProxyServer",strProxyServer,REG_SZ
		'CreateValue HKCU,HKCU_IE_PROXY,"ProxyOverride",strProxyOveride,REG_SZ
		CreateValue HKCU,HKCU_IE_PROXY,"ProxyEnable",1,REG_DWORD
		MsgBox "Proxy Enabled and set to:" & vbcrlf & "(" & strProxyServer & ")", vbInformation, "Proxy ON"
	End If
End Sub

Function CreateValue(Key,SubKey,ValueName,Value,KeyType)
	Select Case KeyType
	Case REG_SZ
		CreateValue = oReg.SetStringValue(Key,SubKey,ValueName,Value)
	Case REG_EXPAND_SZ
		CreateValue = oReg.SetExpandedStringValue(Key,SubKey,ValueName,Value)
	Case REG_BINARY
		CreateValue = oReg.SetBinaryValue(Key,SubKey,ValueName,Value)
	Case REG_DWORD
		CreateValue = oReg.SetDWORDValue(Key,SubKey,ValueName,Value)
	Case REG_MULTI_SZ
		CreateValue = oReg.SetMultiStringValue(Key,SubKey,ValueName,Value)
	End Select
End Function

Function DeleteValue(Key, SubKey, ValueName)
	DeleteValue = oReg.DeleteValue(Key,SubKey,ValueName)
End Function

Function GetValue(Key, SubKey, ValueName, KeyType)
	Dim Ret

	Select Case KeyType
	Case REG_SZ
		oReg.GetStringValue Key, SubKey, ValueName, Value
		Ret = Value
	Case REG_EXPAND_SZ
		oReg.GetExpandedStringValue Key, SubKey, ValueName, Value
		Ret = Value
	Case REG_BINARY
		oReg.GetBinaryValue Key, SubKey, ValueName, Value
		Ret = Value
	Case REG_DWORD
		oReg.GetDWORDValue Key, SubKey, ValueName, Value
		Ret = Value
	Case REG_MULTI_SZ
		oReg.GetMultiStringValue Key, SubKey, ValueName, Value
		Ret = Value
	End Select

	GetValue = Ret
End Function

Categories: Uncategorized Tags: ,

CygWin terminal colors

November 14th, 2010 No comments

Many of the default ANSI colors used by the CygWin terminal are almost unreadable. For example, the blue that is used for directories in a file listing is pretty dark and hard to read on a black background. This limits the color palette that can be used in things such as the PS1 shell prompt. Try the following script (colortable16.sh) to see what your colors currently look like.

The colors can be changed by clicking on the system menu in the upper-left corner and selecting “Properties”. I decided to use the same colors as the Gnome terminal.

This is how the colors are defined in CygWin by default:

Color R G B
Black 0 0 0
Blue 0 0 128
Green 0 128 0
Cyan 0 128 128
Red 128 0 0
Pruple 128 0 128
Brown 128 128 0
Light Gray 192 192 192
Dark Gray 128 128 128
Light Blue 0 0 255
Light Green 0 255 0
Light Cyan 0 255 255
Light Red 255 0 0
Light Purple 255 0 255
Light Yellow 255 255 0
White 255 255 255

This is how they should be changed so that they’re more usable:

Color R G B
Black 0 0 0
Blue 0 0 170
Green 0 170 0
Cyan 0 170 170
Red 170 0 0
Pruple 170 0 170
Brown 170 85 0
Light Gray 170 170 170
Dark Gray 85 85 85
Light Blue 85 85 255
Light Green 85 255 85
Light Cyan 85 255 255
Light Red 255 85 85
Light Purple 255 85 255
Light Yellow 255 255 85
White 255 255 255

Categories: Uncategorized Tags: , ,

X11 remote display

October 31st, 2010 No comments

There’s no shortage of tutorials on how to use X11′s remote display facilities. On modern, properly configured systems, all you need to do is to use the -X or -Y option to ssh, and the magic is all taken care of by ssh and xauth. Unfortunately, sometimes servers are mis-configured and this simple solution doesn’t work.

On one server I was trying to use, sshd was compiled with a hard-coded path for xauth that was incorrect. This is easy to see when adding the “-v” switch to ssh. Look for:

debug1: Requesting X11 forwarding with authentication spoofing.
debug1: Remote: No xauth program; cannot forward with spoofing.

With no admin rights on the server, the normal work-around is to use the XAuthLocation option on the client side, either in the ~/.ssh/config file, or on the command line: ssh -o XAuthLocation=/proper/path. Some versions of sshd (including the one I was using) ignore this option.

Since I was working in a fairly secure environment, I decided to skip ssh/xauth part, and just set the DISPLAY environment variable on the server to “DISPLAY=my.client.ip:0.0″. That’s all nice and dandy, but my Ubuntu client was using Unix domain sockets instead of TCP for X11 so there was no way to connect to it remotely. First I tried modifying /etc/X11/xinit/xserverrc on the client to remove the “-nolisten tcp” option. That didn’t seem to do the trick (there was nobody listening on port 6000 after I restarted X). It turns out gdm has a different configuration file that also needs to be modified. I then changed /etc/gdm/gdm.schemas to:

     <schema>
       <key>security/DisallowTCP</key>
       <signature>b</signature>
      <default>false</default>
     </schema>

After X was restarted, I had X11 listening on port 6000. All that was left to do was to allow the server to connect (xhost +server.name.com) and everything was working like a charm.

Before you open up X11 to remote TCP connections as shown above, make sure you read up on it and understand the security implications.

Categories: Uncategorized Tags: ,