Savagedlight's Road

Anarchy Online merged two live and one previously discontinued dimension yesterday, February 26th, into one new live dimension called Rubi-Ka. More information here, here, here and here. The announced downtime was 24 hours, but the servers came back up after about 11 hours – around 23:00 local time.

This is roughly what happened next:

• I patched my local client
• I started the VM which I use for RDB patch history and patched that client, then archived the data accordingly
• Logged into several bot accounts to add them to the merge queue
• Logged into my main account to add it to the merge queue
• Updated my private items database with the latest patch. Went smoothly!
• Updated aodevs.com icon/texture repository with 18.6 content
• Logged into roughly 15 essential bot characters
• Updated Helpbot to the latest version of VhaBot (Thank you Remco!), made necessary configuration changes, and launched it
• Tower Tracking
  • Spent 15 minutes finding the president of my tower tracking org, to be able to promote the bot to a rank where it can see all tower-related messages
  • Configured and started the tower tracking bot
• Committed changes to Bot# to make it compatible with the new live dimension
• Updated Itemsbot to the latest development version of Bot# (that’s a full year in one sweep)

And that’s where it began going wrong. Everything seemed to be in order, but the bot would forget changes to per-plugin settings upon next restart. I had to start the bot with the Visual Studio debugger to track down the issue. I didn’t get any exceptions, nor any log output indicating anything was wrong. I began tracking code as it executed, line for line. After much longer time than it should have been, it hit me:

private string GetString(Type plugin, string key)
{
	var pc = plugin.GetPluginAttribute();
	string storeKey = pc.Name;
	storeKey += ":" + key;

	StringCacheEntry sce;
	if (this.configCache.TryGetValue(storeKey, out sce))
	{
		if (sce.Age.TotalSeconds < 60)
		{
			return sce.Value;
		}
	}

	StringConfigEntry confEntry = StringConfigEntry.TryFind(storeKey);
	if (confEntry == null)
	{
		throw new Exception("Key does not exist", new Exception(String.Format("plugin: {0}\r\nkey:{1}", plugin.FullName, key)));
	}
	this.SetString(plugin, key, sce.Value, true);
	return sce.Value;
}

Doh. It’s a result of bad re-factoring.  At some point in the past dozen revisions, I had made changes to the first part of this method and forgot to update the rest of it.
This code will try to populate the variable named ‘sce’ with cache information. If that succeeds *and* the information is less than 60 seconds old, it will return the cached value. Otherwise, it will fetch information from the database, and update the cache.

However, this last step is where it goes wrong. It’s trying to populate the cache, and return a value, using the ‘sce’ variable. This means it will return an outdated value (best case), or throw a NullException because the ‘sce’ variable wasn’t set at all. Either way, that’s bad. Very bad.
Additionally, the Bot# configuration UI use the TryGetString() method, which is a try/catch wrapper to GetString(), which returns false (“nope, couldn’t retrieve that setting!”) upon any thrown exception; without logging.
I fixed these problems and added logging to TryGetString(), recompiled, updated Itemsbot again: Works like a charm!

Then, I continued on with updating the Central Items DataBase to patch 18.6.0 and fixing Demoder’s PlanetMap Viewer to be compatible with 18.6.0 (yet to be tested & released) before calling it a night around 02:00 in the morning.