A farewell to the old, and a hello to the new

It’s late August, which means it’s currently late summer, and soon fall, where I live. It’s time to decide on a winter project. Previously, I’ve had coding sprees on various things related to the MMORPG Anarchy Online. Some examples, in no particular order: AOItems (an items database), Towerwar Tracker (mobile) (shows Player versus Player land control fights), PlanetMap Viewer, and more. Although I have to admit some of those stretched into year-around and even multi-year projects. :) In idle moments, I’ve taken stabs at improving things in the FreeBSD land as best as I can, by writing some guides and giving a helping hand on various social media, forums and IRC.

Anarchy Online has been the main focus point of many of my IT-related hobby projects. And much of my home infrastructure, powered by FreeBSD, is the way it is now because it had to be that way to support and run those projects. I started playing the game in December 2004, and stopped playing it early 2009. I have played it a week here or there when large patches have hit, but for the most part, just logged in to check if my AO-related tools were still working, or in some other way related to improving those tools. It’s now August 2015, and it’s six years since I stopped playing that game. I’ve been making tools for the game for more than ten years, and it has been fun. It has been an incredible learning experience. I’ve done many crazy things, such as:

  • Making a multi-process map compiler using PHP and syncing state on disk (4 processes was about 50% faster than one)
  • Porting said compiler to C#.Net, heavily optimized with threads. About 40x faster than single-process PHP version :)
  • Creating a LARGE toolchain (20 libraries/applications) for extracting item info from the game client and making sense out of it, storing it in a DB and displaying it on a website using PHP. Toolchain parses through 14 years of patch history in about 15 minutes. (similar tools were said to be taking weeks to do the same). This toolchain also verifies integrity, and automatically creates reports on human-induced mistakes in the data, for easy and detailed report submission to the games developers.
  • For the planetmap viewer, using a hooking library to run my own code in the context of the game client, effectively using the game client as an API.
  • Much more :)

If it wasn’t for this game, its awesome community, and all the related projects I’ve worked on, I wouldn’t be where I am today. I wouldn’t have known C#.NET as well as I do, if at all. There are many friends I would never have made otherwise, all over the world. If you ever read this, you should know who you are. :)

All that being said, 10 years is a long time. It’s a third of my life on this planet. It was fun, it was quite the experience. It really was. But now it’s time to change focus. I will maintain the existing projects for the foreseeable future, until a suitable successor can be found. No new features are likely to arrive. All the tools except those related to AOItems.com are open source, so anyone who’s up to the task can fork and improve them. I’ll keep AOItems updated for the foreseeable future. If I ever stop maintaining it, I will make the tools available to the community, so that others can pick up the torch where I left it. People can do amazing things if you let them. :)

To the whole Anarchy Online community, who know me as “Demoder”: Thank you for being truly awesome, and inspiring me to do ‘crazy’ things. I’ll try not to be a stranger!

The New

I plan on learning to play the (musical) keyboard. Exercise more. Be more out-going. Maybe quit a bad habit or two. All of those things that people write on their blogs, and sometimes follow up on. And often don’t. But there’s more.

I was introduced to the world of FreeBSD and Linux back in 2001 or so. The story is long and for another blog post, but the point is this: FreeBSD has been with me in varying degrees for nearly 15 years, or half my life, and I’m now at a point in life where I feel like contributing more than I have in the past. I started earlier this year, submitting a PR and a patch for adding LibXo support to iscsictl(8), and then proceeding with a thorough technical review of the book “FreeBSD Mastery: ZFS” by Michael W. Lucas and Allan Jude. And I feel these are the kinds of things I want to do with the large chunk of my spare time which is labeled “geeky things”. Make contributions to something, hopefully making a positive change for other people in the process.

I’ve been using jails for a long time. For the Linux folks out there, think containers. For the Solaris people out there, think zones. I love how easy it is to manage large amounts of data with ZFS, and how trouble-free it is to share this data with the jail environments, with (nearly) no overhead.

The past couple of years, some awesome people have been implementing a native hypervisor to FreeBSD called “Bhyve”. I love it. I really do. It lets me do things that jails wouldn’t. It will eventually let me retire my VMWare ESXi server. There are a few things which are annoying with it though, but most of those are being worked on by very skilled people. The one feature I miss the most,  is an easy and cheap way to leverage ZFS with no performance penalty. Think of jails with nullfs, or having a ZFS dataset delegated to a jail. That’s super awesome.

My primary use case would be to set up a virtualized file server, leveraging ZFS on the host without ZFS on ZFS or similar overhead, and avoiding the growing complexity of things like NFS configurations. As such, my new winter project is FreeBSD-related. It involves Bhyve (hypervisor), file systems, host/vm communication, and simplifying administration. It involves learning C properly, virtio, FUSE, and FreeBSD kernel internals. Its name is Tunnel File System (https://tunnelfs.io/). The goal is to share files between host and guest like you do between host and jails using nullfs. It aims to be simple to configure/use by the system administrator. It aims to be predictable. It aims to make your life simpler. You can read about it in more detail on the project’s site.

I have a pretty good idea (or so i think right now!) on how to implement it. I know what I need to learn. I know I have a LOT to learn. But that’s okay. I LOVE learning. Almost anything fun in life involves learning something new. So I’ll get started on that. And I’ll probably find out that I can’t do these things the way I wanted, but that’s okay too. It’s a learning experience, and I will get to the finish line eventually. :)

PS: If you haven’t read FreeBSD Mastery: ZFS, and use or plan to use ZFS, you should go read it. Even if you’re not using FreeBSD. If you don’t own it, you should buy it. It’s awesome, and it will look good on your book shelf.

Anarchy Online server merge and updates to tools & stuff

Anarchy Online merged two live and one previously discontinued dimension yesterday, February 26th, into one new live dimension called Rubi-Ka. More information here, here, here and here. The announced downtime was 24 hours, but the servers came back up after about 11 hours – around 23:00 local time.

Continue reading

Status Update

It has been a very busy couple of months, especially with getting a new puppy last month.

I’m in the process of restructuring my personal infrastructure! In extremely broad terms, this will simplify and clarify each servers role, hopefully making it easier and more predictable to maintain the infrastructure. I’ll write a in-depth post on this at a later date!

I’ve also been working on a number of projects related to the game Anarchy Online. Probably most notable is the launch of Demoder’s PlanetMap Viewer v1.2, adding many new features which are described in the thread. Work have begun on sanitizing the Itemsbot and CIDB tool chain by making an independent resource database parser which handles all items, nanos, icons and a few other data types. I’ll go into detail on this later. Vha.Net and Vha.Chat is being upgraded from .NET 2.0 to .NET 4.0 (client profile), and is receiving a new packet system in the process. I’m currently on the planning stage for porting Helpbot to Bot#; No ETA yet.

Photography status: Slacking! I’ve taken some less-than-stellar photographs, and have not bothered uploading these; I am probably going for a photo spree this weekend though; Hopefully I’ll be able to take some which are upload-worthy. :)

My game time is pretty evenly split between Civilization 5 and The Secret World – an amazing modern-time MMO RPG. If you didn’t check it out yet, you should – Ragnar Tørnquist, the man behind The Longest Journey, is the main guy behind the game.  The story easily sucks you in like a good book which you just can’t put down till you’ve finished reading. Quests are very original and fun – I end up doing quests not for their XP or loot, but for the story and direct entertainment value.

Coding core features

I’ve somehow slightly started on the scheduled coding spree, half a month early. I’m working on the core aspects of the code; The parts which makes a lot of non-connected events and data come together into a nice, clean interface for plugin developpers to interact with. The better job I do with the core aspect of the bot, the easier (and more enjoyable) it will be for people to extend its feature set.

I kind of figure that I shouldn’t really program any ‘directly usable features’ until all the core features are in place; Much because if I do implement a lot of features at this stage, any change to the core of the bot will involve lots of code change to the features which were prematurely implemented.

Todays most significant change is that plugins are now able to refer .Net and external libraries not referenced by default. To take a simplified example: The bot does not reference a MySQL library by default. This means no plugin is able to connect to MySQL databases without using unsafe code, which is not permitted for plugins.
With todays change, a plugin developer can attach the required MySQL libraries, then tell the plugin compiler to reference them when it does its thing. The plugin can then access MySQL databases.

Simplicity vs Features

It always starts with a simple idea. “I need a chatbot for Anarchy Online!”. The following days will involve a lot of thought, and hopefully, realization that it’s a project much bigger than first anticipated. This is what happened the first time I wrote a bot from scratch for AO (written in PHP, named ‘FLWBot’, using the BeBot-updated AOChat library developped by Auno). And I knew it would happen when I started on Bot#, too.

Continue reading