dinsdag 23 september 2008

Foolishness with NHibernate

Recently I've been working extensively on a web application based on ASP.NET with a C# backend. For data persistence I used Castle ActiveRecord, which is based on NHibernate. I encountered a problem with queries taking too long, or so I thought.

I wrote a bit of code that printed the amount of queries executed on the bottom of each page. I did that by putting a memory appender in my log4net settings.
By doing this, all SQL queries will be logged into the memory appender. At the end of each request I would simply get the amount of entries from the memory appender and clear it, like this:

var app = Array.Find(LogManager.GetRepository().GetAppenders(), a => a is MemoryAppender) as MemoryAppender;
if (app == null) return 0;
var count = app.GetEvents().Length;

Using count as the number of queries, I was able to see how many queries NHibernate executed. To my amazement, it wasn´t all that much (10 queries max, which was acceptable for the situation).

My next step was to profile the database. I used a free SQLProfiler, which worked like a charm. However, none of the queries was causing a lot of problems. I did manage to fix some small performance issues (the mappings weren't written very carefully, so there were a lot of useless joins done. Lazy loading is a blessing!), but it still didn't solve my problem.

I eventually found the problem. I had a root logger configured for log4net, which meant every single logmessage was getting written to an appender. After removing the appender, everything was as fast as it was supposed to. The hardest problems are sometimes so easy to fix that you won't find the solution until you exhausted every other option...

So, if you're planning to use NHibernate in a production environment, be very careful with your logging settings!

woensdag 10 september 2008

Migrating from Subversion to a DVCS part 2: chosing a DVCS flavor

In the process of migrating from one solution a key part is chosing the right solution to migrate to. Why migrate from one solution to another if your end result still isn't satisfying?

First off, I know there are a lot of solutions out there for software revision control, or, as Linus Torvalds put it in his talk at Google, source code management. However, I don't have the time to research every single one of them, so I'll focus on two of the most publicly used ones: git and mercurial.

The first thing that I noticed in my research was that there's a very strong feeling of being on one of the two sides. A bit similar to the Java vs C# camps if you will. Another thing I found is that for basic usage, both solutions are more than adequate. Committing, branching, rollbacks, pushing and pulling, everything is relatively simple for both options.

How to compare?
So, the functionality for basic usage are very similar, which leaves some non-technical requirements to base the choice on:
  • Windows compatible. As I'll be developing on a windows enviroment almost exclusively, it should work well in windows;
  • Small footprint. I want to be able to easily move my local repository to another workspace using an USB stick without having to wait for too long for it to copy;
  • IDE integration. Preferably a TortoiseSVN-esque plugin for windows explorer;
  • Possibility to use with integrated builds and automated testing. Shouldn't be a problem with some scripting I assume;
  • Documentation. I enjoy figuring out things on my own, but not on my third-party tools;
  • Compatibility with other versioning systems (read: subversion). I have the main part of my projects in subversion, and I'd like to use that concurrently, at least for the change-over period;
  • Project activity and coverage. I don't want to invest time in using one system, just to have the developers pull the plug a few months later.
Ok, onwards to the analysis (note: might be slightly subjective)!

The first thing I noticed when looking into Git was the strong *nix orientation of the whole package. The core is written in pure C, which consists of a LOT of low level commands. All high level commands are written as shell or perl scripts, which makes using it on a windows machine hard. I'm not counting running it in Cygwin, as that makes things needlessly complicated. There's a Google Summer of Code project that focused on replacing these by native C scripts, which should make porting to windows a bit easier. So far I've found git being a bit hard to use on windows, while on *nix it's a piece of cake.

I found git to be VERY fast in everything it did, provided you repack your repository routinely. As I tend to forget tedious, repetitive tasks like that, I think that's a downside to git for me. I'll just get fed up with things not going fast enough. The speed when repacked is a definitive upside, as I do intend to use my versioning system extensively. I'd rather commit too much than too few changes.

As far as size is an issue, git has a very small footprint, again, provided that you repack your repository before checking. I wish git would do that automatically. The footprint of the repository is actually smaller than a standard Subversion repository, and also a lot more portable (you can simply clone your repository to another place).

Windows support
Windows support on the whole is marginal at best, which also goes for integration with the windows shell. I'm used to versioning in windows explorer using TortoiseSVN, which makes things a lot easier (especially when you have to resolve conflicts!). The Git equivalent for TortoiseSVN, Cheetah has been on hold since late may 2008, due to the developer getting fed up with people making demands but not actually contributing themselves. Until this is resolved, I don't think windows integration for Git will be something to look forward to, sadly.

According to various sources across the internet, the documentation for Git is not as good as for Mercurial. I have to say that either this has improved greatly over the past six months, or those sources are blatantly incorrect. The documentation for both systems is good overall, with plenty of examples on how to do everything you want with the system.

As far as I'm concerned, Git is well suited for use together with Subversion. Git contains a tool called git-svn, which seamlessly integrates your Git repositories with an existing Subversion repository. With this, you can use Git for your local versioning, while still using your central subversion repository for all the deployment scripts you already had, or for instance when using an online source control provider like SourceForge.

Project activity
Git has several high profile projects in its portfolio, with the linux kernel being arguably the biggest. Seeing how Git is being developed together with the linux kernel (it was written as a free alternative to BitKeeper for use on the kernel repository), I think one can safely say that the project is stable, going forward, and here to stay as long as linux stays.

Mercurial shares some history and goals with Git: it's intended as a free alternative to BitMover's BitKeeper, and was started when BitMover decided to withdraw their free licenses for BitKeeper. Mercurial is written Python, which makes it a bit slower but more portable. That's the first big difference you notice between Mercurial and Git. Git is intended for a *nix audience with marginal windows support, where Mercurial focuses on multi-platform development, which means proper support on all operating systems that can run Python code.

Being written in Python has it's drawbacks: Mercurial is a bit slower than Git on most operations (see bzr, git and hg performance on the linux tree for a comparison). Of course, the differences are marginal, but if you're version control happy like me it could mean minutes on a workday. Minutes spent getting coffee or aimlessly wandering around that is ;-). I don't think it's a really big issue, but Git clearly wins here.

Mercurial repositories are slightly bigger than a Git repository, comparable to one for Subversion. On the other hand, the size of a Mercurial repository is a lot more constant due to not needing to repack routinely.

Windows support
Being written in Python, Mercurial inherently has good multi-platform support. Mercurial consists of a few higher level commands, with multiple option parameters for doing specific things. Like with Subversion and Git, Mercurial has its own GUI tool, called TortoiseHg. Opposed to GitCheetah, this project is actively developed, with the latest release dated at august 8, 2008. It shares a common interface philosophy with TortoiseSVN, so people coming from Subversion should feel right at home.

Mercurial's documentation is top notch. As every popular project nowadays, it has an extensive wiki-like documentation project, which explains everything from the daily usage to the finer details of the system. If that's not enough, there's a book on red-bean, called Distributed Version Control with Mercurial.

Working together with Subversion is not supported by Mercurial... yet. According to this manual page, integration with Subversion IS possible, using a few third-party tools. Too much work for me, I'll wait for the built-in support. Migrating from Subversion to Mercurial using this approach should be perfectly fine though, I don't see a problem in a migration process being a bit complex. After all, it's usually a one-time process (one-time per repository that is!), after which you should be able to continue working on your new VCS.

Project Activity
As with the other versioning systems mentioned, Mercurial has a few big projects running, with the biggest ones being the Mozilla projects (like Firefox), and another being OpenSolaris. That being said, the only thing that could pose a problem for Mercurial's development is other VCS. However, with proper backing from the big projects that are running on it, I don't think it'll go down the drain that fast.

But... Which one to choose!
Having said most things about both options, I'd say they're both pretty even in daily usage. There's a difference in the finer details, which I don't think I can pinpoint without actually using them.
Mercurial has far superior Windows support, while Git is faster and smaller. Git has proper SVN integration, while Mercurial can do it too, but with a few tools. Documentation is adequate for both options, as is the project acceptance and activity. As I implied before, it's more of a 'taste' issue.

I think I'm going to go for Mercurial, with the sole reason being the superior support for non-*nix platforms. I will be doing a lot of development on Windows machines, which means my life will be easier with Mercurial (hopefully).

Migrating from Subversion to a DVCS part 1

Version control. It's absolutely one of the most important parts of a sane development environment. Almost all of my experience with version control is with Subversion (and a bit of Visual SourceSafe, ugh!), but lately I've been thinking about trying out distributed version control. The whole concept of being able to do versioning properly on my local dev-machine is very appealing, and frankly, I'm just interested in learning the inner workings of such a system.

I've set out a plan for migrating one of my current Subversion repositories to a DCVS, probably either Mercurial or Git. To do this, I have a few phases set out:
  • Phase 1: Chosing the appropriate solution. As far as I know, the main options are Mercurial (used by the Mozilla Foundation, among others) and Git (used by the linux kernel development team). I'll have to figure out the pros and cons for these systems and eventually make a decision on which solution to use.

  • Phase 2: Creating a new repository in the DVCS. Basically getting started with a test project in the DVCS I chose in phase 1.

  • Phase 3: Find and use a manual/tutorials on how to migrate my Subversion repository to the DVCS, preferably while keeping the logs I had.

  • Phase 4: Set up hooks for different events, if possible. Continuous integration/nightly builds is something I really want to do for a change.

  • Phase 5: ???

  • Phase 6: Profit!
I'm planning to at least have something running before september, provided that my spare time allows for such a time investment. I sure hope this works out, I hate to invest the time for something I'm not happy with.

zondag 31 augustus 2008

Which tools to use: PHP/MySQL

For my work I have to work in a few different scenes so to speak. On one hand we have the PHP junkies, on the other hand there's the .NET evangelists. I'm sort of caught in the middle, with a unique vantage point on both sides. I can't say it's pretty all of the time.

After having worked with C# and Microsoft SQL Server 2000/2005 exclusively for a few years, I can honestly say I've gotten spoiled with all sorts of cool productivity tools. Snazzy IDEs, database profilers, query browsers, you probably know the drill. How horrified I was when I saw a co-worker use his souped up notepad clone and phpMyAdmin for all his development! Debugging by putting echo and var_dump statements here and there, refreshing the page and trying again.
I couldn't imagine this was the only way to go with PHP, so I started to search for some Visual Studio/SQL management studio replacements. This is what I came up with so far, still looking for a proper mySQL profiler:

  • Eclipse 3.3 (Europa) : Modular IDE which grew from a mainly Java environment to a multilanguage extendible IDE framework. Apparently it has support for C#. If only I wasn't so stubborn in my almost symbiotic attachment to Visual Studio, I might even switch over completely.
  • WampServer : A Windows-Apache-Mysql-PHP all in one installer. Comes with an easy to use system tray utility for doing mundane tasks like restarting your services, adding Apache aliases and toggling PHP and Apache extensions.
  • MySQL's own tools : Contains an admin tool for user management, backup tools, logging etc.
  • XDebug for PHP 5.2.6 : A debugger plugin for PHP, which works well together with Eclipse. With this plugin you can get visual studio-like debugging going in Eclipse, which gets rid of the ugly 'echo and see if it gets there' debugging. It also offers code-coverage and profiling support, but I haven't had a chance to try that out yet.
For installing Eclipse 3.3 with XDebug support, check out this article on Rob's Notebook.

PS: At the time of writing there is a new version of Eclipse available (3.4, Ganymede). However, the PHP development toolkit doesn't quite work yet on that version.

For logging your queries, try the MySQL Log Monitor application. It's kinda crude, but it comes with source code attached. Customize to your heart's content!

woensdag 6 februari 2008

First post!

Well, here we go! Time to kick off my 'professional' blog. I will be posting my experiences with various programming languages here, although it will mostly be about Microsoft's .NET framework.

A little bit about me? Sure!
I've been developing on different levels of complexity in C, C++, Java (1.3 - 1.5) and C# (1.1 - 3.5) for a few years now. I've developed a strong preference towards web based frameworks in the past few years, after working for a company that develops online management tools.
This company (we'll just call it Company X) has a nice package deal on things like CRM, CMS and stock management, in a web based environment. The software started out small, and grew to, dare I say, enormous proportions. Needless to say, it's a great source of all kinds of anti-patterns.
Working with this bunch of spaghetti sparked my interest in proper design, on ALL levels. Nowadays, I want my (x)html output to be clean and validated, my CSS has to be structured, and last but not least, my C# code has to follow strict standards.

Anyway, enough about me, I hope I can find the time to blog about some interesting developments in the .NET framework soon. I'm kinda waiting for the .NET 3.5 extensions RTM (I want my MVC controls!).