Friday, July 10, 2009 
Humongous JIT memory leak

I mentioned three weeks ago that I had just repaved my work dev box and installed the 64-bit version of the Windows 7 RC. Nine or ten years after I first ported parts of IIS to Win64, I am finally running my main desktop on 64-bit Windows. With one exception, it's been painless. Programs have just worked, devices have just worked. There are relatively few native x64 applications, but for the most part it doesn't matter. The cases where it does matter—e.g., shell extensions such as TortoiseSVN—are available as 64-bit binaries.

I briefly flirted with using the 64-bit build of Python, but realized that I would have to recompile several eggs as 64-bit binaries. That was too painful and the 32-bit binary did everything I needed.

Building in Visual Studio 2005 is noticeably faster. I'm not sure how much of it was due to accumulated cruft after 18 months on Vista, but builds there were very slow.

The one exception was a major problem for the first week and a half. Whenever I ran our ASP.NET web application, it would go berserk, eat up all 4GB of my physical RAM, push the working set of IIS's w3wp.exe to 12GB, and max out one of my 4 cores! The only way to maintain any sanity was to run iisreset every 20 minutes to gently kill the process.

WinDbg and Process Explorer showed that the rogue thread was stuck in a loop in mscorjit!LifetimesListInteriorBlocksHelperIterative. I passed a minidump on to my former colleagues in IIS, who sent it to the CLR team. They said:

The only thing I can tell is that it is Regex, and some regex expression compiled down to a method with 456KB of IL. That is huge, and yes 12GB of RAM consumed for something like that is expected.

With that clue, I was able to track down the problem, a particularly foul regex, built from a 10KB string, with 32 alternating expressions, each of which contains dozens of alternated subexpressions. The string is built from many smaller strings, so it's not obvious in the source just how ugly it is. I commented out the new Regex() and my problems went away.

Regardless of how ugly the regex is, this is a major regression in the CLR. This code has been working without blatant problems for two years on the 32-bit flavors of XP, Server 2003, Vista, and Server 2008. I've been meaning to try this code on 32-bit Windows 7, but have been too busy.

(The original, long-gone author was apparently aware that the regex is expensive to create because he runs a background thread to new the regex, which should have told him something. We'll fix the code that uses the regex to do something saner, soon.)

All that aside, I've been happy with the 64-bit version of Windows 7.

posted on Saturday, July 11, 2009 6:12:02 AM (Pacific Daylight Time, UTC-07:00) 
#    Comments [0]
Friday, June 19, 2009 
Blown Capacitors

I complained a week ago about my display driver going berserk. I blamed Windows Update, since it happened within hours of a pile of updates being installed. I upgraded to the latest beta NVidia drivers on Monday and it helped for a while, but by Wednesday, it was almost as bad again as it had been last Friday. It was infuriating and I was both entertaining and alarming my neighbors with my cursing.

Today was the last day of a very busy sprint for me and at last I had the time to dig into it. I opened up the case and took a look at both video cards—I have two dual-head cards connected to three monitors—and one of them had partially blown capacitors like those in the picture. I removed the bad card and did some graphics-intensive things for an hour, and the other card behaved flawlessly.

Oddly, until someone mentioned that it might be a hardware problem yesterday, it didn't occur to me, even though a video card blew in this machine last year. I came in one morning to find a black monitor, and when I pulled out that card, I found that some of the capacitors had popped right open with stuffing protruding.

On general principles, I had been meaning to repave this machine for a while. I've had it since December 2007 and it was still running the original installation of Vista. I booted from a DVD, reformatted my C: drive, and installed Windows 7 x64 RC1.

I finally have a 64-bit OS as my primary Windows desktop, so I'll actually be using the Win64 build of Vim that I maintain. My first impressions of Windows 7 on this machine are very favorable, but there's plenty more that I need to install before the machine has everything that I need.

posted on Saturday, June 20, 2009 4:40:37 AM (Pacific Daylight Time, UTC-07:00) 
#    Comments [0]
Thursday, May 07, 2009 
Windows 7 x64 running in Mac VirtualBox 2.2.2

I ported Vim to Win64 but I don't have a convenient Win64 system to test it on.

I decided to install the Win64 build of the Windows 7 RC on VirtualBox, which has supported 64-bit guest operating systems since version 2.0.

It worked without problems on my MacBook Pro. I used VirtualBox's Virtual Media Manager to mount the Windows 7 ISO and installed from that. See also the handy guide. (Why does Windows 7 offer a choice of upgrading from a previous version of Windows on a virgin disk?) After completing the installation of the operating system, I installed the Guest Additions for mouse pointer integration and other goodies.

As always with VirtualBox VMs on my MacBook, I had to fix the Network settings to work over WiFi. When the VM is turned off, go to Settings, choose the Network tab. Change “Attached to” from “NAT” to “Bridged Adapter” and “Name” from “en0: Ethernet” to “en1: AirPort”. Tip: to get a right-click without a mouse, place two fingers on the trackpad and click the trackpad button, or Shift+F10.

I tried installing the Win64 build of Win 7 on my Win32 Vista desktop box at work. The host system bluescreened within seconds of starting the installer! I filed ticket 3963.

I had inadvertently installed the Win32 build first on my work system. That worked fine. It also seemed to have snappy disk I/O. When I unzipped the Win64 Vim binaries (not having realized yet that I had the Win32 Win 7), it was slower than unzipping in the host operating system, but not unreasonably so. On my MacBook, the details pane from the Win 7 zip extractor said that it was running at a mere 260KB per second, which is pitiful. It certainly wasn't that slow when installing the OS onto the virtual disk.

posted on Thursday, May 07, 2009 7:01:14 AM (Pacific Daylight Time, UTC-07:00) 
#    Comments [0]