How to be a programmer

I saw this on secretGreek today:

   intellisense
        ||
        \/
       code >>> compile >>>>  run  >>> success ;-)
        /\         ||          ||         
        ^^         \/          \/
        ^^      errors       errors 
        ^^          \\       //
        ^^           \\     //
        ^^             google
        ^^              ||
        \\              \/
         \<<<<<<<  cut N paste

I have days where this is my programming model.

Tech Tags: secretGeek How+to+be+a+programmer cut+n+paste intellisense Google

Where is "Step Into" on my Visual Studio 2005 Debug menu?

I was debugging some code in VS 2005 when I noticed that I could no longer find "Step Into" on the debug menu.  I was already used to the keyboard short cut of F11, but I was wondering why it was no longer on the menu.  After some googling, I found a post from Paul Litwin's blog that explained how to fix it.  Basicly, the IDE thinks that you just don't want to see that menu item and you just need to reset your profile.   Here are the steps that Paul listed on his blog:

1. Click the Tools->Import and Export Settings menu item
2. Choose Reset all Settings
3. Don’t save your settings
4. Pick a new profile
5. Profit!

OK, I couldn't resist the last one.  The $64,000 question is why this happened in the first place.  When Paul posted his blog entry, VS 2005 was still new and shiny and the thought was that a prior installation of VS 2005 (beta?) had some how screwed up the settings.  That didn't match my setup, I avoided the VS 2005 beta like the plague.  I do have VS 2003 installed, that may have confused VS 2005.  I also installed SQL Server 2005 before VS 2005, that too could confuse VS 2005.

Tech Tags: Step+Into Visual+Studio+2005 VS+2005 Underpants+Gnomes

Surfing with Takeshi Terauchi and the Bunnys

While skimming Boing Boing, I came across a link to some MP3s by Takeshi Terauchi. These files come courtesy of WFMU's Beware of the Blog. Takeshi Terauchi plays surf guitar music the old schol way, these MP3s were from recordings that are four decades old. They sound the Ventures with backed by a traditional Chinese opera (in a good way). WFMU has two complete albums, This is Terauchi Bushi, and Let's Go Classics.

Let's Go Classics is a surf guitar interpretation of traditional classical pieces like Blue Daube Waltz and the theme from Swan Lake. Very different.

Takeshi Terauchi started out as a sideman in Country/Western band, Jimmy Tokita & the Mountain Playboys. In 1962, Terauchi formed his first Eliki (Japanese slang for Electric) band, the Bluejeans, becoming Japan's first Surf band. In '66, Terauchi quit the Bluejeans and formed the Bunnys, the group behind the two albums on WFMU.

Tech Tags: Takeshi Terauchi Eliki Surf Guitar Boing Boing WFMU Beware Of The Blog Ventures

THIS IS YOUR REPLACEMENT TAG

I'm on my third E-ZPass tag.  The battery in the first one died after about 5 years.  I have had the second one for another 5 years.  This time, the E-ZPass people sent me a new one before the current one died.

They did it the right way.  The new tag came in a small envelope with a letter and a return envelope.  The return envelope was for the old tag so it could be sent back.  It was a pre-paid envelope, a nice touch.  The letter explained that the new tag had to be sent back, or I would be charged for it's replacement cost. 

They clearly labeled the serial number of the old tag and of the new one.  I have two tags in use, I had to check each car to grab the right one.

The replacement tag had a eaisy to peel off sticker with the following text: THIS IS YOUR REPLACEMENT TAG.  Seems kinds of obvious, but if you are fooling around in your garage with the tags, that makes it easy not to mix up the old tag with the new one.

They could have mentioned which car the tag was for.  The only way to tell was by the serial number on the front of tag.  I could read it just fine, but it's in small print and that's not too friendly for AARP crowd.  That would be the only thing I would have changed with how they handle replacements.

More companies should study how the E-ZPass program is run.  The hardware just works, no fuss or muss.  We recentl;y drove from Albany NY, to Corolla, NC and just about every toll bridge and road accepted the E-ZPass.  The only exception was the Chesapeake Bay Bridge-Tunnel, and they are in the process of implementing an electronic toll collection system (E-ZPass).

Tech Tags: E-ZPass electronic toll collection It Just Works

Why you wont see Linux on Corporate America's desktops just yet

I came across this blog post by Robert Scoble while checking something for a GPS data collector that I writing, it has a useful data point to file in the back of the mind.  It has nothing to do with GPS data collection or even what I was googling for, but it explains a very good reason why you won’t see Linux on corporate desktops for the conceivable future.

 

I can sum it up for you with one sentence:  Linux fonts and their technology are no match to Windows and the Mac.

 

I'm not bashing Linux, I have a few things that run on it at home.  Not paying the Microsoft tax makes perfect sense for devices using embedded OS's.  I love my TiVo and the fact that it uses an OS that doesn't blue screen.  My $70 router/access point is running a pretty cool 3rd party firmware that could only be done if it was using Linux.

But on the desktop, I want a desktop that easy on the eyes.  The quality of the fonts and how they are displayed are a huge part of that.  Microsoft recognizes that and they spend a lot of time and money on fonts.  Consolas is a very nice typeface for pounding out code.  It's not everyone's cup of tea, but I like it. 

Tech Tags: Consolas Desktop Fonts GPS Linux Scoble Tea

Project Bluemoon

Dave Zatz uploaded to YouTube a file that's relatively hidden on TiVo's web site which details the mysterious orgins of the TiVo device.  Tune and watch "Project Bluemoon".

You can also download the file as an .mp4 video directly from TiVo's web site (50 mb) from www.tivo.com/bluemoon. 

There looks like there are some subliminal text message in the video, I couldn't navigate YouTube's controls with enough precision to get the frames with the text.  It was a more subtle effect than the blipverts used in some GE commercials than ran back in May.

Tech Tags: TiVo Project Bluemoon Dave Zatz Blipvert One Second Theater YouTube GE

TiVo Series 3: Looks cool, pretty costly

I've been drooling over reading the various articles and blog postings over the soon to be released TiVo Series 3.

It has the stuff I want: Ethernet support built in, dual tuners, cablecard, High Def, big honking hard drive (plus external SATA port).  The rumored list price is $795.  That's a lot of shekels.  I have two Series 2 units and $800 is too much for my pocket.  Maybe next year if the prices drop...

I read a rumor that they are finally addiing WPA support for their 802.11g adapter.  I don't know how accurate that rumour is, TiVo has been planning on supporting WPA for nearly two years.

It's stunning that they have gone so long without it WPA.  When I first bought my TiVos, I had them on the home network using a wireless connection.  I did it a little differently than most people.  TiVo supports a limited range of USB connected wireless network adapters.  TiVo supports WEP, but not WPA.

WEP is fairly trivial to crack, while WPA is pretty secure.  Our neighborhood is pretty noisy on the 2.4ghz spectrum, everybody and their mother has a wireless router/access point.  I had set up the wireless part of my home network to be 802.11g with WPA security.  TiVo's wireless solution was not going to work.

There is more than one way to proivde wireless ethernet.  I picked up a couple of SMC 2870W wireless bridges.  A wireless bridge converts an Ethernet to 802.11 (a/b/g/n) signal.  It works transparently to the Ethernet equipped device.  That device thinks it's on a wired network and there are no drivers to install.  The 2870W supports WPA right out of the box.  The radio on the 2870W is lot more powerful than the run of the mill USB wireless adapters, it's like having mini access points on each machine.

With the TiVo, I use a cheap Linksys USB to Ethernet adapter and connect that adapter to the bridge.  The TiVo thinlks it's on a wireless network and everyone is happy.  There is an additional performance benefit by offloading the 802.11g processing from the TiVo to the 2870W.  The bottleneck becomes the 11 mb/s speed of the USB connection.  The TiVO CPU is not the fastest in the world.  Its roughly equivalent to Pentium 100, no need to have it tackle 802.11 processing if it doesn't have to.  That's probably the reason it didn't support WPA, just not enough horse power under the hood.

The Linksys/SMC combination worked very well, I never had any problems.  I had more problems using a Linksys 802.11g adapter on Windows XP box on the same network and in the same room as one of the TiVos.

I ended up running CAT5e cable through the house when I swtched to FiOS and now the TiVos are wired to the network. If you can do that, life is much easier.  With each TiVo directly on the network, I can transfer shows between the TiVos in real time.  I'm still hobbled by the Series 2 USB speed (or lack thereof), the Series built in Ethernet adapter would be cool to have.

Tech Tags: TiVo Series 3 SMC2870W WPA WEB 802.11g

Diagnostic mode for Ipod Shuffle

Shuffle Lights

Main

Solid orange	Charging, OK to disconnect.
Blinking orange	Mounted, do not disconnect.
Solid green	Connected to charger: fully charged, OK to disconnect. Not connected: play. Light will turn off after two seconds.
Blinking green	Paused: light will turn off after one minute.
Solid orange when you press a button	Hold is on. To turn it off, hold the Play/Pause button for three seconds.
Solid green when you press a button	Normal operation of that button. The light stays on as long as you press the button.
Blinking green and orange	You're trying to play a song, but the shuffle is empty.

Battery Status (when pressed)

Green	Good charge.
Amber	Low charge.
Red	Very low charge.
Off	No charge.

courtesy of Craig A. Finseth

 

Tech Tags: iPod Shuffle Shuffle+Lights

I just took a MSDart to the head and I don't feel so well right now.

One of clients went to run some of our applications on their Windows Server 2003 SP1 box and it wasn't very pretty. They got the error message:

"The procedure entry point DefWindowProcI could not be located in the dynamic link library msdart.dll"

Not a very helpful message. It happens when we access an ADO object within our code. We couldn't duplicate the problem on our W2K3 SP1 boxes. Some searches through Google seem to point the figure at a corrupted MDAC stack with SP1. When our client rolled SP1 back, the problem went away. Microsoft seems to be aware of a similiar problem as they have a KB article, 889114, that describes a similiar issue in Msdart.dll, but not with W2K3. I have also seen references to another article, 892500, but that one refers to DCOM permissions. I'm not sure if that one is relevant.

The question now is how to resolve this issue. How many ways can SP1 be installed on Windows Server 2003? Our boxes got SP1 through Windows Update and they have the right version of Msdat.dll.

Tech Tags: msdart.dll Windows Server 2003 SP1 DefWindowProcI 889114 892500 ADO OLE DB

Leave that thread priority alone

During initial e-Link web service development, I played around with lowering the priority of a background processing thread.  It didn't need to run in real time and having the service handling client requests was more important.  For a background housekeeping thread, I lowered it's priority to BelowNormal.

Well that didn't work quite the way I expected it.  That thread never ran, or it ran so seldom that it was effectively useless.  So I removed the calls to lowering the thread priority and life was good again.  I chalked it up to one of those thread things not to touch and moved on to other tasks.

Today, I saw a posting on the Coding Horror that explained why you should never mess with thread prioirities.  If you have a thread running at a lower priority and it enters a sleep state, it may never wake up if another thread with a higher priority continues to run.  That's an oversimplification, the actual details are described better here by Joe Duffy.

The moral of today's story is "Set up your threads at normal priority and let the operating system deal with scheduling them"

 

Tech Tags: e-link thread priority BelowNormal evil Win32 Delphi C# Coding Horror

Stream reading in C#

I was banging my head against the wall with an odd stream reading problem.  I was making a web service call as straight http, no SOAP, when I hit a snag reading the response back.  I was making the request with a HttpWebRequest object and getting the HttpWebResponse response back by calling the HttpWebResponse GetResponse() method.  From the response object, I was using GetResponseStream() to get at the content.  The data coming back was of variable size.  You would get a fixed size header block, plus a number of fixed sized data entries.  The header block had a field to say how many data blocks there would be.

Naively, I thought I could just use a BinaryReader on the data stream and read x number of bytes in for the header block.  The I would parse that header to the get number of data blocks and then call Read() for that number of data blocks.  Let's say that the header block was 64 bytes in size and the data blocks were 32 bytes.  I had logic like the following:

HttpWebRequest req = (HttpWebRequest)WebRequest.Create(uri);
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();

Stream stream = resp.GetResponseStream();
BinaryReader br = new BinaryReader(stream);

byte[] buff = new byte[Marshal.SizeOf(typeof(MyHeader))];

c = br.Read(buff, 0, 64);

GCHandle handle = GCHandle.Alloc(buff, GCHandleType.Pinned);
MyHeader header = (MyHeader)Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof(MyHeader));
handle.Free();

LogEntry MyLogEntry;

for (int i=0; i < MyHeader.EntryCount; i++)
{
  buff = new byte[Marshal.SizeOf(typeof(LogEntry))];
  c = br.Read(buff, 0, 32);
  if (c == 32)
  {
    handle = GCHandle.Alloc(buff, GCHandleType.Pinned);
    LogEntry = (LogEntry)Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof(LogEntry));
    handle.Free();
  }

}

The problem was that c was sometimes less than 32.  My bytes were disappearing.  I did some quick sanity check code like this:

c = br.Read(buff, 0, 8192);
TotalBytes = c;

while (c > 0)
{
  w.Write(buff, 0, c);
  c = br.Read(buff, 0, 8192);
  TotalBytes += c;
}

When I ran that, TotalBytes had the expected number.  What was I missing?  A little bit of googling found this bit of extremely helpful information from a guy named Jon.  I was reading while data was still coming into the stream.  The Read method is going to return before all of the data has been written to the source stream,  I had to read the stream into a holding array, by reading it as chunks, until there were no more bytes.  Then I could read the data from the array.  This was so obvious, I can't believe I missed it.  The ReadFully() method that Jon supplied worked quite well.

In case you were wondering about the GCHandle stuff, that was needed to marshall C style structures into C# structures.  Getting that bit of code to work right is another story....

 

Tech Tags: C# Binary Data Stream HttpWebResponse GetResponseStream

Vista performance (or lack of)

I just installed the Beta 2 of Vista on one of my dev boxes.  It used to be my primary development box until I got a bright shiny new one last year.  About two weeks ago, the hard drive had a massive failure (it needs to be defraggled at this point) and I needed to rebuild the box.  Since we had the Vista DVD, I figured why not.  We set it up as a dual boot, with XP as the "other" OS.

After a couple of days of use, I've come the this conclusion.  Vista is a pig.  It's slow to boot and slow to run.  It's running on an older box, P4 1.7ghz with 1GB of RAM, but that box is fast enough to run XP without any issues.  It's slow enough that I am not going to use it as a day to day OS.  I'll run XP as the primary OS and I'll manually boot into Vista when needed.  The performance issues come with the beta tag, that's all and good.  I just can't use it.  It felt like OS/2 on a 386.

The fun part was trying to figure out how to get XP back as the default OS.  The new Vista Boot Loader is a strange and wonderful beast, but it's not your father's boot.ini file.  With boot.ini, it's a trivial process to set the default OS.  Vista requires you to use a new command line tool named bcdedit.exe.  With bcdedit, you can specify the default OS, by using the /default parameter and the GUID of the OS to run.

The GUID of the OS?  Where the frack do I get the GUID of the OS? If you run BCDEDIT /enum all, you get a listing of everything BCDEDIT knows how to load and the includes the GUID.  Except for my XP, which didn't get one.  Apparently that's a magic number, if you run bcdedit /default {466f5a88-0af2-4f76-9038-095b170dc21c}, the legacy OS becomes the default.

Since I'm using the Vista Boot Loader, I'll need to remember to restore XP's boot loader before I rip out this Vista Beta for next one.  In the Vista section of Tech-Recipes, they have helpful information on how to do that.  What you need to do is the following:

1. Reboot using the XP CD-ROM
2. Start the Recovery Console
3. Run FixBoot
4. Run fixmbr to reset the Master Boot Record
5. Exit the Recovery Console
6. Reboot

Lifehacker has some tips on how to setup the dual boot here.

Tech Tags: Vista Repave bcdedit Defraggle Tech-Recipes Recovery+Console Lifehacker

Back from vacation

After a couple of weeks of vacation, I came back to the office to find my old development book DOA.  It was worse than a Blue Screen, it was a black screen with Read/Write error message.  My C: partition was filled with CRC errors.  I could copy some files off it, but most were toast.  A second parition on the same drive reported no errors, so I don't think it was a drive failure.  We did have a brown out while I was away, and that box only had a surge protector to save it.  Which it didn't.

I was planning on on installing the Vista Beta 2 in a VMware session on my main development box, but I think it will go on the old box.  I have to reinstall the OS anyways, why not go with the beta.  It's not like I have anything to lose at this point.

This post and the one before it was created with Windows Live Writer (Beta).  It's a pretty cool blog entry tool from Microsoft.  What is really nice about it is that it reads your blog setup and you can edit the posts in the look and feel of your actual blog.  It just works.  It didn't display the map below, but it showed where it would be.

Another cool feature is that you can edit previous posted entries and it replaces the old post with the new post.  This is my 3rd attempt on this post.

We spent a week down on the Outer Banks in North Carolina.  We have been going on and off down there for the last ten years and I like it alot.  This year, we stayed in Pine Island, on the southern end of Corolla.  I played a bit with Wikimapia and got this map of where we stayed:

 

The house in the lower left corner is where we stayed.  I tell you the name, but I want to rent it next year...

Tech Tags: Pine Island Corolla Outer Banks Windows Live Writer Vista

Dates are not numbers

One of the other developers that I work with had a question about inserting some date values into a SQL Server database.  The code in question is doing a batch insert and it was implemented as a series of INSERT statements and they get executed in large batches.  He was having some difficulty in getting the right values for the dates.  He was formatting the INSERT statement with the datetime values being formatted as numeric values.  The end result was that the dates were off by two days.  It was easy to fix and is yet another example of a leaky abstraction can bite you in the ass.

Dates are not numbers.  You have to consider them to be an intrinsic date type, even if the environment handles them internally as floating point numbers.  It's easy to get into the habit of adding 1.0 to datetime variable to increment the time by one day. SQL Server will happily let you do so, and so do many programming languages.  Oddly enough, Sybase Adaptive Server Anywhere wont let you treat datetime values as numbers, they force you to do it the right way.

The problem is how each environment anchors that numeric value to the actual calendar.  What day is day 0?   It depends on what created that value.  For SQL Server, the number 0 corresponds to 1900-01-01 00:00:00.  In the Delphi programming environment that we do a lot of work in, 0 works out to be 1899-12-30 00:00 (a Saturday for those keeping score).  If you pass in a datetime as numeric, when you query it back out of SQL Server, it's going to two days ahead of from your original date.

The 1899 date was define by Microsoft when they defined their OLE Automation data types.  The .NET runtime uses 0001-01-01 as it's starting point.  Those are not the only ways datetime can be encoded.  Raymond Chen did a decent round up on his blog.

The way we usually pass in datetime variables in non-parameterized INSERT statements is to encode the datetime variable as a string in the standard SQL-92 format (yyyy-mm-dd hh:mm:ss).

Tech Tags: SQL Delphi .NET DateTime Numbers 1899 1900 timestamps

Hidden gotcha in FreeAndNil()

Time to go memory leak hunting in my service. I'm using AutomatedQA's AQTime 4, a really cool tool. I've used it's profiling features in the past, but not the memory leak detection. Since Delphi frees up your allocated memory when you exit the app and/or service, it's too easy to get sloppy and not free up singleton types of objects. Well, that makes it harder to find actual memory leaks as AQTime is going to flag everything that wasn't explicitly freed up as a leak. And that will lower the s/n ratio to make the too to hard to use.

So I'm pounding through the code and making sure that everything gets created, gets freed. Great fun, I recommend it for the entire family. I'm starting the service (actually the app version of service, but that's another posting), then exiting it after it initiatizes. That way I can clear out all of the obvious suspects and then turn my attention to the serious memory leaks.

So I'm in the middle of doing this, when one of the objects that I am now explicitly freeing is now blowing up when I free it. And not in a good way. This object, let's call him Fredo (not really the name), owns a few accessory objects (call them Phil and Reuben). In Fredo's destructor, Fredo is destroying Phil & Reuben. In Phil's destructor, Phil references another object belonging to Fredo and blows up because Fredo has gone fishing and doesn't exist anymore.

It took a while to figure out what was going on. You see Fredo wasn't actually fishing, Fredo was still around. Phil was accessing Fredo through a global variable (bad legacy code) because Fredo was a singleton. The variable that reference Fredo had been set to nil, even though Fredo was still in existence.

It took a while, but I figured where and how I had broken Fredo. The code that I had added to destroy Fredo looked like this:

FreeAndNil(Fredo);

The FreeAndNil() procedure was added back around Delphi 3 or so. You pass in an object reference, it free's that object and sets the reference to nil. Horse and buggy thinking for the managed code set, but useful in non-managed versions of Delphi. The problem was that FreeAndNil doesn't exactly work that way. Let's take a quick peek at that code:

procedure FreeAndNil(var Obj);
var
  Temp: TObject;
begin
  Temp := TObject(Obj);
  Pointer(Obj) := nil;
  Temp.Free;
end;

It's setting the variable to nil before it free's it. It's not how it's documented and it caused my code to fail. There's nothing wrong with how FreeAndNil is coded, by setting the variable to nil first, other objects can check to see if it still exists and not try to access that object while it's being destroyed. I just would preferred that the documentation more accurately described the actual functionality.

Tech Tags: Delphi AutomatedQA AQTime FreeAndNil Fredo

Poor man's guide to memory usage tracking

I have a service and I need to make sure that it doesn't have a memory leak after running for a sustained length of time.  I'm doing all the good stuff in code to manage the garbage collection and I have the right tools to check for leaks in my code.  But I still want to monitor an instance of the service during regular usage.  I don't need anything too finely grained, I just want to see if the memory usage is trending upwards.  There's a good chance that I might have to deploy this out in the field, so I want something simple.  I could have used the Performance console, but I wanted something very simple to explain.  You can probably do this with a CScript batch file and WMI, but I wanted something fast to load and fast to exit.

What I want is to log the memory usage to a text file, with each entry timestamped.  I was able to do this with almost all off the shelf parts.  I did have to write the timestamper, but that was a trivial task.  Since the home viewers will not have my service, pick a service or app of your own and play along.  I'll describe what I did using FireFox as a substitute for the actual service.

In the excellent PsTools suite over at SysInternals site, there is a utility named PsList.  It's a combination of the pmon and pstat tools that works like a command line version of the "Processes" tab of Task Manager.  By default it lists information for all running processes, but you can filter it by service name or process ID.  I wrote a batch file to call PsList with the service name and the "-m" command line switch to print the memory usage.  PsList prints some banner information with the details.  Something like this:

PsList 1.26 - Process Information Lister
Copyright (C) 1999-2004 Mark Russinovich
Sysinternals - www.sysinternals.com

Process memory detail for Kremvax:

Name                Pid      VM      WS    Priv Priv Pk   Faults   NonP Page
firefox            3936  108952   41380   32748   36452   140201      8   54

All fine and good, but not pretty enough for a log file.  What I need was just the last line.  So I piped the output from PsList through the good 'ol FIND command with "firefox" as the filter text.  With that, I can redirect the output to a file (with append).  I ended up creating a batch file named memlog.cmd that had the following commands:

pslist -m firefox | find "firefox" >>c:\logs\memuse.txt

That gave me the last line in a file.  But I still needed the time stamp. I thought about going through some script file sleight of hand with ECHO and DATE, but this is the Windows Server 2003 CMD.EXE.  It doesn't have that skill set.  I could do with some 3rd party shells, but the goal is something I can deploy on a remote site without anyone having to pay for a tool or go through the hassle of installing something like Power Shell.

Time to fire up Delphi and create a little command line app that would take text coming in as standard input and send it back out as standart output, but with a timestamp prepended to the text.  The source code has less text in it than the previous sentence.  If you have Delphi, the following code will give you that mini-tool.  I used Delphi 7, any of the Win32 versions should do.

program dtEcho;

{$APPTYPE CONSOLE}

uses
  SysUtils;

var
  s: string;
begin
  ReadLn(s);
  WriteLn('[' + FormatDateTime('yyyy-mm-dd hh:mm', Now) + '] ' + s);
end.

There's no banner or error checking.  I didn't need any of that and I wanted to keep it light.  By adding dtEcho to my batch file like this:

pslist -m firefox | find "firefox" | dtecho >>c:\logs\memuse.txt

I now get output like this:

[2006-07-28 23:14] firefox            3936  536904   61324   51244   57384   445377     90  249
[2006-07-28 23:15] firefox            3936  538176   60844   50764   57384   449193     91  249
[2006-07-28 23:16] firefox            3936  538212   60620   50528   57384   455935     91  249

The output only goes down to the minute, I'm tracking the memory usage every 10 minutes, I didn't need to make the timestamp that granular. If I needed it, I just need to make a slight change the dtEcho source code and it will print the seconds.

To run that batch file, I just used the scheduled tasks control panel applet and set it to run off of my account. For remote deployment, that would probably be the hardest step.

Tech Tags: Delphi pslist sysinternals memory usage cmd memory leak service win32

SQL Server WHERE clause tip (not needed for SQL Server 2005)

One of my services logs every request to a private log table. That table mainly a diagnostic tool to provide some crude performance benchmarks. It's not designed for historical trending, so I have code to purge older records. The service would periodically (twice a day) issue a DELETE statement to the database server to delete records older than 30 days.  Given the following schema (sample, not the actual schema):

CREATE TABLE [MyLog](
       [RecordID] [int] IDENTITY(1,1) NOT NULL,
       [LogTimeStamp] [datetime] NOT NULL,
       [Duration] [decimal](12, 4) NOT NULL,
       [SessionID] [varchar](40) NOT NULL,
       [IP] [varchar](24) NOT NULL,
       [Request] [varchar](80) NULL,
       [Response] [varchar](80) NULL,
       [Error] [varchar](255) NULL,
       [Description] [varchar](80) NULL,
 CONSTRAINT [PK_MyLog] PRIMARY KEY CLUSTERED 
(
       [RecordID] ASC
) ON [PRIMARY]
) ON [PRIMARY]
GO
CREATE NONCLUSTERED INDEX [SK_MyLog_LogTimeStamp] ON MyLog
(
       [LogTimeStamp] ASC
) ON [PRIMARY]

I would execute the following SQL statement;

DELETE MyLog WHERE DATEDIFF(DAY, LogTimeStamp, GETDATE()) > 30

It's pretty simple, use the DateDiff() function to compare the timestamp field with the current date and if it's older than 30 days, delete that record.  I implemented that code in the first go around of the code, about two years ago.  This week, I was in that area code for some maintenance and I took another look at that statement.  That WHERE clause jumped right out at me.  For every row in that table, both the DateDiff() and GetDate() functions are going to be called.  SQL Server will need to compare every value of LogTimeStamp to see if it is older than 30 days ago.  In this case, MyLog has an index on LogTimeStamp, but it will has to read the entire index.   GetDate() is a nondeterministic function, it's going to get re-evaluated for each row in the database.  Since the actual date comparison is against a constant value, I decided to evaluate the comparision date first and change the WHERE clause to a simpler expression.

DECLARE @PurgeDate smalldatetime
SELECT @PurgeDate = DATEADD(DAY, -30, GETDATE())
DELETE MyLog WHERE LogTimeStamp < @PurgeDate

I added a smalldatetime variable and assigned to it date of 30 days ago with the DateAdd() and GetDate() functions.  Now SQL Server can use the value of @PurgeDate to jump into the index and jump out when the date condition no longer matches the criteria.  By I implemented this on SQL Server 2005 and when I evaluated the estimated execution plans for each delete statement, I was surprised to see identical plans.  Both sets of statements spent the same percentage of time doing scanning and deleting.

When I did the same evaluation on SQL Server 2000, I saw different results.  The first delete statement spent 73% of the time scanning the index and 27% actually deleting rows from the table.  The second delete statement spent 19% of the time scanning and 81% of the time deleting rows.  On table that could have a large number of rows, it turned out to be big performance saving on SQL Server 2000 installations.

It's pretty cool that the SQL Server 2005 parser is smart enough to optimize code and recognize a constant expression when it sees it.  My code would have seen a nice little performance boost by moving from SQL Server 2000 to SQL Server 2005.  It's still a better thing to pull constant expressions out of the WHERE clause when you can do that.

SQL formatting courtesy of The Simple-Talk SQL Prettifier for SQL Server.

Tech Tags: SQL SQL Server SQL Server 2005 DateAdd GetDate DateDiff Execution Plan

Migrating to Delphi 2006

I've been working on migrating our Delphi 5 code to Delphi 2006.  For the most part it's been pretty straight forward, the fun part has been dealing with the assortment of 3rd party packages.  Some of them required new versions, others were not available and I had to migrate them myself.  I'll save my DevExpress experiences for another post...

Most of our Delphi code has been in Delphi 5 for a number of reasons.  We mainly stayed with it because it worked and did what we needed.  We did a complete rewrite of flagship app, VersaTrans RP, and Delphi 6 and Delphi 7 were released during that cycle.  Delphi 6 didn't add any features that we needed and Delphi 7 came out during regression testing for RP.  It would have been insane to switch compilers during final testing.  I do use Delphi 7 for our web based application, e-Link RP, mainly because of the SOAP functionality.  Now it's time to round up the children and jump on the Delphi 2006 wagon.

One of the packages that I migrated up was Abbrevia.  This is an Open Source compression toolkit that includes components for handling PKZip, CAB, TAR, and gzip formats.  It used to be a commercial product from Turbopower, but a few years back it became Open Source when Turbopower was bought out by Los Vegas gaming company.  Since going Open Source, development has come to a flying stop and nothing new has been released since 2004.  I had to tweak the package files somewhat to compile with Delphi 2006.

The distribution came with tons of file, and package files (.dpk) for every version of Delphi/C++Builder from 3 to 7.  I took the Delphi 7 package files and edited them outside Delphi to make Delphi 2006 packages out of them.  I probably could have done within Delphi, it was easy enough to do with TextPad.  I changed the names to match the compiler, Abbrevia uses the the compiler version to differentiate between the various package files.  B305_r70.dpk became B305_r100 and B305_d70 became B305_d100.  Except they didn't include B305_d70, I had to use the Delphi 5 version B305_d50.

I also changed the file paths around.  There is a source folder and a packages folder.  I added a bin folder and edited the package files to place all of the compiled units in that folder.  I also need to explicitly add the source folder to the package's source path.  Then I tried to compile.  The runtime package compiled without incident.  I moved it's .bpl file to my system32 folder and moved to the design time package.  It failed to compile because it couldn't find the DesignEditors unit.  The DesignEditors unit contains the base classes for implementing component editors.  You used to be able to have that unit in your runtime code, Borland has stopped allowing it.  It does come with Delphi, just not on the default search paths.

So I figured I could just add the path to DesignEditors.pas to the package search path.  Just add the path and Bob's your uncle.  Nope, didn't work.  DesignEditors.pas has a typo in it. It's missing a comma in the uses clause of the implementation section.  My first inclination was to just add the idiot comma, but it seemed like other things would have failed if that was the case.  I was doing it wrong.  Instead of adding the source path, I'm supposed to have added the .dcp compiled package, which is in DesignIDE.dcp.  Once I added that unit to the package's requires list, everything was good to go.

Tech Tags: Delphi 2006 Delphi 7 Delphi 5 Delphi Abbrevia TurboPower Open Source Code migration

The following could have been coming from my desk (but didn't)

Overheard at work today.

"It's Wise, so I really don't ask questions. I just work around it." — Sam Tesla, 7/12/2006

If you've worked with the Wise for Windows Installer, you'll understand...

[Joe White's Blog]

I finally received confirmation from Wise Tech Support that the last bug that I reported was a real bug. You can't install a service written in .NET 2.0 on a machine with the 1.1 and 2.0 Frameworks installed. Wise runs the installutil.exe from the Framework to install the service and they pick the wrong version to run. Of course, it may be fixed for the next release, but that's tentative. My work around of using undocumented methods to self-install will continue to be used.

I don't care about RocketBoom

I have seen too many blog postings about Amanda Congdon leaving the RocketBoom vlog. There's a he said/she said thing going on between her and the other owner. Since the world+dog seems to have a need to post about, I might was well too.

I first heard about RocketBoom when it became available through TiVo. I subscribed to it for about a week or so, but i cancelled it. I found her schtick to be tiresome and annoying. She's an acquired taste that I just couldn't acquire. Ze Frank is much better at that kind of stuff.

In the end, it will probably work out well for both partners. Congdon has been getting plenty of job offers and RocketBoom has had tons of free publicity.

Tech Tags: RocketBoom Unboomed TiVo Ze Frank Amanda Congdon