Recent Coding Travels

I haven’t posted much code stuff lately. Nothing in particular seems blog-worthy, but hey! after CodePaLousa, I’m making an effort to write about what might be mundane.. in a relatively anonymous way.

Story #1: Performance Improvement

Problem: I had 5 people, all doing the same web service call(s), which were slow – sometimes up to 20, 30 seconds to complete. Except that the system was taking 2 minutes to bring the information back.

Diagnosis: Ants Performance Profiler showed some interesting stuff.

Initial call to get the list of A1,B2,C1,D2 … took 5 seconds
But a background thread to populate A1a, B2a, C1a, D2a .. took another 30 seconds
But there’s a thing, which for any given A, needed the sum of A1a, A2a, A3a even if A2 and A3 were not in the initial result set. THAT one .. for large screens.. going at a speed of about 6 per second.. took up to 2 minutes.

Solution:

Introduce a service that preemptively did the web service calls.. cached results for up to 5 minutes. Service runs every 4.5 minutes. Everybody became much faster, and 5 minutes did not affect their business decisions. Instead of 2-20 seconds, now those calls finished in 7ms .. IF the cache was filled. Which the service ensures that it is. If the service is down, any client can fill the cache as well – they all use the same class.
Removed the background thread that got all the extra A stuff… waited until the user clicked a little expando button that showed that information. THEN go and get it.

Problem introduced by the solution:

Now, if you click “Expand All” on a large screen, you get 200 threads all trying to expand at the same time. It succeeds, but it also thrashes while attempting to resolve. However, that’s a <1% use case, so we can live with that.

If my client is okay with it, I’d love to post some before and after graphs to show how things changed.

Story #2: Idle Crash

Not on mine or the product manager’s machine, but certain machines on the floor, while using the app – leaving it sitting in the background – it would crash.

Diagnosis:

Event log entry #1: .Net blew up, Yo.
Event log entry #2: System.AggregateException()

Research:

in .Net 4.0, if a Task.Factory.StartNew(()=>{}) background thread generates an exception, and nobody reads the exception, … nothing happens.

Until the GC runs. Then the app blows up.

Doesn’t happen in .Net 4.5, which is why my machine ran fine.

Solution:

It was blowing up in the code that I changed in Story #1.
Added a handler for UnobservedTaskException – to log any other things that might happen with a full stack trace so they could be debugged.
It hasn’t shown back since.

Story #3: Iterating on Requirements

Problem:

While working on my normal client, I have another client for whom I (and another guy) are estimating a project. However, this is a large client, and the scope of the project seemed.. fuzzy. Huge. Integrated with a bunch of systems that had not yet been written.

Solution:

Break the project up into Phases. The client came up with the same idea! Yes!
Make the first Phase the smallest it could be. Thus, success on that phase would also clear up architecture problems, and provide a target for other systems to code against. It would also not be dependent on any other system to come online first.
Collect everything into a single document:

Questions
Decisions
Design
Features

Almost Everything started off as questions.

As stuff gets clarified, it moves over to “Decisions”.

“Design” is where larger things (like schemas, interfaces, etc) can be added.

“Features” was an interesting idea:

It is a one-line-at a time, “if the system has all of these things, then it is complete” list, bulleted and indented.
When you first start working with the project, you won’t know what most of that stuff is.
But after you’ve caught up to the “groupthink” on the project, you Will. It’s a checklist, without content.
This is the only part of the document that needs to be “complete”; everything else is just talking points, things in flux.
This is what the estimate actually gets built on.

I can’t say this is my idea. I picked it up from a speaker who talked about his daily process.

So, yep, that’s my software engineering excitement of probably the last 3-4 weeks. If I were asking questions on this blog, I’d ask, what have you been up to?

Feed a Hungry Software Developer: Panda Express

We got a new fast food restaurant in the area: Panda Express. First time visiting.

At noon.
With the parking lot full, the queue of people stretched back to exactly the door, and there were one or two open spots left in the dining room. Ie, they know their flow and sized appropriately.
Time from joining the queue till getting to the counter: about 11 minutes.
Food Cost: 2 entries + white rice + 2 eggrolls + water (no drink) ~= $10. So, not inexpensive.
Food Quality: very colorful; I’d put it higher than your average mom+pop Chinese restaurant, below PF Chang’s / Asia Moon.
They have brewed ice tea, which is important for my wife. I forgot to check what kind of ice they have (cubes or nuggets)
Their fortune cookie doesn’t try to teach me Chinese.

The Oldham County Grand Slam #1: Roman Road 5k

This race holds a dear place in my heart, because it was the first 5k I ever aspired to run. I trained for months leading up to it, and in 2010, I finished it in an impressive (for me) 36 minutes! This year, after a year off from running, I return, with video camera on head.. to pay homage, tribute, fan love, to my favorite series of races: the Oldham County Grand Slam.

[youtube=http://www.youtube.com/watch?v=aX3KN-0bU38&w=448&h=252&hd=1]

*skreetchinHalt* Hey this is a geeky blog. Get to the geek, boy-o!

Making this video was hard. What to include? What not to? Running the course is too many minutes. All of the finishers is too many minutes. The awards is too many minutes. Do I really need to include the bit I did with the guy from Thorntons? (I wanted to acknowledge that they provided the water). In the end, I mixed it up a bit with picture in picture – any one thing would be boring by itself, but having multiple things going on makes it just a bit more interesting.
I tried using stabilization on the running video, but it was too much for my poor computer (and I ran out of time).
I needed to ensure nobody confuses this with something “official”, ‘cause its not.
The storyline of the various teams competing against each other.. well, its interesting, except that the same teams keep winning. And nobody is stepping up to take them on.
Putting the video camera on my head was probably a bad move. I think it might be more stable on my chest. I’ll try that next race.
Had some problems with resolution – the old GoPro takes 720p video. I might have to max-render this at 720, currently both renders are going. (turns out, 1080i not so bad, so rolling with that)
The final video composition was gnarly:

Anyhow, back to the race:

Its cute!
People know me there!
There’s a 1 mile fun run, which is what the kids are getting medals for at the end.
Sliced Oranges & Bananas!

There were a lot of stories that I didn’t try to fit into the video:

Andrew, hasn’t run since high school, showed up at 7am (the race isn’t till 8:30). He won first place in his age division.
A lady and I chatted at length about heart rate training.
The guy with the sign – from a running club in Louisville. They have four signs. He used to run, but can’t anymore, but loves to help out. At the PPJ10M, he goes around and borrows signs from a lot of places and puts them up.
Mark, the guy with the microphone, came up with the idea of the grand slam 12 years ago (or so). Mostly for the family and community of it.

Not sure what I’ll do for the next one. Probably more of the course and the finish, but sped up a lot more. There’s a lot more spectators as we go through downtown LaGrange. Maybe try out the chest strap. We shall see.

Firefall– I am enjoying this game!

In the realm of geeky, one of the things I am doing that I have not talked about here is .. playing the closed beta of Firefall.

Last night my wife gave me permission to upgrade to “Commander” level in my investment – which gives me a lot of in-game currency to buy the “pay-for” builds, and a Motorcycle to get around the map faster with. (they call them LGV’s) On a $/hr enjoyed ratio, I’ve already enjoyed this game enough to justify that investment.

Last night, after the purchase, I started to get serious. I’m building a spreadsheet of what kinds of resources I need to keep my eye out for, in order to build the “better” gear.

My goal: to have decent Tier-II gear so I can solo a Light Personal Thumper.

A quick crash course to frame this goal better:

There are 5 major resources, with 3 sub-species each. (See Xls Screenshot above)
When you mine them, they come in different qualities.
When you build components, the quality of the material you put in affects the stats of the item you build. So if I want more damage from my Heavy Turret II, I need to use a better quality Conductive Metal.
If I’m short on Conductive Metals, I could use another metal. However, a high quality Computational Metal used instead of a Conductive Metal would yield a lower quality.
To get resources, mostly you “mine” them using a “Thumper”
- You run around the map, scanning the ground with something called a “Scan Hammer”. It tells you what the concentrations of stuff are at the spot you’re standing. Concentrations shift over time.
- Pick a spot, and call down a specific size of thumper.
- The thumpers supposedly annoy the surrounding fauna and they come attack the thumper. Your job is to defend the thumper so it doesn’t get down to integrity=0 and blow up.
- The resources mined are shared amongst the people who helped defend it in some manner.
- The smallest thumper (Stock Personal) is easy to solo. I can send it back at Integrity = 80% or so.
- The next size up is the Light Personal Thumper, and I cannot solo that. I can’t kill them fast enough.

There’s other stuff to do around the game as well:

There’s an overall story line of an alien invasion (“Chosen”) with a different atmosphere (“Melding?”) with zombies (“Tortured Souls”) and they occasionally try to take over human territory (“Drop Ships, Incursions”) which makes it quite annoying to try to mine an area. If they take over an area for too long, they convert the atmosphere there and its effectively “lost”. (I don’t know if it can ever be taken back – probably not yet, but maybe in the future)
You can fly around the map using a glider. Gliders do exactly that: let you glide from a high elevation to a lower elevation – the higher off you start, the longer you can glide. There are achievements to unlock here (distances, sight seeing, death from above, etc)
There are roads that can be used to ride your motorcycle (“Light Ground Vehicle”) between places at high speed (which is quite fun).
There is PVP that can be queued for (not my cup of tea)
There’s something called a “Melding Tornado” which involves some dimensional shifting monster spawing blah blah dropping resource blah blah stuff.

A lot of the stuff (Chosen, Tornado) is pretty difficult to solo, however, the population is pretty quick to respond if you ask.. once you get two or three people working on it, most of the tasks are doable.

Things I like about this game:

I like that the difference between a newbie (stock armor, no enhancements) and a veteran (T2 armor and above) is in the same order of magnitude – unlike most other games. The monsters are the same for both – the veterans can handle more of them at a time.
Amazing sound. Explode a grenade and listen to the echo from the walls.
Good Graphics
FPS Shoot’em’up adrenaline
Jumpjets are fun
Good people playing the game. That will change when it goes public, but right now its awesome.
The battleframes are different enough in styles of play – each one has its own fun.

Okay, so done. This is what I’ve been playing for the last month or so … it replaced “European Truck Driving Simulator” as my go-to game. I hope I’ll be playing it for at least another month or two.

Almost all the images in this post are borrowed from other resources, which I have linked them to. This last one is me using GiantManiac’s thumper as a photo shoot.

I work at an awesome company!

I signed up for Short Term Disability & Discombobulation today. No big deal, one might say, but the manner in which it happened is what was different.

Previous experiences with companies: “Hi, we’re the really large Human Resources Department, and we have your best interests in mind, and thus here is the voodoo that you can choose from.” I know they do their jobs and I have to trust them on that, but I definitely did not feel like I had any participation or representation in the choices available to me.

This experience: “We’re thinking about maybe making some of this stuff available, if we sign up as a company we get a 40% discount over straight individual rates, lets meet with them and find out what our interest level is, we need 4 people to make it a go-ahead”. We got 8. I participated. Very informed decision.

(We referred to our Aflac sales team as Mr. and Mrs. Aflac; they are married.)

Kentucky Derby Festival Mini Marathon / CodePaLousa Day #2

“Stop calling it a Mini. There’s nothing “Mini” about 13.1 miles.” — somebody who probably is not a Marathoner.

Well, the second half of the Marathon – making the video of it – Its not “over” because the video isn’t “perfect” and I’m a perfectionist, so I’m admitting defeat and uploading it as it is. Because to succeed would involve more frustration than I pay myself for. Maxim S1: Dial the perfection knob down to 7 unless I’m getting paid for it.

[youtube=http://www.youtube.com/watch?v=lD3T-L5PPzo&w=448&h=252&hd=1]

My wife’s name is Molly but that’s not for her

My hope and intention was to capture some of the “how” it works (for those folks who’ve never been to a bigger running event like this) as well as the “why” its beautiful (the people who run it, the beautiful old people along the road, the general sense of celebration, the life that happens along the way).

I planned poorly – I ran the GoPro at a frame rate (60 fps), shortening the available battery life to 50 minutes, and took too much video at the starting line, so I had to switch to the iPhone for the last bit. I wish I could have included:

The girl who broke her leg, got crutches, and she, and her entire family, are walking the rest of the way to the end. Beautiful family.
The guy who broke his leg, got crutches and a wheelchair, and he, with a buddy following behind him with the wheelchair, are intent on finishing it.

Other little things that are interesting to me:

The lady who ran three halves this year – her shirt said “I can do it.”
If my friend Todd had run 1 second per mile slower, he would not have qualified for Boston.
Running into just about everybody I know who runs, including Nick whom I know from Panera who is 72, race-walks Marathons, and hopes to do 6 M’s in 6 days in the North East this year. And Steve. And Shannon.
I was able to sit with somebody by the side of the road and listen to her. Her knee was locking up on her good leg, and she didn’t know how she could keep on going, and she told her friend not to come back for her but her friend wouldn’t listen, and she was frustrated enough to cry to me, a stranger. Bless you little lady, I hope you did good, whether that was respecting your pain, or finishing your goal. Please don’t injure yourself, its much more important to stay healthy than to finish one race.

Lessons learned:

The most comfortable place for the camera is, in fact, on my head. On my chest, it vies for attention with the heart rate chest strap and its much harder to aim; and there’s no hope for having it “put away” on my belt. If you skip to the end of the video, I don’t look too dorky with it. Although why did I have to dance?
I can probably get a much faster time if I didn’t talk as much, or shoot as much video. Almost everybody I talked with – if you go look up their times via their bib numbers – finished faster than me. Go figure!
Don’t try to mix anything else with a half marathon day. To whit:

CodePaLousa Day #2

No Problem. Car is broken, I’ll just ride my scooter to the half marathon, run 13.1 miles, ride home, let the dogs out, ride back to downtown, and attend the last 3-4 sessions, having only missed a keynote and one session.

Problem Number 1. We parked 1 mile from the start. In the morning, that seemed like a great idea. After the race, not so much. Every sidewalk curb was an oww-portunity.

Problem Number 2. Scooter = Windchill; Windchill against tired fatigued muscles = frozen muscles. When I got off the scooter at home, I was locked in a “Cowboy Stance”. I could hardly get the scooter back on its kickstand. I could not walk up the steps (I had to crawl).

Problem Number 3. Given what my wife calls “Marathon Brain”, I very intelligently put away my scooter lock into the under-seat storage, with my keys still attached. At that point in time, didn’t know where the spare keys were (Separate story, starring the Blinking Red Dot from my unconscious). This makes transport difficult.

However, I did get to wave at two ‘PaLousers (wait that sounds bad) as I jogged past the Seelbach Hotel at the end of the race. And that would be my second day of CodePaLousa.

The good news is, my inability to attend has lead to a grassroots movement – I am dedicating the first weekend in the month to have my own mini-conference – including Pizza, and Board Games, and Microsoft’s Channel 9, and one hour sessions, and a friend or two — to catch up on videos / presentations on various technologies.

Freakin Awesome Software: Greenshot

Maybe I’m late to the party, and everybody already knows about this. I was introduced to it by Michael Thornberry

Why I use it:

Hotkey to take a screen capture => easy to start
Once screen capture is taken, THEN I get to choose what to do with it, including “just save it as a !#!# file NOW” and “I need to touch it up a bit”
In the editor:

My most recently N used colors are remembered!!!! (I use a LOT of color)
I can re-edit annotations after I’ve made them, making it closer to Visio than to Paint.Net in its flexibility
Built-in obsuffofusticate. (stupendulous!)
When I’m done editing, I can re-enter the pipleline of “what do I want to do with this image”

I suspect I’ve donated to them twice now. They are that awesome.

About the only thing I can’t do with them (yet) is to take captures of multiple little areas and put them together to tell a story. As soon as sourceforge lets me, I’m going to search for that feature request and upvote it.

Papa John’s 10 Miler Video

I had some footage I took during the PPJ10Miler sitting around, that I’ve meant to do something with.

[youtube=http://www.youtube.com/watch?v=3M5xtcCmlpQ&w=448&h=252&hd=1]

Scenes from the Papa John 10 Miler 2013

Stabilization is hard! especially on iPhone video, where there isn’t a large forgiving field of view like a GoPro. It took an hour to stabilize the 3 minutes of footage I had, on an i7 processor with 12G of RAM.
I scream way too loud for being a camera person.
I still had some moire develop on the 1080p video, so I took it down to 720p at a drastically different frame rate (25p). I ended up doing 4 different renders trying to find something that worked. 60fps helps with the source of this problem, but is a booger on battery power.
I should do some test stuff with the different modes of stabilize some time. Cool blog post about it for me to digest: http://whoismatt.com/bestwarpstabilizersettings/ In my case, I went with “Position only” to preserve frame space, and removed cropping. Yep, more research needed..

Anyhow. 10 miler is a fun race. Its really nice to see the leaders of the pack – usually I only hear about them on websites and stuff like that. And they have nice technical shirts. Definitely recommend.

Tutoring? Growing? Developers

I volunteered my way into a meeting today with a pretty cool guy: Rider Rodriguez Jr of www.kentuckianaworks.org. We were discussing how to make Louisville a better city as far as Software Developers – there’s a shortage, actually nationwide, and the gap is likely to grown in the future. (I have no references, but I figure I’d trust him on that one).

It was a great meeting! We talked about what actually works, what doesn’t work. As people in the trenches who actually do the work – very often biggity-bigs who don’t do the work, have opinions of how people should learn and are looking for some way to leverage that for dollars. Not efficient. We just love doing it, and we know what it takes to do the work in our industry.

Then he presented to us the plan that he’s shooting for (applying for a grant, etc). I’m not going to try to detail it here, as I’ll doubtlessly get it wrong, but it does involve classroom availability, labs / peer sessions with people in the industry, building a portfolio in skills and abilities that companies actually want, and internships.

Much discussion later, and some white board drawing by yours truly, it seemed like we were all on the same page.

The Whiteboard. Someone noted that it looked like a person’s face. The face.. of the anonymous software developer.. dum dum dum!

I hope this works.. and yes, if anybody *wants* to learn *anything* that I know (including! how to make Instant Mashed Potatoes. I practiced that tonight. My wife thinks I did it wrong. She thinks that by adding liquid to powder, it would be lumpy. She might be right, it was rubbery), I’d love to help them learn it.. just to see the joy in their lives as they figure it out. I am the son of a teacher, after all. (10th from the top)

Database Change Scripts

At CodePaLousa Day #1, I attended a session, Tools and Adoption Patterns for Automating Database Deployment, about doing automated change in databases. That’s more than I need to do right now, however, it gave me a +1 on how I’m currently managing change, which is my default method if the customer doesn’t provide one.

What I do:

For every change, I start or append to a script file.
When a change gets to prod or “moves on” to branch, I start a different script file.
Every script file must be re-runnable.
Every script file must have a precondition check – something that is true of the database before it can run against it.
For SqlServer, if the precondition check fails, I turn noexec on to prevent the rest of the script from running.
For SqlServer, DDL statements are wrapped in exec() so that I don’t get compilation errors.

Here’s a sample, with names changed to protect the client:

SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- This is something from the most recent previous round of changes, which should already exist. 
PRINT 'Precondition Check'
if (not exists (select 1 from information_Schema.COLUMNS where TABLE_NAME='Quadrant' and COLUMN_NAME='ImperialCode')) 
begin
    PRINT '  Failed -- skipping rest of script'
    set noexec on
end else begin
    PRINT '  Passed'
end
GO 

-- In support of TA26 (revision 1683): 

PRINT 'Table NoRebelsAllowedRule'
if (not exists (select 1 from INFORMATION_SCHEMA.TABLES where TABLE_NAME='NoRebelsAllowedRule')) 
begin

    CREATE TABLE [dbo].[NoRebelsAllowedRule](
        [NoRebelsAllowedRuleID] [int] IDENTITY(1,1) NOT NULL,
        [CatalogID] [int] NOT NULL,
        [DeletedOn] [datetime] NULL,
        [DeletedByUserID] [int] NULL,
        [CreatedOn] [datetime] NOT NULL,
        [CreatedByUserID] [int] NOT NULL,
        [UpdatedOn] [datetime] NULL,
        [UpdatedByUserID] [int] NULL,
     CONSTRAINT [PK_NoRebelsAllowedRule] PRIMARY KEY CLUSTERED 
    (
        [NoRebelsAllowedRuleID] ASC
    )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
    ) ON [PRIMARY]
    PRINT '  Created'
end else begin
    PRINT '  Already Exists'
end
GO

PRINT 'Foreign Key FK_NoRebelsAllowedRule_Catalog'
if (not exists ( select 1 from information_schema.REFERENTIAL_CONSTRAINTS where constraint_name='FK_NoRebelsAllowedRule_Catalog' ) ) 
begin 
    ALTER TABLE [dbo].[NoRebelsAllowedRule]  WITH CHECK ADD  CONSTRAINT [FK_NoRebelsAllowedRule_Catalog] FOREIGN KEY([CatalogID])
    REFERENCES [dbo].[Catalog] ([CatalogID])
    PRINT '  Created'
end else begin
    PRINT '  Already Exists'
end
GO

PRINT '04/23/13 Replacing Catalog.CannotFireBlasters with BlasterBehaviorID'
if (not exists ( select 1 from information_schema.COLUMNS where table_name='Catalog' and column_name='BlasterBehaviorID' ) ) 
begin
    alter table [dbo].[Catalog] add BlasterBehavior INT NOT NULL DEFAULT '0'; 
    PRINT '  Added BlasterBehaviorID'
end else begin
    PRINT '  BlasterBehaviorID Already Exists'
end
if (exists (select 1 from information_Schema.COLUMNS where table_name='Catalog' and column_name='CannotFireBlasters' ) ) 
begin
    exec (N'update [dbo].[Catalog] set BlasterBehaviorID=0 where CannotFireBlasters=0');
    exec (N'update [dbo].[Catalog] set BlasterBehaviorID=3 where CannotFireBlasters=1');
    PRINT '  Migrated Data from CannotFireBlasters'
    alter table [dbo].[Catalog] drop column CannotFireBlasters
    PRINT '  Dropped old column CannotFireBlasters'
end else begin
    PRINT '  Old column CannotFireBlasters does not exist'
end
GO

set noexec off
GO

Since the above sample, I’ve started adding dates and work item numbers into the script as well, to make things a bit more traceable. I figured as long as I had a sequence going, I’d append to the same script, though in an environment with multiple developers, you may want to partition more.

Why I didn’t..

RedGate Sql Compare tools: I don’t own them.
Sql Server Project in Visual Studio: When you actually get around to using them against real environments, there are a ton of problems around “Server” projects verses “Schema” projects (or some such) which are worse in 2012 than they used to be.
EF Migrations: Haven’t spent the time to learn them, desire to be productive, sticking with what I already know and can do quickly.