Torn

Last Tuesday I went to http://openhack.github.io/louisville/.

I might have been the only guy who would profess to knowing .Net there.  I felt out of place.

A guy from the other group, http://www.meetup.com/Lean-Startup-Circle-Louisville/ came over and asked if anybody knew C#.  I raised my hand.  I’m glad I helped him.  But it also felt like I revealed myself.. a .Net developer amongst a community of Ruby, Rails, Java, etc..

Its probably all in my head, but it felt like nobody would give eye contact when I came back from helping Joe.  (Btw, He has a nice business model.  But that’s his, not mine to talk about).

What was my intention anyway?   To be famous?  That’s a bad intention.  I went there, that night, to find time to work on a problem that I had, about getting time-per-task out of Harvest when the task was encoded in the comments for each time entry (each task is a user story, effectively).  To compare story points vs actuals.  That’s another topic.

Anyway, its got me thinking.  Here’s this resource of not-.Net people.  A world I know little about. Maybe my next little project, just to shake up my .Net mold a bit, should be in.. something else.

What else? Where the fudge do I start?   Gah!  Ubuntu is probably a good spot.  I have that on a laptop.    Java? Ruby? Python?  Node.js?   What am I trying to solve?

But why?  I already don’t have very much time available.  Do I want to solve a problem, or do I want to “investigate cool new stuff and build new skills”?

Is it even a problem?  Really, keeping track of car metrics is a VERY 1st world problem.  And I have enough other (boring) projects that I have (externally) committed myself to doing, that I really need to finish, that have nothing to do with Coding.

I want to be the kind of guy who can easily grab another language and go do something else.  I know I used to be that guy.  Am I still?  Its been 13, 14 years since I’ve stepped out of the ecosystem that I work in (C#/.Net).  Why should I ? Is it worth it?  To whom?

Possible Answers

The answer is to table it all, for now.  First, I need to finish the last two running video’s, so I can cancel my subscription to Adobe ($30/mo), and I have another household project in the works that is going to take some time and energy.    Let it sit, till next month first Tuesday.

Define my goal.   I want to compare my different routes to work:

  • by Time taken
  • by Fuel consumed

I could get to my end goal in about 3 hours using Powershell and Processing, slightly modifying stuff I’ve already written and hand-editing a few files.    What I guess I’m choosing here is to investigate a different world.. and see what tools there are.

Possible:  I could learn Java & Eclipse the next time I go to @OpenHackLVL.   Java has what I need –  libraries for reading CSV, a local database, and 3D support.   My goal could be made incremental like this:

  • Read in a single file, draw a simple chart (lat+long)
  • Read in multiple files, draw a simple chart (lat+long)
  • Draw a 3D chart.  (lat + long + fuel consumption)
  • Read multiple files into a database (the local kind, that could be embedded within the app)
    • Must be able to grow to the concept of a trip with start/end locations
    • Must be able to grow to the concept of a meta trip which is several shorter trips back to back. (to cover turning engine off at stoplights)
    • Must be able to trim off idle time (engine running, not moving) at the start and end of a trip.
  • Draw the chart from the database.
  • Draw a chart from the database of only those trips which start/end at certain locations
    • Compare different routes from Home to Work.  
  • Up till now, everything has been a console app (I think).   Investigate a Gui App with:
    • import trip(s) that have not yet been imported
    • filter by start/end
    • rendering different types of charts.
    • This would be a “product” that I would consider “releasable”.

I don’t think I’ll get all the way to the end, but this could be a roadmap of something to play with.  

Are there other ecosystems I should consider instead?   I need the ability to process text files, write/read from a (local) database, and render charts.    What could I do?

YMCA Oldham County Days 5k Video!

I did it!   I’ve edited together yet another 5k race video!

[youtube=http://www.youtube.com/watch?v=OUFRNGFNkrk&w=448&h=252&hd=1]
Yet Another Caption

 

This time, my workflow:

  • Don’t pick and choose, stabilize ALL the running camera video.   I carved them up into 2 minute sections, and spent several days clicking stabilize, and walking away.   Some day I should figure out some batch automation for this.
  • Use a 720p format, so I get an automatic clip of the 1080p stabilized footage. 
  • I did not go dual screen this time.  I got everybody on camera as they left, rather than as they came in.  Side effect: less work!
  • Strategic use of “cross” vs “dip” to hint at when a story line was changing.
  • I had some problems with picking a consistent font.   Some day, I’ll find a font I like.
  • Once the music started, I was able to cut/trim/fit the running video to the music. 

I only had two stories I could really tell in interview style:

  • History of the race (thank you Bill!)
  • Hot.

But I managed to fit in some little tidbits:

  • Downtown LaGrange.. Oldham County days.. band playing, etc.
  • The 1 mile is a different route than the 5k for this race.  Which means, you can see the 1 mile winners on their way back!
  • The interrupting Tra…  “Toot Tooot” …ain. 

Guest appearances:

  • The guy in the golf cart is Corey, he runs the indoor soccer program for the Y.
  • Shannon runs Oldham County Physical Therapy, they are wonderful.  They fixed me up after ITBand’11.
  • Bill the sign guy helps with signage at almost all the local races.  Including the PPJ10mi.
  • 1st place winner lady = last year’s overall winner.
  • 1st place winner guy Sean R = former soccer teammate of my stepson’s.
  • A bunch of the winners are related to each other.   There’s a family who is in to running.   I should interview them some time.

It’s a bit harder with the camera on the front.  If I run hard, my hands get in the way.  On the other hand, with the camera on my head, stabilizing the footage is much harder.   I wonder If I can find a nice person with a running stroller for the next race, put the camera on the stroller.. Hmmm..

The next one up is Fastline 5k.  I know almost nothing about Fastline (the printing company) .. I guess I’ll get to find out!

Staying Focused..

I had a horrible day for focus yesterday.   So horrible, in fact, that I ran out of newsblur things to read, and I found myself browsing reddit.  And that, now that I’m better rested, is not okay. 

So I’m trying out @StayFocusd.  It only works in chrome, however, chrome is what I do my personal browsing in; Not as much a “I’m going to limit myself by force” but a “remember, I made an agreement with myself”. 

image

My main distraction points are:  facebook, newsblur, and reddit. 

On the other side of the coin, I will sometimes use the Pomodoro Technique to try to stay focused.  When I do, I use this website as my timer:

image

Here’s to a focused, productive day!

I should probably stop writing blog articles for today.

Nostalgia: Thank you Computerphile

There’s a channel that I enjoy watching called “Computerphile“.   Its a guy, Brady, interviewing all these computer-science related people at the University of Nottingham.  For the most part, its all stuff that I know, but I enjoy listening to it explained carefully.  If I ever have the need to explain something to a lay person, I would certainly send them to this channel.  Occasionally I learn something new, for example, Pick’N’Place machines.

I saw this video last night: 

[youtube=http://www.youtube.com/watch?v=-rPPqm44xLs&w=448&h=252&hd=1]
Nostalgia Caption!

I remember learning Unix on the Iowa State University graduate student unix box.. my older brother was a grad student in 1983 .. and it *was* running on a PDP-1175 or something like that. (I think).   I have direct experience with what this dude is talking about.  And he has no hair.  (Give me 10 and I’ll catch him)

Took a while for the emotion to work its way through my system.   Then I remembered:  I was borrowing access.  I wasn’t actually a grad student.  So, perhaps I was 5-10 years ahead.  Comparatively, “my” generation (if i had become a grad student) would have been Windows 3.1 heading on to Windows NT 3.5.1, and the birth of Linux and perhaps Java.

Still old.  damn.

Compare/contrast with http://openhack.github.io/louisville/

They are probably feeling:

  • the excitement that I used to feel when I was learning Turbo Pascal and figuring out the math to do 3D rendering of a spaceship.  
  • Or the fun that I had writing in an autopilot into a lunar lander game which landed the ship safely every time, with obstacle avoidance.
  • Or the macros I wrote into my TTY program to keep regenerating a character till I got one with at least 3 18’s .. for the game that was like NetHack but was on the VAX/VMS cluster.

I think, if i can get my courage up, I might go to that open hack thing.  It would be an opportunity to work on my car logging scripts and visualizations. 

Also, come to think of it, my memory of the CS program at Iowa State University is stale.   I wonder what they do now?   (Goes off to find somebody to ask)

Crazy Backup Plan using Crashplan

-Original 7/25/2013- 

I don’t have the time left tonight to write this post proper.. so here it is really fast:

  • Hanselman rule of 3
  • 1TB to cloud from WHS Home Server v1 = expensive, no thanks.
  • BuddyBackup hit some problems with scale (possibly) (I tried them first)
  • Crashplan (free) saving the day.

image

Quirks / Details:

  • A Crashplan backup to a USB device, when taken over by sneakernet, can be “Imported” to seed a cross-internet backup.    The caveat is, its not so much an import, as it is a “start using this new location as the place to dump stuff and forget everything you knew before” – which is perfect.
  • Crashplan will not directly back up a network share, but if:
    • You run the Crashplan service as a user who has access to those shares
    • You create a symlink (MKLINK /D) from the hard drive to that share
    • Then CrashPlan can back it up.
    • image
    • image
    • image
  • Crashplan Network bandwidth bottleneck + Schedule to keep relationship with work on a good side.  (And asked for permission first)
  • imageimage
  • Same 3TB Ext HD used for both a WHSv1 Shares backup and a Crashplan Backup.
    • image
  • I had to block WHS from backing up the 3TB Ext HD to prevent needless usage of space
    • imageimage

Its been a fun ride.   Its not 100% synced yet .. but its more than half way there and hasn’t died yet.   Thumbs up

-Update 7/30/2013-

I can confirm that if:

  • you do a backup to a local folder on an external USB hard drive HD1 at \CrashPlan
  • you do an over-the-internet backup to a remote computer that is also saving to a USB hard drive HD2 at \CrashPlan
  • You can then sneakernet HD1 to the remote location, disconnect HD2, attach HD1 as the same drive letter
  • and Crashplan resumes just fine.  Ie, local storage and over the internet remote storage are compatible with each other.
  • Good job on the design, CrashPlan!

However, I did run into a problem at about the 900G mark:

 

The Color Run (Louisville, KY 2013)

It took a while (life) but I finally got the video trimmed and uploaded:

[youtube=http://www.youtube.com/watch?v=NtZyHrOvCrU&w=448&h=252&hd=1]
Ye Ole Caption

 

Yay! I joined one of.. 4000+ results?    Everybody had a video.    And I joined late, so all the people searching for it, have already chosen their favorites and moved on.    Down, Ego, Down..  Pop original motive:  To represent “what I love” about the races I’m running this year.  And give an outlet for my video create/organize/edit urge.

For this video, I had one camera on my forehead, and another on a chest strap / reversed / pointing backwards.  This had some challenges:

  • Chest strap was not made to be worn backwards, and so it rubbed up against my front neck/chest area uncomfortably.
  • Because I could not see where the camera was pointing, and it was at chest level, and the race was mostly populated by women, I got some footage that was, possibly invasive of the person’s space.  This mostly happened at the start.. and at the finish when everybody was throwing color in the air.   I tried to ensure I cut out anything that would be embarrassing, but I wonder if I missed something.   Solution: cut out the backward pointing video as necessary.  
  • Battery life!   Hero 3 chews through it like crazy, compared to the Hero 960.   Observe when the camera got turned off for a bit (not sure why) in the middle of the video.
  • Presentation:  I went the easy way out; I didn’t try to choose front vs back, I just showed the whole thing.  Which actually is accurate – it really is that chaotic there.
  • Stabilization:  Chest moves left/right a lot when I run.   However, that’s easily handled by Warp Stabilizer.. at the cost of 1 hour per 5 minutes of video or so.    On the flip side, Warp Stabilizer against the head-mounted camera didn’t work, the camera view angle moves around too much.

About the race itself:

  • Its not a race.  It’s a casual walk / jog with a bunch of screaming, yodeling, skipping, and general goofing off.
  • Go with goofy friends.  I was in good company.
  • Protect your phone!
  • The color won’t last.  Even if you try to save it with vinegar etc.  So don’t worry about staining stuff too much.  (well, it might give a hint of a shade)
  • There are multiple waves.  I was in Wave 5?   Somebody else I met was in Wave 14, I think they said. 

About the video edit:

  • I tried to keep everybody who waved at the camera in the final edit.
  • Original was 1 hour and 15 minutes of video.  Cut down to 8 minutes.  Yeah baby.

Good fun.  Highly recommend the race to anybody who even considers it .. yes you can walk it!

Backup, Balance, and Cross Apply

My goodness, the weeks have flown.    I haven’t had the time to properly devote to any one geeky project.. but there’s been a bunch of smaller things going on.   I’ll try to sort them from Geeky-est to Meta.

I used Cross Apply for the first time

Its hard to do better than Stack Overflow, so here: http://stackoverflow.com/questions/1139160/when-should-i-use-cross-apply-over-inner-join

My use case was pretty simple:

  • I had a sproc which would return inventory of stocks held.
  • Anybody who had more than 100 stocks,  I was trying to create an option from those stocks (100 stocks = 1 option) (for scale testing purposes)
  • However, to create an option, I needed a valid combination of symbol, expiration date, and strike price (so that web service lookups against the market would return valid data)

The lookup of what is valid is something like SELECT TOP 1 EXPIRATION, STRIKEPRICE from <stuff we already know about> where SYMBOL=@symbol

The cross apply becomes something like this:

INSERT INTO (new thing) 
SELECT (bunch of old things from S, LOOKUP.EXPIRATION, LOOKUP.STRIKEPRICE) 
FROM inventory of stock S
CROSS APPLY (  select top 1 ... ) AS LOOKUP

I had to reformat my W7 box at home

2013-07-12 09_30_12-IGNEW - dev.mycareernetwork.com_4156 - Remote Desktop Connection
Something went haywire, and the computer would freeze every time a video played on a web page.  Even ads.  I tried all kinds of stuff around uninstalling and reinstalling drivers…  SAFE mode worked ok, but coming out of safe mode = kaboom.

So I decided to restore my machine to a previous known good version, thanks to Windows Home Server.

But I forgot that my machine had a RealTek integrated network adaptor, which the WHS restore CD doesn’t know about.. so no connection for restore!   I tried looking for the drivers CD, couldn’t find it in 5 minutes

So I reinstalled Windows 7 from scratch.   And still no Network driver.   I had to go get the drivers on a different machine, and bring ’em over by sneakernet.   At this point, i was committed, so I continued the reformat.

I currently have a robocopy going from my WHS backup to my new working directory.  My incredibly awesome nomenclature is:   “c:\2013“.  That’s where I put my stuff.    If a project spans years, it gets copied from one year to the other.   One folder per project.

Life Balance

  • Family Reunion Road Trip
  • 2nd swim time with Father in Law.. I’m getting better at this treading water stuff.
  • Not much running.. 5 miles here, 0.6 there, 2 more squeezed in
  • Fourth of July = dogs went crazy while we were not home = broke through drywall to get out of basement and tear through the house
  • Dryer motor getting repaired, thanks to a shoe.   And a dryer vent, thanks to a dog (it was next to the drywall above)
  • Crazy dogs led to walking the dogs in the morning.  I am not a morning person.
  • Today is door replacement day, thank you Lowes
  • Back deck is done!  Thank you Steve Ader.  He does good work.
  • 3 birthday parties … ages:  30, 33, 1.. and an upcoming 86.

Looking forward

  • I’m hoping to get my Android + OBDII set back up again.  I want to see how much gasoline I’m spending over a 2D map.
  • I could write about tracking down some performance problems in EF .. mostly around context.Attach() .. and how I replaced it with a single update statement.
  • I could write about my current project, the “feel” of it, as it went from “we don’t trust the system” to “hey, this is working pretty good”.
  • I need a different home backup solution.    If I upgrade one more hard drive, my WHS will be at reasonable-capacity, and honestly I just use it as a NAS.  I have a 1 year old off-site backup, which is not automated enough for me.    I have only used WHS’s restore the way it was meant ONCE – when my laptop got unstable, and i restored it to itself — and that was a beautiful thing — but once in 6 years?   Compared to 7 or 8 reformat’s (changing OS’s usually, or new owner for a machine)?   … contemplation is needed.

peace out.

 

Meta: Spread too thin, like a little butter, over too much bread

Derek Sivers wrote a blog post – “Seeking Inspiration?”  – which talks about the ratio of consuming to creating.  My interpretation — Some people consume a lot, don’t create, and end up constipated.   I think what he was actually saying is, the moment of their inspiration is lost, because they don’t turn it into action.  

I’ve been looking at my life lately, and best I can figure:  I’ve been creating too much – in areas of my life that, while they are important (mostly, videos and songs), they don’t apply to my work (security) – and as thus, I’ve had less “bank” available for creating at my day job.

So, while I have a list of projects that I would love to work on:

  • Cam’s last baseball game of the season ..  I have some awesome footage!!!
  • The Color Run 5k – I had a front and back camera through the race.  I suspect people were being goofy behind my back.   I have not reviewed the footage yet.
  • Song for family member #1 – definitely high energy – June 19th
  • Song for family member #2 – them requested is “Motown” – June 21st
  • Song for family member #3 – theme as yet undecided – early July (5th?)
  • Find, and hook up, and create, some DVD’s that have been requested since January
  • Old Family Photos from the 1950’s thru 80’s – interview and create adjoining commentary – create family video

I also have some other things that are important for me to do:

  • Visit mom next weekend
  • Start swimming.  To build core muscles.  I have a weekly date with my father in law and his pool that we need to start taking action on.
  • Get back to being regular on nutrition
  • Get back to being regular on working out & running (superset of #2)

And then there’s the thing that (I feel) has been lost:

  • Be creative and awesome at work, rather than just reacting.

There’s some things that I really don’t need to spend any time on, other than as brain-rechargers:

  • Play Firefall the Game – this usually takes up more time than I intend to allot to it.

So, I have to put my non-work creative efforts on hold, and do maintenance actions for a while.   At least a week, maybe two.   Then phase them back in, one at a time. 

Recent Coding Travels

I haven’t posted much code stuff lately.  Nothing in particular seems blog-worthy, but hey! after CodePaLousa, I’m making an effort to write about what might be mundane.. in a relatively anonymous way.

Story #1: Performance Improvement

Problem: I had 5 people, all doing the same web service call(s), which were slow – sometimes up to 20, 30 seconds to complete.   Except that the system was taking 2 minutes to bring the information back.

Diagnosis: Ants Performance Profiler showed some interesting stuff.

  • Initial call to get the list of A1,B2,C1,D2 … took 5 seconds
  • But a background thread to populate A1a, B2a, C1a, D2a .. took another 30 seconds
  • But there’s a thing, which for any given A, needed the sum of A1a, A2a, A3a even if A2 and A3 were not in the initial result set.  THAT one .. for large screens.. going at a speed of about 6 per second.. took up to 2 minutes.

Solution: 

  • Introduce a service that preemptively did the web service calls.. cached results for up to 5 minutes.  Service runs every 4.5 minutes.    Everybody became much faster, and 5 minutes did not affect their business decisions.     Instead of 2-20 seconds, now those calls finished in 7ms .. IF the cache was filled.   Which the service ensures that it is.  If the service is down, any client can fill the cache as well – they all use the same class.
  • Removed the background thread that got all the extra A stuff… waited until the user clicked a little expando button that showed that information.  THEN go and get it.

Problem introduced by the solution:

  • Now, if you click “Expand All” on a large screen, you get 200 threads all trying to expand at the same time.  It succeeds, but it also thrashes while attempting to resolve.  However, that’s a <1% use case, so we can live with that.

If my client is okay with it, I’d love to post some before and after graphs to show how things changed.

Story #2: Idle Crash

Not on mine or the product manager’s machine, but certain machines on the floor, while using the app – leaving it sitting in the background – it would crash.

Diagnosis:

  • Event log entry #1:  .Net blew up, Yo.
  • Event log entry #2:  System.AggregateException()

Research:

  • in .Net 4.0, if a Task.Factory.StartNew(()=>{}) background thread generates an exception, and nobody reads the exception, … nothing happens.
    • Until the GC runs.  Then the app blows up.
  • Doesn’t happen in .Net 4.5, which is why my machine ran fine.

Solution:

  • It was blowing up in the code that I changed in Story #1. 
  • Added a handler for UnobservedTaskException – to log any other things that might happen with a full stack trace so they could be debugged.
  • It hasn’t shown back since.

Story #3: Iterating on Requirements

Problem:

While working on my normal client, I have another client for whom I (and another guy) are estimating a project.  However, this is a large client, and the scope of the project seemed.. fuzzy.   Huge.    Integrated with a bunch of systems that had not yet been written.  

Solution:

  • Break the project up into Phases.  The client came up with the same idea!  Yes! 
  • Make the first Phase the smallest it could be.  Thus, success on that phase would also clear up architecture problems, and provide a target for other systems to code against.   It would also not be dependent on any other system to come online first.
  • Collect everything into a single document:
    • Questions
    • Decisions
    • Design
    • Features

Almost Everything started off as questions. 

As stuff gets clarified, it moves over to “Decisions”.  

“Design” is where larger things (like schemas, interfaces, etc) can be added. 

“Features” was an interesting idea:

  • It is a one-line-at a time, “if the system has all of these things, then it is complete” list, bulleted and indented.  
  • When you first start working with the project, you won’t know what most of that stuff is.
  • But after you’ve caught up to the “groupthink” on the project, you Will.  It’s a checklist, without content.
  • This is the only part of the document that needs to be “complete”; everything else is just talking points, things in flux.
  • This is what the estimate actually gets built on.

I can’t say this is my idea.  I picked it up from a speaker who talked about his daily process.

 

So, yep, that’s my software engineering excitement of probably the last 3-4 weeks.     If I were asking questions on this blog, I’d ask, what have you been up to?  

Feed a Hungry Software Developer: Panda Express

We got a new fast food restaurant in the area:  Panda Express.   First time visiting.  

  • At noon.
  • With the parking lot full, the queue of people stretched back to exactly the door, and there were one or two open spots left in the dining room.  Ie, they know their flow and sized appropriately.
  • Time from joining the queue till getting to the counter:  about 11 minutes.
  • Food Cost:  2 entries + white rice + 2 eggrolls + water (no drink) ~= $10.  So, not inexpensive.
  • Food Quality:  very colorful; I’d put it higher than your average mom+pop Chinese restaurant, below PF Chang’s / Asia Moon.
  • They have brewed ice tea, which is important for my wife.  I forgot to check what kind of ice they have (cubes or nuggets)
  • Their fortune cookie doesn’t try to teach me Chinese.
image image

image

 image