C# code against Gmail

I wrote some code and I FINISHED IT

UPDATE – After writing this blog post, about two hours later, I got the stuff working for it to be considered “DONE”. Pretty happy with it. The rest of this post, though, accurately reflects how I was feeling at the time, so i’m leaving it as written.

I wrote this code, and I am Sad. I’m sad that i got started on this silly project (that i’ve thought about for several years), and I got far enough that I can see the end … and now:

ONE: I’m out of time. I did this while we were on our 1-week-away vacation… it started slowly, but got faster and better. This is probably 6-7 hours in? Spread over 3 evenings? I rediscovered so many things (list later), but .. I have a maximum of 4 more hours left before I return to normal life which has NO room for focus time like this (which is something I could change/prioritize, but I don’t because I am finally prioritizing adequate sleep).

TWO: I’m not done, and I’m already rewriting and improving it. So why finish it in this incarnation? I realized with the approach I’m taking, it deals with “recent” emails pretty good, and it does “discovery” pretty good, but it does not handle “the massive backlog of crap that could be deleted”. Nope, that would be a different approach .. in the above screenshot, i think this would be the “Discovery” tab and then there would be another “Purge-atory” tab for the deep cleanse cycle.

Anyway, what is it? Source is here: https://github.com/sunnywiz/gmail-filter-2024/tree/blog-post-1

  • It uses OAuth to connect to Gmail
  • It downloads the last N days of email and stores just a bit of it in a local cache
  • It looks for senders who send me lots of stuff.
  • It lets me choose to keep only the most recent N things per that sender and delete the rest (from that sender)

Along the way, I got to revisit some technology stuff:

  • I started in Console for .Net 6, switched back .Net Framework 32bit talking to Outlook via interop for just a bit, realized that was also painfully slow, and switched back to Gmail’s nuget package in .Net 6 with WPF. Thank God, at my workplace i’ve been entirely in “Framework” and its nice to see where (what was .Net Core and is not .Net) has gotten.
  • I wrote it somewhat with JetBrains AI Assistant, mostly looking up sample code on how to do stuff like get the list of emails from Gmail, wire up a TwoWay binding, etc.
  • I had to reremember a lot of the WPF stuff.. haven’t worked in that since 2016? 2018? … There was this trick of setting up a Debug Type converter that I had to use to fix a binding, turns out I needed to change the UpdateTrigger to be on property change rather than loose focus, everything was ok otherwise.
  • I remembered a trick of using a FlowDocument to lay out controls quickly in a way that looks okay-ish and is usable, rather than trying to get all those !@#!@# grid columns right.

So, for 6 hours spread over 3 days.. not too bad. Not 100% of where I wanted to get to. But i have a few more hours left yet. At half an hour per “session”, I might be able to get these things done – by the time you read this, this drama will be over:

  • I need to remember what the settings are for different from: email addresses, and save them, so i can auto-apply them later
  • I need a hands-off mode where it spins up, grabs email, and purges stuff.
  • I need a “keep 2 weeks” kind of thing for stuff like Paypal and Venmo transactions, and Patreon posts. I usually check on those things within 2 weeks.

And then there’s the nice-to-have:

  • Show the category that an email is in
  • Show if an email is read or unread
  • Option for “Mark all as Read” – stop cluttering my inbox notification counter!!!!
  • As mentioned above, the “Deep Clean” cycle – where it applies the rules to EVERYTHING from those senders – individual searches per sender, only get the details on the messages to keep, and everything else, i can send the messageId’s to the Trash() method.

Or, I could eat that Cinnamon Roll that I got and watch Youtube and call it done for the weekend.

Making Educational Videos for Code Louisville

I volunteered to be a mentor for the latest session of CodeLouisville, on the .Net side.

No big deal, I thought.. and then, TreeHouse was unable to get their Entity Framework videos ready on time, so we, the mentors, volunteered to pick up the slack.

Naturally, I don’t do well when given an open ended problem, so I went into crazy-detail-mode.    I paid $15 for a 1-year subscription to https://screencast-o-matic.com/home  (VERY GOOD), and I started making videos.

And I ran out of steam.  Because, of course, I chose the slow, measured, one-thing-at-a-time approach, and I created far too much work for me to do myself.

Luckily, two other mentors suggested that we do X for our mentoring session – so if what I was doing was from top down, they were going bottom up.  Basically, show the fast way to create an MVC website, Code First, Scaffold, and let the students backtrack from there.

This worked VERY well.   It boot-kicked the students into effective land.  At least one of them showed me some working code in their project in the same week.

Lessons Learned

I am too. detail-oriented for my own good.

When in doubt, ask people what they need.

Sometimes, you just have to show the finished product and NOT explain everything, then go back and explain what is needed.   Its okay, people are resilient, they will survive Smile.

I like screencast-creation, but not more than a few hours a week: For a 20 minute screencast, it takes about 40 minutes of recording, about 2 hours of editing, another 20 minutes of rendering and uploading.   So basically plan on 4-5 hours for a 20 minute video on a topic.    Thus, I can probably sustain generating 10 minutes of educational content a week.

What Did I make

This is a walkthrough of LocalDB:   https://docs.google.com/document/d/1tTDKTQsfGPr_nFSbEifGQRc8dc-FfrxbPUfnc7CyKoc/edit?usp=sharing

This is a walkthrough of C# talking to SQL:  https://docs.google.com/document/d/1lSt-C5-L3VwLLGE6oJO3A_UOpIu2lYA-EWMhzAo-Ye0/edit?usp=sharing – it has links to 5 videos, and (eventually) links to code in github.

Candy Crush Helper

I was listening to Tim Ferris interview Jane McGonigal on the neuro-programming that occurs around playing games.  One of the games mentioned was Candy Crush Saga.  Curious, I decided to start playing it – and now I find it relaxing, a good place to put my mind when my mind is obsessing about other things.

But of course, being me, I started to wonder, “can I program a solution for this?”    Well, of course not – the boards are actually random – but I could write a helper to detect available moves, and possibly future moves.

Turns out a guy already wrote one.  See his video explaining how it works here.   However, I couldn’t get it to run, and it required me running candy crush on my laptop.  And, it didn’t look ahead moves – just the currently available ones.  Still, an impressive piece of work. 

2015-09-12 22.36.39So I started coding one for fun.   I started with this screen:

Question:  How many possible moves do you see?  (A move for the uninitiated, is swapping adjacent candies so they form at least 3 in a row or 3 in a column).  I counted 11 on my initial count.

I wrote some code to load this into a class, and wrote some methods to load candies.   It was a very zen-kata-exercise thing to figure out how to save the board – turns out I didn’t know 2D arrays in C# very well at all (and I still don’t, I chose a different method): image
I wrote a some “possible candy swap” code, and wrote some stuff to find solutions, and wrapped that in a test: image
It says there are 14 moves. What?  I don’t have a clean way of outputting the moves yet, so I will have to highlight them by hand. image (89 ms)

Here’s what it found: 


  • Yeah. Bug in my code.  I thought I was doing a copy from one board to another, but it was a shallow copy, so all the “possible” moves were being remembered one after the other, rather than one at a time.  PANDA-MONIUM!

My attempted fix of the code didn’t work.   And its getting late.

I believe the test needs to Assert(13, allPossibleMoves.Count):


Or am I missing something?   I will resume this.. some day. 

In the mean time, it raises interesting things like:

  • How to represent a very complicated system in a way that makes sense while developing?
  • How to load test data into complicated systems?

Happy crushes.  I’m at level 35, FWIW.

Oh, and I’m using the test-adapter to run NUnit Tests using Microsoft’s test-running tools.

Thoughts about being a Tutor – 2015

A while ago, I was a mentor for CodeLouisville.    It was fun, but it was also Front-End design – something I needed to learn on the fly to keep up with the coursework that I was mentoring.  I met a lot of folks who were excited about learning.

There was some criticism of the program, though.  One such criticism was, Why aren’t you teaching .Net?  Louisville has a lot of .Net work available.  The answer, for CodeLouisville, was a lack of teaching resources.  They are reliant on “free” resources (as in, LFPL paid TreeHouse and thus it was now free for Louisville residents and workers), and at that point in time TreeHouse did not have any .Net curriculum.  (They have since hired somebody and they are working on it).

This annoyed me, and so I (along with some other folks) tried to design a .Net curriculum.  I think the design was good, but the problem we kept hitting was the lack of consistent free curriculum.   And under the umbrella of CodeLouisville, I couldn’t venture into paid curricula like Pluralsight, for example.

The other thing holding me back was, adjusting to the speed of a class of people.   Some folks go fast, some folks go slow.   How to enable success?

And finally, there is the conversion of time to money.   I used to not have any free time, but thanks to kids getting older, I’m finding myself able to reclaim an evening or two a week.  I have my own projects I’m super-interested in, however, I’m also interested in having spending cash to buy coffee and gadgets with. 

So here’s the pitch:

  • Hire myself out as a private .Net / C# Tutor.
  • I get to help a person 1-on-1 in the direction / goal of their choice.
  • I get spending money.  Cash, Starbucks Gift Cards, or Amazon Gift Cards all work for me.  

How much $ do I charge?

  • For folks with hardship, I don’t know yet.  
    • Definitely won’t be enough to replace my day job Smile
    • Maybe the first time around, its on a pay-as-you-can basis as I get the kinks ironed out of the process? 
      • I want to help people get better, so I’d discount myself for folks in hard spots.
  • In cases of non-hardship, I would not discount myself.  For example:
    • Employers hiring me to train-up their employee
    • Parents hiring me to tutor their high-school kid.    
    • I’m thinking $50/hour for these instances, that feels right.  
      • I reserve the right to change my mind before or after a session;
      • I will not change my rate during a 2-month session.

What’s the curriculum and method of study?

  • Depends on where the student is at, and where they want to get to.
  • I would steer the student towards Pluralsight.  Its got the best content that I know of, as a learning tool.  I would, in fact, discount the cost of pluralsight from my tutoring fee.  But if that’s not how they learn, we can find other ways for them to collect information.
    • There is a TON of content out there, but the quality varies.
  • I would guide the student into picking a personal project that is interesting to them, that is not too hard, yet not completely ridiculously easy.
    • The personal project must involve at least two related sets of data.  We need some parent/child complexity to be real-world complex.
  • We would meet once a week for 1-2 hours, and pair-program (or side-by-side program) our way towards completing that project.

.Net is Huge, what specifically would I cover?

  • I’m an old fogey, so I’d start the student off with a console application.
  • Get them to put data in a database.  Start of with SQL Express.
  • Then either go with a Windows/WPF version, or a WebForms/MVC version, of their app.
  • We can also visit other topics like web services, javascript/jQuery/Ajax, SSRS, stored procedures, optimizing SQL, profiling, EF, etc.
    • I am intentionally wanting to take the student the “hard way” through doing some of the stuff (the way it used to be done) before we switch to the easier way which hides the complexity.  For example, SqlConnection before EntityFramework.
  • If desired, we can visit “advanced” topics like:
    • Unit testing, IoT/Dependency Injection
    • Integration Testing, Automation best practices
    • Scale testing (I’d need to refresh myself on this)
    • Waterfall vs Agile vs Scrum vs …
    • Art of Software Estimation
    • I suspect these are more appropriate if I’m hired by an employer to up-train an employee.

What I cannot do:

  • I am not a UI / CSS / Make it Look Pretty person.    I would not dare teach best practices around that.
    • I am, however, good at detecting UX problems (usability). 
  • I am not a mobile-dev person at this time.
  • I cannot teach an uninterested person.  You have to want to learn, be curious.
  • I cannot guarantee employment if you are un-employed. 
    • However,  if you show me the job description your are interested in, or if you are an employer, tell me what you want your candidate to be like, I can teach towards achieving those skills.

Am I a good instructor?

  • Rider Rodriguez, who ran CodeLouisville, thinks I am.
  • Most of my students during my single stint at CodeLouisville as a mentor think I am.
  • I have a very good intuition / radar: I can detect where people are at in their understanding of something and then guide them to getting new things understood.  
  • I’ll hazard a Yes.
  • If I’m not, I’m open to feedback, and I can learn from feedback.

Does this take away from CodeLouisville?

  • First, CodeLouisville does not currently cover .Net programming.  So, no, I don’t think so.  If a person really wants to learn .Net, and thus spends their time with me instead of them, then who is being served?
  • Also, I’m doing this one-on-one.  If I were teaching a class, then that might be different, but I’m really trying to focus on making one person shine at a time.
  • If CodeLouisville does pick up .Net, I definitely would not want to detract from them getting students and/or compete with them.   I will visit that thought in the future when it becomes reality.


What do you think?   Is this feasible?  Do You know somebody who would want to hire me in this manner?   Find me on twitter at @sunjeevgulati if you are interested.  

Currently available for May/June/July 2015.

Code for doing Explosions

Playing FortressCraft Evolved, I played with explosives.  I was not particularly happy with how the explosives were handled – and I was on vacation, and I had no internet access, so my creativity got into my brain and I tried writing code that would handle explosions better.

I shoved the code here:  https://github.com/sunnywiz/explosion2d

Here’s the basics of what I did:

  • I did this in 2D instead of 3D because easier to Console.WriteLine()
  • I created a TwoDSpace<T> so I could store an arbitrarily large number of T’s in a 2-D plane
  • I created a function which mapped out, given a center 0,0, what the distances were to each X,Y. 
    • I kept distances as squares, so I didn’t have to do square roots.
    • I did it with an initial max-radius, but the final version in my head would jit-figure out what it needs and save the resulting calculations in a static – it only needs to calculate it once.  Bigger explosion might need to calculate some more.  
  • That function can then call to a lambda, giving it:
    • the X,Y of the point that is being visited by the explosion
    • an array of parent points for the explosion and the ratios by which to weight them
  • The caller then says, “circle visit an explosion starting from here”, and keeps track of the force per X,Y that is being experienced.  The caller is responsible for all the code that says “Ores are hard, but dirt is soft”, for example, or “Diamond cannot be damaged by the explosion”
  • I think its pretty fast. O(r^2), which given its a 2D explosion in 2D, is expected. 

Fun Fun stuff.   I miss the days of mud coding.   You just don’t need to do explosions like these in real-world paid programming.

OpenCV: Or Not

I was hoping this would be a blog post about some wonderful stuff I was doing in OpenCV about detecting the location of soccer balls in video.. for the purpose of writing a robot that controls the Pan-Tilt-Zoom of the video camera to automatically follow the action in a soccer game so that I don’t have to (Lazy)

However, the program keeps hanging.

I started commenting out code to figure out what I was doing wrong.

Here’s the final version that hung:

using OpenCvSharp;
using OpenCvSharp.CPlusPlus;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace OpenCvTest
    class Program
        static void Main(string[] args)
                //using (var cap = CvCapture.FromFile(@"00026.MTS"))
                //using (var cvWindow = new CvWindow("output"))
                //    var size = new Size(9, 9);
                //    while (CvWindow.WaitKey(10) < 0)
                //    {
                //        using (IplImage imgSrc = cap.QueryFrame()) 
                //        using (IplImage imgSmall = new IplImage(new Size(640, 480), BitDepth.U8, 1))
                //        using (IplImage imgGray = new IplImage(imgSmall.Size, BitDepth.U8, 1)) 
                //        {
                //            cvWindow.ShowImage(imgSrc);
                //            Cv.Resize(imgSrc, imgSmall);
                //            Cv.CvtColor(imgSmall, imgGray, ColorConversion.BgrToGray);
                //            Cv.Smooth(imgGray, imgGray, SmoothType.Gaussian, 9);
                //            using (CvMemStorage storage = new CvMemStorage())
                //            {
                //                CvSeq<CvCircleSegment> seq = imgGray.HoughCircles(storage, HoughCirclesMethod.Gradient, 1, 100, 150, 55, 0, 0);
                //                foreach (CvCircleSegment item in seq)
                //                {
                //                    imgGray.Circle(item.Center, (int)item.Radius, CvColor.Red, 3);
                //                }
                //                cvWindow.ShowImage(imgGray);
                //            } 
                //        }
                //    }
            catch (Exception ex)

  • If you look closely, you’ll realize that all the lines are commented out.    And it still hung.  
  • 1 Reboot later:   Nope, still hanging.
  • Remove references to OpenCV:   Still Hangs.
  • Turns out if I run it without the debugger, its okay, but running it under the debugger, it hangs.

Well, La De Derp.   I don’t like it when things like Debugging don’t work.   Makes me .. trust things less.  Granted, this is Visual Studio Express.  But still!    Back to Console.WriteLine debugging for now…

Here’s some links to some code:

What I’ve learned:

  • In the future, I’d probably use another library – EmguCV?   Seems more mature?  
  • I might be able to use a <insert technical terms here> (other forms of detection – FAST something? BURPY something?  I remember not exactly) .. something which can use a single image to train from using 64 dimensional wavelets.  Woo.    Not today.
  • I have not researched PTZ Apis
  • I don’t have a solution for splitting the stream from capture to video and capture to ball detection
  • I’m not sure about the easing functions that would be involved, there’s quite a delay from detection (if I ever do) to movement
  • I’m not sure what to do with the ball kids playing with soccer balls at the sides of the field.

Day at the Office: Naming Anonymous Types

I ran into a problem yesterday.  I’m in the middle of a large routine.  It looks at a set of projected changes and determines, if the changes were applied, what would the errors be.  Most of these errors have to do with balances going out of bounds, so there are several balances rolled up at various levels.

Where It Started

At first, I created several Dictionary<??,decimal> of balances like this:

var balance1 = .... 
 .GroupBy(t=>new { T.ClientId, T.CommmodityId, T.BucketId })  

The Problem

The code grew, exceeding 300 lines of code in one routine.  Time to refactor.  However, when I refactored, it did not know what to name my anonymous type, and I got this:

void PerformFancyMath(Trade t, Dictionary<??,decimal> balances)

More detail and other options here:   http://stackoverflow.com/questions/6624811/how-to-pass-anonymous-types-as-parameters

Not A Problem!  I’ll stop using the anonymous type, and create a helper class for it.

My first attempt was something like this:

public class DooHickey { 
    public int ClientId { get; set; }
    public int CommodityId { get; set; }
    public int BucketId { get; set; }

However, this does not work, because DooHickey does not define Equals() or GetHashCode(), so the dictionary doesn’t know how to match keys together.    

I hate writing those boilerplate bits of code – I like F#’s idea where it defines these for you, same as C# anonymous types.

My Solution

Rather than writing out that code, this is what I did instead:

public class DooHickey : Tuple<int,int,int> { public DooHickey(int a, int b, int c):base(a,b,c) { } /* This bit is not really necessary, but could be done

public int ClientId { get { return base.Item1; } } public int CommodityId { get { return base.Item2; } } public int BucketId { get { return base.Item3; } }

*/  }

  • I didn’t realize I could have a Tuple with 3,4,5 or more constituents.
  • The Tuple handles GetHashCode() and Equal().

Now armed with my helper class, I was able to refactor my code and bring my methods down to under a screenful each. 


Mocking Time at 1000x speed

I had a complicated bit of logic that involved multiple threads, waiting for the right time, checking for time expiration, etc.   I wanted to unit test it, which meant mocking Time.  And not in a simple way, but in a complex, works-across-threads way.

This is what I ended up with:


That’s pretty simple.  The fun stuff happens in the Stub version:


Thus my logic component, which spawns multiple threads, each of which take a section of work to do, and each unit of work has “cannot start before” and “must not end after” – when it runs it against unit tests, it runs it at 100x or 1000x normal speed – but the simulation is accurate across all the threads. 

Having Fun with SSIS

My new project involves getting data from 500+ remote sites into a target database; in parallel, quickly.  Thanks to some proof of concept work done by my coworker Mike “The Devastator” Hurd (I made up the nickname), I got to learn SSIS, and I’m cursing that I had not picked it up earlier. 

What Is SSIS?

  • Imagine you wanted to write a generic program to get data from point A to point B, but modify it along the way.
  • You wanted pluggable components that could talk to each other.
  • You wanted anything that could be done in parallel to be done in parallel.  Ie, read from two tables at the same time if you have bandwidth.
  • You want it to work buffered rather than row by row.  But dynamically figure out buffer sizes so that they fit well to the page size on the computer (to minimize swapping)
  • You want a GUI that you can configure these components with.
  • You want every component to have a configurable “if things fail” state, to redirect failed rows to other components, etc.
  • You want it to be solid – no leaking memory, etc
  • You want to parameterize the stuff getting into the components
  • You want it to be runnable from a command line, with ability to override parameters and connection strings.
  • You want the whole configuration to be saveable as an XML file, or hosted in a SQL Server
  • You want to be able to schedule it to run from SQLAGent (SQL Server’s “Cron” thing)
  • You want to be able to run it directly from C# via managed code.

Then you would write SSIS.

What Does It Look Like?





Initial Gotchas

SSIS requires some level of licensing, so this does NOT work:

  • Install SQLEXPRESS
  • install Visual Studio 2012 Premium
  • Install SSIS add-on
  • Install SSDT (“database tooling for VS2012”)

It successfully loads, and you can debug packages within Visual Studio, but you cannot run them outside via dtexec and dtexecui (the command line thingies for SSIS).  Instead, you need to add:

  • Install SQL Server Developer Edition (which includes SSIS)  (or standard edition)
    • Select Integration Services & Data Tools
    • Do NOT have to install the engine.
    • Might have to install an updated SSDT to get VS2012 support

Then everything runs fine.

The other thing I quickly learned is it is very Schema-dependent.  The reason for this is it allocates row counts into buffer sizes based on the size of the rows – so a Varchar(MAX) field is very different from a Varchar(250) field – and the schema will fail to validate if the underlying data source it is reading from does not match perfectly.

The solution I had to apply was to do a select cast(x as varchar(250)) as x to extract the data to guarantee the schema coming in to SSIS.  


Very glad to have this tool in my arsenal.   Thanks Mike!

Day at the Office: Syncing Systems

I have been on a coding adventure for the last two days. 

Customer has a system, which involves holdings, and then proposed trades on those holdings. However, there’s also a master system that sends a new list of holdings on a daily basis.  The master system does not know about the proposed trades, it says “here’s the new list”.


The algorithm when updating the list of holdings to the new holdings went like this:

  • delete anything that doesn’t have a proposed trade on it
  • copy in the stuff from master, creating new ones, except…
  • if an existing holding is found, instead of creating a new one, update the holding to look like what the master says it should (with various keys to try to grab the right one)

The Problem

There was a holding of Quantity = 20 that had proposed trades of –10 against it.  The incoming holdings were of quantity 3 and 15.   It updated 20 to 3, causing a math imbalance in the system.

Further problem on more investigation

It was also iterating the incoming list, so if I had trades against a holding H1, and H1 (or something like it) was not included in the new list, then the trades would get left behind against a stale holding.

My Solution Take 2

  • Delete any holding without a proposed trade on it
  • Figure out our maximum id so far
  • Always insert all data from master as new.   (Now we have duplicate copies)
  • Walk through all the old holdings and proposed trades; try to match the proposed trades up against the new holdings
    • If no match found, log it and delete the proposed trade
  • Having moved all the proposed stuff over to a new parent, now delete all old holdings <= maximum id (no more duplicates)

Why I thought it was Clever

Suppose the load fails. 

Since I’m only either updating or deleting proposed trades, I will not have any duplication of proposed trades.

Since I’m deleting all holdings prior to where I started inserting the new holdings, as long as I successfully complete a load, I will delete any old bad data from a previously failed load.

The Matching Algorithm

The general idea is:  Sort “available buckets”, sort “Proposed Trades”, and then start matching trades to buckets.

  • imageI tested this out on paper – as in, I cut out little pieces of paper, and tried the different options; I determined sorting low to high on buckets and high to low on trades was my best bet. 
  • I used a fit option of f(x1,y1,x2,y2):decimal to give me, for any trade, which bucket it would best fit in (after sizing was taken care of).  (the X’s and Y’s are acquisition dates and execution prices, both of which can be “touched up” on the master system, making an exact match every time non-feasible)

To Translate This To Code

Because the matching stuff could get complicated, I made its own little command for it (see command-query separation).


I was not shy about creating more DTO’s for the specific inputs and outputs needed by this command. 


I wrote enough tests to verify that it was doing the right thing.  When I do unit tests, Iassume only developers will read them, so I don’t spend a lot of time making the names super-descriptive.  Some-descriptive, yes, not super-descriptive.

Embedded image permalink image

And then I wired it up.   Turns out, I wired it into location #1, and that turned out to not work – transactions, entity framework, new transactions, missing ID’s, etc.   So I ended up wiring it into location #2 (after the fact), and that worked a lot better.   Good thing I had black-boxed its meaning, it was easy surgery.