SpiritFarer – Spreadsheet

Loving the game SpritFarer. Resource gathering, with a cute little story line, until you read the back story, and then who cut the onions.

Playing it my way — i got too many people on the boat, restarted, and now I’m separating out “gathering” from “action”. Using a spreadsheet to guide me on what I need to get more of.

View only link to spreadsheet is here – as I satisfy things i cut and move them down. I thought about writing a helper app, but i don’t have that kind of free time.

Speaking of free time.. this is the first evening in i-don’t-know-when that i have had an hour or two to myself, AND i’m not exhausted and just need to veg out. Hence the post.

If i had all kinds of time, i could post about these technical things:

  • Powershell script to automate connecting to cisco vpn
  • Powershell script that runs a query against a database then checks if matching images exist and flags the ones that don’t.
  • Surprise that you can’t connect to a Azure File System from Windows Server 2008R2 because SMB 2.0 vs 3.0. Even more reason to upgrade those servers. The plan is to move those to the cloud.
  • The world of Epicor P21.
  • Fun times working with a company called Heraflux and its chief geek David Klee. That guy knows his stuff, he’s the next level of whatever perfmon skills I used to have.
  • Designing a task-tracking system – we’re separating out “popcorn” from “projects”. Sounds familiar.
  • What happens when a SQL Server doesn’t have enough RAM: the 88G vs 8G typo problem. 🙂
  • Various small problem – small tool/feature wins. But that would reveal too much of the business.

I could also post about these other things:

  • Color Changing Disney Pixar Cars .. the magic temperature is between 60 and 90 degrees, I think.
  • Challenging a grandchild to eat – if you finish one of these three things, you can choose another: 1 pretzel, 3 grapes, 1 pepperoni.
  • Outdoor pools might be simpler than I thought.
  • Awesome channel – Perkins builder brothers, YouTube.
  • I have seen Model Y in person and I still want it.

Be well!

on BLM and protests and stuff

Stuff I learned / re-learned. If you’re not familiar with any of these, ask me privately and we can chat. I am by no means an expert, just a student.

  • 1970’s stuff to discredit Black Panthers. I’ve only heard it mentioned, haven’t delved deeply.
  • Today’s social media – much more raw message, much harder to contain. This also means a lot less “marketing buzzword blitz” and you have to work to tune in to the message, or follow somebody you trust to parse it for you. Also means lots of information warfare can jack with the signal.
  • Analogy of “There’s a house burning on the street”. All houses matter. Right now one is burning.
  • Analogy of “All folks are hungry”. And one person has no food.

Some personal historical notes:

As a child in Liberia, there was a different kind of racism. Is it racism? Or does some other word apply? Cultural non-melting-pot? There were native locals, there were the descendants of freed slaves, the African Americans we had heard of but never met, all the white Americans whom we had met, the folks from other African countries neighboring us, the Indians, the Lebanese, etc. All these had different .. relationships? expectations? with each other.

I remember the first time I realized that my race-thoughts were wrong. I was hanging out with Charles “Flip” M. in Washington D.C. I said something about “those Americans” .. and Charles stopped me in my tracks: “We’re American”, and he might have also said Dog. He is Black.

I had never considered it before. The way I was raised, all Americans were White, and everybody else was just lucky if they got allowed in to eat a slice of pie. Possibly very delicious pie. Even now, as a naturalized citizen, that programming is strong inside me – at any point, “Spanish Inquisition” and I’ll be stripped of everything and thrown out on the street. So DON’T BUCK THE SYSTEM BOY or YOU WILL PAY something like that.

But what do I do about it now?

My spiritual path pays special attention to “What do I choose to create” and “Who do I choose to be Now.” So, take a sheet of paper, and make a grid. (Sorry, WordPress styling not so good)

TopicInitial programmingIntermediateCurrent Thoughts/ActionWant to Claim

This is a private exercise to share with someone you trust. But basically, i can put in, for a given topic – say, “who can be American”. What was my initial programming? And in the 2nd to last is .. based on my current actions and thoughts, what do I currently believe in / what “world” am I choosing to create around me. And the last one, that’s between you and God. But its what you would aspire to be.

For example, my second to last column was something like this: “Yes, its all fucked up, and its going to stay that way, because the folks who want to keep it that way have the power to do that and can rewrite narrative.” I kid you not, that’s the negative ass story I was telling myself.

And then people started protesting. And whether those protesting did the boat rocking or not, or others joined in and started throwing rocks at the boat, nevertheless, the boat did rock, and some acts started to get cleaned up. Maybe not all of them, but some did.

Which means .. I was wrong. My 2nd to last column was wrong. Dare I accept a new truth? Dare I write a different next column?

And then if you’re like me and you’re doing this electronically, change the font for all the old ideas and make them strikeout.

This way, when the old ideas come up — and they do, all the time — at least for me, I’ve put the old ideas and the new ideas next to each other, and I’ve revisited my decision of what do i want to create now, and I can gently forgive myself for the old ideas and surrender to God to water the new ones.

Hope that helps.

America is … for the people, by the people?

America is principles before personalities?

Does this relate to the sanctity of the law, putting law above individual people?

American Government is to serve the people, not to strike fear in the people?

American Government is a sham placeholder to police the world’s oil supply for the wealthy and silly things like people’s right don’t matter other than to placate the population?

American people is all the people, not just the wealthy?

American people includes the homeless? Or is it only those who agree to certain norms like street addresses and social security numbers and those who pay taxes?

What is your picture of America?

Covid-19 – why are we so vigilant when seems like nobody has it?

So, there’s this other thing going on – the protests. And restaurants are open. And it feels like there is social pressure .. don’t wear a mask, you look stupid. Maybe just me. Feels like, “If this were a real problem, more people would have it.”

Human feelings are nice, but.. they don’t scale well.

What’s the right percentage of people who should be sick in order for it to “feel” right? Lets say.. 10%.

At 4.4 Million people in KY.. wait, no, that’s assuming even spread. Lets just go with, “In the metropolitan area that I live in, 10%”.

At 600,000 people, that means 60,000 people would be exhibiting symptoms.

According to the most recent numbers (link), 6700 active cases, 500 hospitalizations, 70 in the ICU. So about 7% hospitalized, and 1% ICU. So if 60000 folks where “active”, then 5000 hospitalizations, and 700 in the ICU. We have (link) in the magnitude of 2500 hospital beds across the state of KY and 1500-ish ICU rooms.. but that’s for 4.6 MILLION people. I can’t find the exact numbers for Louisville, but basically:

If “enough” people around us seem sick, then the hospital system would be overwhelmed and we’d be seeing the 10%+ death rates that were seen in other places that got overwhelmed.

So, if we win, it looks like: Nobody around us, nobody we know, gets the virus. And still a ton of people die.

I guess i’ll continue to wear my mask and try not to be a vector, even if 99.9% of the time, i’m not near anybody with the virus, and I’m probably not carrying it either.

Crystal Reports Fun

I haven’t posted, or talked much, about my new work .. basically, I joined a company whose primary job is things, and most work focuses around an ERP, and its all about stabilizing and optimizing flows of information around processes. Edit/Update: I wrote this in, like, February. A lot happened since then.. CoViD19 especially. It is now June.

One of my new skillsets is Crystal Reports. To which my developer peeps say Ewww, usually. Eh, it works. The main thing is, the ERP system that everybody is logged into, has a way to embed Crystal Reports into it. That ERP System (Prophet 21) is also backed by a SQL Server database, and has a decent table structure to it, and a decent user-defined-field add-on strategy to it, making many tasks.. sane.

However, navigating the environment for what actually works has been hard. There were not a log of prior art examples which did things at the architecture I would like, so I’ve had to fumble around. Here’s what does NOT work:

  • Using anything other than a particular driver (I forget which one) for connecting to SQL. Specifically, not to use the SQL Native driver. This is because when the ERP is hosting the report, they do a switcharoo on the connection to connect to the right “instance” of the ERP, and if you use the wrong driver, that doesn’t work. You don’t find out till you try to run the report from the ERP.
  • Directly using a stored procedure to do heavy lifting for a report. You can, and it auto-creates report parameters for you, but those report parameter types lead to less-than-optimal user dialog (think dates plus times instead of dates).
  • Using a parent crystal report to include a subreport to get around the previous thing. Works great for a crosstab, but page headers not so good in a grid style report. However, I am able to bind a parameter from the parent report through a calculated field to plug in to the subreport (and thus the stored procedure).
  • Also, if you have a parent report that only calls sub-reports, and doesn’t actually connect to the database itself, the ERP system doesn’t like that because it cannot find the database connection to override.
  • Not choosing a printer when designing the report. Apparently this affects font choices and Arial looks better than Device CPI 10.

Here’s what does seem to work:

  • I can use Views to encapsulate business logic, example “The View of xxx customers” where xxx is a particular program that customers can enroll in.
  • I can use stored procedures to D.R.Y., for example, the stuff to get the raw data of the number of designer frames sold per customer within a time period.
  • I can call stored procedures from a “command” custom SQL block from Crystal Reports. In that block, I can: declare @t table (…) and insert into @t exec SPROC {?param1} {?param2} to get data from a stored procedure.
    • For example, there are two reports: One is a CrossTab that breaks out customers across brands and # of frames, and another is a detail report. The detail report goes into how many DIFFERENT brands were sold + the number of frames sold — these numbers roll into a formula describing what % back the customer gets, per the rules of the contract(s). Both of these reports use the same stored procedure to get the underlying data.
    • However, using this method, I have not yet been able to go (user input) to (calculated field) to (input into command) to (call stored procedure), yet. On the other hand, I can do a lot of manipulation in T-SQL, so that should be fine.

I’m continuing to learn a lot of things.. next week looks like it will be learning the ways of our EDI interfaces with some bigger customers, like the names you’ll find at a Mall. (I don’t know how much I can talk or not about our customers). (Edit: It is preferred that I do not.)

Side note – We have added datadog to our infrastructure, and monitoring is making our lives better, I think. Separate blog post, but in summary: immediate notify on errors, and notify on lack of success. Except on weekends.

Looking back at this post I wrote 4 months ago.. wow, there’s so much detail I could go into about all the little things that I’ve learned and tweaked. Like some powershell to inspect bad emails in an SMTP dead letter folder. And a powershell to automate connecting my Cisco VPN connection. Messing with Redhat 8.1 re-learining all the unix things including SMB shares. However, .. that’s another post; coming up shortly.

Using Timesheet data from Harvest to create a Force Directed Graph Animation

This is too complicated to try to put into words, so there’s a screencast instead.

Video of the final product:

https://www.youtube.com/watch?v=_TJi9yAm_kM

Video explaining how to do it:

https://youtu.be/DrrA9sd6AAQ

Source: https://github.com/sunnywiz/HarvestToGephi/blob/master/HarvestToGephi/Program.cs

In text form:  C# code to convert a Harvest CSV extract into a node and edges CSV file.  Then in Gephi, import the two files, convert the start/end dates into an interval, and set up the prettyness.  Record a long video with lots of stabilization and then speed it up.

Sorting Watch Later by Video Length

Something I do often – I sort my Watch Later list in Youtube by how long the videos are.  I think its “shortest video to get the quickest entertainment bang for my time” (which is semi sad).

I wanted to automate it.  Found several dead ends:   Youtube doesn’t let you use their App Scripts API to modify the Watch Later list, and the page doesn’t let you run a bookmarklet to load an external js file. 

However, Chrome allows for snippets.. and some folks pointed out that I could use various other softwares (Greasemonkey? another name was given) for running arbitrary javascript on a page.   

Ctrl-Shift-I, Sources, >>, Snippets

https://gist.github.com/sunnywiz/064a4d07f8469074356e73eba5dc1215

That’s my solution for now.   It gets what I need done.  I wish I could have done that more generically.

Thanks to Matt H. for pointing me down the path of Chrome snippets.  That dude is bright.

STAQS

As part of EvolveKY.org, I got to do a table and speak at STAQS – Southern Transportaiton Air Quality Summit – here in Louisville.  Learned a lot of things:

  • Mobile, Moves – models used in calculating air pollution.  I think these are permissible to use instead of sampling when reporting or calculating impacts of a project.
  • CMAQ – its a way of funding things
  • MPO  – Metropolitan Planning Organization.. can span many states and partial counties, example Kentuckiana and Cincinnati and St Louis
  • VW Settlement – learned a lot more about it, the kinds of things that States can fund or not fund.

Leaving this note to myself so I can remember this fun learning time.

Tree of Directions Revisited

image

We are in the process of possibly changing residences, so (during my coding fun times on Wednesday mornings with a friend) I revisited this project.  When I left it, it looked like this:

image

In fact, that was so not-cute that I had tried to go 2D with it instead.   

I revisited it, and went through several stages. 

First, I had to re-do how I called Google Maps – they no longer had a “free” tier that was usable, but they now had a “pay as you go” tier which equated to $5 for 1000 route requests.   No problem.  But, to keep costs low, I added a cache strategy so I didn’t ask for the same thing twice.  (Caveat: I think I forgot to turn off traffic, so different requests at different times were delayed by different amounts).

Then, I tried to create a surface underneath the plot.  I did this using some electrical engineering stuff I once helped somebody with – Here’s the plot + a surface underneath it.  Better, but not the awesome I was hoping:

image

I called that the “Minecraft” look.  (That’s a very rough draft with only 10×10, Its about the same at 50×50 except the squares are smaller). I went for something where I calculated the midpoint and drew a polyhedron from each of the four vertices, it looked a lot better:

image

(The previous were github previews of STL files; this is 3D Model Viewer built in to Windows).  

But, its still to .. Jaggedy.  So, I did a few things: First, I made it so rather than a spiderweb, I did a “ramp” effect (filling in underneath the path), as well as, I trimmed off all the residential streets (<30mph) at the ends of the routes.  This gave me a much better print, which is closer to what I had in mind when I started: something that showed “Which were the best ways to get places”:

image

However, I didn’t realize it, but I had done something even better.  Here’s Just the Ramps:

image

This is what I had been going for!  To heck with the surface print part!  You can see the mountain AND the detail!  This is commit a75a67 at https://github.com/sunnywiz/TreeOfDirection/.

Next direction — where I do the same thing for both addresses, and then figure out a way to do stuff in two colors (ie, two prints, but that join together).    In order to do this, I have to first force the bounding boxes to be exactly the same.

Also other things learned – 3D slicing has come a long way, I don’t need things to be perfectly combined as long as they don’t have holes in them.

pruning promotions in GMail

I finally got around to writing some code – turns out its super easy, there’s something called script.google.com, and you Resources|Advanced Google Resources and add the Gmail resource, and then you can write functions to prune your inbox arbitrarily:

function prunePromotions() { 
  var response = Gmail.Users.Messages.list('me',{'q':'label:promotions', 'includeSpamTrash':false});
  if (response && response.messages && response.messages.length > 0) { 
    // messages seem to come in from most recent to least recent
    Logger.log('inspecting '+response.messages.length+' messages'); 
    var collected = {};
    for (var i=0; i < response.messages.length; i++) { 
      var message = response.messages[i]; 
      if (message.id) { 
        var message = Gmail.Users.Messages.get('me',message.id);           
        if (message && message.payload && message.payload.headers && message.payload.headers.length > 0) { 
          for (var j=0; j < message.payload.headers.length; j++) { 
            var header = message.payload.headers[j]; 
            if (header.name=='From') { 
              var from = header.value; 
              if (!collected[from]) collected[from] = []; 
              collected[from].push(message); 
              break; 
            } // if header is from
          } // for each header
        } // if message.payload.headers
      } // if message.id
    } // every message
    for(var from in collected) {
      if (collected[from].length > 1) { 
        for (var i=1; i < collected[from].length; i++) { 
          var message = collected[from][i];
          Logger.log('trashing '+message.snippet); 
          Gmail.Users.Messages.trash('me',message.id);
        }
      }
    }
  } 
}

Unfortunately, I get a lot of promotions, and this only works on the most recent 100 messages or so, so it might be of limited use in this state.   But its a start.

Learned some IoT today

My head is swimming with new information, so dumping it here in a bit, but first, a meta- of what’s going on.

1.  I often feel “left behind” and “not having time to play with things”.  I decided to try to make it a priority by committing with a friend to spend an hour learning new stuff..  together.  So the idea is we spend half an hour on my project, and half an hour on his project, in a pair-program kind of way.

2. We did the first non-planning session of it today.  Well, it was still a lot of planning.  And what happened is:  the 30 minutes were just enough to get me unblocked and to learn which directions to go.. and then we switched over to his project.

3. I’m looking forward to revisiting this next week.

3.97.  if this works well over time, we might polish it up a bit and open it up to more folks. 

The Details. 

First, His Project:  (topic not revealed, that’s his story)

  • I saw Jupyter for the first time in action.  It was amazing.  I think I’ll use it the next time I need to go spelunk and chart in a RDBMS (instead of query, copy-to-excel, create chart)
  • Revisiting OpenCV for image manipulation and stuff
  • There might be some ML in that project’s future.  That’s up to him, of course.  My job is to provide interest and provide support and be a rubber duck.
  • I hope to learn more about running things in docker containers from him.

imageMy Project – Temperature logging for my A/C, Furnace — what I learned

  • I got an LED to blink and read a photoresistor via a WIFI connection (via particle.io and Tinker)
  • Kinda-sorta why we need resistors going to ground to make circuits work.  At least, I saw that repeated pattern.
  • What a breadboard is, and how it is wired
  • Digital vs Analog ~= 3.3V and ranges, and HI=3.3V.  Some specific pins have better A-to-D converters and can read ranges up to 4096.
  • The temperature sensor I have uses something called a One-Wire protocol, and I have to do a library-add to get the right library to read it.  It involves scanning for devices, multiple devices can be on the same wire.    However, there’s two ways of powering it – parasitic and non-parasitic.
  • It might be that when I plug power into the photon, if the other end is my computer, i might be able to debug print via a serial connection.   That will help with the one-wire scan.
  • Turns out particle.io does all kinds of cloudy-mc-cloud stuff so once i have my number, all i need to do is publish(“temperature”, value) and it heads up to the cloud.   Then if I add the integration to Azure IoT, it goes into Azure where I can do other things with it.  AWS doesn’t seem to have that integration.   There are other folks who do have ways to graph things.  I need to analyze patterns over a 6-month time period, involving the drop or rise in temperature from ambient to heated/cooled. 
  • For one-wire stuff, i need a 4.7k ohm resistor, and i’ll probably need breadboard cables.  I didn’t have them yet.  I ordered them.   Should be coming along with a few more thermistors.
  • Using one-wire, i might be able to have a separate sensor for “ambient temperature” so that i can diff the two easier and just take the delta.
  • I read up on the spec for the “Wiring” language, which is basically C/C++ but scaled down. 
  • I learned there’s a web, a desktop, a CLI, as well as a VsCode development environment.  The vscode one seems to have a debugger.  Don’t think I’ll need that, as long as I can get serial.print() to work.

Side note – I didn’t have LetsEncrypt set up to auto-renew my cert, so the https cert for this blog expired.  I had to go find my blog post to find the link to find the commands to renew it.   Done.