Latest Round of Harvesting Car-Tracks

In the past, I used Android – Torque, and an upload to Dropbox, to gather car-tracks.

I’ve started up that project again – this post focuses on my solution for gathering car-tracks for later processing.

Android-Tablet-Always-On

I got a Samsung Galaxy Note Tablet used, and I’ve stuck it in the glove compartment of my car. Its plugged in to power, but power only runs when the car is turned on. Click on image to zoom.

I use automate-it to run a few rules:

When power goes off, go into airplane mode.
When power comes on, turn off airplane mode, and start MyCarTracks

This seems to work as long as I drive a good amount each day. Then again, when I went to write this blog post, I found the tablet powered completely off – not enough charging? temperamental battery? Looks like not enough charging is the culprit.

It would be awesome to completely shut down the tablet when power is lost, and then power on when power is applied; however, I’d have to jailbreak to get that, and my 15 minutes at attempting to jailbreak, didn’t work, so, meh.

MyCarTracks

I could totally get by with the free version of MyCarTracks. Its an excellent product! It has these features which I use:

Auto-record car tracks – after you reach 6 miles per hour
Auto-stop recording car tracks – when still for 5 minutes.

I can then get access to my tracks via an “Export All” feature, which will let me export to CSV, GPX, or KML. GPX is the winner for me:

<?xml version=”1.0″ encoding=”ISO-8859-1″
standalone=”yes”?>
<?xml-stylesheet type=”text/xsl” href=”details.xsl”?>
<gpx
version=”1.0″
creator=”MyCarTracks”
xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance”
xmlns=”http://www.topografix.com/GPX/1/0″
xmlns:topografix=”http://www.topografix.com/GPX/Private/TopoGrafix/0/1″ xsi:schemaLocation=”http://www.topografix.com/GPX/1/0 http://www.topografix.com/GPX/1/0/gpx.xsd http://www.topografix.com/GPX/Private/TopoGrafix/0/1 http://www.topografix.com/GPX/Private/TopoGrafix/0/1/topografix.xsd”>
<trk>
<name><![CDATA[2015-04-15 19:52]]></name>
<desc><![CDATA[]]></desc>
<number>29</number>
<topografix:color>c0c0c0</topografix:color>
<trkseg>
<trkpt lat=”38.242019″ lon=”-85.72378″>
<ele>132.89999389648438</ele>
<time>2015-04-15T23:52:25Z</time>
</trkpt>
<trkpt lat=”38.241821″ lon=”-85.723613″>
<ele>131.89999389648438</ele>
<time>2015-04-15T23:52:29Z</time>
</trkpt>
<trkpt lat=”38.241649″ lon=”-85.723491″>
<ele>129.1999969482422</ele>
<time>2015-04-15T23:52:32Z</time>
</trkpt>

MyCarTracks.Com

But Wait There’s More!

I went ahead and paid them $16 for a 1-year service for a small fleet, which gives me access to my tracks online (up to 2 years old) for quick viewing. In order to make this happen, I sometimes hook up the tablet to my WIFI and say “synchronize all”. There’s also an option where I can say “sync between 2 and 3 in the morning”, and I configure automate-it to take airplane mode off from 2-3, however, that’s hit or miss.

Once the tracks are loaded up to MyCarTracks.Com, I can browse them on a pretty nice map. (picture at the top of the post).

What’s Next

My intention is to load these GPX’s into a small sql-server database, using Spatial (Points), and then come up with little data sets of “here’s all the tracks that passed through these two points”, etc.

I then want to take those tracks and convert them to a 3-D rendering with Z-axis = time, to compare various paths with each other.

And I want to convert that into a 3-D sculpture. Because, art. My art. Representation, archival – these I love.

But, one thing at a time. I’m always welcome to shelf my projects; I only work on them when they are attracting my soul. Might be a bit before I get there. I do have a start on the gpx-parsing code, though: https://github.com/sunnywiz/cartracks2016

SqlServer on a Ram Drive: Fast or Not?

TL;DR: Don’t bother, its not.

I’m at it again .. writing what I call integration tests, which are effectively database-friendly-data-setup-and-teardown tests. As can be imagined, its definitely much slower than unit testing; however, I love doing it, and there’s a lot of sprocs and other stuff that its really nice to get some tests around. (most of the bugs that led to this investment in time, were in the sprocs).

Since I had the RAM available.. decided to try to have SQLSERVER (Developer Edition) run against databases that were stored in RAM. How does that compare?

The database is about 4G in size.

SqlServer Developer, 10G RAM Drive, In a VM , Tests run by R#

SqlServer Developer, using MDF files against C: in a VM; VM is on SSD; R#

SqlServer Enterprise on VM in Azure (network lag); R#

One test failure was because this server’s copy of the invoice database was not complete; and the test was set to be readonly against this server (local sqlexpress = much happier about dropping and re-inserting records).

I could not ping all the way to the database server, but Client Statistics would indicate a ping of probably 150ms?

SqlServer Enterprise at client location via VPN; R#

This is definitely faster than our azure-hosted SQL – Looking at ping, says it’s a 50ms round trip.

Pinging taacasql01.triaa.local [10.120.0.10] with 32 bytes of data:
Reply from 10.120.0.10: bytes=32 time=50ms TTL=127
Reply from 10.120.0.10: bytes=32 time=44ms TTL=127
Reply from 10.120.0.10: bytes=32 time=45ms TTL=127
Reply from 10.120.0.10: bytes=32 time=45ms TTL=127

SqlServer Enterprise local network (client site); Teamcity

This is almost on-par with local SqlServer. However, I don’t know if the machines are faster.

Summary

Ram-Disk Sql-Server didn’t help. Or maybe its that, even on a regular hard drive, it was able to load everything into RAM, and got quite fast.

Local Sqlserver vs LAN SqlServer were close enough; I’d use one as a substitute for “how would it perform” on the other. Helps that the customer is (probably) running both VM’s on a Hypervisor, and network communication between the two machines is … superfast.

WAN SqlServer was definitely the dumps; however, that’s good for posing artificial limits on ourselves, to make sure we’re not doing too many round trips, etc. Nevertheless, our main cloud sql server is slower than a VPN into our client; that doesn’t seem right. Or, our client is that awesome. It could be the latter.

Not shown above, but if you drill in to some of the tests, you can see the cost of setting up an Entity Framework context the first time. It seemed to take about 8 seconds against my local server. Once set up, subsequent tests were less than a second each. However, the set-up would happen again for every other test fixture – apparently whatever its doing to cache things, got dropped between fixtures. Possible optimization, hang onto it in a static? *food for thought*

Methodology Notes:

I ran each full suite twice. Sometimes three times, till I got what seemed to be a stable number.

I used SoftPerfect for the RAM disk. Its set to sync ramdisk to disk every hour or so. After seeing that it didn’t really improve things, I deleted it.

I drank 3 glasses of Tea, 1 Spicy Hot V8, and ate 3 pieces of chocolate, and 1 cream cheese snack, while running all these tests.

Amazing DIY Backup Camera Lines with Turn-Indicators v2.0

I’ve had two coworkers exclaim that this was an awesome idea, so I figure that makes it blog-worthy.. See if I can get an animated gif of this going .. eh, not that good, but I’ll leave it.

How did I do it?

Step 0.1. Acquire tablet screen protectors. Beautiful Wifeling, who loves hunting for stuff, found them at a dollar store.

Step 0.7. Apply tablet screen protector to backup camera screen. This involved some scissor work. Be amazed at how the polarization is just a little different, when seen through polarized glasses.

Step 1. Find empty abandoned parking lot. Where the parking lines all line up. In my case, its at my work-building.

Step 1.1 Acquire dry erase markers and sharpie.

Step 2. Get the Backup Lines figured out. Line up the car with the wheels touching one line evenly, and then trace onto the screen protector with the dry erase marker. Note that due to lens configuration its not a straight line, its slightly curved. If you’re really good, you could probably lift a fingerprint from this image.

Step 3. Get the hard turn lines figured out

Start out parked in a parking spot.
Pull out, avoiding hitting the imaginary parked cars next to you, and turning the wheel as far as possible when possible, till you’re in the street and lined up.
Trace the outline of the parking spot with dry erase marker.
Test it out by trying to back into the spot. You may have to adjust.
Once you’re sure, then go over it with a sharpie.

Step 4. Profit

I have earned at least 27 self-credits at the good reverse parking jobs I’ve done. Go me. I’ll spend them on some ice cream.

Strength Progress

Very quickly – don’t have time to make this too pretty..

This is an overlay of two charts.

The blue dots comes from a mousotron log.

Import to excel, Fixed width

Formulas (click to zoom) – Hint: Don’t reset the counter, its easier if you don’t.

The lines come from measuring my strength using two household items.

I do a full-hand test, I do a per-finger test averaged; and I do that for both left and right hands.

The set of lines along the top = Left hand, my good hand.

The set of lines along the bottom = Right hand, my recovering hand.

Summary

I think what I’m seeing is, I had a nice bit of recovery, and then I started using my hand more, and so my recovery has slowed down, but its still recovering.

Backup Strategy Revisited: v2016.05

A while ago I wrote about using crashplan as my backup strategy. That didn’t last long. Crashplan (free) eventually stopped syncing. I did not have time to look into it.

Then I listened to Mac Power Users Episode MPU318: Backing Up and I got re-inspired to pick up the gauntlet again.

I recently re-acquired my desktop machine at home (it had become my wife’s in an emergency move when her machine started BSOD’ing), and I decided to try Yet.Another.Backup.Solution; this time going back to what some readers had suggested (see comment section here), using BitTorrent Sync.

So far so good

(click image for larger view)

I keep all active (personal) code bases in their own folder, c:\code,which rely on github, visual studio code, beanstalk, etc – various source code repositories – for their backups.
I keep a Dropbox folder somewhere on all my machines. Dropbox is my main “My Documents” folder, for ad hoc type stuff. But, it does not get stuff related to a project involving media – for example, a video edit from my GoPro. That’s too large for it.

I chose dropbox because I used them first, and I’ve stuck with them so far. I believe I’m at 7.5G, free. There are other services that provide similar…
I do an occasional 1-way copy from dropbox to a spot in my big share. Just in case.

I set up a Windows Storage Space on my big home machine, across the top three drives from my former home server – 2TB, 1.5TB, and 1TB … which combined give me a 2TB 2-way-mirrored array. Any one drive dies, no problem.
I set up BitTorrrentSync Pro (paid) ($40 personal one time) to set up a personal cloud:

1.2TB of data between my home storage space and an external hard drive hooked up to my work computer, over the internet.
Several per-year folders (2016, 2015, 2014, etc) that are much smaller that live on every machine I am productive with. (my convention is /yyyy/projectName for any given project, copy-forward if it goes between years).

I’m doing a lot less video work now, so the sizes are quite manageable. 2014 was the peak, that was 283G. That’s been pruned down and put in the big sync now. Most I’m doing for a year now is 15G, because of a GoPro video.

If I have a project that exceeds the size of one of the laptop drives, I’ll probably break it down into its own little BTSYNC cloud (“2016ProjectName”) and share that amongst the machines that care.

I’m still relying on Dropbox for my phone photo backup, although I am doing a iCloud backup as well.

I am currently not backing up my mom’s computers, because she has 4(!) of her own, with Dropbox set up to keep track of important stuff. Go mom! (I need to check on that soon, verify health).

On Seeding Large BtSync Share

I started out with a single copy of my files. USBHD1:\SHARE
Copied them all onto USBHD2:\SHARE
Walked USBHD2 to the office, bypassing the internet.
Set up a BitTorrentSync folder on USBHD1:\BTSYNC1 (initially empty)
Sync that to USBHD2:\BTSYNC1 (initially empty)
At home, copy USBHD1:\SHARE to USBHD1:\BTSYNC1 – 5 years!!!! (usb 2.0 is not great for disk speed)
At work, copy USBHD2:\SHARE to USBHD2:\BTSYNC1
Within 24 hours everything was synced.
I compared files against the original shares and there were no problems

Now that I’m more familiar with it, I could have just told the work computer, “really, use THIS folder to receive the files, yes, I know it already has stuff in it”, and it would have worked fine.

Moving the data from the USBHD to the StorageSpace was easy:

Copy the data
Stop BtSync
Unplug the Ext Hd
Start BtSync
Tell BtSync where to find the files again.
Profit.

What Did Not Work Well For Me With BtSync

Using my iPhone to browse my big share – Nope, it tried to download the big share to my phone (in selective sync mode) (which does one place holder file for every file) (too many files!)

Selective Sync in general – I did not like the idea of placeholder files. I’d rather have placeholder directories.

In Conclusion

So.. there you have it. I have my mirrored local, I have at least 1 offsite backup, several protocols, and an offline backup in case of ransomware.

We’ll see if its still working fine a month or two from now.

Blog Neglected; Adventures

Hey folks; I’ve had to neglect my blog lately – with my RSI, I’ve been saving keystrokes for work.

The above was put together via code2flow.com – source in smaller thumbnail picture (click to zoom in). I’m really enjoying them for putting together small diagrams.

While I’ve been out, I’ve had some adventures. A Summary of Adventuring:

Adventure: Macbook

I inherited my wife’s 2008 Macbook; upgraded it to max ram and an SSD, and wow that’s a cool machine. I would definitely use a Macbook as my main driver in the future. I even liked the O/S; it made sense once I learned how to think like it. I did have to install VMware Fusion to do some of my work-necessary stuff though. (Macbook has been wiped and returned to wife in exchange for desktop computer)

I did get to learn about installing OS/X El Capitan, and rescue boot disks, and Option-R, and several other things that help me be much more comfortable around a Mac.

I’m also glad that the maximize icon on a mac, now actually goes fullscreen in most apps, instead of the full-vertical crippled thing that it used to do.

Adventure: Upgrading the rest of my laptops

Success with the Macbook lead to me looking at all the hardware I have. Alas – most of it is max spec.

One machine I could up to a 64GSSD, so I did .. it can go from 4G to 8G ram, but we’re using it as a conference room machine, so no bit deal there. Most anybody does with it is browse the web and remote to another machine.

Another machine, I just need to pitch. Something is wrong with it. BSOD’s and stuff during installs, did not run ubuntu! Craziness.

My main work laptop, at Windows 10, is unstable; needs to go back to Windows 7. Samsung Series 7 Chronos. Takes 3-4 minutes to boot. I’ve tried the solutions that people mention, but no luck so far.

Adventure: Offsite Backup

With my desktop back, I’m revisiting using bittorrent sync to set up an offsite backup of my terabyte-size store of stuff. ETA: 13 days to sync at the moment, but I have it throttled; will unthrottle overnight.

Adventure: Adapting to RSI recovery

This includes Dragon Naturally Speaking, as well as some home-brew methods of strength testing. I have a chart. Will post later when chart grows more.

Adventure: Giving a Talk

I gave a 10? 11? minute lightning talk on my RSI discoveries. Link to event, which has a link to the slideshare – http://www.meetup.com/Louisville-DotNet/events/228974003/

That’s it for now. There are so many cool threads going on, but none of them is developed enough to do a good post about it. More later.

My Right Forearm is Toast

I am writing this blog post mostly using a microphone.

I think I’ll try to do this in one shot, clean it up later but keep the original version around, so that the blemishes and problems with voice dictation can be seen.This is not a good sample though, because I am not using a headset: my Turtle Beach microphone has a TRS connector and the Samsung Surface doesn’t have that input.

I am having difficulty using my right hand (lateral epicondylitis) and in an effort to heal I am choosing not to use it for at least 3 weeks. This has led to several discoveries:

Taking things for granted Discoveries:

There are several things built into the muscle memory of my right hand. For example in Visual Studio, Intellisense. When a suggestion shows, to choose it is apparently down arrow enter, but when my left hand tried, it tried pressing enter and did not know about the down arrow involved.

I discovered that my brain would output code at the same speed that my hands could put it in. (Perhaps my brain puts it out faster than my hands can put it in and that is why I am having a problem). Now down to typing with one hand, I simply cannot keep up! I get frustrated, and the thoughts that were in my brain disappear leaving me wondering what I was typing.

I have to reevaluate how I am going to be effective at coding over the next few weeks.

Directions and Challenges

I am going to see a doctor whom I trust and see where she thinks I should go. I am hoping that rest will let my body self heal, but if there are major tears in the tendon etc that might need a different level of attention. I am very reluctant to go there. I have heard that once you invade, complications arise. Bottom line: I need diagnostic stuff, x-ray or something, to determine what the state of things are under the skin.

I have been trying to find resources online that state exactly what the mechanism is for ice and compression leading to healing. I see a lot of sources that indicate that inflammation in the body’s attempt to recover in which case how can I get more inflammation? 🙂 (A: Graston Method is one.)

I have another problem as well: by using my left hand more now, it too is in danger of burning out. Friday I think my active left arm hurt more than my slinged right arm.

I have watched videos of people using various speech synthesis program to do programming. my reaction was: OMG that is so slow! I am hoping that if I do something like draw onto paper first and get the code out quickly, then I will be able to take my time typing it into the computer in a relaxed and optimal way. I wonder if I could hook up a midi keyboard to something that types in code for me. lots and lots of macros.

Something very nice about my current employer colon he has had to go to this kind of pain as well, about two years ago.

Tools:

Here are the various tools and tricks that I have learned:

Typing one handed with a living room style small chicklet keyboard: this minimizes travel distance between keys; however I had a hard time finding a small keyboard which also had function keys. Wife and I found one, will try it on Monday.

I am now much more aware of the muscle tension in my forearm while typing. Normally I would tense up everything and then out would come a burst of keystrokes. I am now trying to be aware of keeping everything relaxed and returning to zero after every keystroke.

Four ways of limiting motion:

1. Tennis elbow brace – bad. cut off a lot of blood circulation and ended up being less than effective, because my problem is along the entire length of the muscle / tendon and not just in the elbow region.

2. Sling: a lot better. the first two days using the sling, my hand definitely went into repair and recovery mode. however, the sling dug into my neck of lot and can be very uncomfortable.

3. Athletic Tape: bind my right hand fingers together to prevent the temptation to use them.

4. Wear a glove on my right hand: this is very effective in stopping me from using it accidentally. However, it is not as healing for my hand as a sling, but it is better on my neck.

I find it very funny that my attempt to say third came out as a turd

Another thing that deserves a metnion is KT tape. I used it for a week or so prior to becoming serious and using a sling. It provides a lot of good support, but at the expense of skin sensitivity, as the support comes from elasticity of the skin. Eventually the skin rebels.

And now I have a cat who is trying to use the microphone as well

Emotion:

So I have this amazing opportunity color to re-evaluate how I work, as well as something that forces me to choose what I do with my time. I am certainly not going to play much Elite dangerous, nor am I going to work on my side project, until this is better – trying to save my hands for generating income. I could probably get around playing American Truck Simulator with my steering wheel and using my left hand just like driving my car. However, there’s only two cities left to visit, I’m probably going to wait for the DLC / Arizona to show up before I play it more.

No, I now have an opportunity to catch up on several books that I have not yet read, and to focus on working out and nutrition and things like that.

However, right now I am sad and just want to say cluck it all and watch TV shows and hide. and that’s okay, it’s just part of Grief. I am not sure I am out of denial yet. Maybe I’m near the pyramids. get it? Denial? the Nile? Ed used to make that joke all the time.

The other disadvantage of a voice dictation is that this laptop keeps thinking I am not doing anything, and shutting off in the middle of a sentence

Wish me well! I am now going to go back through this post and use my left hand to edit it so it is readable. I also had a cat volunteer to sit on my right hand helping me not use it.

Original Text – before lots of editing.

I am writing this blog post mostly using a microphone.

I think I’ll try to do this in one shot and then clean it up later but keep the original version around so that it’s blemishes can be seen. space this is not a good sample though, because I am not using a headset comma because my Turtle Beach microphone has a TRS connector which My Little surface Samsung sing sing thing doesn’t have an input for

I am having difficulty using my right hand insert black muscle Palma and in an effort to heal this hand I am choosing not to use it for at least 3 weeks. this has led to several discoveries hole in the line

There are several things that I take for granted that are built into the muscle memory of my right hand for example in Visual Studio Computing intelligence is apparently down arrow enter, but I didn’t. Know that my right hand you that when my left hand try to do the same thing it tried just pressing enter and did not know about the down arrow involved

Other thing I discovered is that for many years, my brain would output code at the same speed that my hands could put it in. perhaps my brain puts it out faster than my hands and put it in and that is why I am having a problem. Now down to typing with one hand, I simply cannot keep up the mycohl I get frustrated, and the thoughts that were in my brain disappear leaving me wondering what I was typing. I have to reevaluate how I am going to be effective at coding over the next few weeks.

My solution plan is currently very simple: I am going to see a doctor whom I trust and see where she thinks I should go. I am hoping that rest will let my body curious, but if there are major tears in the tendon or something like that color that might need a different level of attention. I am very reluctant to go there. Center Center

I have another problem as well: by using my left hand more now, it too is in danger of burning out.

I have watched videos of people using various speech synthesis program to do programming. my reaction was color oh my God that is so slow exclamation I am hoping that if I do something like hold onto paper first and get the pot out quickly, then I will be able to take my time putting the code in to the computer in a relaxed and optimal way.

Something very nice about my current employer colon he has had to go to this kind of pain as well, about two years ago

Here are the various tools and tricks that I have learned colon um

Typing one handed with a living room style small chiclet keyboard color this minimizes travel distance between keys semicolon however I had a hard time finding a small keyboard which also had function keys

At first using a tennis elbow brace, however. Cut off a lot of blood circulation and ended up being less than effective, because my problem is along the entire length of the muscle / tendon and not just in the elbow region

Second attempt is using a sling, and that has worked a lot better. the first two days using the sling Kama my hand definitely went into repair and recovery mode. however, the slang dig into my neck of lot and can be very uncomfortable align

Turd was to use either some athletic To find my right hand fingers together to prevent the temptation to use them, or to wear a large fingerless mittens on my right hand colon this is very effective in stopping me from using it accidentally. However, it is not as healing for my hand as a link, but it is better on my neck

I find it very funny that my attempt to say third came out as a turd

I have been trying to find resources online that state exactly what the mechanism is for ice and swelling leading to Healing. I see a lot of sources that indicate that inflammation in the body’s attempt to recover in which case how can I get more information? 🙂 New York

And now I have a cat who is trying to use the microphone as well

So I have this amazing opportunity color to re-evaluate how I work, as well as something that forces me to choose what I do with my time. I am certainly not going to play much Elite dangerous, nor am I going to work on my side project until this is better Kama trying to save my hands for generating income. I could probably get around playing American Truck Simulator with my steering wheel and using my left hand just like driving my car. Uline no, I now have an opportunity to catch up on several books that I have not yet read, and to focus on working out and nutrition and things like that.

However, right now I I am sad and Hi just want to say cluck it all and watch TV shows and hide. and that’s okay, it’s just part of Greece. I am not sure I am out of denial yet. Maybe I’m near the pyramids. get it? Denial? the space and I on? The Nile River? Ed used to make that joke all the time.

The other disadvantage of a voice dictation is that this laptop keeps thinking I am not doing anything, and shutting off in the middle of a sentence

Wish me well! I am now going to go back through this post and use my left hand to edit it so it is readable.

dotnetmud: spacemud: optimizing network traffic

I spent some time getting the network load produced by the game down. The rest of this post, I test out the changes to see how much better everything became.

Methodology

Two ships connected; I’m going to leave their starting locations as random. I’m going to have them turn in circles and constantly fire. This will yield some explosions against the planet, and others against each other, but most of the missiles will be flying out into space.

Running against a deployed web server in the cloud

Using WIFI and my home internet connection.

Collecting data via portal.azure.com’s App Service Monitoring graph set to minutes. (This didn’t work, I had to go with my local wifi connection)

https://github.com/sunnywiz/dotnetmud2015/tree/chatty1

This is where we started at. Sample packet and network load:

https://github.com/sunnywiz/dotnetmud2015/tree/chatty2

The main change was to give custom JsonProperty names, as well as to reduce the number of decimal places being transmitted.

I use Decimal instead of Double because in double’s, there’s a chance that the underlying representation isn’t quite so digit friendly; decimals are exact for every digit. However, decimals are harder on the processor for doing math – supposedly. I haven’t tested that.

The result: Not too much better. 5.5 MB came down to 3.1MB on the receive side.

https://github.com/sunnywiz/dotnetmud2015/tree/chatty4

chatty3, which I’m skipping, changed the method signatures to two-character method names. Not too much saved there.

chatty4 added “blanking out” data that doesn’t change much. It does this by keeping track of what it believes the client thinks the state is, and doing a diff:

In order to pull this off, I had to go to a dictionary of objects to render, rather than an array. I also kept a “full” frame every 10 frames, similar to MPEG encoding G vs I frames.

I thought I’d get fancy and send nulls if various values (DX, DY) had not changed (for example, they do not change for bullets) – however, JSON sends a “:null” which can be longer than just sending the value. The more advanced version of “don’t include the property if it hasn’t changed” is possible, but it got too complicated, so I ignored that for now. So this version only does the image and name attributes, but that’s enough to get a nice reduction in size:

The savings: another 30%. Its still a lot of data. Not quite the gains I was hoping for.

At this point, an average entity on the screen is taking 100 bytes or so, instead of the 500 they were before.

Network Optimization: Where to go from here?

I’m going to call this good enough for now. The directions to go from here for network optimization:

Instead of using the system JSON serialization, write my own serializer. This could do things like “If It hasn’t changed, don’t send it”.
- so instead of: {“DR”:0.0,”DX”:17.109,”DY”:299.512,”ID”:0,”IM”:null,”N”:null,”R”:446.7,”RA”:3.0,”X”:37.0,”Y”:1474.8}
- “dx17.109dy299.512id341r446.7x37y1474.8” that’s about half the size. But needs a lot more code to massage it.
- Could use a binary packing method as well. Quick search doesn’t find anything that is happy in both C# and Javascript.
Put some logic in as to whether something needs to be sent every time or not. For example,
- bullets pretty much go in a straight line once fired. They don’t need to be sent every time. In fact, if the current X,Y is approximately where the previous update X,Y + DX,DY would put it, then there’s no need to send those values.
- Planets currently don’t move. (That will change). This is a broader case of “nothing has changed in this object from what the client expects, so just acknowledge the object still exists, nothing more”.

What’s next?

I think that authorization / identity of clients (and dealing with disconnection better) is in order.

Then I can do the “single person logged in, multiple clients” code

And then we can launch a spaceship from the text game.

And then we can design an actual playable game. (oooo!)

dotnetmud: spacegame: Timing Loops 1: Client / Server timing loop

I feel stalled in my project. It might be that I need to write about the important stuff done so far before I continue to redo/optimize the game; setting the stage to understand the need for some of those optimizations.

Reference code for this post (tag): https://github.com/sunnywiz/dotnetmud2015/tree/blog20160302.

The playable game (which might be updated, not frozen to this post) is: http://dotnetmud.azurewebsites.net/Home/SpaceClient

Game state at this time: Multiplayer, can shoot missiles, missiles hit things, score is kept.

There are 3 interacting timing loops:

Client / Server Update Loop

I made the client “in charge” of this. Starting in Javascript, the client requests an update from the server:

Along with the request for game data, the client is also sending the state of controls for the user. I went with “how many ms has the thrust key been pressed” as my input – there’s a separate set of handlers which watch key-up and key-down events and long how long keys have been pressed for.

This then travels up through Signal/R, and shows up at the server 20-50ms (an eternity) later:

The SIgnal/R side of the hub takes the connection, looks up which server-game-object that connection is tied to, and asks that game object (which happens to be a Ship.cs) to craft what it thinks the world looks like.

The Ship has several things it has to do.

First is to look at the keypresses and turn those into thrust amounts. I went with a % of thrust:

There is similar code for “can I fire another missile yet” (not pictured).

Then, it makes a copy of all the stuff that the client needs to draw the world:

It records itself, all the other 2D objects in space, some scores, and what the current “Server Time” is (and rate of change of server time, which is a YAGNIY for when the server gets overloaded and has to go into bullet time).

It packages all that into a return (of type object, which gets JSON-serialized), and SpaceHub happily sends it back to the client:

Then, 25-50ms later (another eternity), the client receives the game-update:

Green – the client updates what it knows about what’s going on on the server. I tried to keep this in its own “object space”, to indicate that the server is the authority here.
Blue – the client updates various things that are of interest to the client
Purple – there’s some bulk copying going on that isn’t very sexy and is very network-inefficient.
Red – and the client immediately requests another packet.

Places to Improve: Network Traffic.

The server sends ALL the data, ALL the time. A network packet looks like this (raw, pretty printed):

There is a LOT of stuff in there that is needed once (like image URL’s), and then does not change. Also, many property names, while nice for humans, is not optimal for network traffic (words like “ClientREquestsPollFromServer”) for example.

“No Big Deal?” — Doing a test, holding down the fire button and shooting the planet, for a few minutes, in Azure Data:

Doing some math:

If I have a successful game, and there’s about 1 combat going all the time, we’re looking at $18 per month.

The solution for this is to only send diffs across the network. That’s on my list for pretty soon. But it will require some hand-crafting detail. And some before/after testing. This will be its own blog post.

Places to Improve: Network Lag

This model is similar to how Kermit used to work – there is a LOT of dead time between updates.

One way to improve this might be to have the client keep track of the lag between server updates, and then, in its request to the server, ask “please give me additional packets at the following times, until you hear from me again”. So, the server could send 2 game-updates for every 1 client-inputs-update. Its on my list, sometime after sending diffs. Have to be careful and mark the intermediate frames as intermediate, so that only the major frames cause the cycle to continue.

Another direction we could go is completely decouple the client-input from the server-response. I’m not going to do that, because I like how the client has to constantly remind the server of its existence – if the client goes haywire, I don’t need to write additional server code to detect that and shut down the server sending stuff to the client. I’m also lazy. Smile

In Conclusion

So, there’s the ugliest timing loop in the game so far. The other two are relatively easy, may not get written about soon.

dotNetMud: Its Alive! (on azure)

The link: http://dotnetmud.azurewebsites.net/

From there you can either get into the old-fashioned text-and-chat mud:

(Valid commands are who, look, shout, say, and east and west)

Or you can get into the space-mud side of things:

(Valid controls are W for thrust, A and D for rotate)

Neither of these are really interesting until you have more than one client (browser, tab) logged in. In the case of the latter, perhaps from different computers, with different network speeds.

Corresponding source code: approximately here (as of these screenshots): https://github.com/sunnywiz/dotnetmud2015/tree/blog20160220

Where to go from here?

I could try to make the space game more fun – adding missiles and lasers. More realistic game. You can’t have a space game without combat.. can you?

I could try to make the space game more single-player interesting – adding gravity, multiple planets. (your mission: try to get into orbit around a planet). This lends itself to “more realistic server side computational load”.

I could try to make the space game more efficient – optimizing network traffic (right now, VERY large messages).

I could try to make both clients work better – by having “session” be the unifying factor, and the text game launches the space game, both in the context of the same player object, somehow. This would probably involve hooking up to a database for persistence of user information as well. This is complicated… this is probably where I should go if I’m focusing on the “sample game” side of things.

I could try to make space more interesting – by voronoi’ing it, so that it can have 10000 or more objects, but its very easy to find the objects that are within, say, 100000 units of you. So that I can do hyperdrive between star systems that you can drop out of and be in the middle of nowhere. Also complicated.. probably comes after multiple planets.

I could try to stress test it – get a bunch of computers attached to it, see how it performs on the azure side. I have it running on a Free web-node at the moment, and its surprisingly happy – 1.9% CPU.