Updates to twit-sort

I am currently on vacation!  In Florida!  We drove!

I took the opportunity (in the early mornings / late nights when everybody else is asleep) to work on some code that I wanted to update – I don’t normally get time to do this.   Its my twitter-reading app.   Which only I use.  But hey, that’s fine, I haven’t tried to market it.

Changes

Change #1:  I kept seeing “…” with truncated tweets.  A little research led to some interesting stuff — basically following retweets to get down to the original tweet to get the text from there.

Along the way I tried to build up a “who quoted who” breadcrumb:

image

The code is a bit wonky:

image

However, its confusing – many people (or clients) retweet without actually having a retweet link.  There’s “quoting” which is different ..  eh. whatever.  If I’m given it via the API, passing it through, figure it out later.

Change #2: I surfaced the like count and favorite count, as well as a link per tweet to open the tweet in its own window (as hosted by twitter). 

image

Complications

I have the code in github, which is public, and I don’t want my access keys and tokens and stuff checked in there.   So, I had to fudge around a bit – this was my solution:

image

  • I created a branch with all the passwords and stuff, and worked from there.
  • Once it was done, I either did a rebase (with cherry picking) or a direct cherry pick to move commits over to master, which I pushed up.

Granted I could use my azure visual studio hosted git, but I wanted the code to be visible / usable.  So.. if anybody has other ideas of how to do this better, please let me know.

What Next (with this project)

Not much else I can do with this in its current codebase.  I could certainly make it prettier, but .. eh, that’s not me.

If I had unlimited time, I’d rewrite it – make it so it did all its fetching and filtering and sorting locally (in javascript).  I could get a LOT fancier then – things like pulling out hashtags into groups, etc.  Maybe adding some sentiment analysis things.

However, I have so many other projects on the burner.. this one won’t make it for a while.  My need has been solved, so there’s very itch here to scratch.

The url of the site:   http://twit-sort.azurewebsites.net

The code: https://github.com/sunnywiz/twit-sort

Goodbye to old code and dreams of immeasurable wealth

imageIn the Beginning

I would always hear about people who wrote simple pieces of software, who were in the right spot, at the right time, and their stuff got used and they became famous, and.. perhaps even rich. 

Every time I heard such a story, my baby tyrant would say: “I want that!  Lets DO that!”

To which my Fuddy-grownup would say, “Honey, you probably won’t become famous, and it probably won’t work out.  Are you sure?”

And the Couch-Buddy would say, “Ah, too much work.  Lets read some more facebook.”

I Made a Decision

I started coding this thing that I thought would be a good start of things.  It was an app to make reading twitter easier – less context switches.  It was also an experiment in using Azure, Visual Studio Online, and a little bit in starting bootstrap from scratch.

I got it working.

I started the code 7/26/2015, and by 9/22/2015 I was ready for the big time.   This was mostly an hour or two during a workweek in the evenings, and maybe an hour or two on a Sunday morning.

I had a logo created, bought a bootstrap style, I had added what I thought were the key features I needed, I rebranded it, and I bought a domain name.

image  image

I stopped.

And then life got complicated, and I let it sit – costing me monthly $, btw.  $17 per month to keep it hosted at the cheapest level I could get away with, AND have a domain name.  I used it for a while. 

Eventually, I got cheap, and work distributed a full MSDN license to me with an azure subscription, so I nuked it.

I’m letting it go.

Very recently, I put it back online under my MSDN license –  You can use it here:

http://twit-sort.azurewebsites.net/

I’ve cleaned up the code that I deployed, removed all the passwordly bits from it, and uploaded it to github.  Here’s the guts of it:

https://github.com/sunnywiz/twit-sort/blob/master/azuremvcapp1/Controllers/ReadController.cs

Letting go the dreams as well

I would have liked to have seen this thing become better.

  • I could have done a face lift on the front page.  Too many words.  Replace with screenshots of the configuration page and the read page.
  • I could have made it more colorful. Orange and Blue!  You can see this in the icon a bit.
  • I could have made it front-end js only, with no server side talking to twitter, using local-storage for persistence
  • I could have added “click hashtag or username” to create additional groupings on the fly.  delete groupings on the fly as well.

The good news is, all these dreams live on, in a future project – that works with Facebook, instead of Twitter

Conclusion

Letting this one go to make psychic room for other things that interest me.   May it bless others.  If you write a good one like this I’ll use it.

Figuring out Owin Twitter Authentication

I have been working on an Asp.Net MVC project (personal) that needed to do Twitter authentication.  I learned some things implementing it – presenting here so that I don’t forget them, and also in case anybody else runs into the same problems.

Rather than do a bunch of links below, doing it here – this code depends a lot on LinqToTwitter (https://linqtotwitter.codeplex.com/)  to talk to Twitter after getting logged in.   Also, for all the snippets of code – I could have used the “C# colorization” CSS stuff, but I chose not to.  My wordpress is set up a bit weird, and it doesn’t work quite right.

What I started out with

I created an ASP.Net MVC app with the standard starter template, which gave me an Owin layer that did registration, and allowed for adding an external login to the system.

It didn’t work well for me

  • I couldn’t follow the code well – it seemed almost magical how the authentication came back.  For another thing, I needed to grab two of the claims, save them, and use them for getting tweets from Twitter.
    • The “built-in” way to do that with LinqToTwitter relies on Session State.   And it worked well, except that, I was deploying to a “free” class Azure website, which kept recycling, and thus loosing session state.
      • Which lead to an investigation of session state in Azure.  Which led me to a pre-2012 and a post-2012 difference – SqlSession vs Azure In-role cache.  Which lead to a difference of how you tool websites, I had the wrong tooling, didn’t have Roles to add an in-role cache to.
        • All very frustrating.  And trying to get something working “in an hour” sitting at Starbucks before dinner and getting directed all different ways by various well-meaning authors, didn’t help.   Everybody had the perfect solution for their own little universe, but the black boxes they were working with didn’t line up with my black boxes.
  • So, I decided to rip it all out and start over.
  • Spoiler, and then it made sense and it worked.

Best Source of Info

I have to give kudos to @EdCharbeneau. I know him personally, and I was quite surprised to find he had written a blog post about Owin that was the perfect post for me:  https://www.simple-talk.com/dotnet/.net-framework/creating-custom-oauth-middleware-for-mvc-5/.   It cleared up several things that were not obvious to me.

Code from “scratch”

Here’s how the flow works:

// GET: Read
[Authorize]
public async System.Threading.Tasks.Task<actionresult> Index()
{

You want in, but [Authorize] says you should be authorized.     So, do something to authorize the user.

At this point, we have to step back and go to the startup.auth that MVC / Owin likes to do.  I won’t get into the details of IAppBuilder and what Owin is, but here’s the relevant configuration code:

app.UseCookieAuthentication(new CookieAuthenticationOptions {
    AuthenticationType = DefaultAuthenticationTypes.ApplicationCookie,
    LoginPath = new PathString("/Login"),
    ReturnUrlParameter = "redirectUri",
    Provider = new CookieAuthenticationProvider() {

    }
});

We’re telling OWIN that we want to use a cookie to prove that the user is logged in. If the user aint got the cookie, then send them to /Login to go get them the cookie. When sending them to /Login, pass a parameter of ?redirectUri=xxxx to tell Login where to send the user back to.

So the user shows up at /Login. What happens next? LoginController:

public class LoginController : Controller
{
    // GET: Login
    public ActionResult Index(string redirectUri)
    {
        var whenDoneLoggingInUrl = Url.Action("ExternalLoginCallback", "Login", new { ReturnUrl = redirectUri });
        return new ChallengeResult("Twitter", whenDoneLoggingInUrl);
    }
    ...

So the way that Oauth works – every provider, has a slightly different way of initiating the go-log-thyself-in thing. In order to work with everybody – there’s a particular type of response that needs to get sent back. In order to hook how to send the right kind of challenge back, we have to write a custom ChallengeResult where we can execute some code as we’re building the result to send back to the browser. This is almost directly lifted from the MVC boilerplate app that ASP.Net provides:

internal class ChallengeResult : HttpUnauthorizedResult
{
    public ChallengeResult(string provider, string redirectUri)
    { 
        LoginProvider = provider;
        RedirectUri = redirectUri;
    }

    public string LoginProvider { get; set; }
    public string RedirectUri { get; set; }
    // public string UserId { get; set; }

    public override void ExecuteResult(ControllerContext context)
    {
        var properties = new AuthenticationProperties { RedirectUri = RedirectUri };
        // this does the 302 or whatever the login provider wants you to do. 
        context.HttpContext.GetOwinContext().Authentication.Challenge(properties, LoginProvider);
    }
}

Ah, but that LoginProvider — how is there code that knows what to do there? That comes in a bit more of startup.auth.cs code:

var options = new TwitterAuthenticationOptions()
{
    ConsumerKey = ConfigurationManager.AppSettings["TwitterConsumerKey"],
    ConsumerSecret = ConfigurationManager.AppSettings["TwitterConsumerSecret"],
    Provider = new LinqToTwitterAuthenticationProvider()
};
app.UseTwitterAuthentication(options);

TwitterAuthenticationOptions is a microsoft-provided class that does the right challenge to send stuff to Twitter; it also handles when twitter redirects back to our app, and handles getting the claims and all that. Perhaps. The LinqToTwitterAuthenticationProvider might do some of that as well, I haven’t delved too deeply into what that does differently from the microsoft provided one.

There’s one more little bit of code that needs to happen to get things to work:

app.UseExternalSignInCookie(DefaultAuthenticationTypes.ExternalCookie);

This tells the authentication provider how to hold on to information as its doing the login process.

So where does this leave us at? Well, the negotiation with twitter happens; it comes back to the Oauth provider (which we can’t see the code of), the user gets approved, and we pick up some claims from twitter, and everything calls back into the “whenDoneLoggingIn” url we defined above. That leads us to:

public ActionResult ExternalLoginCallback(string returnUrl)
{
    var owinContext = HttpContext.GetOwinContext(); 
    var loginInfo = owinContext.Authentication.GetExternalLoginInfo();
    if (loginInfo == null)
    {
        // did not survive the login process
        return View("LoginFailure");
    }

    // log them in persistently here! 
    // we have to create a new claims identity with authentication type cookie. 
    // i could probably copy the claims over easier... 
    var claims = new List<Claim>();
    foreach (var externalClaim in loginInfo.ExternalIdentity.Claims)
    {
        claims.Add(externalClaim);
    }
    var id = new ClaimsIdentity(claims,
                                DefaultAuthenticationTypes.ApplicationCookie);
    // https://stackoverflow.com/questions/23180896/how-to-remember-the-login-in-mvc5-when-an-external-provider-is-used/23228005#23228005
    owinContext.Authentication.SignIn(
        new AuthenticationProperties
        {
            IsPersistent = true,
            AllowRefresh = true
        }, id);

    return new RedirectResult(returnUrl);
}

Whoo that’s a lot of code. Lets walk it.  Side note:  Every time I find an answer that I use on StackOverflow, I try to attribute it in the code that I generate, and up arrow the answer I used.

First is, the possibility that we didn’t get a successful login from twitter. If so, login failed. Boom. Done.

Otherwise, we have a list of claims about the user. Twitter gives us about 6 claims –

image

I need to copy these claims over to a new list of claims, create a new ClaimsIdentity based on those claims (but with a different AuthenticationType – remember that you can be logged in with 3 identities at the same time), and then log that new identity in; the user gets logged in via CookieAuthentication.

The result of this is a persistent cookie in the browser which is an encrypted version of the 6 claims, plus an expiration time and several other things:

image

And finally, the user is re-directed back to where they started – the Controller which had the [Authorize] attribute that intercepted the request.

This time, when the user requests the page, the cookie is decrypted and unraveled into a list of claims that we know about the user. Then, in the page that was requested, if I want to talk to twitter to get stuff, I can do the following:

var userInfo = _cq.GetLoggedInUser();

using (var twitterCtx = await _cq.GetTwitterContextAsync())
{
    var tweets = _cq.GetHomeTweetsQueryable(twitterCtx).ToList(); 
}

The heavy lifting I’ve put into a CommandsAndQueries class (* controversial; seprate topic) –

public LoggedInUser GetLoggedInUser()
{
    if (_context == null) return null;
    var owinContext = _context.GetOwinContext();
    if (owinContext == null) return null;
    var authman = owinContext.Authentication;
    if (authman == null) return null;
    if (authman.User == null) return null;
    // C#6 will make this prettier. 

    var loggedInUser = new LoggedInUser();
    foreach (var claim in authman.User.Claims)
    {
        // Claims: 
        // http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier = 152333310 
        // http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name = sunjeevgulati 
        // urn:twitter:userid = 152333310 
        // urn:twitter:screenname = sunjeevgulati 
        // TwitterAccessToken =  ; 
        // TwitterAccessTokenSecret = ;
        switch (claim.Type)
        {
            case "urn:twitter:userid": loggedInUser.TwitterId = ulong.Parse(claim.Value); continue;
            case "urn:twitter:screenname": loggedInUser.TwitterScreenName = claim.Value; continue;
            default:
                continue;
        }
    }
    return loggedInUser;
}

This is code which looks at the logged in user, interrogates the claims, and if it finds the claim that has a particular piece of information, extracts just that claim, into a little DTO helper object.

A more complicated one to get a LinqToTwitter context:

public async Task GetTwitterContextAsync()
{

    var credStore = new SessionStateCredentialStore();  // i frequently loose session, so need to rejigger this
    if (!credStore.HasAllCredentials())
    {
        if (_context == null) return null;
        var owinContext = _context.GetOwinContext();
        if (owinContext == null) return null;
        var authman = owinContext.Authentication;
        if (authman == null) return null;
        if (authman.User == null) return null;

        foreach (var claim in authman.User.Claims)
        {
            switch (claim.Type)
            {
                case "urn:twitter:userid": credStore.UserID = ulong.Parse(claim.Value); continue;
                case "urn:twitter:screenname": credStore.ScreenName = claim.Value; continue;
                case "TwitterAccessToken": credStore.OAuthToken = claim.Value; continue;
                case "TwitterAccessTokenSecret": credStore.OAuthTokenSecret = claim.Value; ; continue;
                default:
                    continue;
            }
        }
        credStore.ConsumerKey = ConfigurationManager.AppSettings["TwitterConsumerKey"];
        credStore.ConsumerSecret = ConfigurationManager.AppSettings["TwitterConsumerSecret"];
        await credStore.StoreAsync();
        if (!credStore.HasAllCredentials()) throw new NotSupportedException("Could not get credential store to have all credentials!");
    }
    return new TwitterContext(new MvcAuthorizer() { CredentialStore = credStore });
}

Its the same idea, except that it needs a few more claims, as well as the two things stored in AppSettings.

Finally, the call to get tweets for the home screen, lifted almost directly from the LinqToTwitter documentation:

public IQueryable GetHomeTweetsQueryable(TwitterContext twitterCtx)
{
    return (from tweet in twitterCtx.Status
            where tweet.Type == StatusType.Home
            && tweet.Count == 200
            && tweet.IncludeContributorDetails == true
            && tweet.IncludeEntities==true
            select tweet);
}

Conclusion

That’s a long way to go around; but, that is what it takes, with all these different black boxes talking to each other, to get the app to work. Hope this helps you save some time doing your own things.

Trying to find a Twitter app for an Android tablet

sad-twitter-bird.jpg

I have been searching for a Twitter app for my Nexus 10 Android tablet for a few weeks.  None of the ones I have tried have worked for me.   Here’s the list so far.  My apologies to a developer if I misrepresent your app, kindly correct me and I’ll retest.

Echofon for Twitter (Beta)Echofon: While I love this app on iOS, the android version does not [yet] mute by client.  Its not very tablet-friendly either.  When they do, I’ll likely switch back to them.

Plume for TwitterPlume: The inline rendered pictures are far too wide (and not tall enough) to be useful.  Still too wide in three column mode;  and I wish that the three columns could all show information from the same stream.  But no, I have to look at 3 month old direct messages instead.

Flipboard: tries to be smart about what it will show and what it won’t, resulting in missed stuff.  And doesn’t mute but heck its like its muting everything. Very pretty though it has the best UI.

Twitter: I refuse to use it.  If its like their website, it has “sponsored tweets”  .. no thanks.  And, it doesn’t have mute.

Seesmic: So far, its working okay.  It does previews, it does mutes, and while it is not intelligent about using all the space available on the tablet, it’s the least obnoxious one (apart from Flipboard). 

What am I looking for?

  • Must have the ability to mute by keyword search.  I really don’t want to see all the GetGlue, Foursquare, and Untappd things that my friends post, while I do want to see their other stuff.
  • Must have the ability to preview pictures and/or article text.   I don’t want to waste time following the link.
  • Must show a lot of information in a small space.
  • Must not skip information.

If I were to dream I would want an app that did the above and:

  • Configured against multiple sources (twitter, facebook, etc)
  • Rendered the view using little squares
    • Either picture or text in a square
    • 2×1 Rectangles possible, depending on the form factor of the picture / content. 
  • Allow square sizes to be variable – zoom in/out to get more/less information on the page
  • Allow grouping by chronological or by user or by popularity (or both)
    • both:  map chronology across the X axis; and user across the Y axis;  give everything a gravitational pull that yanks them together and/or tries to keep them in order, and see what happens.
  • Allow grouping by slicing first
    • 1st priority is: all stuff by my wife;
    • 2nd priority: stuff by my family;
    • 3rd priority: stuff that’s popular;
    • 4th priority: everything else
    • This is only possible if the time frame to slice over is configurable on the page; defaulting to “since the last time I checked”, or maybe “the oldest unread thing up to a week ago”.
  • Mark things as read vs unread so that if I don’t find it one view, I can find it in the other. 
    • Either hide things that are read, or display them dimmed, configurable.

I would probably call the app “SquareReader” or something like that.    Except that there’s a credit card processing tool that seems to have the dibs on the word “Square” at the moment.

Ah yes.  If somebody would pay me for the time, I’d write it.   Smile