Evented Github Adventure - Temporal averages

Carrying on in the EventStore series...

• Introduction to the EventStore
• Pushing data into the EventStore
• Projections in the EventStore
• Re-partitioning streams in the EventStore
• Creating a projection per stream
• Pumping data from Github into the EventStore
• Emitting new events from a projection
• Who is the sweariest of them all?
• Temporal queries in the event store
• Projections from multiple streams
• Temporal averages

If we can emit temporal events by partitioning streams and building up state over time, then we can probably go a bit further and build up averages over "How long things usually take", let's have a look at the temporal projection we used to emit time-based events:

"PushEvent": function(state, ev) {
  if(state.lastPush) {
    var newDate = new Date(ev.body.created_at)
      , lastDate = new Date(state.lastPush.body.created_at)
      , difference = (newDate.getTime() - lastDate.getTime()) / 1000

    if(difference < 120) {
      emit('paranoidpushes', "ParanoidPush", {
        first: state.lastPush,
        next: ev
      })
    }
  }
  state.lastPush = ev
  return state
}

What we can actually do here is instead of checking for a difference, simply emit an event for every time we receive a PushEvent notification which happened after another one we already saw:

fromCategory('pushesbyrepo')
  foreachStream()
  .when({
    $init: function(state, ev) {
      return {}
    },
    "PushEvent": function(state, ev) {
      if(state.lastPush) {
        var newDate = new Date(ev.body.created_at)
          , lastDate = new Date(state.lastPush.body.created_at)
          , difference = (newDate.getTime() - lastDate.getTime()) / 1000

        emit('successivepushes', "SuccessivePush", {
          first: state.lastPush,
          next: ev,
          difference: difference
        })
      }
      state.lastPush = ev
      return state
    }
  })

This can actually lead to better analysis of our data, for example

What are the average times between pushes across the different languages

So, this time we'll partition by language and work out an average time between pushes per language

fromStream('successivepushes')
  .when({
   "$init": function() {
     return {}
   },
   "SuccessivePush": function(state, ev) {
     var langState = getPerLanguageState(state, ev)
     langState.total++
     langState.totaltime += ev.body.difference
     return state
   }
  })

Now how I got to re-use that per-repo stream I created earlier.

My old version of this chart had C# developers at the right hand of the chart, but more data showed this was not the case (which is a shame because I was going to make a snarky remark about build times)

As it happens, the ML folk are the folk who push often and furiously - I can't explain why this is but this is from over 3000 pushes and is quite representative. (Anybody who works in standard ML care to comment? I didn't even know this was a popular thing...)

VimL being on the right hand side makes a lot of sense - how often do we change our configs after all? :)