Election Day at The News Journal was a pretty solid success.
We used PHP and internet duct tape (iframes) to scrape and display results for big races live on our homepage and results for all races on another landing page.
Unlike the Primary Election, our cron job ran smooth all night and my stress level wasn’t through the roof.
Click here to check out a “still-live” version of our election night banner on our testing stage. [UPDATE: 02/05/2013] That link is a bit busted right now because our PHP server was hacked and we lost a few resources.
The day before, though, was a little more difficult.
Along with likely every other newsroom in the country, we’ve been having a lot of fun with campaign finance data around here. Last week, we tore into Gov. Jack Markell and GOP challenger Jeff Cragg’s reports and were experimenting with how to present the data online.
Our initial idea was just a map because the story focused heavily on geography after Markell asserted that most of the out-of-state money from surrounding states came from people who commuted and worked in Delaware. Leaflet took care of that quickly. I built the map in ArcMap, transferred it to GeoJSON and it’s straight to the web.
"But wait," we told ourselves. We’ve cleaned up all this data, let’s hand it over to readers and make it searchable. In comes DataTables. I’ve used DataTables a lot since I joined The News Journal. It’s a fantastic jQuery plugin, and I love that I can easily plug it into a MySQL database with a PHP script.
The News Journal now has a PANDA, a “newsroom data appliance” that will help fuel our reporting and hopefully make lives easier.
Needless to say, I’m pretty stoked. At the NICAR conference last February, I attended a presentation by the developers and knew right away that our newsroom would benefit from a PANDA. It took us a little while to track down a local server we could use, but we ended up re-purposing an old advertising server.
If you’ve never heard of PANDA before, it’s essentially a data “library” for the newsroom. Reporters can load data - or “feed the panda” - and it indexes it and makes it searchable. Already, we have loaded the registered voters database, state credit card payments and a few campaign finance tables. So, now all we have to do is type in a name and PANDA will tell us which databases that person is in. A search for Gov. Jack Markell’s last name shows entries in the registered voters database, the credit card data and the campaign finance reports for U.S. Senator Tom Carper. Once we get more data uploaded, we could ostensibly do a relatively thorough background check on sources just by searching for them with PANDA.
In a nutshell, this is going to greatly enhance our newsroom’s ability to capitalize on all the data we collect. Who wouldn’t be excited for that?
Oh IE, how the world hates you.
<meta HTTP-EQUIV="REFRESH" ...> </HEAD> <BODY> </BODY> </HTML>
- Unfortunately, Microsoft has added a clever new
- "feature" to Internet Explorer. If the text of
- an error's message is "too small", specifically
- less than 512 bytes, Internet Explorer returns
- its own error message. You can turn that off,
- but it's pretty tricky to find switch called
- "smart error messages". That means, of course,
- that short error messages are censored by default.
- IIS always returns error messages that are long
- enough to make Internet Explorer happy. The
- workaround is pretty simple: pad the error
- message with a big comment like this to push it
- over the five hundred and twelve bytes minimum.
- Of course, that's exactly what you're reading
- right now.
I’m cleaning and preparing school test score data for later this week. We are ranking schools for each subject and grade level based on average test score. The somewhat annoying challenge in excel is getting the built-in RANK() function to work on filtered data. When trying to input a range of scores into the RANK function, excel is including the hidden rows. The solution is courtesy of Damon Ostrander (yeah, he’s a pianist as well as a VBA pro).
Function Vis(Rin As Range) As Range Dim Cell As Range Application.Volatile Set Vis = Nothing For Each Cell In Rin If Not (Cell.EntireRow.Hidden Or Cell.EntireColumn.Hidden) Then If Vis Is Nothing Then Set Vis = Cell Else Set Vis = Union(Vis, Cell) End If End If Next Cell End Function
I very very rarely mess with VBA (though I should find some time to learn it), but this was just too perfect. All you have to do with this function is go from =RANK(a1, $a$1:$a$99) to =RANK(Vis(a1), Vis($a$1:$a$99)). Drag the auto fill down the column and Presto!
As always, if you have any ideas or know better ways to do this, leave me a note in the comments.