Category Archives: Journalism

Journalism, Software Development

Using Python to scrape website data loaded through JavaScript

Leave a comment

As more sites are leaning on JavaScript to load dynamic data, web scraping is hitting new hurdles. Purely using the urllib2 library from Python will return the site without the dynamic data available to gather if a JavaScript library is used to populate site information.

Hitting this wall, I decided to take a deep dive in scraping data from a local paper who I freelance for’s rosters to create code replacement files to use in Photo Mechanic in this upcoming season. This makes captioning faster and easier, making photos available quicker to the paper.

Continue reading Using Python to scrape website data loaded through JavaScript →

Journalism

Katy Carron’s sports wild side

Leave a comment

I can’t count how many football and basketball games I’ve photographed, but when it comes to my first time shooting sports such as swimming or pole vault, I’m lost.

The idea is simple enough: swim faster, use a stick to throw yourself over a bar. But this past Saturday while shooting pole vault, I had to pick up the phone and figure out how you actually win.

Katy Carron in a cross country uniform
Katy ran cross country and pole vaulted at Francis Howell. Family photo

A few seasons ago, I had no idea what a swimming medley relay was or, as embarrassing as it is, the difference between a breaststroke and the butterfly.

Luckily, my fiancée had the answers.

Katy Carron may have one of the most obscure high school sports resumes around. A season of cross country. A season of pole vault. A season as a hockey cheerleader. Two seasons of swimming — including competing in the 500 meter freestyle. Continue reading Katy Carron’s sports wild side →

Journalism, Software Development

Building a digital newspaper machine

Leave a comment

Click here for a video of the finished product running.

Awhile back, a friend of mine, Nate Papes, ran across Scott Walker, director of enterprise and investigation at Alabama Media Group, who had retrofitted a newspaper machine to display newspaper front pages digitally. (Update March 13, 2016: This link is dead; it appears Walker removed the post.) We talked about creating our own, but never followed through.

A few years later, Nate found a New York Times newspaper machine in a Joplin, Mo., junkyard and began to work on his own machine. I offered to write the software to make the box run since Walker’s was written for a Macintosh and we both were toying with Raspberry Pi computers. In the end, Nate kept his box as original to the Times box as possible and I wrote the software to run as JavaScript in the browser and pull the pages down.

Continue reading Building a digital newspaper machine →

Journalism

Diving into Django, jail data

Leave a comment

My weekend project is to dive into Django. I get the concept and have been it in action, but I’ve never actually worked with it directly. It’ll be part of my jail project, which is going very well. (An amazing number of people go to jail daily in Greene County, Mo.)

My goal is a “dashboard” of jail statistics of frequent flyers, latest people booked along with some fun, and interesting, statistics of eye color, hair color, etc.

For example, right now in my 1,798-row data set there are:

  • 824 brown eyes
  • 499 blue eyes
  • 278 hazel eyes
  • 179 green eyes
  • 5 black eyes
  • 3 grey eye
  • 10 unknown or not specified eye color

Useful? Probably not in reporting, but certainly interesting for readers.