Spark Structured Streaming from Kafka by Maria Patterson
I’ve also been looking at how to use Spark Structured Streaming with Kafka, a new streaming platform from Spark.
Spark Streaming from Kafka with Avro data by Maria Patterson
I’ll show here an example of using Spark Streaming to process Avro formatted data from Kafka.
Spark Streaming from Kafka by Maria Patterson
I’ve been looking at how to use Spark with data streaming from Kafka, a data streaming platform.
Counting change - image analysis in Python by Maria Patterson
One of the nice perks about working at a University is the opportunity to go to a wide variety of all kinds of classes and workshops. I just went to a really great workshop about Image Analysis in Python, given by Brian Keating, a UChicago RCC staff member. The materials for the class are on the lecturer’s Github:
It's Hard to Sell Statistics by Jeff Coughlin
I recently held a press conference with a colleague for NASA’s Kepler mission. In it we announced two items, with my colleague announcing (1) and myself (2):
Training smarter for a half marathon with my Nike+ data by Maria Patterson
I’ve started to train for a half marathon, just finishing week 5 of the 12 week Nike+ running coach half marathon program. I just want to know- how quickly can I expect to improve my running pace? I’m NOT a runner, and I’m very slow. By “not a runner”, I mean I had never run more than 4 miles at once before this program and not a single mile at all in the last two years. By “slow”, I mean that I am a 5’2″ girl who has a hard time keeping up with people I walk next to. To clarify further, because I recently read a comment somewhere on the interwebs about someone who was “slow” with a “sluggish pace” of 9’45” (HA, that’s my fastest mile!), I’ll just let you also know that my all time average pace with Nike+ Running app so far is 13’12”.
Exoplanet Habitability – Full of Hot Air? by Jeff Coughlin
In the search for exoplanets, we have primarily focused on finding planets around stars like our Sun, called G-dwarfs. While it makes sense to search around what we’re familiar with, the problem is that stars like our sun are in the minority — the vast majority of stars in our galaxy are smaller, cooler stars known as M- and K-dwarfs. Furthermore, recent discoveries by the Kepler Mission have shown that M- and K-dwarf stars are more likely to harbor rocky, terrestrial planets like Earth, compared to gaseous giants like Jupiter and Saturn. Thus, wouldn’t it make sense to expand our horizons and focus more on M- and K-dwarfs in our search for Earth-like planets?
Reflections from an astronomer at the Grace Hopper Celebration of women in computing by Maria Patterson
Last week, I traveled to Phoenix (along with 8,000 other women and a few men), supported by a generous scholarship sponsored by Target, to attend the Grace Hopper Celebration of women in computing, put on by the Anita Borg Institute. This was my first time in attendance, and as someone moving away from astronomy more towards the computing field I was unsure of what to expect.
Mapping data across the US by Maria Patterson
Recently, I’ve been working on analysis of geospatial data. I found that there was a bit of a sharp learning curve just to be able to do some preliminary visualizations of spatial data, so this post will be a tutorial in some simple map visualizations with R. Note that this is really just useful for exploratory visual analysis.
Building a Better SOFIA Flight Map, Part 1 by Ryan Hamilton
The Stratospheric Observatory for Infrared Astronomy (SOFIA) is a one of a kind airplane: it has a 2.5 meter diameter telescope in it! We fly it around in the stratosphere at night, pointing the telescope at targets chosen competitively by the astronomical community. You can read more about why exactly we have a telescope in an airplane here.