Monitorama 2014 wrapup

I’m just settling back into the daily routine after RelEng/RelOps’ workweek and then Monitorama back-to-back.

Videos will eventually be posted here.

I thought it was awesome the conference started with some #hugops.

Here are my highlights:

  • I gave a talk about crontabber! I have my speakers notes if you’re interested!

  • Dan Slimmons gave a nice talk about basic probability and how understanding the difference between sensitivity and specificity can help you choose more useful alerts. It was super basic stats stuff, but a good foundation for building up stats competency in teams.

  • James Mickens gave a hilarious talk about the cloud that is well-worth finding when it goes up.

  • Ashe Dryden gave a talk about gender issues and “our most wicked problem”. It was very well-received by the audience, which was gratifying for me personally. I think the audience walked away with some very practical things to do: speak up among peers when someone says things that make you uncomfortable and ask questions about equal treatment in your company for things like salary, perks and benefits.

  • Several talks were given about monitoring and managing ops inside companies. My favorite was from Daniel Schauenberg (contributor to statsd) of Etsy. and Scott Sanders spoke about similar topics in this presentaton on Github’s outage lifecycle. And related, but not at the conference, Heroku just published an incident response runbook.

  • There was a hilarious lightning talk about the failure of the Swedish ship Vasa as an object lesson for massive project failure. Here’s a link to the case study the lightning talk was based on.

  • Larry Price (@laprice) gave a 5-minute talk about Postgres autovacuum tuning, which was awesome, and I hope he posts the slides. It reminded me that I should do a couple brownbags about Postgres config this summer!

  • I was struck by how many people said they used Postgres in production. Someone else asked the question during a talk, and nearly half the audience raised their hands.

  • InfluxDB, a new timeseries database emphasizing an HTTP API (remind anyone of CouchDB? :D), seemed interesting, although maybe rough around the edges when it came to documenting useful features/best practices. When I mentioned it on Twitter, I found a few folks already trying to use it in production and got at least one bug filed. 🙂

  • I also saw an amazing demo of Kibana, which seems like a very interesting dashboard/investigation/querying interface to Elastic Search. I watched a friend deploy it in about an hour to look at their ES systems last Wednesday.

  • Dashing from Shopify was also very interesting, although a rubyist project, so not easy to integrate with our Pythonic world. However, putting on a contributor relations hat — it could be a wonderful and beautiful way for contributors to interact with our many APIs.

I’m looking forward to the videos coming out and a list of slide decks, as I missed a few talks during hallway track conversations. I met several people who are managing similar or larger event loads than we do with Socorro, so it was fun swapping stories and seeing how their software stacks are evolving. RabbitMQ was a weapon of choice for reporting environments, along with Storm. Lots of love for Kafka was out there for the people dealing with real-time customer response.

Overall, highly recommend attending Monitorama to dip a toe into the state of the art with regard to system operations, monitoring and ops management.

Recent talks: How to get a job like mine, Command-line essentials, Restore FTW

Here are a few of the talks I’ve given recently here in Portland. I’m trying to give more talks locally, and happy to speak at your Portland User Group. Just drop me an email.

  • How to get a job like mine. This talk was given to PSU students as an encouragement for them to get involved in free and open source software. Toward the end, we did a brainstorming session on the reasons why they didn’t contribute, and tried to come up with projects each person in the audience might be interested in learning more about
  • Command-line II. I’m writing up my notes from this talk, hopefully to turn it into a real tutorial that others could copy. My goal this year is to give a tutorial every other week, and I’m hoping to have at least 10 lessons come out of that work. It seems like I need to give each lesson twice to really get the hang of it. Which means I aught to get out of this experience with 26 lessons… but trying to stay realistic about my time.
  • Restores FTW at PDXPUG. This talk is about backups for PostgreSQL and how to get your teams to come up with restore plans that exercise databases as part of normal operations. I’m trying to switch talks about Backups to being talks about Restores. The next time I give this, I think I’ll change the order of the “restore patterns” to be at the start of the talk, and the discussion about planning for backups/restores to the end. I plan to do a Mozilla brownbag that covers these topics and also goes through a live demo of backing up, restoring and testing PostgreSQL with the new 9.2 tools.

Day 2 at PgConf.EU: hallway track and the marketing of Postgres

The hallway track is always my favorite part of the conference. I had to give a full-length and a lightning talk today, so much of my time was spent making sure I was really prepared and then giving the talks!

But between talks, I got to chat with Heroku, 2ndQuadrant and EnterpriseDB folks about what they think is coming next in the world of enterprise development and Postgres.

One topic that I touched on in those conversations and my lightning talk (Postgres needs an aircraft carrier) was that our plan for world domination needs to get quite a bit more specific and actionable.

For the open source community, the right question is not “are we ready to tackle the enterprise?” — the right question is: Which market segment and customer group are we going to target for complete market domination?

One area that we definitely already dominate is online poker. We have had a few blog posts about it, but not a whole lot else. Another is GIS through PostGIS.

I created a survey to try and capture some scenarios from the developers who work with customers every day solving problems. We need to know more about the people using Postgres and the way that they use the database.

If we can get 30 responses, I’ll publish the results. It’s a bit long, and requires some thought, so I imagine it will take some time to get them all.

If you have a customer that you think represents a good target market for Postgres, take 10 minutes and fill out the survey for us!

9.1 presentation at Windy City Perl Mongers

I recently updated my PostgreSQL 9.1 slides for a presentation at the Windy City Perl Mongers.

We discussed 10 features that the Postgres community decided to emphasize in our press releases. The crowd was primarily people who had never used Postgres before, which was a bit of a different audience for me.

It was great to be able to compare notes with folks who are supporting Oracle and SQL Server, and see a lot of excitement for trying out 9.1.

When I’m traveling around, I’ll be looking for more non-Postgres user groups to give talks like this. Let me know if you’d like me to come speak at yours!

OSCON: Postgres represent! And my links for Harder, Better, Faster, Stronger talk

I’m giving a couple talks at OSCON this year. The first is on Tuesday, 10:40am room C123: Harder, Better, Faster, Stronger: Postgres 9.1. The other is Mistakes were Made, Wednesday at 1:40pm in room D136.

My colleague Robert Treat is giving a Pro PostgreSQL workshop Wednesday at 1:40pm too, room 204. He’s also giving a Scalability Patterns talk at 4:20pm Tuesday. I’m sure his talks will be awesome. 🙂

And here are the rest of the talks tagged with PostgreSQL.

Also remember — there’s a PgDay tomorrow at the Oregon Convention Center!

I’m pushing my examples for my 9.1 talk into a github repo. It should be populated with whatever I decide to use for the talk by Monday evening.

Building 9.1 for me on Mac OS X (leopard!) involved the following:


git tag -l | grep REL9_1
git checkout REL9_1_BETA2
./configure --with-perl --with-python --prefix=/opt/pg91beta2 --with-readline
make
make install

Normal caveats apply – you need X Code of a reasonably recent version, and a bunch of support libraries to make this happen. I haven’t rebuilt from scratch on OS X in a long time, but now I realize that maybe I aught to go through the pain and document this again.

But I digress!

I have a long list of resources for this talk and wanted to share. Probably in the slides for the talk, I’ll provide shortlinks so that people can pull them up and read instead of listening to me 😀

Here’s my links:

And if you’re wondering about the title, I took it from an great Daft Punk song that fans have created some epic videos of:

Explaining MVCC in Postgres: system defined columns

I’m playing around with some diagrams for explaining MVCC that I’ll be posting here over the next few days. Not sure if I’ll end up giving up on slides and just use a whiteboard for the talk. I made an illustrated shared buffers deck to go along with Greg Smith’s excellent talk on shared buffers a while back. This is the beginning of a talk that I hope will emulate that.

Here are my first few slides, showing the system-defined columns. The next few slides will describe optimizations PostgreSQL has for managing the side effects of our pessimistic rollback strategy, and reducing IO during vacuuming and index updates.

Off to MySQL Con

I’m presenting a 25-minute talk on “Thinking about Performance” on Wednesday, around 5:25p at the Percona Performance Conference. My plan is to talk about performance testing and how to go about useful optimizations. This will be in the context of the PostgreSQL Performance Pad testing, and the performance-focused talks Mark Wong has been giving at Portland State University. Stop by if you get the chance – and you’re in San Jose! It’s free.

Robert Treat will be giving a talk on PostgreSQL at 11:15am that should be good, and we’re having a BoF during MySQL Con at 7:30pm. Quite a bit of Postgres-focused content at MySQL Con on Wednesday 🙂

I’m sad that I’ll be missing the Open Source Bridge Town Hall tomorrow night, but you should go! 🙂

Greg’s THREE talks at PostgreSQL Conference East

Greg Sabino Mullane will be presenting three talks at PostgreSQL Conference East this weekend in Philadelphia, at Drexel University. The talks are listed on the site, and here’s what he’ll be speaking about:

Bucardo
April 5, Sunday, 10am
Bucardo is a replication system for Postgres that uses triggers to asynchronously copy data from one server to many others (master-slave) or to exchange data between two servers (master-master). We’ll look at replication in general and where Bucardo fits in among other solutions, we’ll take a look at some of its features and use-cases, and discuss where it is going next. We’ll setup a running system along the way to demonstrate how it all works.

Monitoring Postgres with check_postgres.pl
April 4, Saturday, 2:30pm
What should you monitor? And how? We’ll look at the sort of things you should care about when watching over your Postgres databases, as well as ways to graph and analyze metadata about about your database, with a focus on the check_postgres.pl script.

The Power of psql
April 4, Saturday 10:30am
All about everyone’s favorite Postgres utility, psql, the best command-line database interface, period. We’ll cover basic and advanced usage.

I’ve seen a few of Greg’s talks — The Magic of MVCC, Cloning an elephant and a few others. He’s a great speaker and cool guy. And he’s my boss. But I’m not just saying that because he’s my boss! Really!

He doesn’t like to brag about himself, so I’m gonna help him out. He maintains DBD::Pg, check_postgres.pl, Bucardo and has had MANY patches committed to PostgreSQL. He’s also a volunteer for the PostgreSQL sysadmins team, and specifically helps maintain the git repo box. He’s a contributor to the MediaWiki project. He’s on the board of the United States PostgreSQL Association. He’s basically awesome.

If you’re gonna be there, you should check out his talks. And if you can’t make it, here’s hoping Josh Drake records the talks and shares them with us all! 🙂