twitter

twittering on 2011-04-07

portland , , , 1 Comment

PDX11 Civic Unconference Hacking report

Last weekend we had more than 70 people out for a civic hackathon associated with PDX11, the joint effort between community, the City of Portland and the Portland Development Commission to promote software development in Portland. We kicked things off last Friday with a 30-minute, whirlwind status update since the kickoff of three initiatives in December 2010. Scroll down for a video! The next morning, Saturday April 2, we had another orientation, but focused on people’s hacking projects and an unconference. Many thanks to Kris Wetzel of Emma who organized all our food and drink Friday and Saturday, and to Tropo, who sponsored Saturday’s lunch. A big thanks to the project coordinators who prepared feature lists, wrote out tickets, and mentored others during the event:

  • Aaron Pareki
  • Amber Case
  • Audrey Eschright
  • Igal Koshevoy
  • Kyle Drake
  • Reid Beels
  • Kirsten Commandich
  • Don Park
  • Rafa Gutierrez

There were definitely more folks working on things — sorry I didn’t get all your names. Here are the project reports!

Continue reading →

twitter

twittering on 2011-04-06

postgres, postgresql , , , 3 Comments

Getting ready for the first PostgreSQL track at MySQL Users Conference!

Back in February, I blogged about all the sessions related to PostgreSQL at MySQL Users Conference this year.

My stickers just arrived:

Stop by the PostgreSQL booth during the MySQL Expo, or come to one of my talks if you’d like one.

I’m giving two – one is about Emma’s database systems and the problems with scaling huge schemas. And the other is a community talk about managing competing business and community interests inside of the Postgres community, and lessons that other open source communities can learn from our successes and failures.

We’re having a free Birds of a Feather session Tuesday, April 12 at 7pm at the Santa Clara Convention Center. Please stop by!

postgres, postgresql , , , ,

Where PostgreSQL succeeds and what to do next

Response to my earlier post about meritocracy was overwhelming.

Also, Robert posted a response focusing on code, and how the PostgreSQL project works around Commitfest.

Addressing some criticisms

I talked to Bruce Momjian about a few things that I said toward the end of my earlier post. Things that may have offended people in our community.

We focused mainly on that I brought a discussion of the outside world into the microcosm of PostgreSQL. And that I brought two things together: intrinsic ability of an individual to succeed, and the value of an individual’s contribution to Postgres itself.

I talked about a world that is filled with people who are poor, uneducated or disenfranchised who we, as a project, probably just can’t reach. And that by mentioning these facts, which Bruce and I agreed were facts, I was confusing and insulting people who contribute so much time and hard work to our project.

What PostgreSQL does well

To clarify, PostgreSQL does an admirable job of promoting and encouraging the work of the people who step up and contribute code. Robert’s post about Commitfest shows how much effort goes into finding and encouraging the type of people that we’d like to contribute more code, review our code and document it.

As a project, we also do pretty well with encouraging non-code contributions. In particular, I think we do very well with conferences: finding creative ways of sponsoring them, seeking out and developing new speakers, and helping start user groups. The focus has always been on finding the right people, in the communities that we see growing, and encouraging them. Today, we see conferences and great Postgres representation in Japan, Russia, Canada, the US, China, Cuba, the European Union and Brazil. And there are more.

So, I think that we (Postgres) are succeeding, and growing.

I brought up my criticisms in the context of Robert’s original post, and a request that I lay out my concerns about invoking meritocracy. The concerns I expressed are more about the outside world, how that world impacts Postgres, and how Postgres can impact the rest of the world.

I do think we can do more to create structures that encourage participation, the Commitfest being a great example of how to implement and succeed in the future. I’ve seen a few people step up and offer help in the last couple weeks, and I’ll encourage them in their work. And hopefully talk about their successes here.

What do we do next?

What I wanted to do was provoke a larger discussion about what we could be doing. I didn’t offer any particular solutions. I just asked that we think for a moment about what we might be able to do

And that, magically, happened.

David Fetter asked: “Which of those barriers do you see as important to address first?”

I’d like to connect Postgres more with the people in regions that our community doesn’t yet reach.

So, I’ve put up a survey asking people who live in high population regions that our community doesn’t really serve at all – most of Africa and the Middle East.

Please take a moment and let us know how you use Postgres, and what ways the Postgres community can connect with you.

The plan over the next six months is to both find ways of getting Postgres experts to give talks in those regions, and to find ways of supporting more people who want to be advocates for Postgres.

twitter

twittering on 2011-04-05

  • UpdatePDX tonight! At P.I.E. http://www.meetup.com/updatepdx/events/16416694/?a=md1.2p_evn&rv=md1.2p #
  • I'm experiencing an eventsplosion. Not sure how I'm going to sleep for the next two weeks. #
  • "You can't tune away the horror of 300-line SQL." -@lusciouspear #updatepdx #
  • learned that we needed to hire CS people #updatepdx #nosql #
  • "Once you stop answering email in 24-hours, they start sending more email." -@sarahnovotny #fail #updatepdx #
  • "you need to check that you have backups BEFORE you do your maintenance." -@sarahnovotny #backups #updatepdx #
  • "You don't have valid backups. You have valid *restores*." -@sarahnovotny #updatepdx #
  • "Twitter is really a series of caches." -@al3x #updatepdx #
  • "We had no idea how much we had to learn." -@al3x #updatepdx #
  • many <3s for computer science education today #updatepdx #
  • shout out for Pivotal Labs /cc @sarahmei 🙂 #updatepdx #
  • "both readthrough and writethrough caching was happening at the same time." #fail #updatepdx #
  • "But the issue was not technology, but responsibility in the organization." #fail #updatepdx #
  • "Only use NOSQL if you reach a certain point of despair." -@timanglade #updatepdx #
  • "If you ask for NOSQL by name, you probably don't need it. " -@timanglade #updatepdx #nosql #fail #
  • "Usually they should just hire a MySQL consultant." -@timanglade #updatepdx #
  • "Trust no one. that goes double if they talk about CAP." -@timanglade #updatepdx #
  • I see a niche market for database and devops-specific comedy. #
  • "Get a fresh set of eyes on the problem." (story of replacing data stored in Riak w/Postgres) #updatepdx re/collab while scaling @al3x #
  • talking about learning culture, passion for tech and how to hire when jobs are no longer silo'd between dev, ops, dba #updatepdx #
portland , , ,

UpdatePDX: NOSQL, operational complexity and hiring

Last night, Tim Anglade, Bradford Stephens, Sarah Novotny and Alex Payne put together a three-part discussion to talk complexity, caching and collaboration, and in some cases, skewer popular notions of problem-solving around NOSQL.

Big thanks to Michael Schurter and Rick Turoczy for organizing and providing us space at PIE.

Tim was the thinker and designer behind the event. Many thanks to him for putting so much energy and time into it, and helping many leave inspired.

“Catchy phrases are red herrings.” -@timanglade

The night began with stories of failure: Bradford touched on the horror of 300-line SQL and the value of applying computer science to solve problems; Sarah recalled not having restorable data in an emergency because someone had tested a new backup strategy – for six weeks; Alex talked about the hard problems in computer science (ok, caching) and the value of culture and mentoring.

What I loved about this part of the evening, is that each person had a story. Technical presentations sometimes lack guideposts, but these stories all had villains, heros and a lesson.

“Usually they should just hire a MySQL consultant.” -@timanglade

And then Tim followed with a wickedly funny sendup of “the NOSQL movement”. The thing that struck a nerve and made the audience laugh uproariously was: “Only use NOSQL if you reach a certain point of despair.”

His other points included: Never forget the operational complexity; Some things will always be better achieved with an RDBMS; Distribution Model vs. Data Model vs. Disk Data Structure (invoking the Moon Methodology); Hardware will always help, but it will never save you; Given enough time, most NOSQL projects gravitate towards a MapReduce-like model for computations (and querying); Trust no one. That goes double if they talk about CAP.

And then we talked.

The audience had plenty of questions, starting with how do we address collaboration in the context of scalability?

The answers from the panel, disappointingly, seemed to came down to separation of databases and people. While it’s true that it is easier to give out more databases than trying to communicate, it shows how far we have to go as an industry. Another point made was that it’s not typically possible to repurpose a DBA to maintain something like a Hadoop cluster.

In my notes, I wrote: “when problems are of a certain size, and affects the DNA of a company.” And when that happens, the typical separation of responsibilities between developer, DBA and operations break down.

Alex made my night by mentioning that sometimes solving a problem just takes getting a fresh set of eyes on it. He talked about replacing a particular bit of technology by applying some features of Postgres, with the help of a new hire.

I asked the final question – how does a company hire for the types of skills needed to solve these types of problems? Most agreed that finding someone with a passion for learning was critical. There was a dissenting voice – I think Bradford’s. My notes trailed off at that point, unfortunately. My guess is that he spoke up for logic and patience, and that some of the problems companies face have been largely solved, if you spend the time to study the science.

Epilogue

We wrapped up the night out at Little Big Burger and the Teardrop. Several people commented on how lovely Portland was – even in the rain. But damn, they wish they could hire the types of people they needed here.

So, people who are interested in big data: consider that a call to action.

Visit us for OSCON Data this summer. Then, move to Portland.