Here’s an edited version of Thompson Morrison’s presentation about the software industry’s response to a series of surveys. The original presentation is available here. One of the key slides was about what folks here value:
The point I appreciated about this clip is that Portland’s software community *is* our competitive advantage.
I edited the video most for audio coherence. Sorry about the audio being a little out of sync.
I am working on editing out the pithier parts of the 1 hour 45 minute video that the city released of the PDX Software Summit.
Below is Mayor Sam Adams’ introduction to the process and how the City is engaging the public to make the software strategy happen.
Open data is really only as interesting as what we can do with it!
One sweet thing about ScraperWiki is that it enables quick creation of visualizations called ‘views’ from inside the wiki. They’ve got templates that use Google Visualization to help the process along.
I made the following today (from this datasource):
I don’t have the entire data set, but this graph indicates that the recession had a significant negative impact on the creation rate of new businesses in Oregon.
I just started a new scraper job to pull more information about people and the places where the businesses are located. When that job is done, I hope to create a few more fun visualizations with this data.
UPDATE: I’m playing around more, and here’s the embedded version of the graph if you click through (takes a while to load!).
Continue reading
At the Portland Software Summit on Thursday, a couple people mentioned that it was hard to keep track of new businesses that pop up, and that merger and acquisition activity wasn’t being sufficiently publicised.
I thought – maybe we could get this information in an automated way!
I started with the state of Oregon’s business registry search site. Unfortunately, they limit search results for business searches to 1000, and they don’t paginate their results. So, we kicked ScraperWiki into gear, and wrote a very simple scraper with @maxogden: http://scraperwiki.com/scrapers/oregon_business_registry/
Next, I wanted to find out information about businesses specifically in Portland. The City releases information about this, but in PDF form: http://www.portlandonline.com/omf/index.cfm?c=32192
I wrote a quick and dirty Python script to scrape out information, and am getting probably 250 of the 300+ businesses in the November release. Next, I want to cross reference this data with what’s in the Oregon site. I’ll be publishing the Python scripts over the weekend. Hopefully ScraperWiki will add pyPDF to their Python repo support and I will be able to publish the transform there so it can be easily linked to the Oregon data.
Two lessons today:
The alternative to scraping the state of Oregon’s site is to order a CD-ROM for $50. I think this is such a stupid profit center for the state. I’d be interested to know how much money they’re really making off of it, and whether they could take a page out of Metro’s book and find a way to share the data with a different, more useful service.
So, the software summit happened this evening. Mike Rogoway posted his brief overview here: http://blog.oregonlive.com/siliconforest/2010/12/portland_gathers_software_deve.html. The back of my head, Teyo and Amy are featured on the bottom of the photo 🙂
I livetweeted the event, so tomorrow there’ll be a cavalcade of tweets here: http://www.chesnok.com/daily/2010/12/03/twittering-on-2010-12-03/
Big ideas:
My takeaway from the Q&A was that we’ve got this huge biz/developer divide. We have management talent – but maybe not the big M&A talent that the VC-oriented folks are after. We have loads of developers. We need to make more effort to get those folks connected to one another in a way that makes sense!
We have tons of events that are advertised on calagator.org, and the city is making an effort to make itself a hub (hosting Lunch 2.0, and now this software summit in the City Council Chambers). Overall, I really think the City is doing a great job, and I can’t wait to see what the citizen initiatives come up with.
So if you want to get involved, subscribe to our lists:
http://lists.pdx11.org/mailman/listinfo/pdx11-announce (announce only!)
http://lists.pdx11.org/mailman/listinfo/pdx11-financing
http://lists.pdx11.org/mailman/listinfo/pdx11-mentoring
http://lists.pdx11.org/mailman/listinfo/pdx11-knowledge
Find out more at: http://pdx11.org
UPDATE:
Portland Business Journal coverage!
Slides from the City’s presentation
Video from Software Summit
Alright, PDX11 is alive!
Tonight’s the big software summit! If you didn’t already sign up, it’s going to be live streamed, and I’m sure there will be tons of tweets and blog posts after. PDX11 will have followup information, and announcements related to the next steps each group involved will be taking.
If you want to get a head start — sign up for the mailing lists!
For announcements: http://lists.pdx11.org/mailman/listinfo/pdx11-announce
For more information about today’s meeting (4:30pm at City Hall!) check out Mike Rogoway’s coverage: Portland wants to transform its software culture into an industry
What else should we have on the site? Want to help make it better? Leave your thoughts in the comments.
We’re looking for a few good OPML files!
Tell us where you go to find information about tech in Portland! I’m interested in software companies, non-profits, developer blogs and culture resources that are relevant to tech businesses. Off the top of my head: Silicon Florist, Mark Rogoway’s blog, BikePortland and Sarah Mirk’s twitter feed all are relevant. Urban Airship, Puppet Labs, Bank Simple, Small Society and Cloud Four are all local businesses and startups I’m keeping an eye on.
And what will I do with your contributions? I’m going to pick the best-of and curate a feed for the pdx11.org site.
Let me know what’s in your feed reader below!
So, last night a few Portland tech community folks got together at a bar and talked about the upcoming software industry summit on November 18th. You haven’t heard about it? Well, here’s the lowdown:
Mayor Adams, the Software Association of Oregon and the Portland Development Commission cordially invites you to participate in the development of a key software cluster strategy for the Oregon/SW Washington region. By attending this informative meeting you will play an instrumental part in the process with other top business community leaders.
The deal is: you show up, you get to help direct the resources dedicated to software industry development from the Portland Development Commission, the Mayor’s office and the Software Association of Oregon. That’s pretty much it. The people who go to this meeting are the folks who will be directing the future of the PDC’s “Portland Software Mandate”, which is part of that whole industry “cluster” initiative that targets active outware, advanced manufacturing, clean tech and software. More broadly, this is part of the 5-year plan to guide Portland’s economic growth.
Anyway, you really should attend this meeting next Thursday. If you can’t attend, see if you can get someone you know or a friend to attend on your behalf. We want to pack city hall.
That’s right — we’re meeting in the City Council Chambers. It’ll be great. Please attend! It starts at 4:30pm and goes to 6:30pm. And, if you show up we can lead you over to a sweet afterparty. Seriously. It’s gonna be awesome.
But — back to the PDX11 thing. One thing we talked about during the meeting was that we’re having problems communicating to everyone what it is that we’re doing with this whole “portland/pdx software cluster strategy”. It’s a mouthful to say that, and once you take another step and try to describe who is involved and why, things get muddy.
What if we just made a hashtag for it? What if we created a community site to start hacking on our city’s approach to software industry? To have public conversations about what’s next, and to bring to the table all aspects of development: education, small businesses, big businesses, hobbyists, investors and an epic number of volunteers.
Our first stab at this is to create pdx11.org. There’s nothing there yet (sorry, just bought the domain name), so this is your opportunity to help create this thing.
Last night, as we tried to come up with a concise name for what we were talking about, people started to giggle, and laugh at “turn it up to 11” jokes. Next year is 2011, so this thing has an expiration date (deadlines – my favorite procrastination ending technique). Because really, we’re kicking off this thing and wanting to deliver in 2011 on some initiatives. Which if you’re not aware are:
These conveners are here to be project managers in the best sense — to find out what it is that *you* want to have happen, and then to communicate, document and follow up with public and private partners who are *making* these initiatives happen.
But back to PDX11. We wanted a name, a brand, something that made it easier for us to talk about with and recruit people inside the community, a connection point for locals, and a launchpad for outsiders to check us out.
If you’re up for helping to build the site, contact me directly (selenamarie -at- gmail -dot- com) and we’ll get going.
Otherwise, we’ll see you next thursday!
UPDATE:
For those not familiar with the PDC surveys, here are links to the results in SlideRocket format (meh):
A couple weeks ago, Mark Wong and I took a field trip over to the Database Reading Group at Portland State University. It’s a group of students and professors that meet weekly throughout the school year to go over research papers. The papers are picked by the participants, and vary in topic from obscure to very practical.
This week’s paper reading was led by Professor Len Shapiro, and titled “Online Aggregation“. The paper is considered a foundational paper about SQL aggregates (like COUNT() or AVERAGE), and was published in 1997 by researchers from UC Berkeley and IBM. It’s also precursor to research into query parallelization and streaming databases. It was also awarded the SIGMOD “Test of Time” award in 2007, and is cited by over 170 other papers in the ACM archive.
The basic idea behind the paper centered around how to improve user experience in reporting results of aggregate queries – asking questions about how to solve three key problems when solving aggregates: blocking, fairness and control (from a user’s perspective). Roughly: Blocking is what happens when some part of the system waits and doesn’t return results to the user as a result of the waiting. Fairness concerns whether certain types of operations prevent certain groups of data from being processed (the example given had to do with GROUP BY and groups being processed one at a time). Control concerns whether or not a user can exert control over the speed of computation applied to a group (example given being a lever that “speeds up” processing of a set).
One insight from the paper is how online aggregates should be treated differently than traditional query processing – which might favor expensive plans involving sorts so that the output is ordered. When you’re dealing with online aggregates, you prefer unordered, or ideally random order, because your intermediate results will be more representative of the ultimate result. I guess that’s probably obvious once you think about it, but the paper provided some concrete examples.
Another interesting thought experiment involving the planner is how you pick plans that favor non-blocking, fairness and user control. Each of those properties is not narrowly defined, and changes based on individual user expectation. Professor Kristen Tufte mentioned that she’d be interested in how the ideas presented in this paper would be applied today, and Professor David Meier brought up that we might most be interested in applications involving managing Hadoop.
Prof Meier also brought up an interesting paper involving alternating nested loop joins during a discussion about optimizing JOIN algorithms for online aggregates. Another cool thing about the paper is that it involved modifications to Postgres! Granted, it was Postgres95, which doesn’t resemble the modern PostgreSQL 9.0 very much. But it was nice to revisit research that used Postgres that’s still relevant today.
I’m nearly ready for PDXPUG Day 2010.
We’ve got some fun talks lined up from leaders in the Postgres community in town for OSCON. It’s free to attend. We’ll at the Oregon Convention Center, room D131.
Stop by sometime during the day if you’re in town!
Here’s our schedule:
10am: What’s new in 9.0 – Selena Deckelmann
11am: ORMs and Their Discontents – Christophe Pettus
1:30pm: Mining Your Logs For Fun and Profit – Josh Berkus
2:30pm: PORTAL – Dan Colish
3:30pm: An Introduction to Managing and Troubleshooting PostgreSQL on Windows – Tim Bruce
4:30pm: PostgreSQL in Brazilian Army and Air Force – Luis Dosso
6pm-10pm: Party at the Gotham Tavern!