Document your API with Style

Leave Ego at the Door

One of the hardest steps to take towards being selfless in development is to not let yourself get in the way of making your apps better. You put a little of yourself into everything you make, but it’s extremely important to not let your infatuation with your first build lead you to viciously defend it like some kind of maternal nesting animal.

The first step towards becoming more selfless in development is to forget your ego and remember your purpose: to build applications that solve problems and do what they were meant to do. Everything else will ultimately be an extension of this concept.

The way you respond to criticism and feedback is a good gauge of how closely you cling to your ego. When people can’t figure out basic features or functionality of your application, it’s easy to get frustrated having to explain why you built something the way you did. Ego can get in the way here; many developers take on a condescending role as they explain simple functionality to “lesser” life forms. The important thing to remember here is that if those close to you (coworkers, friends, bosses, etc.) can’t figure out how to use your application, chances are the end user will have an even more difficult time. Learn to take any and all feedback gratefully and respectfully; you need to foster an environment where criticism is a positive thing, another chip off the stone block that will one day become a masterpiece sculpture.

Another indicator of whether ego is getting in the way is how much you resist change. When new features are required or new technology is mandated, the easiest but least helpful reaction is to search for reasons why these changes aren’t as critical as everyone seems to think they are.

Finally, remember the hard truth that you have an extremely limited perspective as a developer. As much as you might think you can place yourself in the shoes of your users, it’s extremely difficult. In fact, it’s impossible. There are companies and job titles dedicated solely to figuring out what users are thinking. That’s not really your job, but realizing that you don’t have all the information is a step towards taking yourself out of the way of progress.

 

Many people focus purely on the speed of search, often neglecting the quality of the results produced by the system. In most cases, people test out some small set of queries, eyeball the top five or ten and then declare the system good enough. In other cases, they have a suite of test queries to run, but they are at a loss for how to fix any issues that arise.

To solve this relevance problems takes a systematic approach, a set of useful tools and a dose of patience. This article will outline several approaches and tools. The patience part will come from knowing the problem is being looked at in a pragmatic way that will lead to a solution instead of a dead end.

Table of Contents

Introduction

Determining Relevance Quality

Log Analysis

Empirical Testing

Debugging Relevance Issues

Examining Explanations

Finding Content Issues

Other Debugging Challenges

Tools for Debugging

TREC Tools

Analysis Tools

Using Luke for Debugging

Dealing with Precision/Recall Conflicts

Conclusions

Resources

I'm working on Lucene integration and thought this might be a good guide when improving the relevancy of the test results. ATM I'm still on the level of 'good enough' and came across this write while searching for Lucene scoring.

Cloudata is a new open source implementation of Google's BigTable paper. It can be found on Github here. It appears to be the project of a Korean developer named YKKwon.

As noted at MyNoSQL, there are only a couple commits and it's not clear how serious this project is. But it will be of interest to big data, MapReduce and BigTable buffs.

Photon is an open source PHP "micro framework" using Mongrel2 and ZeroMQ. It was created by Loïc d'Anterroches, who claims that Photon is three to 10 times faster than frameworks such as Zend. "Photon does not try to do everything, the core is small, easy to understand and without magic," d'Anterroches writes.

As an example, d'Anterroches posted the code for a real-time chat application built with Photon. "This is effectively the entire "server side views handling the jsSocket connection," he wrote on Hacker News. "The front end is a copy/paste of the chat available in the Mongrel2 examples."

I’m fascinated by the way Facebook operates.  It’s a very unique environment, not easily replicated (nor would their system work for all companies, even if they tried).  These are notes gathered from talking with many friends at Facebook about how the company develops and releases software.

• by default all code commits get packaged into weekly releases (tuesdays)
• with extra effort, changes can go out same day
• tuesday code releases require all engineers who committed code in that week’s release candidate to be on-site
• engineers must be present in a specific IRC channel for “roll call” before the release begins or else suffer a public “shaming”
• ops team runs code releases by gradually rolling code out
  • facebook has around 60,000 servers
  • there are 9 levels for rolling out new code
  • [CORRECTION thx epriest] “The nine push phases are not concentric. There are three concentric phases (p1 = internal release, p2 = small external release, p3 = full external release). The other six phases are auxiliary tiers like our internal tools, video upload hosts, etc.”
  • the smallest level is only 6 servers
  • e.g., new tuesday release is rolled out to 6 servers (level 1), ops team then observes those 6 servers and make sure that they are behaving correctly before rolling forward to the next level.
  • if a release is causing any issues (e.g., throwing errors, etc.) then push is halted.  the engineer who committed the offending changeset is paged to fix the problem.  and then the release starts over again at level 1.
  • so a release may go thru levels repeatedly:  1-2-3-fix. back to 1. 1-2-3-4-5-fix.  back to 1.  1-2-3-4-5-6-7-8-9.
• ops team is really well-trained, well-respected, and very business-aware.  their server metrics go beyond the usual error logs, load & memory utilization stats — also include user behavior.  E.g., if a new release changes the percentage of users who engage with Facebook features, the ops team will see that in their metrics and may stop a release for that reason so they can investigate.
• during the release process, ops team uses an IRC-based paging system that can ping individual engineers via Facebook, email, IRC, IM, and SMS if needed to get their attention.  not responding to ops team results in public shaming.
• once code has rolled out to level 9 and is stable, then done with weekly push.
• if a feature doesn’t get coded in time for a particular weekly push, it’s not that big a deal (unless there are hard external dependencies) — features will just generally get shipped whenever they’re completed.
• getting svn-blamed, publicly shamed, or slipping projects too often will result in an engineer getting fired.  ”it’s a very high performance culture”.  people that aren’t productive or aren’t super talented really stick out.  Managers will literally take poor performers aside within 6 months of hiring and say “this just isn’t working out, you’re not a good culture fit”.  this actually applies at every level of the company, even C-level and VP-level hires have been quickly dismissed if they aren’t super productive.

It’ll be super interesting to see how Facebook’s development culture evolves over time — and especially to see if the culture can continue scaling as the company grows into the thousands-of-employees.

What do you think?  Would “developer-driven culture” work at your company?

 

Well, look who just oozed into being, Google's latest flavor of Android, the tablet-friendly 3.0, Honeycomb. We heard talk that 3.0 would be exclusively for tablets, as in not for phones, and that looks to be played out with the text "Entirely for Tablet" seen early in the video. Perhaps the best thing to say is that this looks more or less nothing like Android. Sure, the browser is the same, and the Gmail app will be familiar to iPad users, but trust us when we say there's a lot of lovely UI in the video above to admire...and a lot of glowly blue lights, too. Apparently we weren't the only ones who saw Tron:Legacy on opening night.

We spend a lot of time gathering metrics for our network, servers, and many things going on within the code that drives Etsy. It’s no secret that this is one of our keys to moving fast. We use a variety of monitoring tools to help us correlate issues across our architecture. But what most monitoring tools achieve is correlating the effects of change, rather than the causes.
Change to application code (deploys) are opportunities for failure. Tweaking pages and features on your web site cause ripples throughout the metrics you monitor, including database load, cache requests, web server requests, and outgoing bandwidth. When you break something on your site, those metrics will typically start to skew up or down.

At Yahoo, user-facing software was controlled by product managers and designers. The job of programmers was just to take the work of the product managers and designers the final step, by translating it into code.

In technology, once you have bad programmers, you're doomed. I can't think of an instance where a company has sunk into technical mediocrity and recovered. Good programmers want to work with other good programmers. So once the quality of programmers at your company starts to drop, you enter a death spiral from which there is no recovery.

 

 

It is no longer worthwhile for the average 12-year-old to make 20 bucks for cleaning someone's car. It is certainly pointless for prepimpled youths to make a little pocket money for, say, cleaning the house or, perhaps, washing their armpits.

Not when there's Mozilla ready to give them $3,000 for a little overdue diligence.

According to the San Jose Mercury News, Alex Miller is one 12-year-old who knows exactly which side of his bread he wishes buttered. The moment he saw that the benevolent bosses of Firefox were offering 3,000 big ones in order to eradicate just one bug, Alex bored down into the Internet's ether.
It used to be that Mozilla offered only $500 to those who happened to find major security flaws in its digital innards. Two months ago, management deemed it wise to multiply this figure by 6. Perhaps the recession is, indeed, over.

1985: The first official reference guide for the C++ programming language is published. The author, Bjarne Stroustrup, is also the language’s creator. Stroustrup had been hacking away at his replacement for the C programming language at AT&T Bell labs since 1979, where he and his colleagues in the research department were given free reign to experiment with new ways of building software. C++ went on to become one of the most popular programming languages ever created. It was designed to be a “general use” language: It can be run on just about any platform, and it shows up almost everywhere, especially in videogames and embedded systems. “If I had thought of it and had some marketing sense,” Stroustrup says, “every computer and just about any gadget would have had a little ‘C++ Inside’ sticker on it.” After publishing The C++ Programming Language, Stroustrup continued to stay active in the C++ community, working on the language’s design, writing compilers and helping to make the language an ISO standard in 1998. He is currently a distinguished professor and holder of the College of Engineering Chair in Computer Science at Texas A&M University.

+1

‘If someone claims to have the perfect programming language, he is either a fool or a salesman or both.’

I'm one of the developers for Twitterrific and to be honest, I can't tell you how many hours have gone into the product. I can tell you everyone who upvoted the estimate of 160 hours for development and 40 hours for design is fricken' high. (I'd use another phrase, but this is my first post on Stack Overflow, so I'm being good.)

Twitterrific has had 4 major releases beginning with the iOS 1.0 (Jailbreak.) That's a lot of code, much of which is in the bit bucket (we refactor a lot with each major release.)

One thing that would be interesting to look at is the amount of time that we had to work on the iPad version. Apple set a product release date that gave us 60 days to do the development. (That was later extended by a week.)

We started the iPad development from scratch, but a lot of our underlying code (mostly models) was re-used. The development was done by two experienced iOS developers. One of them has even written a book: http://appdevmanual.com :-)

With such a short schedule, we worked some pretty long hours. Let's be conservative and say it's 10 hours per day for 6 days a week. That 60 hours for 9 weeks gives us 540 hours. With two developers, that's pretty close to 1,100 hours. Our rate for clients is $150 per hour giving $165,000 just for new code. Remember also that we were reusing a bunch existing code: I'm going to lowball the value of that code at $35,000 giving a total development cost of $200,000.

Anyone who's done serious iPhone development can tell you there's a lot of design work involved with any project. We had two designers working on that aspect of the product. They worked their asses off dealing with completely new interaction mechanics. Don't forget they didn't have any hardware to touch, either (LOTS of printouts!) Combined they spent at least 25 hours per week on the project. So 225 hours at $150/hr is about $34,000.

There are also other costs that many developer neglect to take into account: project management, testing, equipment. Again, if we lowball that figure at $16,000 we're at $250,000. This number falls in line with Jonathan Wight's (@schwa) $50-150K estimate with the 22 day Obama app.

Take another hit, dude.

Now if you want to build backend services for your app, that number's going to go up even more. Everyone seems surprised that Instagram chewed through $500K in venture funding to build a new frontend and backend. I'm not.