Monday, July 28, 2014

The Three Laws of Software Development - Conway's Law

During a recent class, Peter Saddington referenced three laws which collectively are knows as the Three Laws of Software Development:
  1. Humphrey's Law
  2. Ziv's Law
  3. Conway's Law
I wrote about Humphrey's and Ziv's Laws in my previous posts. Conway's Law is perhaps the one that is most well known of the three.
Looking up "Conway's Law" you'll find it in Jeff Sutherland's book "Exploring Scrum: The Fundamentals." Conway's Law is an adage named after Melvin Conway who first mentioned the idea in 1968. It states "Organizations which design systems...are constrained to produce designs which are copies of the communication structures of these organizations" or it's also written as "If you have four groups working on a compiler, you'll get a 4-pass compiler" (sic). That second version is a bit dated by today's standards. I believe Peter stated it a bit more colloquially as "Software always mirrors the group that built and designed it - if the company is dysfunctional due to poor communication then the software will have the same dysfunction" or simply "everyone needs to communicate better."
This law is as proverbial as the previous two and like them, is more of a valid sociological observation. I found this diagram by Manu Cornet on Bonkers World that I thought was pretty funny and actually a fair representation of Conway's Law...

Manu's diagram isn't specific to Conway but you can infer what you like. The Microsoft and Oracle diagrams are especially telling. Peter tied this law into the need for excellent communication and collaboration as a necessity in producing great software. I've personally seen this law represented myself in the many start-up companies that I've had the honor of working with and the opposite end of the spectrum, large corporate software endeavors (at some point I'll relate a project that had over 100 sign-offs).
Early stage Start-ups are typically small and have a tiny organizational chart - the software produced is usually fairly simple structurally and the communication is more one-to-one due to the tight org structure. This ability to be nimble due to size produces many advantages, especially the ability to go to market quickly. Larger organizations begin struggling to produce software with any speed - there are so many check-points and stakeholders involved in the production that there are often small changes that impact both the software in complexity and the company's ability to market quickly. Large corporate entities are often bound to many revisions, checks-and-balances, approvals - you get the idea. Not only does the software fill with complexity but releases become a nightmare.
Tying back to the class, the emphasis is to make small incremental changes and expose those changes for approval back to the stakeholders early and often so the team can adjust and do a better job meeting expectations. Not only do you produce better and more efficient software, but the release becomes less problematic - there are fewer surprises. I don't know how many times I've worked on a project where the smaller changes truly delight the customer while the larger changes produce confusion and revolt - this is usually ascribed to poor training but even taken that as a viable excuse, doesn't it amount to a lack of communication?

Monday, July 21, 2014

The Three Laws of Software Development - Ziv's Law

(previously published on LinkedIn)
During a recent class, Peter Saddington referenced three laws which collectively are knows as the Three Laws of Software Development:
  1. Humphrey's Law
  2. Ziv's Law
  3. Conway's Law
I wrote about Humphrey's Law in my previous post.

Looking up "Ziv's Law" you'll find it in Jeff Sutherland's book "Exploring Scrum: The Fundamentals." Ziv's Law states that software development is unpredictable and that the documented artifacts such as specifications and requirements will never be fully understood. First we need to all agree that new product design is the fundamental nature of software development - and that basically whenever we're creating something new, no matter how close it is to something we've done before, it's impossible to be confident of all the constraints and needs of this new development. Sure we can do some affinity comparisons to something we've done previously, but there's always some variability (if there wasn't then why didn't we just use what was built the first time?), and it's virtually impossible to predict the outcome with much accuracy due to this variability. And further, the more the new development diverges from the previous, the more unlikely the accurate prediction of the outcome.

This law, much like Humphrey's Law, is about uncertainty. It's all about the precision of functional and non-functional requirements in regards to software development projects - no matter how well they are understood and written during the time of scoping and initial refinement, things tend to change while the actual development is going on (whether the business need changes or there are changes inherent to the development process itself that impact the complexity). This law is important because it's fundamental to understanding how the agile process, and scrum in particular, addresses this particular issue.

With scrum, because the intervals are fixed and the stages of development divided into bite-sized chunks that can fit into the interval, there's more room to make adjustments in the subsequent intervals (sprints) to allow for what is discovered. This also brings into play the experiences of those working on the user story (i.e. your technical team) so you're already getting additional input into the outcome. All this leads to a much more satisfying final product, both for those involved in the process and for those who are supporting or needing the results of those efforts - and after all, that's what we are trying to do, delight the customer (general term for whom we're doing the effort).

So why is this law important? It has to do with failure and what we do as a result of failure. If we accept that what we are building in any one sprint isn't perfect, then we'll understand that the feedback we receive from the customer for what's been done can be incorporated into the next sprint, allowing us to make adjustments to a final, satisfying outcome. Small failures are OK and should be expected - it's the ultimate result adjusting to those small failures that will make us successful.

-- John

The Three Laws of Software Development - Humphrey's Law

(previously published on LinkedIn)
During a recent class, Peter Saddington referenced three laws which collectively are knows as the Three Laws of Software Development:
  1. Humphrey's Law
  2. Ziv's Law
  3. Conway's Law
I had never heard of these observations referenced as "laws" and wanted to explore them further with my own research. Apparently Conway's Law is used in some scholastic software development classes but the other two are a bit obscure to me. We'll start with Humphrey's Law.

Looking up "Humphrey's Law" you first encounter a Wikipedia entry regarding something called "The Centipede's Dilemma" - which is something psychologists call the centipede effect or the centipede syndrome. The reference has to do with what happens when a normal action is interrupted by the person's own awareness of the action. An example might be a baseball player who normally uses his training, muscle memory and a bit of instinct to swing and hit a home run, but because he's consciously aware of what he's doing, he basically over-thinks the action, causing him to strike. The effect is also called hyper-reaction or Humphrey's Law, named for English psychologist George Humphrey, who wrote about the effect in 1923 using a poem by Katherine Craster, usually titled "The Centipede's Dilemma." - yeah I know I'm getting fairly esoteric in this but I think it's good to understand where this stuff comes from.

The Centipede's Dilemma
A centipede was happy – quite!
Until a toad in fun
Said, "Pray, which leg moves after which?"
This raised her doubts to such a pitch,
She fell exhausted in the ditch
Not knowing how to run.

While the poem is interesting it does illustrate the main point, which is "Thus, the eponymous "Humphrey's law" states that once performance of a task has become automatized, conscious thought about the task, while performing it, impairs performance. Whereas habit diminishes and then eliminates the attention required for routine tasks, this automaticity is disrupted by attention to a normally unconscious competence." - rather obscure, right? So the question remains, how does this fit into the Humphrey's Law as referenced by Jeff Sutherland, co-creator of Scrum and where the Three Laws of Software Development idea was presented?

Jeff Sutherland's version of Humphrey's Law states that "users don't know what they want until they see working software" or to paraphrase Peter Saddington, "People don't know what thy want until they see what they don't want." This seems to be the opposite of what "Humphrey's Law" as referenced by psychologists means, although they do share some common threads:
  1. Both ideas are about human response to exterior stimuli
  2. Both ideas are observations which seem true.
  3. Both contain an element of subconscious objectivity
  4. Both share the same name
I had original concluded that they are the same law, only cross-referenced from the perspective of the observer for the Software Development reference, and from the subject (actor(s) aka centipede) in the psychologists' version. Of course this was purely conjecture - and someone commented on my thread on LinkedIn to say that it's actually a different Humphrey - a software developer.

So what about application? These types of laws represent an observable problem, so using the adage that one must first recognize the problem before it can be addressed, the next step is to identify how Humphrey's Law is impacting the development of good software and using various methods to resolve the issue. Peter did a great job illustrating the problem using a "I want a car" technique that I hope to demonstrate to the product guys and my teams at Altisource Labs. I'm working through this now in the form of an interactive class and demonstration which I'll share with you in a subsequent post. In the meanwhile, I'll continue exploring the other two Laws in my next posts.

-- John

Building Personas

(also published on LinkedIn)
From my last post you may have garnered my desire to begin using some of the Pragmatic Marketing training and incorporating the best ideas into my current projects. Specifically, I began creating personas to use within the user stories and to support each development effort by providing additional context for the team members. When I broached the idea with the development team I initially was met with silence. On further discussion with some of the team members I received a rather interesting response "Well, yeah those are fine for UI but they really aren't useful to a developer." I met this with a bit of dismay - I really thought they would buy into the idea of providing additional context, as the current development groups (we went from a highly efficient team to several inexperienced teams) need all the context they can get. The counter-argument I received had to do with security settings and how typical UI-type personas did not provide enough information to determine access controls. It's interesting how some of those same members who desire efficient agile teams argue against using tools to make it so.

To reinforce the idea, a recent CSPO class emphasized the same thing - provide personas that can be used to help get the team into the mind of the user. At this point I could use some hard data to support the idea (mostly to get buy-in from the team) so anyone out there reading, if you have some fact-based analysis I that I can use it would be helpful in supporting my ideas. In the meanwhile, I'm continuing to build my personas to support epics/stories going forward, hopefully this will nudge the team into being more accepting. As to the question of security settings and access controls, it seems that if the personal has a well defined description and these are of importance, the information can be added into the personal description, no?

In any case, I'm currently defining personas for about a dozen different user-interacting roles into the current application. More updates to follow.

-- John

Saturday, July 19, 2014

Pragmatic Marketing and Scrum

(a slightly different post published on LinkedIn)
For the past 13 or so years I've been working in a variety of software development shops practicing Agile, and generally some modification of scrum. My focus has always been to take the ideal scrum implementation and continue to make it better - I can say that I've only found a very few shops that practice the "ideal" version of scrum as conceived by the founders of that term. At some point I'll get into that a bit more but the purpose of this article and a few to follow are to take what I've learned most recently from the Pragmatic Marketing training, and apply bits-and-pieces to my current job situation.

First a bit about Pragmatic Marketing.
I believe I first found Pragmatic Marketing in the early 2000's - at the time I was a Director of Technology (my role was more of a Product Manager one, I just didn't realize it yet) for a small start-up that had begun making some headway - in fact we were just turning a profit and had begun to expand the various departments of the company, including Product Management. We hired a very good upper management PdM person who, when presenting his ideas, etc. suggested that we might want to take a look at the materials provided by Pragmatic Marketing (I'll use PM going forward) to guide our expansion plans - at the time you could sign-up for a free PM print magazine that was published, I think either quarterly or bi-monthly and by signing up you had access to the other materials and past issues as PDFs (I may still have those stored in my files somewhere). After reviewing most of what was there we started to employ many of the ideas and also the terms associated with the course - it basically gave us a basis of terminology that was previously missing. Of course, being a start-up it was hard to justify taking the paid-course, but I had always been curios about it.

Next, about the current implementation of scum in my current position.
I function as a Product Manager for two different development groups. The first is a legacy platform that is primarily in maintenance mode, so it's a rather small team practicing kanban and mostly doing user stories relating to reporting needs to compliance and regulatory changes. At some point this team will be absorbed into other efforts, but at present there's enough work to keep them busy. The legacy platform has been running for over 10 years and frankly many of the efficiencies from 10 years ago have been exceeded by company demand which leads to the second group. Group two is a complete redesign using a different technology stack and a common services platform. The second group has gone from one team to five, then shrunk back to a single, consolidated team and now is expanding again into three teams. I share the product owner role with another product manager and we're lead by a Director who acts more as a principal product owner (his focus is more on the strategic direction and he has responsibilities tied to the P&L). From my scale, these newly defined teams are about 70% where I think it should be from a classic scrum perspective (it continues to get better but there's still a bunch of work to do). The group two team started with scrum and was relatively successful up to the initial release of the new product. Recently the expansion of the team into two groups (and building to three) has lead to inefficiencies and our velocity is shot.

A few weeks ago I attended three Pragmatic Marketing courses: Foundation, Focus and Build.
The main thing that I took away from the classes was that I already knew much of the information (you would expect that after 13 or so years doing product, right?) - the classes actually provided a framework of using what I already knew that was a bit more organized. It also provided some insights and approaches that I hadn't tried. Now we get into what I've learned and what I intend to implement to improve our teams.

There was an interesting graph as part of the Build class that I want to share, that has relevancy to what I've so far described. This has to do with context, and how it impacts the effectiveness of teams.

Basically, the first tenant is that the more context a team has the more efficient they become in executing what the market needs.
The better the context, the less that needs to be expressed as any type of requirements for a team - that's not to say that there doesn't have to be documentation, but due to the past shared experiences of the core team, the high trust levels between all participants, and the familiar processes, the teams reacted well, quickly and accurately to requests coming from the market. This is really the ideal state of a good scrum team, where from a PdM perspective, just enough detail is included in a user story. This is well illustrated by the previous velocity of the teams while executing the initial release of the product.

The second tenant is that the less context the team has, the more content and requirements product needs to produce for the team.
The analogy is to take some software feature to a contract shop - because they have very little context, they need much more information in order to be successful - this ends up being lots of artifacts, requirements and specifications. You can see that by expanding our previous team with many new, novice team members we should have adjusted as product managers by providing much more context, to help them ramp up and be successful. By having the unlikely expectations that these newly formed teams would magically become as quick and efficient as the original teams (the idea is to spread the experienced members across the new teams as leadership, so they can impart the technical information needed to get the teams up to speed) within a couple of sprints, we didn't consider that the amount of context provided to the team by product needed to correspondingly increase. So one simple graph and about 5 minutes of discussion in my PM class provided a world of context for me, in first identifying the problem and then taking some steps to improve the situation.

So what to do?
There were several recommendations that I drew from the classwork - remember that the main thing I'm trying to fix is a problem of context - how to get all the business information that product has packed into our brains and disseminate it to the new team members. When I describe a new feature idea I usually use flow diagrams to show how the process could be defined and wasn't spending much time describing the why - I'm now scheduling more time to go over the market need so that the teams are all on the same page. That's a start and seems obvious, but until I recognized that there's a problem how does one improve?

One thing that was emphasized by the instructor, Steve Gaynor (who was fantastic, by the way) was that there was a lot of information addressed in the class, and instead of trying to apply everything, just pick out 2 or 3 things and focus on making them work. So the other classroom idea I found interesting in regards to the problem was the concept of personas.

As with most of my Product Management friends, I usually include the "actor" (a representation of who is interacting with the feature) in my user stories. The concept of persona is a bit different - it's to create a fictitious "who" that's more defined. Using real data (surveys from the market was one suggestion for garnering the information), the product group creates a persona or personas who would represent the most likely candidate(s) for the use of the product. Because there's a lot more information provided as background of the persona, the user becomes more than an actor and the background suggests the needs of the user to the team doing the user story. So instead of "As an admin user" you might say "As Greg Jones" as the beginning of your story statement. There's an entire persona called "Greg Jones" that's been created that represents the characteristics of a typical admin user, including experience with the software and business, background and expertise, the amount of time spent on tasks, etc - whatever is relevant. By expressing personas that can be identified as a real entity, the team has a better experience when developing the feature and thus satisfying or delighting the persona, thereby receiving context from the very beginning of the story. Sounds good, right?

So I started stubbing in the personas for a particular business unit late last week and will continue this coming week. I'll report on this and some of the other ideas that fell out of the PM training in my next post.

-- John Eaton

Tuesday, July 1, 2014

The Accidental Product Manager

I've been doing Product Management for over 13 years. When I reflect back upon my entry into that field, I find that there isn't any single event that led me there. Even my entry into software development was roundabout - having come from a graphics background. I've also found that this is a common trait among other product managers I've met or worked with - it seems we all started elsewhere then gravitated towards what we like best about software development, the actual design and creation of something new. Continuing on this thread, I haven't seen any specific courses for product management (sure there are certification courses by companies such as Pragmatic Marketing, but I've never seen any specific course work offered in universities to prepare a student for software product management). I have encountered some common characteristics of the best PdMs that I've met worth sharing:
  1. Most of the professional product people I've met enjoy a high degree of creativity. The best excel at designing new products that not only provide a great deal of interactivity with the user, but also exceeds the perceived need - it takes genuine creativity to accomplish both goals.
  2. Most product managers are super-detail oriented, with the ability to talk high-level, but then dive deep into the minutiae. There's no room for sloppiness or a lack of details and you'll rarely find a PdM that exhibit those traits.
  3. The best product managers make the user the primary driver of the application and strive to enhance the user-experience. If you haven't heard of heuristics you probably shouldn't be a product manager.
  4. Contrary to the belief of many, a good PdM also talks frequently with his/her technology teams to make sure what's being asked for doesn't have an extreme cost (not just money, but also time, resources and a consideration of the overall technical design). If you're not talking to your teams you should consider doing something else.
  5. Product managers should strive to stay abreast of the latest technology - at minimum to keep things fresh and ideally to take advantage of new opportunities. This also includes training in disciplines outside of his/her comfort zone. If you haven't done Agile you should look into it and learn about being a product owner in a scrum team.
  6. I don't think I could ever take myself seriously, or call myself successful without having actually taken a product from design all the way to market. I both count my wins and learn from my mistakes.
Just a few thoughts - thanks for reading!

-- John Eaton