Scrum Confessions – Part 1

Agile development and scrum are no-brainer-choices in almost every IT department in Europe. Over the past 2-3 years it became almost a religion within the most tech communities, every executive and every proficient IT professional starts setting the scene by focusing the importance of applying the scrum process properly and effectively for success.

I do think that scrum has changed the way software is built ,  supported the development processes  in a very positive way and it opened a new era of engineering enabling the alignment of business and engineering much better than it used to be.  On the other hand, having gone through some scrum transitions in several working environments and based on my observations from the management balcony until now I know that there exist some typical and chronic problems around scrum that bother the most of the stakeholders.

Now, I don’t mean the problems that are  related to the suboptimal implementation of the scrum process or not understanding the philosophy. Indeed, probably the first thing  you would hear when your project fails, delivers late or actually gets way to expensive to finance would be: “there must be something wrong with your people and the way there are doing scrum”. However, I believe ,scrum process and roles have weak points and fundamental issues causing productivity and quality disadvantages.
I think many executives and computer scientists working with scrum do know that it cannot be efficiency or the people every time.  We know this funny feeling that there might indeed be some stuff fishy and not mature about the beautiful scrum methodology. Somehow, we usually prefer not to talk about these thoughts and experiences. It is an unwritten law not to question the process and always praise the practices thought to us by the gurus of the Scrumentalists. So I decided to write a series of blog articles which will reflect my observations and criticism about scrum framework, its deficits and risks that every IT manager should consider.

Confession 1 : The product owner will always fail

We expect product owners to act like product managers and drive the business while being fully aware of the customer expectations and competitive market out there. Easy said than done, this means that a PO needs to have strong external orientation, time and skills to drive the KPIs and strategic decisions related to the product. They are also need to have project management skills and fully control the product backlog and prioritize tasks for the team, write the stories and be responsible for translating end user requirements in a way that the team can understand and correctly implement. She has to judge operational and emergency issues against new feature development and act as a leader to keep everybody focused and energized. Which actually requires that she has to be technically  quite knowledgable  about the software development technologies, be internally oriented, have at least some engineering skills and do lots of things that is outside the scope of product management or business analysis functions of waterfall days.

There is hardly anyone on earth who can juggle so many balls in so many disciplines. The consequences of such illusional expectations for the companies are really serious and it is a huge problem for many executives. Because we want convincing market analysis about the products we decide to invest in, see the KPIs and reports that enable us to measure success and of course(!) there must be somebody who is accountable for the optimal delivery of new features while fixing the right bugs and giving the necessary support for operations. The bill is mostly charged to the product owner in cases of failure or problems. “There must be something wrong with her” or “we need to improve how the role is defined and understood” or “ it is a product owner who did not understand/do this and that” bla bla bla. It is an endless and useless conversation! I have known many POs with extraordinary skills and amazing performance but the PO defined above does not and will not exist.

So what could be a solution? : Personally I believe mid and large size companies need to separate the business and technical process side of the product management and install a separate set of professionals who are accountable for the product KPIs, market competitiveness, benchmarking and external analysis of opportunities and risks for the business. These people do not have to cuddle with the scrum team on a daily basis and they do not need to know all the user stories and technical details about the product. Call them product managers, business analysts or anything you want. POs on the other hand can focus on the internal and more technical aspects of software development and backlog management; they get the product delivered fulfilling the defined KPIs by her partner with the help of her team.

Confession 2:  Scrum=software and processes without documentation & lots of risks

Agile manifesto says “we value working software over documentation”. What happens in reality is that over 90% of all scrum projects are not or very poorly documented. There is no incentive, recognition, rules or any process for documentation in the framework. What is even worse is that the framework turns developers into professionals who do not value documentation and find the related tasks boring and inferior. And it hurts, it hurts you a lot

  • When you have attrition in your core team and lose the valuable know-how
  • When your services team have no documented and structured data about what peculiarities and operational requirements a new piece of software on production has
  • When a developer needs to maintain the code he has not implemented and it costs way too much for the company
  • When you need an overview of your services or product value chain and you don’t know what processes are mapped to your running software in what way. (This will bother you a lot in case of ERP,SOX or CMMI implementations, mergers, attempts to scale the business)

There have been discussions in the communities about the need of a technical writer in the scrum teams to tackle this problem. My reaction to this as a manager would be “what???!!”. Seriously I don’t get why managers have to be satisfied with the output of what scrum delivers to them and find additional ways of obtaining the product attributes they require for their business.

Management does not have to care what possibilities scrum has. It needs product with proper functionality, the required user manuals and documentation for internal operations, it is part of the order and it needs to come in the package. Instead, a manager will likely to hear comments like “if you need documentation that will decrease out velocity”, “ you have to invest more into this project, much much more” “ we have higher priorities than this” or “it is not part of feature development of our scrum process” from the team.

I believe ,again the problem is not about management’s understanding or the teams. Scrum fails to provide the practices and incentives for defining and delivering the documentation with the required amount of detail and quality to an enterprise. It needs to be tackled and implemented in each environment and become a mandatory part of the software engineering process.(If you think you can commit such a sinn against the scrum religion)

Coming up next soon : Scrum Confessions – Part 2 with issues about scrum master role, software architecture problems and application lifecycle challenges..

About fuesunw
Computer Engineer & IT Manager

2 Responses to Scrum Confessions – Part 1

  1. Simas says:

    Hi Füsun, good point about the challenges of the PO role. I do agree that it would take extraordinary people to do everything that you listed as attributes of a good PO. In practise, however, I’ve seen that it is always the team that succeeds or fails. If the PO is less technical, but the developers understand the product and its stakeholders – it can work too. With the right balance in the team, it doesn’t take a superhero PO to succeed.
    I can think of one other problem though, the one that I’m facing right now. In the software world, the product is not finished until it’s no longer supported. Once the active development is over, the team has capacity to potentially start working on something new, and it often does. From my experience, while development effort decreases significantly after product is successfully launched, the PO still has a whole bunch of tasks to do, such as measuring the product, keeping the stakeholder communication alive, finding small improvements and so on. Now repeat the cycle 10 times and if all products were successful, the PO will quickly run out of capacity. And that’s where the problem lies: the POs don’t scale as well as developer teams. The dev teams have the luxury to commit to a certain number of user stories and care of nothing else, while the POs don’t. I think the scrum solution to this would be to have more POs and smaller teams, some focusing purely on keeping existing stuff running. I personally think this is an expensive, but I am a big fan of horizontal scaling.
    Regarding your idea to scale PO by splitting her into two – sort of a vertical scaling. I think it’s difficult and risky, because it adds another layer between the customer and the dev team, which is never good. And I feel that one of the two would in the end de-facto assume the full PO role: either the technical PO will start talking to the customer or the business PO will start talking to the dev team. To successfully have a duality like that, both POs must think and act as one, which will be very difficult to achieve, given such different nature of their work.

  2. James says:

    Hi Fusun, I had a good read of this article; it is fair to say that these are regular occurrences in day to day software development teams who use SCRUM the world over.
    Here is my take on your 2 points.

    The Product Owner
    The role is clearly defined and can be broken down into 2 parts.

    Responsible for the product vision
    * Understanding the „WHAT „ of a product (User Stories, Acceptance Criteria, Backlog)
    * Understanding where THE PRODUCT “VALUE” lies (Measurement (KPIs), Revenue streams?, Process Improvement, Backlog Transparency)
    *Understanding the market environment i.e. the Ecosystem upon which this product lives(Competitors, Product Development, User intent)

    Representative of the Stake Holder
    * Within the Scrum Team context (Communicating Stakeholder Feedback, Backlog Prio, Deriving Value, Working with the Scrum Master)
    * Communication with Stakeholders – (Managing expectations, Gauging Feedback, Agreeing on Deadlines)

    In reality, you find POs who assume themselves the role of “chief cook and bottle washer!” in a Scrum Team as you mentioned ie. They want to do and own EVERYTHING!!

    Areas such as energising and motivating the team or being highly technically proficient re The Dev Lead and worst yet Project manager are not requirements.

    Overly technical POs constantly struggle to reign themselves in within the team dynamic, not only do they want to dictate the WHAT of a product but also HOW it is implemented (which the team is responsible for) and causes real friction and wastes time.
    Simply put POs try to do too much on their own without collaboration and loose focus of the core principles as defined in scrum; Defining Product Vision/Value and The Stakeholders

    Documentation

    The biggest mistake of the Agile manifesto IMO was “we value working software over documentation”. It left itself open to the interpretation that NO DOCUMENTATION is a requirement when this was not implied intention of the statement.

    The driving force behind Agile principles in Rapid Inspection & Adaptation, meaning if you are going to succeed, succeed quickly. If you Fail; fail even quicker.

    Traditional waterfall princples frontload alot of work in the form of documentation especially in the Analysis, Design and Specification phases..Huge docs being created without a single line of code being written.
    I myself have been a victim of huge specification phases (1,2,3) which runs over months only to be told that the product will be shelved for whatever reason. This is a waste of time and a real motivation killer.

    An Agile mindset dictates that we can collaborate on a idea, build a backlog with user stories (a representation of the product) start creating shipable value then test if it works in the ecosytem or not then decide upon cutting early or continue early. This was the idea in mind.

    In terms of creating quality documentation re ERP,SOX or CMMI Agile allows us to control our documentation quuality in our “Definition of Done” criteria.
    This means; the team or the organisation can decide what is acceptable quality documentation for each story and that can be applied and adhered to on a daily basis per story per product as DONE::: You can decide.

    What in reality happens is the focus is always on building software and documentation is often left as last tasks. Most times the devs are then relocated to other products hence docs remain incomplete..

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: