Saturday, 30 November 2013

On Big Data and Travel

I've been meaning to write about "Big Data" for a while, especially when as an Artificial Intelligence graduate I saw the Venn diagram at the start of a recent Amadeus sponsored report included both machine learning and natural language processing. Also in Bain & Company's report their research showed that analytical capability had a strong correlation with top performing companies. So I thought I'd sum up some of the articles out there on the subject at the moment.


Is it just hype?

Gartner's analysis is that Big Data is currently at the peak of inflated expectations and heading towards the trough of disillusionment -  and Wired's article asking Is Big Data in the ‘Trough of Disillusionment’? showed areas that is already more advanced.

Is the case that
as the Japan Times OpEd Deflating the hype on big data claims
"big data isn’t much more than a sexier version of statistics, with a few new tools that allow us to think more broadly about what data can be and how we generate it."

I would suspect so from the travel industry articles I'll mention later on.

other articles on the hype aspect:

Problems

The Dangers of Faith In Data by Scott Kerbun describes 6 reasons we should be wary of trusting data:
"1. The Data Paradox. No matter how much data you have, you will still depend on intuition to decide how to interpret, explain and use the data ...
2. No team or organization is Data-driven. Data is non conscious: it is merely a list of stupid, dead numbers. Data doesn’t not have a brain and therefore can’t drive or lead anything...
3. Data is a flashlight. Data gives you specific information about a singular vector of information...
4. Ban the phrase “The data says.” Data can’t say anything for the same reason it can’t drive anything: data is inert. People, including data experts or growth hackers, can never speak singularly for the data...
5. Cognitive Bias pollutes our view of data. We know our brains are kludges, vulnerable to optical illusions. We also have blind spots in our cognition called cognitive biases...
6. Cui Bono -”who benefits?” Who paid for this data? What was their reason for paying for it? What ambitions do they have? ..."

People

Swati M. writing on Medium seems to hit the nail on the head about the hype and problems in We’re Missing the Human Side of Big Data
"We seem to think that Big Data technologies alone will solve or provide insight into business problems. But let’s get back to reality for now - data technologies are merely tools with capacity - they require people to have impact."

But which people? Over at the Harvard Business Review Matt Ariker, Tim McGuire, and Jesko Perry write about the Five Roles You Need on Your Big Data Team

"1. Data Hygienists
2. Data Explorers
3. Business Solution Architects
4. Data Scientists
5. Campaign Experts"
again they talk about the right people and creating the right culture for a successful big data project. This is echoed by Forbes in their Top 10 Strategic CIO Issues For 2013
     "In the past year or so, much of the talk about Big Data has obscured the fact that the real issue is enabling intelligent and instantaneous analysis to provide optimal insights for business decisions. CIOs need to ensure they’re looking at these high-volume, high-velocity challenges in the right way: as business enablers, not tech projects."
So it's not all about the technology and just getting Hadoop installed and a large dataset thrown at it won't solve all your problems.


Strategies

The ThoughtWork's blog Successful Strategies for Analytics and Big Data by John Spens makes the point that
 "Big Data Analytics must focus on insight and action."
How do we achieve that? Well Agility, Big Data and Analytics by Ken Collier (also at ThoughtWorks) details an agile approach and makes the point
"There is a temptation to think of data science as an isolated precursor to application development. This isolation is akin to doing big design up front and is anathema to agile development. If your software solution includes an advanced analytics capability, then data science is best viewed as a role within a cross-functional agile team rather than an isolated specialty."
In The Era of Big Data is For Real Tripp Babbitt has faith that Big Data can help us find the answers to what is currently "unknown and unknowable" and suggests that 
".. how we go about finding and collecting the right data still seems the most worthwhile path."
ThoughtWork's agile approach, with cross functional teams working closely with the business, could be a good method to utilize that focuses on that. As part of a conversation on twitter Tripp also made the following observation
the hype does seem to me to be about "doing" Big Data rather than strategies to deal with Big Data "happening". 

Travel Applications

Working in the travel industry "how can we use this?" is the most interesting question to me. Travel gets a mention in Information week's 6 sectors that are deriving value from Big Data, which is a promising start. 

Within the industry Amadeus certainly think that there is a payoff from big data for the travel industry and have published a report written by Professor Tom Davenport, noting that take up isn't even across the industry for example hotel chains making more use than airlines. The report (available via the article in the previous link) has various examples within the travel industry and I would expect travel management companies to become increasingly apply Big Data Analytics. In a follow up article On value, hype and zettabytes: A big data Q&A with Professor Tom Davenport  he notes online travel agencies as being leading users but most of the industry are behind the finance and retail sectors.


A couple of links that show more practical analysis of data in the travel industry are How big data influences design at Hotels.com, which shows an example of how data is being used for actionable insights, and again in Analysis: dealing with big data the travel companies interviewed seem to be ignoring the hype and investing in the analysis and making best use of the data that they have.

My view is that looking at data is a good way of guiding innovations and finding gaps in the market, does this need to be "Big Data" though? I think that depends on the market and type of question posed, for example in On the optimal distribution of traffic of network airlines by Xavier Fagedaa and Ricardo Flores-Fillolb they use a relatively modest dataset to build a model, which could be used to aid the choice for regional jets or a low cost brand in certain routes.

Update: A couple of hours after hitting publish I read Mark Van Rijmenam's article How Your Travel Journey Can Be Improved With Big Data. This puts forward a vision of airlines and airports sharing data together, which allows the whole journey to be improved both in terms of the service offered and passenger experience.

Other travel related links:

...... and finally 25 Cartoons To Give Current Big Data Hype A Perspective

Monday, 26 August 2013

On innovation and management

In the past month I've been reading a lot and two particular articles caught my eye. The first article I found interesting was a case study by Suleimanagich  (2013) of how even successful implementations of new technology, e.g at the project level, doesn't guarantee success if the company strategy isn't correct.This is the story of how Kodak had invented a digital sensor before any digital cameras were available on the market. This seems to be a common assessment of Kodak's fortunes - they slipped up by not pursuing the innovation - for example "Kodak's Missed Opportunities" by Bergstein (2012) or The Economist's (2012) "The last Kodak moment?".

So at this point in my journey I was thinking, why did Kodak fail? Which brings me onto the second article I found interesting this month. Avlesson and Spicer (2012) puts forward a stupidity-based theory of organisations, could this offer an explanation on why Kodak failed? Did Kodak suffer from what they describe as a "functional stupidity"? They describe this as:

"functional stupidity is inability and/or unwillingness to use cognitive and reflective capacities in anything other than narrow and circumspect ways. It involves a lack of reflexivity, a disinclination to require or provide justification, and avoidance of substantive reasoning. It is related to the intertwined elements of cognition, motivation, and emotion. In many cases functional stupidity can produce positive outcomes in the form of significant benefits to organizations and employees. The narrow and circumspect use of reason, high levels of means-ends oriented intelligence, and the partly positive outcomes, differentiate functional stupidity from ‘pure’ stupidity."
(Avlesson and Spicer, 2012, pg. 1201) 

Going by the CEO's interview in 1995 they certainly seem to have had the his backing, although there is evidence of some lack of reasoning through this quote from a former engineer:

“He told my boss to tell me to stop writing computer proposals, because Kodak would never be a computer peripheral company…not on his watch, at least.”

by why would a senior executive be so against opening up a new market? one clue is earlier on the article where an 800% mark-up on the production costs of a roll of film is mentioned. In that environment you need what the Agile community calls safety - a sense that some risks and the potential to fail are not allowed by how you learn on your way to becoming successful.This blog post by Elssamadisy (2013) describes safety and how productive and high quality agile teams need it.

An alternative theory based on work by Teece (1986) is that for an organisation to make maximum advantage of innovation it needs control of complimentary assets, for example knowledge or products. In this case the sensor and digital image recording is only one component of a camera, Kodak did partner with a Nikon and Canon to produce early high end cameras but they were bulkier re-badged versions of the original cameras. Both companies soon developed in-house digital ranges as the technology matured, Kodak didn't have control of the product so it was easy for the market to slip away from them.
Reading around the subject a bit more I have found a Fortune article from 1995 (Nully and Rajiv 1995) where apparently the main issues were known by the CEO at the time and he did want to build on the digital products. The Economist (2012) is rather damming in its assessment of Fisher's leadership in this period and the outcome for Kodak. Perhaps that's the key, the management at the project level worked, in getting the first still digital camera system, but the leadership throughout Kodak was not suited to what they had. More on innovation and leadership later ...

References

Alvesson, M. and Spicer, A. (2012) "A Stupidity-Based Theory of Organizations". Journal of Management Studies, 49: 1194–1220. http://onlinelibrary.wiley.com/doi/10.1111/j.1467-6486.2012.01072.x/abstract 

Bergstein, B. (2012) "Kodak's Missed Opportunities" 19 January 2012, MIT Technology Review Available at http://www.technologyreview.com/view/426647/kodaks-missed-opportunities/


Elssamadisy, A. (2013) "Why Are Most Agile Adoptions Failing?", 20 June 2013, Industrial Logic, Available at http://www.industriallogic.com/blog/why-are-most-agile-adoptions-failing/

Nully, P., Rajiv, R. (1995)  "Digital Imaging Had Better Boom Before Kodak Film Busts". Fortune, Vol. 131 Issue 8, p80-83.

Suleimanagich, K. (2013) "Kodak’s Problem Child: How the Blue-chip Company Was Bankrupted by One of Its Own Innovations", petapixel.com, http://petapixel.com/2013/06/10/kodaks-problem-child-how-the-blue-chip-company-was-bankrupted-by-one-of-its-own-innovations/

Teece, D. J. (1986) "Profiting from technological innovation: implications for
integration, collaboration, licensing and public policy", Research Policy, vol.
15, no. 6, pp. 285–305.


The Economist (2012) "The last Kodak moment?" 14 January 2012, The Economist, available at http://www.economist.com/node/21542796

Tuesday, 20 August 2013

On user experiences and needs

The Amadeus blog recently posted an interesting take on the fad "less is more" in a post entitled Why the ‘less is more’ concept in software design often falls short of its intentions. At first I agreed wholeheartedly as the title and intro had lead me into some nice confirmation bias ("grrr! gosh darn those gurus selling us different flavours of snake oil every year!"). Then I thought about it a bit more and the new feature being discussed was meeting a need; people wanted it and were pleased it was now present. That made me think that there was possibly some effective product management going on .. lo and behold the post was written by a product manager! This sentence was the most telling :

"how we define the worth of adding more features. For me, it is worth it if you are doing something for the human being who will be using the feature."
So a focus on user need is important, as if by magic later that day I read the brilliant post  How to Work with Designers that suggests the way to get the most out of working with designers is to focus on user need, for example:
"... the language here matters. Make it easier for users to sign up vs. Optimize the conversion rate on the sign-up flow. One approach speaks to the value for the end user. The other approach focuses on what the company needs to do to be successful. Designers generally think and operate in the mindset of the user."
I think this is actually useful advice for all the project team, let what the user will see and/or experience be the lens through which you see that fancy new feature.

Looking at the other side of the fence from a developer's point of view Sometimes less is more in language design probably explains some of the original intentions above. Simplicity should be the base and complexity layered on top as needed, you shouldn't start off complex otherwise things will get unwieldy.

An example from my real life, recently I've moved office building and the new kitchen has a "fancy" microwave. Most microwave ovens I've used have two dials - time and power - and at most two buttons - start, stop/open door. These covers pretty much anything I'd want to set on a basic oven. Unfortunately the makers of this microwave had decided it would be useful to let the user alter the time and power in combination according to the weight and type of food being heated up. This means it has various buttons and an LCD display. It took multiple attempts (including resetting the clock!) before the floor's receptionist rescued me and told me I had to press start for each 30 seconds I wanted to heat it up and then wait, after a short pause of no buttons being pressed it would heating. Here more is most definitely less, it doesn't meet any unmet need I had and is much harder to use the basic functionality.

Another more practical example of when less really is more in the world of IT products came in the Tnooz article Why we ditched mobile and hitched to tablet gives a good example of how when you know the market and have data on how it works you can make decisions to concentrate on features that are useful and meet a need. Although going mobile seems a no brainer and a good feature to have, that's if your product is in advanced booking it might not be the best option. Here is Anshuman Bapna describing how took the decision and viewed the experience they wanted to create:
"we always thought that travel planning should be a lean-back, sit-with-your-partner-on-the-sofa, browse-a-magazine kind of an experience. That screamed tablets, not mobile phones.

So to begin with, we decided to ignore mobile."

So, "less is more" seems to be a poor phrase to use as a guiding principle in product development; and can lead to frustration and miscommunication between different stakeholders. If you're going to use a single phrase I would prefer "build the right thing", the right thing can be a bit vague but I'd try and use these two principles to find out what's right for now:
  • focus on the user needs
  • use your experience in the market place to the maximum

Further reading

Prioritizing Joy By Drew Colthorp at Atomic Object (Posted August 20, 2013) a slightly different look at building the "right product, instead of merely the most feature-complete product." including brand and emotional response.

Sources

Why the ‘less is more’ concept in software design often falls short of its intentions Amadeus blog by Jean-Noel Lau Keng Lun Head of Product Management Corporate, Services and Security, Amadeus IT Group (posted August 1, 2013)


Why we ditched mobile and hitched to tablet Tnooz "Special Nodes" blog post by Anshuman Bapna, CEO and co-founder of mygola, a trip planning portal. (posted August 15, 2013 )

Sometimes less is more in language design "Haskell for all" blog post by Gabriel Gonzalez (posted August 2, 2013)

How to Work with Designers Medium post by Julie Zhou (posted August 15, 2013)

Confirmation bias entry on Wikipedia

Friday, 21 June 2013

BRIEFING: ThoughtWorks' QTB on Mobile and the “shattered future”



Some notes from last night’s ThoughtWorks' Quarterly Briefing on Mobile and the “shattered future”

Session 1 : Giles Alexander
Mobile Lead (Europe) at ThoughtWorks

Talked about move from

Clients -> Apps -> Products

Clients are the old world desktop thick clients that started to get replaced but apps and everyone needed an app on their phone. But now increasingly the world is multi-screen and product is across devices, these are just ways of getting to product. It's not just a case of laptops, smartphones and tablets. Also have to consider other input types/devices like siri, Google glass. there are multiple channels and accessibility is important.

Is Google glass just Segway for your face? e.g. Siri is a bit of a gimmick, not used for regular/serious use, is Google Glass the same?

Couple of examples:

realestate.com.au is a property listings site that started in print media. Business suited the internet well then along came mobile. While developing the app they realised that their asset wasn't the website, it was the property listing DB. The website was just a channel. The natural next step was to do widgets that could be used on other people properties - all just a channel back to their main asset the DB.

Instagram – the app is not the product, social network is but phone form factor isn't great for that. Instead of developing the Android app they could've gone for a desktop web app around the social network side, this could then have been integrated and packaged up with picture taking app into a new iPad app - gain a new way of using product and two new markets.

What is next?

Is payments the next big mobile thing? Giles told an anecdote about being in San Francisco and seeing a Google Wallet NFC reader. He asked what is was and then how often people used it, the answer to the first question was every couple of weeks someone asks that and then about every 4 weeks they use it. Google with Wallet has the technology and channel but no product. Compare this with Apple and iTunes where it has product but not the channel or technology. As a side note aside from people selling NFC solutions I’m not hearing a lot of love for it from the techology industry and nothing at all from actual users, e.g. see http://techcrunch.com/2013/06/12/nfc/ for an example of tech press reaction.

If you are in a pub (or other social setting :) with people with same phones it means nothing, it doesn't open up any extra functionality. Why not? Potential here for direct phone to phone functionality? Apple AirDrop is a move in this direction with wifi direct a potential channel that could open up "next big thing in mobile"

in the Q&A session W3C was talked about especially how around HTML5 they have moved from trying to predict and shape the future with standards to distilling down after the fact the best practices. Like a clean-up operation linking up the technology into a coherent whole. (and yes they will always lag behind as a standards body)

Session 2 : Chris Cheshire
Product Designer at ThoughtWorks

Not many notes here as mainly visual presentation going through agile product design process. Interesting example task around designing a log in screen and then getting feedback from neighbour - all within 60 seconds.

couple of points I did pick out are:

·         Growth leads to opportunity
·         Constraints lead to focus

(the constraints here being around mobile form factor - screen size, input methods etc.)

Went through the Build -> measure -> Learn -> build and repeat cycle from the lean start-up world. Pointed out that "build" doesn't have to be code. Design can be sketch for UI. quote of the evening for me was in this section:

 "lo-fi deliverables, with high quality thinking"

Session 3 with Giles Alexander and Jill Irving (front-end web deisgner) going through some highlights in ThoughtWorks Tech Radar.

HTML5 storage (trial)

* either local or session storage
* For replacing cookies - gets around EU cookie legislation and no back and forth with server - allows more data to be stored - so richer experiences are possible, e.g. in a SPA

Responsive Design (trial)

Although trail in the radar, a few cons on this one were illustrated by Jill's work on the Chime For Change project (http://www.chimeforchange.org/):

* huge development time
* type of application (may not always be appropriate)
* Limitations with automated testing - still very new
*Lack of tooling - again still new

Calatrava - http://calatrava.github.io/  (asses)

* mobile web framework
* gives  you some patterns built in

SMS & USSD as a UI (adopt)

* feature phones still important (about 48% of global market) especially for example in Africa or South America.
* example project was the "M-pesa" money system in Kenya, 1/2 GDP of Kenya goes through the system. No paper and limited infrastructure needed.

Browser-based templates (asses)

* Instead of generating on server and sending to client send a template and JS engine to render.
* Makes SPAs more responsive
* less data, you're operating more on a service level
* Mentioned a couple of frameworks in this area icanhazJS (http://icanhazjs.com/) and mustache (http://mustache.github.io/)

Javascript as a platform and MV* frameworks (trial)

* to be honest not too sure what pros cons were but angular.js (http://angularjs.org/) was recommended (think term data lining came up?) and backbone.js is on hold, as it was first in this space starting to see issues with it.

SASS (adopt)

* SASS - http://sass-lang.com/ - was used on the Chime For Change project
* becoming de facto standard?
* speed in development was increased compared to hand written CSS3 used on projects either side of Chime for Change
* Powerful when used with Compass (Rails framework - https://github.com/Compass/compass-rails)
* Twitter Bootstrap (http://twitter.github.io/bootstrap/) was mentioned as being on hold - mainly as it enforces layers of transformation whether you need them or not.


Summary

Good event and an interesting topic, especially good hearing some context on the tech radar items from people using them. The only downside was travelling to/from London as I work in sunny Brighton, so I didn't get to spend as long talking and networking after the sessions as I would've liked - it was a school night after all!

Wednesday, 19 June 2013

On pets and projects


While watching the Little Cat Diaries the other day, the experiment comparing cats and dogs behaviour to an earlier experiment with children stuck me. In the original experiment the children were pleased to see parents who left then returned to the room as they saw them as a provider of security.

They repeated this experiment first with dogs, who used their owners as points of reference and providers of security. The cats however, were different. They showed more interest in the stranger present in the room if anything. The conclusion was that the cats saw their owners as providers of resources.

I like to think we can make a (fanciful?) analogy here with project managers and the team. Formal command and control processes guide project managers into becoming dog owners. I would suggest that agile techniques put them in the position of herding cats and therefore "merely" providing the resources needed to achieve the team goals.


Be interested to hear peoples thoughts on this, first time I've tried a pet/project management analogy.

Edit 18/02/2015: Since writing this I think it could apply to many knowledge work scenarios with the process owners or management figures as the pet owner type.

Monday, 17 June 2013

On research and application

I am currently lucky enough to be working on a project where the developers are looking into what they can learn from academic computer science and spiking various options to explore the solution space. This got me thinking to how relatively rarely you see this, in my experience programmers tend not to read industrial journals in the same way that say civil or electrical engineers might. 

One thing that I would expect to see more in agile literature are Grice's conversational maxims (Grice 1975)

Maxim of Quantity:

  • Make your contribution as informative as is required for the current purposes of the exchange.
  • Do not make your contribution more informative than is required.
Maxim of Quality:

  • Do not say what you believe to be false.
  • Do not say that for which you lack adequate evidence.
Maxim of Relation:

  • Be relevant.
Maxim of Manner:

  • Avoid obscurity of expression.
  • Avoid ambiguity.
  • Be brief (avoid unnecessary prolixity).
  • Be orderly.
For me these are very good guidelines when producing user stories. I'm not sure if it is wishful thinking but I'm sure I have seen a reference to Grice's Maxims in relation to writing good user stories, but I can't find it now so any help in tracking down an example would be greatly appreciated.

Another piece of research that we can look for lessons in producing good user experiences is the work of Cherry (1953). This was a great favourite to quote at university in our HCI classes, so has stuck in my mind. Here Cherry introduced the cocktail party problem; you can see similar examples in the mix of fonts, colours and style present in poorly designed brochures, presentations and web sites across the corporate world - as parodied by 27bslash6.com

I am sure that there are plenty more examples of more research that isn't more widely known, I did consider talking about deep structure but Chomsky's papers on the subject aren't as easy to read for a non-linguist as Grice. I will be on the look out for papers in fields such as linguistics, psychology and knowledge management that I can learn interesting lessons.

Further Reading

On a slightly tangential note which kind of research is best for industry? applied or pure? Shaun Coffey has written a couple of good blog posts on this
Another set of interesting blog posts are Linguistics for software developers I,II and III at The World of Yesterday.

References

Cherry, E. C. (1953). "Some Experiments on the Recognition of Speech, with One and with Two Ears". Journal of Acoustic Society of America 25 (5): 975–979.
Grice, H. 1975, 'Logic and Conversation', Syntax And Semantics, 3, pp. 41-58, MLA International Bibliography

Monday, 6 May 2013

On impact and behaviour

Something a bit different, some thoughts on impact mapping and a workshop that I was recently lucky enough to attend. One of the first product owner survival camps run by Gojko Adzic with Dave Evans. The workshop description starts with ...
So you're a product owner with an agile team? Congratulations, you are the winner of the world record for the worst named, worst understood and most confusing role in software delivery. Apart from the fact that the PO almost never actually owns the product, they are pushed into an unrewarding role that is mostly about managing "someone else's problem".

That certainly rang true for me ;) how can you take a backlog of items that you are positive that you need right now and prioritise for small, sane iterations. Further, how do you avoid WaterScrumFall in this world? The answer presented here was around impact mapping and expressing things in terms of business goal and how much the "customer" was willing to invest in that goal. Notice the lack of features or cost there? Interesting tack to take in integrating agile projects into an upfront approval/waterfall world.


One way of avoiding the focus on features is to think in terms of the behaviour that you want to change. I think that innovation requires a change in behaviour, so with this method the development cycle should be free to concentrate on innovative solutions to problems not just incremental product development.

The workshop format worked very well for me, talking in terms of business goal and behaviour have seemed easy in theory but going through the process outside of your day-to-day domain and projects provides a practical way to reflect on the process.

Another factor I liked was the intention that the course is the start of building some kind of community. Where the attendees can share how we got on and discuss topics that we didn't have time for on the day. One piece of homework I have for this is writing up my five key learnings from the book "The Connected Company" by Dave Gray.


Edit 07/05/2013
Gojko mentioned a couple of tools for capturing the maps and examples:

  • SpecLog - which I think we are already looking in to at work
  • effectcup - that looks very interesting, but might be another tool too many at this point in time!

Further "reading"

Impact mapping site
BDD and The Agile Business Analyst - Chris Matts (video)

Finding the Truth behind the Story - Dave Evans (video)
Make impact, not software - Gojko Adzic & Dan North (video)
They’re not User Stories - Liz Keogh

Tom Gilb - Evolutionary project management: focusing on the top level critical objectives and delivering real value early in software IT projects & programmes 

Edit 10/05/2013 - couple of extra innovation links
Innovation is about behaviour not products - seems to me impact mapping is all about behaviour change driving the product
3 Keys for Innovation: Why Lean Startup Isn’t Enough - again I see impact mapping as a way of approaching this in a holistic way

Sunday, 10 March 2013

On deployment and enviromental risk

One hot topic that you often get when you have project teams delivering functionality for user acceptance testing while support are testing ad hoc fixes for production issues is "Who owns the environment and how do we stop support and project work from interfering with each other?".

I feel that all code changes and deployments to customer facing sites should have some kind of change management paperwork raised. Because of this you are able to catch scheduling/work conflicts in the change advisory board. A good change control process should help you think about the risk to your project and also help flag to other stakeholders that they might have a risk to consider.

A very basic rule of thumb for deploying anything to production in a manually controlled environment is don't expect that no other changes to have been made since you took your base to work on and always check before merging in. Here I think a "manually controlled environment" is somewhere that developers and application support have write access directly to production and UAT servers and that deployment is typically done manually by a developer or application support developer copying files and installing etc.

In this environment I would consider it a project risk that any UAT sites have the potential to have emergency support changes made and/or in test. A Few things that I would do to help mitigate this in the short term:


•    Use the CAB process as fully as possible, this makes everyone aware of what you intend to change and when 
•    Confirming with support before proceeding and referencing the CAB form – do you need to do anything? Is there any reason we shouldn’t proceed?
•    Check what is there as you deploy – is it what was there before we started development? Have any CABs been raised for these clients/environments that might impact our project?

This is assuming that any inter-project team conflicts over environments can be resolved in regular PM/programme management meetings - I would place the onus on the project work as I would have thought that project deployments should be able to be planned far enough in advance to do this. IMHO it is typically a lot easier for project teams to check and plan for this than it is for support given the nature of the work.

In the longer term it makes sense to migrate those clients to an auto-deployment structure from a "source of truth", for which I would strong recommend a good source control system. Then place stricter controls on environment access at the server level. Ideally I wouldn't want anyone other than operations staff performing anything other than infrastructure tasks directly on the server. For things like logging there are enough ways of piping logs off and examining Windows event logs remotely that developers and application support shouldn't need much - if any - server access.

Monday, 4 March 2013

On documentation and organisational knowledge


It's been a while since I've written up a work orientated blog post. At the moment two (relatively ;) exciting things are happening. One is the introduction of JIRA, Confluence and GreenHopper and the other is working on a series of projects to upgrade customers, getting them onto the latest version of our company's main product. I'm working along the lines of getting three categories of info in JIRA/confluence:

1.       Jira work items
2.       Confluence (upgrade) project wiki
3.       Confluence wiki page per customer

The first item should be quite straight forward and contains information needed for the life time of that work and a point in time record of the system (i.e. what was the state of play when the JIRA was raised/worked on?). For example, a story on a feature has the handover documentation for the original implementation attached as word files, so the current state of play now is easily accessible. I've also then created a subtask to update a linked version of this held in the wiki (the word document content as a wiki page, to do this I've used the standard "Import Word Document" function to upload the handover documents).

The upgrade project wiki holds the "project headlines" of the project, e.g. quick overview of the client and how they will benefit from the upgrade, and any analysis that comes up out of it. This is a record and audit trail of the project (at a higher level than any JIRAs above). This is a natural extension of the project headlines that we had been producing as Word documents moved into JIRAworld.

Finally under the main product project a single page exists per customer and then sub-pages under that per piece of functionality/project. This should document the live version of the customer's system as it is now. This is currently less than ideal, for example looking at one piece of functionality for a client. I've imported the handover documents as they were delivered by project to support, and as you might expect each project only details the work they have done - so you need to read and understand all three links to get the overall picture. Hopefully the upgrade projects will smooth over some of this and detail the whole functionality as upgraded. But this may also require a slight culture change with on-going projects and their PMs to encourage and guide their teams to update the documentation (and not just add their own bit as a new page). This is the new bit where we can add value to what we already do, with very little extra effort.

As you can see the three layers build up in order of longevity and should give oversight about what was done, by whom and when. Also being a wiki we can go back in time and have a look at the page histories to see what the state of play was at any time this can be useful for comparing what a project changed at a high level, by looking at the versions of the customer page before the started and after the work caused the page to be updated (if that makes sense?)

Oh, one more thing I am trying to work on is getting pages that contain guides to some customer specific logic to be aware of and also the projects that either didn't get signed off or aren't quite in the core product yet (added with a warning as reference).

Once pages are in Confluence they can be moved around, so putting them in there isn't wasted effort as and none of this pre-supposes wherever the company decides where our source of truth is, each Confluence page has "Export to Word" and "Export to PDF" options, so we can take versions and save them wherever our hearts desire ... a network drive, Zendesk, Sharepoint, MySpace, Yammer, Salesforce .... the list goes on (as do I ;). This is one of the reasons I’ve pushed on getting it all in Confluence now, I know that I can export updated documentation as required at the end of each upgrade project and that it will be in a similar format to what we have now (if that's required).


Further Reading

One thing that is important is having a common language used by all the company's employees. The next step would be to also build up a glossary within the same wiki framework. OpenBet's approach to knowledge management uses a third party plugin to extend the basic Confluence pages, read about on the Atlassian blog here and part two here

Starting text mining with R

Like a lot of Product Managers I use Excel with tools like Google Analytics a lot. Probably like many people I find Excel very frustratin...