Sunday, 26 November 2017

R for Product Management


Photo by Štefan Štefančík on Unsplash
Since my previous blog post I have made some progress on being able to replace most of what I currently use Excel for with R scripts. I have also launchjed a project to collect together some useful recipes for other Product Managers on GitHub called "R for Product Management"

So far I have samples that cover the following data sources:


  1. Google Analytics - to my previous learning have added analysis of browsers used by site visitors and an example Shniy app for data exploration.
  2. InfluxDB - Working with time series data generated by the product, the two examples here are API response times and feature usage. This includes an example of manipulating time series data to set missing values to 0 for plotting.
  3. UptimeRobot - Simple example of taking error data and using a pivot table to explore the data, after some cleaning and filtering. This kind of workflow can be useful with large data sets.

One thing that I really wanted to get working was Elasticsearch, to get an example analysis of components and logging output, including relative error rates. Sadly the only wrappers available at the moment stop just before the version of Elasticsearch we use. Elasticsearch is also the first NoSQL system I have used, so it takes a bit of getting your head around how everything hangs together. Already having example visualisations in Kibana was helpful in working it out though.

Another thing on my to do list is getting standard pirate metrics out of Google Analytics and then format into a nice report. The R Markdown components come with a range of templates that you can use, for example based on the Bulma framework, or you can create your own with your own branding.

This exercise has been really helpful in getting disparate sources of information into a standard reporting framework. Previously I would have tackled this by building some kind of "data warehouse", but actually as long as I can query well enough to get a format that can then be manipulated into a report then it feels like an extra step. Especially where you don't need to drill down further in the report. If something needs more investigation I'll likely go to the source anyway. 

I hope that someone out there finds some value in my R scripts. Please feel free to fork and send my pull requests with any improvements! I am also always happy to talk to data-driven members of the Product Management community who might be interested in using R.


Further reading


Friday, 3 November 2017

Starting text mining with R


Like a lot of Product Managers I use Excel with tools like Google Analytics a lot. Probably like many people I find Excel very frustrating. So having been technical in a previous life, I decided to give R a try. What is R?

 R is an open source programming language and software environment for statistical computing and graphics that is supported by the R Foundation for Statistical Computing. The R language is widely used among statisticians and data miners for developing statistical software and data analysis.
After taking a simple intro course, I started to look around at examples of doing more interesting things. From a work point of view Topic modelling seemed really interesting. Unfortunately a lot of the example code missed a large step (or two) in being useful! 

So I forked the most complete example that I could find in GitHub called "Text-Mining". This did lead me down a bit of a rabbit hole but eventually I had an R script that collected data, cleaned it before doing some analysis. Success! I'm almost a data scientist! ;-) The link has all the details of what I did to get started and some pointers to how you can experiment.

Looking around RStudio the possibilities of what I could do with this were interesting. In particular the R markdown files. The programmatic aspect makes automated report generation a lot nicer than Excel. I think with the chart wizards that Excel is good for one off charting etc. For repeated processing of similar data being able to copy and paste, then tweak in a text editor is much quicker.

With that in mind, the next step was to look at more real world problems/analysis that I wanted to do. So I hunted for some Google Analytics packages. The first I tried was RGoogleAnalytics. This had issues in authenticating and the guide was completely different to the Google site. A little bit more searching and I found googleAnalyticsR, and after installing I managed to get a session. It was at this point that I realised what a faff the Google Analytics API was!


After some work I did manage to get some data out of Google Analytics. This featured a dynamic relative data lookup for the past 200 days. Then group the sessions by day. Turn it into a time series, where I could run the Holt Winters method to get a prediction for how many sessions we would get in the next week.

I also had some mind bending sessions debugging Spanish code and variable names to update an interesting blog on visualising Google Analytics data. I'm looking to see if the sample code is on GitHub anywhere so I can send a pull request with my updates. Otherwise I'll share my own adaptation with some useful R Markdown reports.

Getting stuck in with the "not quite working" Text Mining code was really useful in learning R. Even though it was a toy scale problem, it gave me something practical to develop skills. I'm almost at the same level as Excel now. 

Further reading

Wednesday, 18 October 2017

An evening with Monzo

A meetup that was a bit different for me last night, organised by the Monzo bank's community team. They have had events in their office in London and have expanded this to a roadshow visiting different locations around the UK. Luckily for me they came to Brighton a couple of streets away from my office.

Monzo is a new take on banking that makes use of technology to innovate. It's similar to Root, which is really bare bones "build your own bank", or Entropay, which is going for those that want to easily create online virtual cards. But Monzo is going for a standard bank product, at the center of your finances and re-imagined in a swish app way. It's an overused cliche, but in terms of experience it's like Uber comparing to traditional taxis. I like the product and what they are try and do to help people.

The two talks were “Inside the Partnerships Team” by a developer in that team and "Working from home - not remotely difficult!" (Remote working culture) given by a customer operations team member.IN the first talk Robin from the partnerships team discussed the history of the team and its goals. He then gave a run down of how an upcoming feature went from ideation to development. One thing I took away was the team autonomy, but also how each job function from each team meets to do reviews. From the designers doing critiques, to the product managers in the product council. We have a similar system at 15below for requesting comments, but I really liked the format that Monzo used with nice use of icons to give them a bit of identity. 

The second talk by James about remote working revealed a bit more about how that process was supported by the culture. From the "remote first" approach that leads to consistent and constant recording of what's going on. I though it was interesting that he talked about "1 to 1s" to keep in touch with others referred to anyone in Monzo. I have worked remotely before and work with remote people now and haven't quite experienced that. In my limited experience that kind of regular touch points are with team members or line managers.

It was an interesting evening, I do wonder how much they can grow before the culture changes. But they seem to be making a lot of effort to keep it working as they grow. I think the way that they approach the community model like a startup rather than a bank, will also help this. For example the community voted on how they wanted ATM fees to work, I can't imagine Lloyds doing this! I can't even think of many app companies that collect feedback on their business model that directly.

Saturday, 7 October 2017

First mistakes and successes in the Bashfully MVP process

Since writing about Public beta and starting the MVP process I have gained more insights about our Bashfully launch. These are around what we have got right vs wrong.

Successes

  • Having a pre-release landing page
  • Having a baseline twitter campaign 
  • Using the above to get a baseline conversion ratio

The Twitter campaign had a segment list, budget, and set of content we could re-run. This allowed us to see that the Twitter engagements this time weren't being matched with similar conversion rates.

Exactly what we got wrong is what are trying to prove. But one thing is that we inadvertently changed a variable on the landing page.



This took the logo away from the landing page, moving it to a reusable sign in screen. This also made the registration button a generic button as below


Another theory is that the copy around the registration button made a difference. Luckily this is easier to do a simple A/B test around. So we'll tackle that first!

Whatever the cause another thing we did right was "releasing" early. First by having the equivalent of a download page to gauge interest in the idea. The second was then release a functioning version of the profile. This had about a third of the intended features. The main aim here was to gauge interest and to get feedback on the remaining features.


Overall, I'm please with the progress so far. Trying to adopt an experimentation and data-informed mindset is paying off already. We are able to treat any solutions to problems we find as hypothesis and test them. In the next phase we are also going to step up some qualitative user research.

Further reading


Monday, 2 October 2017

Public beta and starting the MVP process

So deciding to "go live" shouldn't be a difficult decision ... in fact getting people interested enough to use an early stage project is hard enough!

This is the latest instalment talking about going through the MVP process with Bashfully - my side project that hosts online profiles.

Once we did though we got to see some real behaviour. So far this has lead to us:

  1. Cutting two stages from the on-boarding process
  2. Improving the layout and language in on-boarding
  3. Adding a page describing the profiles in Bashfully
  4. Made the "Call to actions" more prominent on the home page
  5. Prioritised some mobile layout fixes after seeing higher than expected mobile usage
Using FullStory has been invaluable here. Being able to play back and share session, to see what people actually do across all users was so helpful. With our initial batch of beta testers (thanks Shaun, Chris and Ed!) we were able to ask what was going on, then check the video. We could then look for the same behaviour in the all the visitors.

Having tweaked and tested with real life testers, we then started sharing to a wider audience. First a mail campaign to the people who had pre-registered. Next creating a profile on BetaList to get a wider range of testers. This has a month's waiting list. So, to generate more interest we have tried a limited Twitter ad campaign.

We have also got into a rhythm of smaller updates to improve the functionality and usefulness of the site. Two big features we are looking at for the next stage, based on feedback are:
  1. GitHub integration
  2. Tailoring profiles and sharing specific versions of them
The first one really fits in with our vision to tell a story and link experiences in one place. With Open Source Software, developer and designers experience is a lot more transparent. We can tap into that, and use authentication from GitHub so we aren't adding to the number of passwords people have to remember!

The second was a requested feature before we even launched the beta. It makes sense, and is one of LinkedIn's weaknesses. We are really giving people the control of their story and how they share it. The ability to remove experiences that aren't as relevant and highlight those that are.

Further reading

Sunday, 13 August 2017

Building an onboarding process for a green field product

Photo by Riku Lu on Unsplash
Onboarding is an important part of B2C and pure "pay to play" SaaS. With so many tools to use, why would any user put effort into getting up and running? With B2B SaaS platforms that include professional services to get up and running then this isn't important. The same impact comes through the service that accompanies the product.

I have been lucky to experience a great onboarding experience when evaluating product management software for work. The best by a country mile was ProdPad. This had two elements, the first was the trial and flow to get up and running. The second was the emails the accompanied it. My previous post discussed the efforts to rectify the previous mistake, in getting email engagement up and running pre-launch, and how we use email in the onboarding process.


So the first decision was how to to go from registering an account to getting to the profile page. Should we create a blank slate and allow the users to build it up? Or should we go to a wizard process to collect details and finish with at least a partially complete profile? Given that the vision was to guide people we went with the wizard approach. 

Kick starting the experience

Next how could we kick start the process? Data entry is usually dull and most people already have some kind of CV or job/education history profile. How could we use this? Well, the most requested import in the beta survey was from LinkedIn. Luckily LinkedIn have a PDF profile download that we could use for the initial version of the registration process. 

Because this is the first experience people are going to have of the app as a user we really wanted it to be the best it could be. So it was the first full user journey where we've created wire-frames. This has gone through a couple of iterations now and is taking shape. It's still a way off where I'd like it to end up, but given how little time we get to spend on it I'm happy with the framework we've built to go through the MVP process.

Lessons learned

The structural bits are relatively easy to get going. We know from the design what the profile will look like at the end. The next challenge is get the guidance and feedback for the content down, but for that we need to launch!

... which brings me onto the most important lesson from this phase. Delivering something now, when you probably won't have any users, is better than delivering "the perfect process" next year. When you probably still won't have any users but you could have built some up and got a year's worth of valuable feedback. 

Also a great way to learn about onboarding is to do a product evaluation and try a number of products solving the same problem and meeting the same need. This really brought home to me how different ProdPad was. It also was interesting to see how empty space affects this experience. One of the products I used had a very minimal design. This looked great when it contained data, but as a new user and no data I felt a bit lost. Almost randomly clicking to find what to do next.


Further reading

Sunday, 6 August 2017

Brand - what is it good for?

Brand is a large part of what people see and remember about products and services. Here I talk about the first couple of iterations of building a brand for Bashfully.


Initial logo





When starting out I created a logo in one evening from some clip art and an online tool. It was this rather fetching and bashful apple to the left in purple. This was used to give the twitter and git accounts some character. But it was always known that it was a placeholder logo.

This became more obvious when the first version of the landing page was created and it didn't really fit anywhere. As we have prepared the design for launch day we have solved both problems, A bit more effort into the design and what it says, plus bringing in the "brand" colours to the site design.

Second iteration

We have moved from the excellent responsive templates provided by html5up to a custom design using the Bulma framework. The html5up site was good as it allowed us to get a decent looking site up and running while we brushed down our rusty CSS skills, and concentrated on the functional side of the site and user research. Now we are getting form to fit the function and it feels the right time to spend more effort on the design and structure.

I have also put some more thought into the logo. We both liked the purple, so have settled on the exact shade. But given the vision I wanted to convey more the "singing the praises" angle. We also both liked strong letter logos like Facebook or Google. I also wanted a font where the B would look a bit like a person ... not sure if I've achieved that!

Another subtle nod is the colour purple, we are helping people channel their inner "purple squirrel" and tell their story.


Building a style guide

I have also started a style guide as we go along, I know what a pain this can be to create afterwards. Here I am using the Mockflow Styleguide app. This has made it dead easy to create. Although one hiccup I did have was in uploading a font file - make sure you don't have a full stop in the name!

It may not look too much at the moment, but as we add more icons and colours into the palate it's going to be really helpful in making sure we stay consistent. The other thing that is going in here are text guidelines as we develop the tone of voices and brand language.

Although I have been through whole brand development at work with the Harrison Agency and at a lower level with the 15below module names, it's not my main focus. So, it has been fun taking more of this on. Another thing that I;ve picked up is that red and green are good, vibrant colours for print work but in digital product you have to be careful. Red is usually used to signify destructive actions or failure. Green means success or creative actions. Likewise blue for info and orange and warnings. Not to say that you can't use them, just more care has to be taken.

My main advice for greenfield projects is, get something as a placeholder. Don't spend too much time on it until the last responsible moment.Then build the style guide as you go along. This bit can be tedious data entry, so do little and often! :-)

Further reading

R for Product Management

Photo by  Štefan Štefančík  on  Unsplash Since my previous blog post I have made some progress on being able to replace most of what I...