Starting text mining with R


Like a lot of Product Managers I use Excel with tools like Google Analytics a lot. Probably like many people I find Excel very frustrating. So having been technical in a previous life, I decided to give R a try. What is R?

 R is an open source programming language and software environment for statistical computing and graphics that is supported by the R Foundation for Statistical Computing. The R language is widely used among statisticians and data miners for developing statistical software and data analysis.
After taking a simple intro course, I started to look around at examples of doing more interesting things. From a work point of view Topic modelling seemed really interesting. Unfortunately a lot of the example code missed a large step (or two) in being useful! 

So I forked the most complete example that I could find in GitHub called "Text-Mining". This did lead me down a bit of a rabbit hole but eventually I had an R script that collected data, cleaned it before doing some analysis. Success! I'm almost a data scientist! ;-) The link has all the details of what I did to get started and some pointers to how you can experiment.

Looking around RStudio the possibilities of what I could do with this were interesting. In particular the R markdown files. The programmatic aspect makes automated report generation a lot nicer than Excel. I think with the chart wizards that Excel is good for one off charting etc. For repeated processing of similar data being able to copy and paste, then tweak in a text editor is much quicker.

With that in mind, the next step was to look at more real world problems/analysis that I wanted to do. So I hunted for some Google Analytics packages. The first I tried was RGoogleAnalytics. This had issues in authenticating and the guide was completely different to the Google site. A little bit more searching and I found googleAnalyticsR, and after installing I managed to get a session. It was at this point that I realised what a faff the Google Analytics API was!


After some work I did manage to get some data out of Google Analytics. This featured a dynamic relative data lookup for the past 200 days. Then group the sessions by day. Turn it into a time series, where I could run the Holt Winters method to get a prediction for how many sessions we would get in the next week.

I also had some mind bending sessions debugging Spanish code and variable names to update an interesting blog on visualising Google Analytics data. I'm looking to see if the sample code is on GitHub anywhere so I can send a pull request with my updates. Otherwise I'll share my own adaptation with some useful R Markdown reports.

Getting stuck in with the "not quite working" Text Mining code was really useful in learning R. Even though it was a toy scale problem, it gave me something practical to develop skills. I'm almost at the same level as Excel now. 

Further reading

Comments

  1. samyuktha13 January 2020 at 10:00

    Ink for all expedite my workflow for posting blog articles to our static site with its integrated Markdown export tool. You can export Hugo-compatible docs as well

    ReplyDelete
  2. Computer Cord13 May 2020 at 18:34

    Great Article. Text mining is really a big thing and I always use R for this.
    Btw, you can add this article: Projects of Data Science in the resources. Its a easy and fun list of mini-projects.

    ReplyDelete
  3. FavTutor4 May 2021 at 05:32

    Great One !!
    please check this out Data Structure and Algorithm

    ReplyDelete
  4. FavTutor24 May 2021 at 06:37

    Nice one!!
    check out this alsoData Wrangling

    ReplyDelete

Post a Comment

Popular posts from this blog

CONFERENCE: TTI Summer Forum 2017 – Getting to Grips with GDPR

Image
T his week I was at the Travel Technology Initiative's Summer Forum . The subject was the new EU (and UK!) data protection laws. These are due to come into force in May 2018. It's a large topic where the individual member state's laws and guidance are evolving. There were three main points that I picked up from the presentations. First from Dai Davis , GDPR expert at Percy Crow Davis & Co. His lively presentation talked about how a large part of the change is in how rights communicated. Before this was by "fairness" through registration of usage with a central body. The shift is to transparency by informing individuals directly. This means that the consumer mindset could then shift to match how legislation is framed. For example, with the repetition of many companies holding personally identifiable information now having to inform what they are collecting and how they are processing it. Currently, awareness isn't high and the compensation not high eno...
Read more

On HBX and online education

Image
I have written this blog post as a reflection on what has occupied me a lot in the past 3 months. After finishing my Post graduate Certificate in Technol ogy Manage ment with the O pen U niversity I was looking at what to do next in continuing my education - being a life long learner! I was looking at different online plat forms for interesting courses.  I was looking at sites like Coursera and  Future Learn , when I came across a new extension to Harvard Business School and a course they run called HBX CORe. This one paragraph pitch they provide sums it up pretty well: HBX CORe (Credential of Readiness) is a 120-150 hour certificate program on the fundamentals of business from Harvard Business School. CORe is comprised of three courses - Business Analytics, Economics for Managers, and Financial Accounting – developed by leading Harvard Business School faculty and delivered in an active learning environment based on the HBS signature case-based learning mod...
Read more

Getting email up and running for side projects

Image
This is  part of a series  about my side project  Bashfully , which aims to give graduates and other new entrants to careers a seasoned professional level way of expressing themselves through the super power of story telling. Following the core principles of being discoverable, personalised and guiding in approach. Photo by Songeunyoung on Unsplash Rectifying my previous mistake - not building community by using email engagement - has proved to be a learning experience! Since our investment is minimal (and we have no real users yet!) the tools that we use all come with some constraint/trade-off. This next bit explain this is going to be slightly technical ... With Heroku it has surfaced as putting in CNAME records in the root of the domain. With register.ly this then stops us adding any other root records, for example mail servers. Cloudfare has a service that will flatten CNAME records for you into A records so everything plays fits together. So a switch ...
Read more