Skip to content

what is big data and what is people data?

Published by orgvue

*This is the third blog post in a series of 6, if you haven’t already please read Part 1 and Part 2. Enjoy.

Defining Big Data and People Data

What is the difference between ‘Big Data’ and ‘People Data’? Big Data is defined as data of high volume, high variety and high velocity. People Data can be defined as datasets describing people in organisations. The two types of data will overlap in some cases, and I’m sure there will be other interpretations of the differences. I have found it useful, however, to try to specify what’s different between two important sources of data for analytics.

Dimension Big Data People Data
Structure Big datasets are better structured People data frequently changes its structure, with new dimensions being added
Stability Big datasets are more stable People data is less stable, with the frequent creation of new datasets
Size Big datasets are larger, extending to billions of rows People datasets are smaller, usually ranging from 100 to 100,000 rows
Systems Big datasets are stored on systems specifically configured for the dataset by database professionals People datasets are created and stored on a mix of systems, including many in excel
Cleanliness Big datasets are relatively clean – with partial records cleansed or omitted People datasets often have high percentages of gaps and errors
Interaction Big datasets tend to be used for reporting People datasets are often edited and added to with changes or comments which become part of the dataset (e.g. manager scoring of performance and notes on employee performance)
Source of Value Big data can be valuable due to sheer volume, e.g. allowing regression analysis of the optimal care pathway for small subgroups amongst millions of healthcare patients. It can be valuable via linked analytics – for example, when Amazon nudges you with the suggestion that ‘People who bought this also bought…’ . The value of people data is in overcoming the incompleteness of data, visualising it in meaningful ways, supporting interaction, modelling and allowing linked analytics: ‘Which site manager training course was linked to the greatest increase in site profitability?’

Distinguishing the Output of Organisation Design from the Process of Organisation Design

Organisation Design can be understood as an output (the noun – the thing you produce) or as an activity (the verb – something you do). I’m arguing that of course Big Data might change the output, in the sense of needing different numbers of people in different places. I don’t see that Big Data change the process. People data are different.

For the Organisation Design process, it is not the large volume of data that makes a difference but the more unusual properties of People data – for example, of being Imperfect, needing Visualisation, being Reflexive, and having linked relationships. These ideas are captured in the table below:

Where ‘People Data’ might affect the Organisation Design process:

What makes it People data? Example Change to the structure? Change org design process? Comment
Imperfect data – incomplete or of doubtful veracity People data might be incomplete, but might still need to be used for org design Could allow simpler and more flexible structures Incomplete data could be used for incomplete design – x% adaptable Organisations have always been re-designed on the back of napkins! However, it would be innovative if organisations  were designed consciously to cope with incomplete data.
Visualisation of data People data particularly relevant for expressing via colour, size, shapes and hierarchical structures No direct impact on structure Adjustmentmore likely if managers see where costs, skills, customer impacts are. Visualisation could affect organisation design – by giving a sense of the organisation more intuitively, it might be more possible to achieve an organisation design that makes sense to more people
Reflexivity of data People datasets can affect themselves – as expressed in the feedback loops in Silverman Research’s Social Media Garden Doesn’t change structure directly Yes – the org design process can evaluate its own progress and adapt An exciting extension of the idea of group training environments where the group is explicitly invited to reflect on its own process, take ownership of it, and improve how it operates
Linked relationships People data is unusually highly linked – to processes, costs, customers, skills, engagement etc. New role for strategy team / MI team? Yes –monitor systemic impact during the change process It has always been hard to link and process the data. As this becomes possible on an ongoing basis, people will be more able to reconfigure their organisations as needed

Imperfect or incomplete data in organisation design:

Incomplete data might be seen as a problem for organisation design. After all, an organisation design is meant to treat the whole organisation as a system – linking people to roles to processes, to competencies to client deliverables and objectives. For example, if we do not know how people currently use their time, how can we understand the ‘As-Is’? How can we instruct people to use their time in the ‘To-Be’?

Linked datasets are valuable in addressing incomplete data because they expose gaps. We understand 100% of our costs. But in a changing world, can we link them to processes? Can we link them to clients? We have listed our organisation’s risks but do we know who is responsible for each one? As people’s roles change and outside factors alter risk levels, can we track who is overloaded? It is easier to sense-check this kind of analysis through linked datasets than it is through simple ‘one aspect’ datasets.

Incompleteness may be deliberate. Google’s 20% time is based on the idea that the most valuable innovations may come from unexpected areas. Google has empowered its employees to spend part of their time on whatever seemed to them to be the most valuable use. It is arguable that this ‘unstructured’ time is actually structured and managed highly effectively. The work done, the choices made and the results achieved are reviewed by peers in briefing sessions. Google’s 20% time is an example of how organisations can be designed flexibly to include information gaps, to convert unstructured innovation to structured value.

For both kinds of organisation design – a fully ‘defined’ organisation or a deliberately incomplete definition – linking is powerful. Linking one dataset to another helps the organisation to be conscious of where it has specified its activities, skills, deliverables and risks, and where it still has gaps.

Visualisation of data in organisation design:

Our recent work with OrgVue has shown us that visualisation affects how organisations can design and manage themselves. The example below shows a business which had historically visualised its cost in tables of numbers or bar charts per division. We mapped people to processes allowing the business to see the cost per process (in this case, a simple example for the cost of HR processes). The impact for the business was to enable the organisation to see itself as a system, and to start making choices around processes that needed to improve and work it wanted to do in new ways:

OrgVue - Visualisation of data in organisation design

Reflexivity of data in organisation design:

Silverman Research’s ‘Social Media Garden’ allows a large group to consider ideas reflexively – not only giving their suggestions, but considering each other’s suggestions, and responding, to develop ideas over several loops and allowing the socially constructed mass of ideas to evolve:

[column col=”1/2″]OrgVue - Reflexivity of data in organisation design[/column][column col=”1/2″] OrgVue - Reflexivity of data in organisation design questions[/column]

This methodology for gathering group ideas genuinely differs from surveying due to its looped nature and differs from a ‘town hall’ meeting because of its greater potential scale and anonymity. It could be applied as a new method for organisation design, to surface issues and evaluate options.

Linked Relationships in Organisation Design:

During the organisation design process, we have found it critical to be able to link aspects of the organisation system to one another, so that impacts can be understood properly. In this example of an IT infrastructure supply company, we mapped (1) customers into customer segments, (2) processes to customer segments and (3) people to the processes that they carried out for customers. This allowed us to understand the connections between the customer segments and the true underlying cost, either at a customer level or a process level. This was vital for understanding true cost to serve per customer. It also helped drive real thinking about opportunities for process redesign.

Sample of process cost linked to Customer Segments:

OrgVue - Process cost linked to customer segments

*This is the end of part 3, please subscribe or visit our blog for the fourth part, “Why is there no economic theory of Organisation Design?”.