Big Data Play

Introduction With the world gravitating towards digital-everything, connectivity will rise exponentially, and so the use scenarios and the underlying operational data. CSPs are best positioned to be key services...


With the world gravitating towards digital-everything, connectivity will rise exponentially, and so the use scenarios and the underlying operational data. CSPs are best positioned to be key services enablers and custodians of customer data, but for this to happen there must be prudent means to discovery and leverage of data without compromising privacy and security concerns of consumers/customers.

Big Data is no more a hype, but a reality depending on the degree of capability organizations are willing to harness from existing sources. For CSPs such sources include network data, customer events, usage data etc and use towards strategies to boost customer profiling, open data interchange and more.

The limits of Big Data – Leading case

The complexity of consumer behavior and the need to ensure predictability by big Enterprise has led to the new niche for “Big Data” capability and services. The science of Consumer behavior rallies a bet on the ability to collect and collate multiple Consumer events and data “breadcrumbs”, especially across multiple dimensions of consumer activity in order to be able to determine what, who, why, when. For anyone wanting to really understand how to use Big Data, the most practical comparison today, is to observe the intelligence communities like the NSA or CIA (US) based on the US PATRIOT Act and the “Planning tool for Resource Integration, Synchronization and Management” (PRISM) data collection programs or similar initiatives by national intelligence agencies near you.

Given the impasse with Snowden leaks, you’ll understand the principle of the Information Systems used by these corporations to support their decision-making, as formed on the premise of  “Big Data”. The data sources are disparate (variety), happening in instants that require high-speed interception and use (velocity), the data activities themselves are huge (volume) and  representing actual facts at those moments (veracity).

“It is telling that some of the same agencies that spearheaded the creation of previous waves of technology, most notably the relational database, are at the heart of PRISM. If we thought before the PRISM story made the headlines that big data was solely the domain of social networking companies and Internet giants, we certainly have a different understanding now.

However large the big data challenges faced by technology bellwethers are, PRISM is addressing ones that are far greater. We’re increasingly seeing endless real-world uses for big data applications. In the case of PRISM, these uses are both concrete and serious.

Put aside all the noise about PRISM, and it becomes clear that it is a validation of the applications at the platform’s core: namely, cloud and big data. This program uses technologies that the rest of us would recognize as cloud (massively distributed, hardware abstracted, commodity component-based) and big data (Hadoop, machine learning and pattern recognition) on a massive scale.” ~ By Dan Rosanova, How PRISM validates Big Data

Exhibit 1 – Example of a Big Data strategy: US NSA’s Prism Data collection program



Given the peek into the key ideas and boundaries of Big Data, we can appreciate the reasons why big Enterprise today is keen at the need to tap into the consumerization of usage and content data in order to understand consumers, target them with new opportunities, predict likely services of value or enable actions to be able to manage organizational resources more effectively.

With the world pushing towards 50 billion connections by 2015, it means that, practically, a single individual may have up to 5 or more connections or interact with up to 5 digital or connected services one way or the other. These figures are skyrocketing with the introduction of new digital services to replacing traditional services provided by public and private sectors.

In the intelligence community, data sources range from public Wi-Fi, traffic cameras, e-mail, telecom services, banking records – including credit card usage, travel service providers and many more. Imaging attempting to collate all these disparate sources of data, with some being structured and others being un-structured, together to create an informed decision, or measure lagging or leading measures to, for example, make an arrest or intercept a criminal activity. It is an immensely huge and impressively complex undertaking that reduces “casualties” and improves results with precision benefits.

The ability to harness the data generated from the various sources in structured and unstructured formats is primarily what Big Data is about.

Though CSPs have handled large amounts of data for years, the confluence of the telco variety in data sources hasn’t driven any game-changing use until recently. Now “Big Data” provides an avenue to derive new insights – real-time or near real-time – to become more competitive by capturing or creating new business value.

With every new age or new technology, there businesses and individuals who pry on the lack of adequate understanding of technology, its use and impact, on unsuspecting businesses, governments and individuals. The “April fool” of Big Data is not understanding what it is, but how to realize it in your domain, most importantly, what use case to gravitates towards. Depending on the veracity sought,  “Big data” investment can be huge with practically poor return on investment (ROI). The question enterprises and businesses must understand is investment made in “Big Data” must have relevant bases in current and growing business. From a purely storage point of view, data storage is money, and becomes more expensive when it’s from a variety of sources with poor adequacy in use. How much data is stored today that is truly relevant or essential to practical business use tomorrow? Collecting and collating data sources requires ensuring that data can be used to create informed decisions now and in the future, it must have relevancy.


Turning the data into profitable actionable ventures (PAV)

Transform from today’s  Meta-Data analytics to Big Data Analytics (BDA)

The key possibility Big Data brings to traditional Business Intelligence and Analytics is the ability to harness, largely so, the two types of data structures that typically has risen due to the increasing digitization of lifestyle and commercialization and industrialization of everything. The structure of data generated in a CSP’s operations can be split into both structured and unstructured variants:

  1. The Unstructured variant (can be considered the content of a data flow) is basically heterogeneous in principle and doesn’t fit the enterprise relational model or have a clear structure, relationship or consistency. It is heavily text based with a need to implement specialized or sophisticated processing in order to create sense of it. Examples include voice calls, voice, texts, blog posts, web content, file downloads, applications, media content etc. Unstructured data, according to IDC in 2011, will account for 90% of the data that will be created in this decade and remains the new data asset to tap into.
  2. Structured variants, a.k.a meta-data, is composed of data that are easily processed because there is already forced structure and consistency in format, like with XMLs, spreadsheets, databases and sample data sources including billing records, call records, simple network management protocol (SNMP) Information Base records (MIBs), location records, inventory etc.

With many CSPs having already embarked on their journeys to turn the wealth of data they generate into new ventures, Big Data represents an opportunity as well as a challenge to these businesses, IT as a function and indeed the customer. It requires a change in approach in the capture, storage and analysis of data, both structured and unstructured.

When the circle of value is considered with Big Data, the value is based on the confluence of the four V’s highlighted earlier – Volume, Velocity, Veracity and Variety. Value must not only be seen to serve a one-sided focus, where CSPs concentrate on “WIIFM” but also how to serve the data to subscribers and adjacent industries – Open data. This potentially creates new avenues to deliver Data-as-a-Service (DaaS) back to users / subscribers, enterprises by  expose the non-confidential, non-private part of it for cross industry and societal benefity.

“Open data and open knowledge are fundamentally about empowerment, about giving people – citizens, journalists, NGOs, companies and policy-makers – access to the information they need to understand and shape the world around them.” – Rufus Pollock, Founder and Director of the Open Knowledge Foundation


 a. Capturing Value – Play Now

Capturing value attempts to step out of the “rat race” to emerge silently powerful in maximizing customer life-cycle value (CLV), assets and resources to the best of a CSP’s market position. Capturing the residual value in the existing pool of data available to a telecom operator or CSP is an important step to vitalizing value across the CSPs operations first and foremost. For CSPs, this is an important tactical move to re-position their existence in the value chain of subscribers. The act of investing in capturing value from the data generated today has numerous advantages and benefits to customers as well as the society. In particular, it facilitates development, helps to determine resources to guide customers in their choices, enable retailers and enterprise to develop services that drive value and potentially can impact on the whole digital economy in profitable, ethical and responsible services.

“We are at the end of the age of executive gut feeling,” said Sandberg. “Our data can help us answer important business questions. We can stop spamming customers and reach the right ones instead. This data can also be used for societal development, identifying disease outbreaks or predicting impending economic downturns. But it’s important for us to find the right balance between using this data, respecting laws and regulations, and taking care of our customers.” ~ Bjørn Taale Sandberg, Telenor’s Head of Research and Future Studies

The hidden value in the volumes of data generated by CSPs is a sleeping “giant” waiting to be awoken, but through building-blocks of incremental strategies. Also, value can be extended by de-limiting the use of internal existing data through enrichment by to expand the value horizon and helping guide investments around Big Data. Some opportunities as part of the wider set of possibilities include:

–          Helping CSPs to be on the “right” side of the customer, by providing “what” customers just need and not more. For CSPs, this helps to avoid unnecessary capacity investments that impact on their bottom line, and in addition this demonstrates CSPs’ commitment to responsibility in guaranteeing use-case privacy for the data. –          Harnessing internal data sources (IPDR, CDR, SMS, Key-press events, Location data etc) by looking not just at CSPs’ operations but rather how to begin exposing these data to subscribers other sectors such as Healthy, Device Manufactures, Transport etc –          Using the data to influence own day-to-day operations and how customers can leverage CSP services for more (operational excellence, customer engagement and revenue increment) –          Big Data profiling, gaining a richer view of the customer/subscriber over and above pure characteristics into their social likes and dislikes and more (customer intimacy).

At a Telco Big Data event, Sprint claimed they were already generating $10 million in revenue by selling, externally, market insights about their customers. Many other CSPs declared double-percentage-point increases in product sales achieved through better targeting. Guavus, a vendor of telco-focused analytics software, claimed a reduction of network equipment costs for one customer from $1bn to $54m based on better understanding of how the network is being used. And these are just a few of the opportunities that Big Data play 1 can bring.

 b. Creating Value – Play More

Using the existing data and complimenting it with additional external data sources to extend the value creation landscape, Big Data expands the limits of possibilities in combining data sources and structures from the perimeter of the CSP to the community at large. It’s no longer about one-dimensional data but a multi-verse of cross-industry data flow. The Play More era leads to:

–          Combining existing data with new and external data sources (working with cross-industry partners and ecosystems), where CSPs can leverage data from other industries and vice-versa and not just be the end-user, but also expose patterns to subscribers to build their own analytical insights into their day-to-day. –          A multi-lateral exposure of data mining and analytics beyond the borders of CSPs to other industries, the enterprise, healthcare, educational institutions and even users etc. Each play requires taking into consideration the concerns of customer privacy and security, and also ways to leverage security of customer data and lifestyle.


Big Data Profiling (BDP) – Achieving practicality now.

How can processing data from a CSPs operation drive Big Data? How can a CSP achieve this capability given the level of data it already generates and possess, and what use is this in driving Big Data Profiling (BDP)? To appreciate the dimensions of these two questions, we need to understand the degree of profiling that is essential to enable the CSP to capture or create value, and that value, will differ from current and future intentions (strategy), the market and environment and how it sees itself in the bigger play of the digital era.

There is undoubtedly a granted recognition of the potential value in customer data, and the key to this is profiling. However, there is also the potential risks involved in attempting to monetize, even through legitimate means. CSPs are in the best position to be custodians of customer data in the developing “Digital Economy”. With global CPSs recognizing the era of the one-sided business model as not likely sustainable, a more partner oriented side-by-side model brings shared advantages of business-to-business and business-to-consumer engagements. This now represents the single biggest opportunity whose “fruits are ripe for picking”.

There is also recognition (cited from the recent World Economic Forum) that Telco (or CSP) CEOs fear a ‘3 Mile Island’ scenario, where a poor strategy or operation to tackle privacy risks openly, could hold back or even destroy this nascent information services opportunity.[1]  With any increase in inertia by CSPs to take advantage of the side-by-side model, adjacent players increase their propensity and ability to compete head-on with CSPs.

The significant risk of an unnecessarily restrictive regulatory intervention driven by concentrated fears of privacy abuse is a key challenge that needs to be balanced by understanding of possible benefits to users and consumers, organizations and the wider economy. But there are yet greater opportunities that has produced by fair, functional and legitimate means digital business opportunities that can drive better understanding of CSPs subscribers and customers.

  Exhibit 2 – The Telco data: Ubiquitous, multi-functional, mobile


Source: Telco 2.0 Customer Data and Privacy Report

Big data profiling (BDP) focuses on the four “w” – what, who, why, when in a CSPs domain. It uses data from any usage on the network to determine how to improve efficiency in operations, engage the customer better and effectively grow business. How CSPs potentially can create BDP on subscription usage are not limited to such usage that include; (1) mobile wallet and NFC m-commerce activities; (2) mobile health services; (3) cell-location services based on nodes or GPS for location information; (5) other telematics services such as utility services; (6) calling services (voice, video etc); (7) internet services (on-line movies, e-tickets); (8) Social networking and more The data already exists in meta-data or content-data formats one way or the other and provides a rich understanding of usage, customers and their needs.

In order to drive and extend BDP, customer data must be understood from two dimensions: how the data was obtained and how it is to be used. In Exhibit 2, we use the viewpoint from Telco 2.0 by Marc Davis of Invention Arts, as illustration based on analysis of categorization of customer data at the Telco 2.0 brainstorm summit in Orlando 2012,

The sources of personal data can be split into three groups: (1) declared by customers; (2) observed; and (3) inferred. The last group includes many lifestyle characteristics, interests, tastes, communication and relationship preferences, attitudes, beliefs and behavioral patterns. Turning to consideration on how the data is to be used can also be split into four types: (1) internally by the entity collecting the data; (2) externally for services that do not actually share the data but make use of it for supporting third party organizations (Google search is an example of this); (3) externally as aggregate, anonymized data that cannot be attributed to an individual; and (4) sharing Personally Identifiable Information (PII) data externally with third parties. These classifications, collectively, represent a view of potential exploit across customer life-cycle of customer data with varying degrees of customer awareness and implicit or explicit accord.

  Exhibit 3: Customer Data analysis framework – Making sense of consumer data


Source: Telco 2.0 Customer Data and Privacy Report   By using the Telco 2.0 customer data analysis framework, BDP can provide a richer and more vivid view of the customers and enable service delivery in a less intrusive and more constructive way across the customer life- cycle.

This framework paves the way to tap into: (1) the description of customers (demographics, psycho-graphics, behavior); (2) location – locate customers by finding places customers are attracted to, whether a physical location or a cyber space (where do they hang out, where they gather, what they read online and offline, what they search for online); (3) understand their purchasing processes (where do they begin their search, define their problems or needs, benefits to finding solutions); and (4) connect with current customers for additional insights into the what and why behind certain actions by conducting personalized surveys. for example, to improve response from CSP (driving engagement) and achieving intimacy (experience).

This opens up a window to sharing data between CSPs and adjacent players like Web 2.0, Content, traditional service brokers or Internet players to bring potentially huge opportunities to enrich the life of subscribers, customers and society at large. To enable these opportunities, BDP must be initiated as a start of the CSPs enterprise wide marketing strategy.

The data sources for CSPs continue to expand with new services for digital lifestyle being introduced either through partnerships or by OTT conduits. These sources are potentially enriching the capability for BDP bases, but they must be harnessed, and very much so with a customer-centric mindset grounded in privacy, confidentiality and accuracy.

  Exhibit 4 –Communication Services Providers data sources


CSPs are already equipped with multiple data sets through Operational Support Systems (OSS), Business Support Systems (BSS) and Customer Relationship Management (CRM). “Fishing” for patterns from these data sources requires implementing the right customer life-cycle framework, investing in the life-cycle management, aggregation of essential triggers (meta-data and content-data) to actualize essential building blocks for decision and insight. This essentially turns data into assets for and to support customers and the society.

Building a profile on a subscriber in Big Data is no different from the traditional approach. However, Big Data brings technologies and capabilities to extents to which we had not in the past envisaged or included in the delivery of value to both customers and big Enterprise / business.

Today, CSPs lack the capability to deeply extend cross-industry value (intra / inter) which can change with investments to; (1) present valuable data to subscribers; and (2) actualize beneficial use with cross-industry. There is a need to collaborate with the adjacent “parties” to create value from Big Data. However, for that to happen, CSPs must start capturing value from the existing data now as a way to showcase the capability and thus lay down a foundation for newer business models to fruitfully emerge.



Big Data represents both an opportunity and a challenge and there is a need for CSPs adopting the terminology in their business not to be fooled by it. It requires re-thinking what data is needed, security implications, privacy concerns and the required associated technologies.

There is a huge distinction between attempting to implement Business Intelligence and analytics in your current relational data environment and harnessing deep trenched and non-intrusive content to better gain benefits of Big Data. Using Big Data to gain visibility into both CSPs’ business operations and customer usage is no small task. The capabilities of Big Data with CSPs are enormous and spans beyond the perimeter of todays use. The plays of Big Data are clear now and in the future strategies involving capturing value internally through richer reporting into discovery and creating value externally by leveraging avenues around deep-webbed profiling of subscribers/consumers for cross industry use.

Today, CSPs are in a better position to providing closer-to-real customer profiling views. With the growth of mobile and digital lifestyle already heightening exponentially, there is a yet note of caution regarding the need to spell out a middle ground for WIN-WIN across customers and industry. To do this, will require defining the limits of content and meta-data use – Promoting customer confidence, securing data in a new “Big” visibility domain and an assurance mechanism that is overarching the capability.

With Big Data Profiling (BDP), an essential entry point is now open for CSPs to discover essential data assets, capture and creating value that will fuel new opportunities. Big Data “Play Now” is already in force with forward looking CSPs who are taking advantage of data they have today to improve customer use of services as well as facilitate better customer value lifestyle.

Taking the customer lifecycle into consideration, there is an opportunity to transform the era of purely meta-data driven CSP business intelligence into a combined data discovery services that comes from enriched visibility that includes content-based business intelligence in order to better the “Know Your Customer” (KYC) campaign. This will lead to providing relevant services and packages to consumers, reducing the complexity of choice consumers need to make for new services and keep customers fully engaged. Big Data Profiling is a step in the right direction to “Play Now” on your information assets wholly, in order to enrich position of CSPs and pave the way to harness the benefits of the digital economy across customers, CSPs and the society at large.

  [1] Source:


Related By:

  • Performance “manage” to “value” autonomous scenarios in the eye of customers. Why?

    Measuring is a key part to managing. The famous statement, "if you don't measure, you can't manage"....
  • Xpeng’s NGP User Interviews

    黑人工程师克服挑战直至遇到“干杯和烤鱼” 程雷(Emmanuel)已经来中国5年多了。解决问题,适应环境,他一直乐此不疲。 在此之前,他是某公司的首席信息技术官,喜欢“去做别人都不愿意做的”棘手事情。换工作的时候,华为和IBM都找到了他。但是,程雷想要尝试一个更有开创性的领域,所以他选择了前者。 他说,自己骨子里就喜欢挑战,喜欢逆袭的感觉。来自加纳的程雷,正是依靠着新兴科技适应了中国的一切。 作为一名来自加纳的工程师,程雷觉得,人们经常会因为他是黑人而有误解。有一次他去试驾,销售员一来就给他找说唱嘻哈音乐,令他哭笑不得,“他觉得我会喜欢吧”。 语言不通,程雷却靠着百度翻译在中国生活了那么多年。他经常说,挑战促使他往前走,“正是因为这一切,事情才有趣”。 他一直就是这样,灵活适应着周遭的一切。闲暇时候,他会去游泳、骑车、做设计,弹首曲子或者下个厨房,还会去旅… Read more at:
  • The 2021 Contenders Racing for Consumer-ready L3+ Autonomous Vehicles

    Standby as I compile my first-hand experience of the three top contenders with consumer-ready level 3 autonomous cars....
  • Navigation Guided Pilot (NGP) – An Xpeng Highway Self-drive Capability

    XPeng Motors laid out the company’s roadmap for fully autonomous driving implementation on its Tech Day recently. The XPeng’s ADAS system, called XPILOT, is currently on SAE 2.5, which...