InsideBIGDATA Guide to

Big Data for Manufacturing

by Daniel D. Gutierrez

BROUGHT TO YOU BY

Guide to Big Data for Manufacturing

Big Data for Manufacturing An Overview

Manufacturing concerns consistently
have sought ways to reduce waste and
variability in their production processes to
dramatically improve product quality and
yield (e.g. the amount of output per unit
of input). Further, these companies need
a granular approach toward recognizing
and correcting manufacturing process
flaws. Big data technology provides
just such an approach and many hightier manufacturers possess a significant
degree of interest and motivation in
adopting the big data technology stack.
Big data analytics refers to the application of
tools based on principles of computer science,
statistics, data mining algorithms and mathematics
to enterprise data assets with the fundamental
goal to assess and improve business practices.
In manufacturing, operations managers can
use big data to drill down into historical process
data, discover previously unidentified knowledge
among discrete process steps and inputs, and
then optimize the factors that are shown to have
the greatest effect on yield. Many manufacturers
across a broad range of industries now have an
abundance of real-time shop floor data and the
capability to conduct sophisticated statistical
learning assessments. They are taking previously
siloed data sets, aggregating and joining before
analyzing them to reveal key insights.

Even when considering manufacturing operations

that are thought to be best in class, the use of big
data may reveal further opportunities to increase
yield above industry benchmarks. In addition,
companies can reduce their waste of raw materials,
reduce energy costs and thus increase profitability
by rigorously assessing production data, all without
having to make additional capital investments or
implementing major change initiatives.

Most manufacturers collect vast

troves of process data but typically
use them only for tracking purposes,
not as a basis for improving
operations.
The essential first step for manufacturers that
want to use big data to increase yield is to consider
how much data the enterprise has at its disposal.
Most manufacturers collect vast troves of process
data but typically use them only for tracking
purposes, not as a basis for improving operations.
The challenge is for these players to invest in
the systems and skill sets that will allow them to
enhance their use of existing process statistics. For
example, it might be prudent to index information
from multiple sources so it can be analyzed more
easily and hire data scientists who are trained
in identifying patterns and drawing actionable
business insights from the data.

Contents
Big Data for Manufacturing
An Overview. . . . . . . . . . . . . . . . . . . . . . . . . 2
Common Big Data Pain Points
for Manufacturers. . . . . . . . . . . . . . . . . . . . 5
Big Data Technology for Manufacturing. . . 6
Analytics Layer. . . . . . . . . . . . . . . . . . . . . . 6
Data Integration Layer. . . . . . . . . . . . . . . . 7
Data Management Layer . . . . . . . . . . . . . 7

Infrastructure Layer. . . . . . . . . . . . . . . . . . 7
Professional Services. . . . . . . . . . . . . . . . . 7
Big Data Technology Stack
for Manufacturers. . . . . . . . . . . . . . . . . . . 8
Adopting Big Data for Manufacturing . . . . 9
Case Study: Omneo. . . . . . . . . . . . . . . . . . 11
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . 12

www.insidebigdata.com | 508-259-8570 | Kevin@insideHPC.com

Guide to Big Data for Manufacturing

Big data savvy is nearly a necessity

these days because many companies
feel if they dont adopt a data-centric
business strategy it could affect their
competitive landscape in the next
3-5 years.
Some manufacturers, particularly those with
lengthy production cycles, have too little data
to be statistically meaningful when put under a
data scientists magnifying glass. The challenge
for thought leaders at these companies will be
taking a long-term focus and investing in systems
and practices to collect more data. They can invest
incrementally, e.g. gathering information about
one particularly important or particularly complex
process step within the larger chain of events, and
then applying sophisticated analytics to that part
of the process.
Big data is becoming a strong competitive
advantage in the manufacturing industry. In fact,
big data savvy is nearly a necessity these days
because many companies feel if they dont adopt
a data-centric business strategy it could affect their
competitive landscape in the next 3-5 years. As a
result, manufacturing businesses, across a wide
range of categories and verticals, are exploring
ways to adopt big data.
Here are just a few areas of interest for big data
under the manufacturers prism:
M
 anufacturers are seeing a higher degree
of visibility into supplier quality levels, and
improved accuracy in predicting supplier
performance over time. Using big data,
manufacturers are able to view product
quality and delivery accuracy in real-time,
making trade-offs on which suppliers receive
the most time-sensitive orders.
Selling only the most profitable customized
or build-to-order configurations of products
that impact production the least. For many
complex manufacturers, customized or build-

to-order products deliver higher-than-average

gross margins yet also costs exponentially
more if production processes arent well
planned. Using big data, manufacturers are
discovering which of the myriad of build-toorder configurations they can sell with the
most minimal impact to existing production
schedules to the machine scheduling, staffing
and shop floor level.

Big data can also show quality,

performance and training variances
by each machine and its operators.
This is invaluable in streamlining
workflows in a production center,
and is becoming increasingly
commonplace.
Measuring compliance and traceability to
the machine level becomes possible. Using
sensors on all machinery in a production
center provides operations managers with
immediate visibility into how each is operating.
Big data can also show quality, performance
and training variances by each machine and
its operators. This is invaluable in streamlining
workflows in a production center, and is
becoming increasingly commonplace.
Quantify how daily production impacts
financial performance with visibility to the
machine level. Big data is delivering the missing
link that can unify daily production activity to
the financial performance of a manufacturer.
Being able to know to the machine level if the
factory floor is running efficiently, production
planners and senior management know how
best to scale operations. By unifying daily
production to financial metrics, manufacturers
have a greater chance of profitably scaling
their operations.

www.insidebigdata.com | 508-259-8570 | Kevin@insideHPC.com

Guide to Big Data for Manufacturing

For an important perspective on this new found

awareness, the Industrial Internet Insights Report
for 2015 sponsored by GE and Accenture considers
the rise of the Industrial Internet, the combination
of big data analytics with the Internet of Things
(IoT), and how it is producing huge opportunities
for manufacturers. The report includes a number
of telling highlights for how manufacturers see big
data affecting their bottom lines:
1. 67% of manufacturers report strong boardlevel support as primary influencer for big
data initiatives.
2. 87% of manufacturers report big data is one
of the top three priorities.
3. By introducing big data analytics and
more flexible production techniques, manufacturers could boost their productivity by as
much as 30%.

 0% of manufacturers report interest

7
in using big data to gain insights into
customer behaviors, preferences
and trendsmore than any other
industry.
The goal for this Guide is to provide strategic
direction for enterprise thought leaders in the
manufacturing sector for ways of leveraging the
big data technology stack in support of analytics
proficiencies designed to work more independently
and effectively in todays climate of striving to
increase the value of corporate data assets.
Source: Accenture

4. 70% of manufacturers report interest in

using big data to gain insights into customer
behaviors, preferences and trends more
than any other industry.

Top business priorities by industry shows manufacturing sectors desire to gain insights.

www.insidebigdata.com | 508-259-8570 | Kevin@insideHPC.com

Guide to Big Data for Manufacturing

Common Big Data Pain Points for Manufacturers

With the rise of big data, global manufacturers
recognize the importance of data as the new
currency and as a competitive differentiator. Data is
being created and consumed at rates never before
seen and this effect especially is prevalent in the
manufacturing sector. With this data explosion,
manufacturers are recognizing the need for more
than traditional, structured systems to take control
of their data. But there are significant pain points
in getting up to speed to take advantage of the
data deluge. For instance, many manufacturers
dont have access to the tools that they need for
deploying big data solutions to get results faster,
to perform required calculations regardless of
what theyre manufacturing.
At a high-level, many manufacturers share common
pain points when undergoing a big data initiative.
Here are a few of most frequently mentioned:
Needing to crunch more data in less time
Keep your organization ahead of the data
deluge so that your decisions are based on
information from the analytics. Even the most
advanced big data solution will not benefit an
organization if it takes too long to get insights.
Ensuring the right people have access to big
data results If the right people do not have
access to the right tools to deliver insights,
it will not matter how much data you have.
Effectively handling data quality and
performance The goal must be to build a
big data infrastructure that aligns with your
business goals and delivers actionable, realtime business insights that you can trust.

If a manufacturer wants to get started with big data

to address their own set of pain points and build
a use case in their environment, one good path
toward success is the Dell QuickStart for Cloudera
Hadoop program which offers an effective pointof-entry for enterprises to begin managing and
analyzing data. An all-in-one system designed to
reduce the complexity of deploying, configuring,
and managing Hadoop systems, includes the
hardware, software and services needed to deliver
a Hadoop cluster that will start organizations
on a proof-of-concept to begin working with big
data. With this program, Dell is building on its
deep expertise and relationships in working with
Cloudera and Intel. The solution represents a
unique collaboration in the big data ecosystem
to collectively deliver an easy and affordable way to
get started with Hadoop.
Dell QuickStart for Cloudera Hadoop enables
organizations to quickly engage in Hadoop testing,
development and proof of concept work. Through
the combination of Dell PowerEdge servers with
Intel Xeon processors, Cloudera Enterprise
Basic Edition and Dell Professional Services,
organizations can quickly deploy Hadoop and
enable development and application teams to test
business processes, data analysis methodologies
and operational needs against a fully functioning
Hadoop cluster.

Needing big data solutions that scale to

fit your business To get the right insights
into the right hands at the right time, you
must have a flexible, scalable big data
infrastructure that can reliably integrate
front-end systems with back-end systems
and keep your business up and running.
Being able to expand your companys data
handling strategy by having the ability
to analyze various types of data including
traditional structured data, but also newly
available unstructured and semi-structured
data sources.
www.insidebigdata.com | 508-259-8570 | Kevin@insideHPC.com

Guide to Big Data for Manufacturing

Big Data Technology for Manufacturing

In order to consider big data solutions for
manufacturing in a holistic manner, the following
diagram divides up big data into four primary
componentsanalytics, data integration, data
management, and infrastructure. In addition, Dell
provides state-of-the-art big data solutions in part
with the help of strategic partners like Cloudera,
Intel, Microsoft, Oracle, and SAP along with its
professional services arm to ensure a quality big
data deployment that can address enterprise data
processing requirements at scale.

Analytics Layer
With analytics, the strategy is to turn enterprise
data assets into better and faster actionable
insights through managing data, processing
data, storing data, and eventually analyzing data.

For analytics, Dells Statistica for Big Data

Analytics platform is a good option as it extends
the Statistica portfolio with advanced natural
language processing (NLP), entity extraction,
interactive visualizations and dashboards, and
distributed advanced analytic models across
Hadoop, databases and database appliances. The
unique aspect of Statistica is that it has over 4,000
different models created for different vertical
markets, including some for manufacturing. The
attractiveness of Statistica is that it has these
models created so manufacturers will not have to
start from the ground up to build models to analyze
their data, whether its a model for manufacturing
efficiency, or a model to look at failure points within
a product.

www.insidebigdata.com | 508-259-8570 | Kevin@insideHPC.com

Guide to Big Data for Manufacturing

Data Integration Layer

In building a solution focused on data analytics, a
manufacturer must first take the steps necessary to
implement the end-to-end solution. One key driver
is how to integrate the data into a suite of big data
tools in order to manage it, process it, store it, and
analyze it. For data integration, Dell solutions like
Boomi and Shareplex resonate in a significant way.
Boomi is a data integration platform and is unique
in that it allows you to integrate any type of data
source (e.g. log data, social media data, sensor
data, machine data, etc.) into an application, plus
you can integrate that data source on premise or in
the cloud. The Shareplex Connector for Hadoop is
another important tool which is key here because
most of the time the first step enterprises take with
big data is to integrate their existing data assets
from a relational database or enterprise data
warehouse. Shareplex enables you to easily load
and continuously replicate changes from an Oracle
database to a Hadoop cluster. The manner in which
the data transfer takes place is not simply a big
download, but rather with Shareplex you replicate
the Oracle data directly into Hive or HBase and
HDFS environments, essentially streamlining this
cumbersome task.

Data Management Layer

When building your big data solution, at an early
point in the journey you must determine where
you will put the data so that you can then manage
the various types of data that will be a part of your
eventual analysis. When dealing with diverse types
of data, one option for the management layer
is to deploy Hadoop as your data management
platform. Hadoop allows you to collect, manage,
analyze and store data in a scalable, flexible and
cost effective solution. The reason this is key and
the reason Hadoop is recommended is because
Hadoop allows you to store the data in its native
format. One of the biggest problems today with a
relational database is before you can actually store
data in the database you must clean it, parse it,
and make it fit in a table, row, and field within the
database. With Hadoop, you will not have to do any
of that initial work. You can deliver the data as is
and eventually when you want to start analyzing
it thats when you begin the cleaning process. This
is why enterprises like it, where organizations find

the value in Hadoop, because whether that data is

structured, unstructured, or semi-structured, you
can put it in Hadoop today. Hadoop is the tool of
choice for the data management layer.

Infrastructure Layer
Successful big data deployments depend on reliable
hardware infrastructures. The Dell PowerEdge
R730xd, based on Intel Xeon processor technology, offers an exceptionally flexible and
scalable, two-socket 2U rack server that delivers
high performance processing and a broad range
of workload-optimized local storage possibilities,
including hybrid tiering. This is a hardware solution
ideal to run the Hadoop distributed computing
platform for a solution to big data problems.
Additionally, Dell, together with Cloudera
and Intel, provides a turnkey, purpose built
in-memory advanced analytics data platform. The
Dell In-Memory Appliance for Cloudera Enterprise
represents a unique collaboration of partners
within the big data ecosystem. Together Dell,
Cloudera and Intel deliver both the platform and
the software to help manufacturers capitalize on
high-performance data analysis by leveraging the
Cloudera Enterprise in-memory features (Apache
Spark) for interactive analytics and multiple types
of workloads. Cloudera Enterprise also features
Impala for fast query and Cloudera Search for
interactive search. The result one tool for both
processing and analytics.

Professional Services
If a manufacturing firm needs professional
services to come in and build the entire
solution, deploy and integrate, essentially
build a solution from the ground up, Dell has
the big data services to deliver that to you. In
addition, there are the Dell Solution Centers
if youre new to this technology and you dont
know how it will act in your environment, you can
go to a Dell Solution Center to do a proof of concept
without making a big investment Dell offers this
as a free service to their customers. Finally, Dell
Financial Services can package up new and creative
ways to finance these types of solutions so you dont
have to take it out of your capital expenditures but
instead make it an operating expense.
www.insidebigdata.com | 508-259-8570 | Kevin@insideHPC.com

Guide to Big Data for Manufacturing

Big Data Technology Stack for

Manufacturers
The available technology stack for applying a big
data methodology to manufacturing applications
is growing by leaps and bounds. Here are the top
level components of a big data initiative.
Big Data Software the early focus is on endto-end solutions for big data starting with
integral software applications like Statistica
for Big Data Analytics. With this content
mining and analytics solution, youll transform
complex and time-consuming manipulation
of web-scale data resources into a fast and
intuitive process. You can harvest sentiments
from Twitter feeds, blogs, news reports, CRM
systems, and other sources, and combine
them with demographic and regional data
to better understand market traction and
opportunities.
Hadoop In order for a manufacturing firm to
extract value from an ever-growing onslaught
of data, the organization needs nextgeneration data management, integration,
storage and processing systems that allow the
company to collect, manage, store and analyze
data quickly, efficiently and cost-effectively.
The Hadoop distributed computing platform
provides end-to-end scalable infrastructure,
leveraging open source technologies, to allow
you to simultaneously store and process large
datasets in a distributed environment for data
mining and analysis, on both structured and
unstructured data.
Hadoop is scalable, fault-tolerant and
distributed. The open source software was
originally developed by the worlds largest
Internet companies to capture and analyze
the massive amounts of data they generate.
Now, manufacturing companies are climbing
aboard the big data bandwagon with Hadoop
as their chosen architecture. Unlike earlier
platforms, Hadoop can store any kind of data
in its native format and be used to perform a
wide variety of analyses and transformations
on that data.

Using Dell | Cloudera Apache Hadoop

solutions for big data, Dell offers three ways to
initiate your journey: deployment of the Dell
QuickStart for Cloudera Hadoop packaged
solution, exploration of Hadoop software via
a Dell Solution Center and on-premises work
with a fully functioning Hadoop environment
via the Dell Hadoop Pod Loaner Program.
Security for Big Data Security is a key
component for all big data projects. All solution
designs must encompass performance,
access, compliance and security. Security
should be defined at all levels of the system
implementation and account for both atrest and in-flight data. Big data systems
introduce new challenges for security that
must be accounted for including data plus
data policies and the handling of documents
that may contain multiple levels of security.
Big data projects are unique and should be
carefully crafted and designed beginning
with a use case definition and then allowing
teams to work with low risk data as the focal
point to enable organizations to become
comfortable with new technologies as well as
to determine how best to ensure the solution
can be implemented to conform to corporate
security policies.
Compliance is an imperative part of the security
of data. Strong tools must be deployed as part
of any big data solution to ensure that all data
access and use can be reported on, and alerts
generated for inappropriate data access. As
data sets become more complex and more
disparate data sets are integrated, ensuring
compliance will become more difficult, but
can be managed if data is integrated in steps,
rather than all at once.
One good option for a manufacturer to
employ for securing their big data solution
is Dell SecureWorks, an information security
service organization. Dell SecureWorks helps
organizations worldwide protect their IT
assets, comply with regulations and reduce
security costs. These managed security
services clients range from small local
manufacturers to global industry leaders.

www.insidebigdata.com | 508-259-8570 | Kevin@insideHPC.com

Guide to Big Data for Manufacturing

Adopting Big Data for Manufacturing

The big data revolution is dramatically changing
the manufacturing industry. The following driving
factors have created opportunities for growth and
have motivated the need for manufacturers to
collect, store and analyze massive volumes of data
leading to the adoption of big data technology:
Product quality/defect tracking
Supply planning
Manufacturing process defect tracking
S upplier/supplier components/parts
defect tracking
C
 ollecting supplier performance data
to inform contract negotiations
Forecasting of manufacturing output
Increasing energy efficiency
S imulation and testing of new
manufacturing processes
Enabling mass-customization in manufacturing
In realizing the above benefits, many manufacturers
have implemented practices using big data
including: log aggregation, monitoring, analysis
and reporting; integrating sensors or embedding
a log inventory system with machines on the
manufacturing floor; this has enabled them to gain
an understanding for what the machines are doing
allowing the data-driven organizations to grow,
protect and bring added value to their business.

Koehler Paper Group

An example of a successful big data
adoption in manufacturing is Koehler Paper
Group. The company streamlined their
data warehouse systems, reducing costs by
30%, giving managers and executives up to
the minute monitoring of manufacturing
processes, allowing for much faster and
smarter decisions to be made. The new
system cuts data loading times from five
minutes to five seconds. Koehler worked
with Dell to deploy a new SAP HANA
database solution on Dell PowerEdge
servers with Intel Xeon processors in just
three days.
The Maturity Pyramid diagram below is a good
way to visualize the big data adoption process.
We can see that every new big data adoptee
starts at a different spot, i.e. every business has
different tools, every business has a unique set
of skills. With the maturity pyramid you need to
take an executive approach to this realization,
e.g. a consultative approach to the process as it is
important to have clarity for where the company is
positioned. The maturity pyramid is a good ground
point for manufacturers to understand how to get
to analytics.

www.insidebigdata.com | 508-259-8570 | Kevin@insideHPC.com

Guide to Big Data for Manufacturing

Recent findings from a survey completed by LNS

Research and MESA International Attitudes
on How Big Data will Affect Manufacturing
Performanceinclude where big data is delivering
the greatest manufacturing performance improvements today. The highlights from this survey
include the following top three areas that big data
can improve manufacturing performance: better
forecasts of product demand and production
(46%), understanding plant performance across
multiple metrics (45%) and providing service and
support to customers faster (39%).
As a clear indication for how solidly Dell is behind
the big data adoption of big data solutions, Dell
boasts several significant results! Dell, as a large
manufacturing company, is taking this big data
journey as well, and the company can speak to how
it has achieved its goals, how the company built its
big data solutions, where it failed, and what it will
implement next time.

SAS to Statistica migration

Dell works with customers in its Solution
Centers to help them leverage Dells own
experience with building big data solutions
Dells own metrics demonstrates success
when adopting big data (see chart below)
Dell has a great deal of expertise in building big
data reference architectures, in that the company
built the first custom designs for big data platforms
as early as 2009. Dell has been in the big data space
for that length of time, since 2009 with the platform
and the first reference architecture in 2011.

Dell is adopting big data solutions together with

internal IT teams in order to enact the following
enterprise initiatives:
ETL offload
SAP HANA deployment
www.insidebigdata.com | 508-259-8570 | Kevin@insideHPC.com

10

Guide to Big Data for Manufacturing

Case Study: Omneo

A division of global enterprise manufacturing software firm

Camstar Systems, now a wholly-owned subsidiary of Siemens
For a compelling example that
illustrates how big data is affecting
the manufacturing sector, we
can consider Omneo, a provider
of supply chain management
software for manufacturing companies. The business need was
to enable global manufacturers
to efficiently manage product
quality/performance and customer
experience. Consequently, Omneo
needed to collect, manage, search
and analyze vast amounts of
diverse data types, and it sought
the right software and hardware
infrastructure to support this
effort. The organization worked
with Dell and Cloudera to build a
software solution on top of the
Cloudera of Distribution Hadoop
(CDH) platform running on a
cluster of Dell PowerEdge C8220
servers with Intel Xeon processors, giving customers total product data
visibility throughout their entire supply chain.
The benefits that the Omneo solution offers to
manufacturers are:
Enables global-brand owners to manage
product performance and customer
experience
Delivers a 360-degree view of supply
chain data
Searches billions of data records in less
than three seconds
Scales to support 300 million records
every month
Allows customers to quickly search, analyze
and mine all their data in a single place so
that they can identify and resolve emerging
supply chain issues
Helps manufacturers and suppliers
detect emerging issues

Searches billions of data records in less

than three seconds
Scales to support 300 million records
every month
Saves millions of dollars and boosts
productivity
Improves product quality, performance,
customer experience and compliance

We are able to help customers search billions

of records in seconds with the Dell infrastructure
and support, Clouderas Hadoop solution, and
our knowledge of supply chain and quality
issues, says Karim Lokas, senior vice president
of marketing and product strategy for Omneo,
a division of the global enterprise manufacturing
software firm Camstar Systems, now a whollyowned subsidiary of Siemens. With the visibility
provided by this solution, manufacturers can put
out more consistent, better products and have
less suspect product go out the door.
www.insidebigdata.com | 508-259-8570 | Kevin@insideHPC.com

11

Guide to Big Data for Manufacturing

Summary
In this Guide we have delivered the case for the
benefits of big data technology applied to the needs
of the manufacturing industry. In demonstrating
the value of big data, we included:
An overview of how manufacturing can
benefit from the big data technology stack
A high-level view of common big data pain
points for manufacturers
A detailed analysis of big data technology
for manufacturers
A view as to how manufacturers are going
about big data adoption
A proven case study with: Omneo

Dell Statistica offers an extensive portfolio of solutions including a complimentary online resource
and textbook on statistics that me be found at http://www.statsoft.com/textbook.
Additionally for manufacturing we offer 15 subcategories of scenarios and solutions outlined at:
http://www.statsoft.com/Solutions/Manufacturing

Intel, the Intel logo, Xeon, and Xeon Inside are trademarks or registered trademarks of Intel Corporation
in the U.S. and/or other countries.

www.insidebigdata.com | 508-259-8570 | Kevin@insideHPC.com

12