Featured Posts (645)

How IoT can benefit from fog computing

fog computing

By Ben Dickson. This article originally appeared here.

What I’m mentioning a lot these days (and hearing about it as well) is the chaotic propagation and growth of the Internet of Things. With billions of devices slated to connect to the internet every year, we’re going to be facing some serious challenges. I’ve already discussed howblockchain technology might address connectivity issues for huge IoT ecosystems.

But connectivity accounts for a small part of the problems we’ll be facing. Another challenge will be processing and making sense of the huge reams of data that IoT devices are generating. Close on its heels will be the issue of latency or how fast an IoT system can react to events. And as always, security and privacy issues will remain one of the top items in the IoT challenge list.

Fog computing (aka edge computing) can help mitigate – if not overcome – these challenges. As opposed to the cloud, where all the computation takes place in a central location, fog computing pushes the computation of tasks toward the edge of the network and distributes it among smart routers or gateways. The term and concept was coined by networking giant Cisco even before the IoT became a buzzword, but it was the advent of the Internet of Things that provided it with true, legitimate use cases.

Here are some of the domains where cloud computing can deal with the challenges of IoT.

Computation and data processing

Naturally, computation problems will be one of the main reasons we’ll descend from the cloud and wade into the fog. A problem lying ahead of us is the sheer amount of computation and data processing that IoT ecosystems will require.

With Machine-to-Machine (M2M) communications accounting for most of exchanges in IoT ecosystems, the amount of traffic that will be generated will be incomparable to what we’re used to deal with in human-machine settings. Pushing all of these tasks to the cloud will overburden centralized computation nodes and require bigger and stronger cloud servers.

The cloud is best known for its huge storage and analytics capacities. Meanwhile, many of the tasks and events that take place in IoT ecosystems do not require such capabilities and sending them to the cloud will be a waste of precious resources and will only bog down servers and prevent them from performing their more critical duties.

Fog computing can address this issue. Small computational tasks can be performed at the edge (IoT gateways and routers), while valuable data can continue to be pushed to the cloud. This way, precious cloud resources for can be saved for more suitable tasks such as big data analysis and pattern recognition. Reciprocally, functionality and policies of edge devices can be altered and updated based on insights gained from cloud analytics.

This model will also help address response time and latency issues, which is discussed next.

Response times and latency

Rather than requiring huge computational resources, many of the transactions and decisions being made in IoT systems are time-critical. Imagine a telemedicine scenario, or an IoT-powered hospital, where seconds and milliseconds can make a difference for patients’ health or life. The same can be said in industrial settings and work areas, where quick response can prevent or mitigate damage and safety issues. A simpler example would be parking lights that would have to respond to passage of cars and pedestrians, but must do so in a timely fashion.

Other settings that require large bandwidth, such as IoT ecosystems involving many CCTV cameras, would also be hard to deploy in environments that have limited connectivity if they rely on cloud computation.

In many cases, it’s funny (and outright ridiculous) that two devices that stand a few feet apart have to go through the internet and the cloud to exchange simple messages. It’s even more ridiculous having to cope with the fact that your fridge and toaster don’t work because they’re disconnected from the internet.

A roundtrip to the cloud can sometimes take seconds – or even minutes, in poorly connected areas – which is more than can be afforded in many of these scenarios. Meanwhile, at the edge, IoT ecosystems can make decisions at the speed of lightning, making sure that everything gets responded to in time.

study by IDC Futurescape shows that by 2018, some 40 percent of IoT-created data will be stored, analyzed and processed at the edge.

Security and privacy

As Phantom CEO Ken Tola mentioned in a previous post, encryption isn’t panacea to IoT security problems. And as a study by LGS Innovations told us earlier, hackers don’t necessarily need to crack into your encrypted communications in order to carry out their evil deeds. In fact, just eavesdropping on your IoT internet traffic – whether it’s encrypted or not – will provide malicious actors with plenty of useful information, e.g. give away your living habits.

Moreover, some forms of attacks, such as replay attacks, don’t require the attacker to have access to encryption keys. All they need to do is to replicate packets that are being exchanged on the network. For instance, with a good bit of network monitoring, an attacker might figure out which sequence of packets unlocks your home’s smart-lock.

Of course, there are ways to mitigate each of these threats, but robust security practices aren’t the greatest strength of IoT device manufacturers, and that’s why we’re seeing all thesespooky IoT hacks surface every week.

Fog computing will reduce many of these risks by considerably decreasing the amount of dependency on internet connections. Moving data and command exchange into the local area network will make it much harder for hackers to gain remote access to your data and devices. Moreover, with device-cloud exchanges no longer happening in real-time, it will be much harder to discern life and usage patterns by eavesdropping on your network.

Overcoming the challenges

Despite all the mentioned advantages, fog computing does have its own set of caveats and difficulties. For one thing, edge devices can’t match the power of cloud in computing and analytics. This issue can be addressed by distributing the workload between the cloud and the fog. Edge devices such as smart routers and gateways can mimic cloud capabilities at the edge location, making optimal use of their resources to respond to time-critical and lightweight tasks, while the heavier, analytics-intensive requests that don’t necessarily need to be carried out in real-time can be sent to the cloud.

Meanwhile, edge software should be designed and developed with flexibility in mind. For instance, IoT gateway software that controls industrial equipment should be able to receive policy and function updates, which will be produced by machine learning solutions analyzing big data at the cloud.

Read more…

Using Mattermarks’s list of the Top 100 IoT startups in 2015 (ranked by funding, published in Forbes Oct 25, 2015) Ipqwery has looked behind the analytics to reveal the nature of the intellectual property (IP) behind these innovative companies. Our infographic presents a general summary of the IP within the group as a whole, and illustrates the trailing 5-year trends related to IP filing activity.

The vast majority of these companies have both patents (84%) and trademarks (85%) in their IP portfolio. There was a sharp and mostly linear increase in filings for both patents and trademarks, from 2011 through to 2014, with a slight decrease showing in 2015. 2016 looks to be on pace to meet or exceed last year’s filing activity as well. All this is consistent with the ever-expanding number of companies operating within the IoT ecosystem.

A closer look at the top 5 patent class descriptions amongst all patents granted or published yields close results between these classes. This is not surprising given the similar technologies behind many IoT products, such that their patents will incorporate the same or similar descriptions within their claims. Comparatively, there is a wider variance in the Top 5 Trademark classes used, but this speaks more to the wider variety of marketing and branding potential than to the underlying IoT technologies. 

What’s striking in Mattermark’s original analysis of the Top 100 IoT Startups is that 30% of all funding raised by this group as a whole has been concentrated in only the top 5 companies; Jawbone, Genband, Silver Spring Networks, View Glass and Jasper Technologies. Ipqwery’s analysis further reveals that only two of these companies (Silver Spring and Jasper) have Top 5 inventors within the group. In fact, Jasper actually has 2 of the Top 5 inventors. The other top inventors come from Hello and Kineto Wireless.=

The broad-strokes approach of IPqwery’s infographic doesn’t directly illustrate the IP held by any one company, but certainly hints at where exactly this type of analysis could be very useful indeed. For where Mattermark sought to pinpoint where the greatest growth potential (momentum) was within the group of companies by looking at the overall IoT funding environment, IPqwery’s analysis of the general IP trends within this group sheds additional light on the matter, and perhaps raises some additional issues. Wouldn’t potential correlations between IP and funding also be a useful measure of momentum across metrics, and thus shouldn’t IP data be generally more integrated into business growth analytics, from the get go?

Here's a link to a new infographic by IPqwery summarizing the intellectual property held by the Top 100 IoT Startups (2015). 

 

Read more…

By Bill Vorhies

Bill is Editorial Director for our sister site Data Science Central and has practiced as a data scientist and commercial predictive modeler since 2001. This article originally appeared here

Summary:  In this Lesson 3 we continue to provide a complete foundation and broad understanding of the technical issues surrounding an IoT or streaming system so that the reader can make intelligent decisions and ask informed questions when planning their IoT system. 

In Lesson 1

In Lesson 2

In This Article

Is it IoT or Streaming

Stream Processing – Open Source

Three Data Handling Paradigms – Spark versus Storm

Basics of IoT Architecture – Open Source

What Can Stream Processors Do

Streaming and Real Time Analytics

Data Capture – Open Source with Options

Open Source Options for Stream Processors

Beyond Open Source for Streaming

Storage – Open Source with Options

Spark Streaming and Storm

Competitors to Consider

Query – Open Source Open Source with Options

Lambda Architecture – Speed plus Safety

Trends to Watch

 

Do You Really Need a Stream Processor

 

 

Four Applications of Sensor Data

 

 

Continuing from Lesson 2, our intent is to provide a broad foundation for folks who are starting to think about streaming and IoT.  In this lesson we’ll explain how Spark and Storm handle data streams differently, discuss what real time analytics actually means, offer some alternatives for streaming beyond open source, and suggest some trends you should watch in this fast evolving space.

 

Three Data Handling Paradigms:  SPARK Versus Storm

When users compare SPARK versus Storm the conversation usually focuses on the difference in the way they handle the incoming data stream. 

  • Storm processes incoming data one event at a time – called Atomic processing. 
  • SPARK processes incoming data in very small batches – called Micro Batch.  A SPARK micro batch is typically between ½ second and 10 seconds depending on how often the sensors are transmitting.  You can define this value.
  • A third method called Windowing allows for much longer windows of time and can be useful in some text or sentiment analysis applications, or systems in which signals only evolve over a relatively long period of time.

 

Atomic:  (aka one-tuple-at-a-time) Processes each inbound data event as a separate element.  This is the most intuitively obvious but also the most computationally expensive design.  For example, it’s used to guarantee fastest processing of individual events with least delay in transmitting the event to the subscriber.  Seen often for customer transactional inputs so that if some element of the event block fails the entire block is not deleted but moved to a bad record file that can later be processed further.  Apache Storm uses this paradigm.

Micro batching:  The critique of this approach is that it processes in batches (not atomic level streaming) but typically those batches are extremely small encompassing actions that occur within only a few seconds.  You can adjust the time window.  This makes the process somewhat more efficient.  SPARK Streaming uses this paradigm.

Windowing:  A hybrid of the two approaches, Windowing maintains the atomic processing of each data item but creates pseudo-batches (windows) to make processing more efficient.  This also allows for many more sophisticated interpretations such as sliding windows (e.g. everything that occurred in the last X period of time). 

All three of these approaches can guarantee that each data element is processed at least once.  Only the Atomic paradigm can guarantee that each data element is processed only once.

 

Consider this Example 

Your sensors are like FitBits and sample data every 10 seconds.  They transmit that in bursts whenever the sensor is cued to dump its data into a Wi-Fi stream.  One user may monitor the results of the stream many times during the day, valuing low latency and causing his sensor to upload via Wi-Fi frequently.  Another user may not be near a Wi-Fi connection or may simply not bother to download the data for several days.  Still a third user may have trouble with a network connection or the hardware itself that causes the sensor to transmit incomplete or missing packets that are then repeated later or are simply missing from the stream.

In this scenario, data from sensors originating at the same time may arrive at the stream processor with widely different delays and some of those packets that were disrupted may have been transmitted more than once or not at all.

You will need to carefully evaluate whether guaranteeing ‘only once’ processing, or the marginally faster response time of atomic processing warrant using this factor in your selection of the Stream Processor.

 

Streaming and Real Time Analytics

It’s common in IoT to find references to “real time analytics” or “in stream analytics” and these terms can be misleading.  Real time analytics does not mean discovering wholly new patterns in the data in real time while it is streaming by.  What it means is that previously developed predictive models that were deployed into the Stream Processor can score the streaming data and determine whether that signal is present, in real time.

It’s important to remember that the data science behind your sophisticated Stream Processor was developed in the classic two step data science process. First data scientists worked in batch with historical data with a known outcome (supervised learning) to develop an algorithm that uses the inputs to predict the likelihood of the targeted event.  The model, an algebraic formula, represented by a few lines of code (C, Python, Java, R, and others) is then exported into a program within the Stream Processor and goes to work evaluating the passing data to see if the signal is present.  If it is, some form of action alert is sent to the human or machine, or sent as a visual signal to a dashboard.

Recently the first indications that some new discoveries can be made in real time have been emerging but they are exceedingly rare.  See more in this article.

 

Beyond Open Source for Streaming

Why would you want to look beyond open source for your IoT system?  Largely because while open source tools and packages are practically free, this is the same as ‘free puppy’. 

Yes these packages can be downloaded for free from Apache but the most reasonable sources are the three primary distributors, Hortonworks, Cloudera, and MapR all of whom make sure the code is kept up to date and add certain features that make it easier to maintain.  Even from these distributors, your total investment should be in the low five figures.  This does not of course include implementation, consulting, or configuration support which is extra, either from the distributors, from other consultants, or from your own staff if they are qualified.

With open source what you also get is complexity.  Author Jim Scott writing about SPARK summed it up quite nicely.  “SPARK is like a fighter jet that you have to build yourself. The great thing about that is that after you are done building it, you have a fighter jet. Problem is, have you ever flown a fighter jet? There are more levers than could be imagined.”

In IT parlance, the configurations and initial programs you create in SPARK or other open source streaming platforms will be brittle.  That is every time your business rules change you will have to modify the SPARK code written in Scala, though Python is also available.

Similarly, standing up a SPARK or Hadoop storage cluster comes with programming and DBA overhead that you may not want to incur, or at least to minimize.  Using one of the major cloud providers and/or adding a SaaS service like Qubole will greatly reduce your labor with only a little incremental cost.

The same is true for the proprietary Stream Processors many of which are offered by major companies and are well tested and supported.  Many of these come with drag-and-drop visual interfaces eliminating the need for manual coding so that any reasonably dedicated programmer or analyst can configure and maintain the internal logic as your business changes.  (Keep your eye on NiFi, the new open source platform that also claims drag-and-drop).

 

Competitors to Consider

Forrester publishes a periodic rating and ranking of the competitor “Big Data Streaming Analytic Platforms” and as of the spring of 2016 listed 15 worthy of consideration.

Here are the seven Forrester regards as leaders in rank order:

  1. IBM
  2. Software AG
  3. SAP
  4. TIBCO Software
  5. Oracle
  6. DataTorrent,
  7. SQLstream

There are eight additional ‘strong performers’ in rank order:

  1. Impetus Technologies
  2. SAS
  3. Striim
  4. Informatica
  5. WSO2
  6. Cisco Systems
  7. data Artisans
  8. EsperTech

Note that the ranking does not include the cloud-only offerings which should certainly be included in any competitive comparison:

  1. Amazon Web Services’ Elastic MapReduce
  2. Google Cloud Dataflow
  3. Microsoft Azure Stream Analytics

Here’s the ranking chart:

 

It’s likely that you can get a copy of the full report from one of these competitors.  Be sure to pay attention to the detail.  For example here are some interesting observations from the numerical scoring table.

Stream Handling:  In this presumably core capability SoftwareAG got a perfect score while Impetus and WSO2 scored decidedly below average.

Stream Operators (Programs):  Another presumably core capability.  IBM Streams was given a perfect score.  Most other competitors had scores near 4.0 (out of 5.0) except for data Artisans given a noticeably weak score.

Implementation Support: data Artisans and EsperTech were decidedly weaker than others.

In all there are 12 scoring categories that you’ll want to examine closely.

What these 15 leaders and 3 cloud offerings have in common is that they greatly simplify the programming and configuration and hide the gory details.  That’s a value well worth considering.

 

Trends to Watch

IoT and streaming is a fast growth area with a high rate of change.  Witness the ascendance of SPARK in just the last year to become the go-to open source solution.  All of this development reflects the market demand for more and more tools and platforms to address the exploding market for data-in-motion applications.

All of this means you will need to keep your research up to date during your design and selection period.  However, don’t let the rate of change deter you from getting started.

  • One direction of growth will be the further refinement of SPARK to become a single platform capable of all four architectural elements:  data capture, stream processing, storage, and query.
  • We would expect many of the proprietary solutions to stake this claim also.
  • When this is proven reliable you can abandon the separate components required by the Lambda architecture.
  • We expect SPARK to move in the direction of simplifying set up and maintenance which is the same ground the proprietary solutions are claiming.  Watch particularly for integration of NiFi into SPARK, or at least the drag-and-drop interface elements creating a much friendlier UI.
Read more…

For IoT and M2M device security assurance, it's critical to introduce automated software development tools into the development lifecycle. Although software tools' roles in quality assurance is important, it becomes even more so when security becomes part of a new or existing product's requirements.

Automated Software Development Tools

There are three broad categories of automated software development tools that are important for improving quality and security in embedded IoT products:

  • Application lifecycle management (ALM): Although not specific to security, these tools cover requirements analysis, design, coding, testing and integration, configuration management, and many other aspects of software development. However, with a security-first embedded development approach, these tools can help automate security engineering as well. For example, requirements analysis tools (in conjunction with vulnerability management tools) can ensure that security requirements and known vulnerabilities are tracked throughout the lifecycle.  Design automation tools can incorporate secure design patterns and then generate code that avoids known security flaws (e.g. avoiding buffer overflows or checking input data for errors). Configuration management tools can insist on code inspection or static analysis reports before checking in code. Test automation tools can be used to test for "abuse" cases against the system. In general, there is a role for ALM tools in the secure development just as there is for the entire project.
  • Dynamic Application Security Testing (DAST): Dynamic testing tools all require program execution in order to generate useful results. Examples include unit testing tools, test coverage, memory analyzers, and penetration test tools. Test automation tools are important for reducing the testing load on the development team and, more importantly, detecting vulnerabilities that manual testing may miss.
  • Static Application Security Testing (SAST): Static analysis tools work by analyzing source code, bytecode (e,g, compiled Java), and binary executable code. No code is executed in static analysis, but rather the analysis is done by reasoning about the potential behavior of the code. Static analysis is relatively efficient at analyzing a codebase compared to dynamic tools. Static analysis tools also analyze code paths that are untested by other methods and can trace execution and data paths through the code. Static analysis can be incorporated early during the development phase for analyzing existing, legacy, and third-party source and binaries before incorporating them into your product. As new source is added, incremental analysis can be used in conjunction with configuration management to ensure quality and security throughout. 

2023315?profile=RESIZE_1024x1024

Figure 1: The application of various tool classes in the context of the software development lifecycle.

Although adopting any class of tools helps productivity, security, and quality, using a combination of these is recommended. No single class of tools is the silver bullet[1]. The best approach is one that automates the use of a combination of tools from all categories, and that is based on a risk-based rationale for achieving high security within budget.

The role of static analysis tools in a security-first approach

Static analysis tools provide critical support in the coding and integration phases of development. Ensuring continuous code quality, both in the development and maintenance phases, greatly reduces the costs and risks of security and quality issues in software. In particular, it provides some of the following benefits:

  • Continuous source code quality and security assurance: Static analysis is often applied initially to a large codebase as part of its initial integration as discussed below. However, where it really shines is after an initial code quality and security baseline is established. As each new code block is written (file or function), it can be scanned by the static analysis tools, and developers can deal with the errors and warnings quickly and efficiently before checking code into the build system. Detecting errors and vulnerabilities (and maintaining secure coding standards, discussed below) in the source at the source (developers themselves) yields the biggest impact from the tools.
  • Tainted data detection and analysis: Analysis of the data flows from sources (i.e. interfaces) to sinks (where data gets used in a program) is critical in detecting potential vulnerabilities from tainted data. Any input, whether from a user interface or network connection, if used unchecked, is a potential security vulnerability.  Many attacks are mounted by feeding specially-crafted data into inputs, designed to subvert the behavior of the target system. Unless data is verified to be acceptable both in length and content, it can be used to trigger error conditions or worse. Code injection and data leakage are possible outcomes of these attacks, which can have serious consequences.
  • Third-party code assessment: Most projects are not greenfield development and require the use of existing code within a company or from a third party. Performing testing and dynamic analysis on a large existing codebase is hugely time consuming and may exceed the limits on the budget and schedule. Static analysis is particularly suited to analyzing large code bases and providing meaningful errors and warnings that indicate both security and quality issues. GrammaTech CodeSonar binary analysis can analyze binary-only libraries and provide similar reports as source analysis when source is not available. In addition, CodeSonar binary analysis can work in a mixed source and binary mode to detect errors in the usage of external binary libraries from the source code. 
  • Secure coding standard enforcement: Static analysis tools analyze source syntax and can be used to enforce coding standards. Various code security guidelines are available such as SEI CERT C [2] and Microsoft's Secure Coding Guidelines [3]. Coding standards are good practice because they prevent risky code from becoming future vulnerabilities. As mentioned above, integrating these checks into the build and configuration management system improves the quality and security of code in the product.

As part of a complete tools suite, static analysis provides key capabilities that other tools cannot. The payback for adopting static analysis is the early detection of errors and vulnerabilities that traditional testing tools may miss. This helps ensure a high level of quality and security on an on-going basis.

Conclusion

Machine to machine and IoT device manufacturers incorporating a security-first design philosophy with formal threat assessments, leveraging automated tools, produce devices better secured against the accelerating threats on the Internet. Modifying an existing successful software development process that includes security at the early stages of product development is key. Smart use of automated tools to develop new code and analyze existing and third party code allows development teams to meet strict budget and schedule constraints. Static analysis of both source and binaries plays a key role in a security-first development toolset. 

References

  1. No Silver Bullet – Essence and Accident in Software Engineering, Fred Brooks, 1986
  2. SEI CERT C Coding Standard,
  3. Outsource Code Development Driving Automated Test Tool Market, VDC Research, IoT & Embedded Blog, October 22, 2013

 

Read more…

By Bill Vorhies.

Bill is Editorial Director for our sister site Data Science Central and has practiced as a data scientist and commercial predictive modeler since 2001. This article originally appeared here

Summary:  In this Lesson 2 we continue to provide a complete foundation and broad understanding of the technical issues surrounding an IoT or streaming system so that the reader can make intelligent decisions and ask informed questions when planning their IoT system. 

In Lesson 1

In This Article

In Lesson 3

Is it IoT or Streaming

Stream Processing – Open Source

Three Data Handling Paradigms – Spark versus Storm

Basics of IoT Architecture – Open Source

What Can Stream Processors Do

Streaming and Real Time Analytics

Data Capture – Open Source with Options

Open Source Options for Stream Processors

Beyond Open Source for Streaming

Storage – Open Source with Options

Spark Streaming and Storm

Competitors to Consider

Query – Open Source Open Source with Options

Lambda Architecture – Speed plus Safety

Trends to Watch

 

Do You Really Need a Stream Processor

 

 

Four Applications of Sensor Data

 

 

Continuing from Lesson 1, our intent is to provide a broad foundation for folks who are starting to think about streaming and IoT.  In this lesson we’ll dive into Stream Processing the heart of IoT, then discuss Lambda architecture, whether you really need a Stream Processor, and offer a structure for thinking about what sensors can do.

 

Stream Processing – Open Source

Event Stream Processing platforms are the Swiss Army knives that can make data-in-motion do almost anything you want it to do.

The easiest way to understand ESP architecture is to see it as three layers or functions, input, processing, and output.

 

Input accepts virtually all types of time-based streaming data and multiple input streams are common.  In the main ESP processor occur a variety of actions called programs or operators.  And the results of those programs are passed to the subscriber interface which can send alerts via human interfaces or create machine automated actions, and also pass the data to Fast and Forever data stores.

It is true that Stream Processing platforms can directly receive data streams, but recall that they are not good at preserving accidentally lost data so you will still want a Data Capture front end like Kafka that can rewind and replay lost data.  It’s likely over the near future that many stream processors will resolve this problem and then you will need to revisit the need for a Kafka front end.

 

Stream Processing Requirements

The requirements for your stream processor are these:

  • High Velocity:  Capable of ingesting and processing millions of events per seconds depending on your specific business need.
  • Scales Easily:  These will all run on distributed clusters.
  • Fault Tolerant:  This is different than guaranteeing no lost data.
  • Guaranteed Processing:  This comes in two flavors: 1.) Process each event at least once, and 2. Process each event only once.  The ‘only-once’ criteria is harder to guarantee.  This is an advanced topic we will discuss a little later.
  • Performs the Programs You Need for Your Application.

 

What Can ESP Programs Do

The real power is in the programs starting with the ability to do data cleansing on the front end (kind of a mini-MDM), then duplicate the stream of data multiple times so that each identical stream can be used in different analytic routines simultaneously without waiting for one to finish before the next begins.  Here’s a diagram from a healthcare example used in a previous article describing how this works that illustrates multiple streams being augmented by static data, and processed by different logic types at the same time.  Each block represents a separate program within the ESP that needs to be created by you.

 

There are a very large number of different logic types that can be applied through these ESP programs including:

  • Compute
  • Copy, to establish multiple processing paths – each with different retention periods of say 5 to 15 minutes
  • Aggregate
  • Count
  • Filter – allows you to keep only the data from the stream that is useful and discard the rest, greatly reducing storage.
  • Function (transform)
  • Join
  • Notification email, text, or multimedia
  • Pattern (detection) (specify events of interest EOIs)
  • Procedure (apply advanced predictive model)
  • Text context – could detect for example Tweet patterns of interest
  • Text Sentiment – can monitor for positive or negative sentiments in a social media stream

There is some variation in what open source and proprietary packages can do so check the details against what you need to accomplish.

 

Open Source Options for Stream Processing

The major open source options (all Apache) are these:

Samza:  A distributed stream processing framework. It uses Kafka for messaging, and YARN to provide fault tolerance, processor isolation, security, and resource management.

NiFi: This is a fairly new project still in incubation.  It is different because of its user-friendly drag-and-drop graphical user interface and the ease with which it can be customized on the fly for specific needs.

Storm:  A well tested event based stream processor originally developed by Twitter.

SPARK Streaming:  SPARK Streaming is one of the four components of SPARK which is the first to integrate batch and streaming in a single enterprise capable platform.

 

SPARK Streaming and Storm Are the Most Commonly Used Open Source Packages

SPARK has been around for several years but in the last year it’s had an amazing increase in adoption, now replacing Hadoop/MapReduce in most new projects and with many legacy Hadoop/MapReduce systems migrating to SPARK.  SPARK development is headed toward being the only stack you would need for an IoT application.

SPARK consists of five components all of which support Scala, Java, Python, and R.

  1. SPARK:  The core application is a batch processing engine that is compatible with HDFS and other NoSQL DBs.  Its popularity is driven by the fact that it is 10X to 100X times faster than Hadoop/MapReduce.
  2. ML.lib: A powerful on-board library of machine learning algorithms for data science.
  3. SPARK SQL:  For direct support of SQL queries.
  4. SPARK Streaming:  Its integrated stream processing engine.
  5. GraphX:  A powerful graph database engine useful outside of streaming applications.

 

Storm by contrast is a pure event stream processor.  The differences between Storm and SPARK Streaming are minor except in the area of how they partition the incoming data.  This is an advanced topic discussed later.

If after you’ve absorbed the lesson about data partitioning and you determine this does not impact your application then in open source SPARK / SPARK Streaming is the most likely choice.

 

Lambda Architecture – Speed Plus Safety

The standard reference architecture for an IoT streaming application is known as the Lambda architecture which incorporates a Speed Layer and a Safety Layer

The inbound data stream is duplicated by the Data Capture app (Kafka) and sent in two directions, one to the safety of storage, and the other into the Stream Processing platform (SPARK Streaming or Storm).  This guarantees that any data lost can be replayed to ensure that all data is processed at least once.

 

The queries on the Stream Processing side may be extracting static data to add to the data stream in the Stream Processor or they may be used to send messages, alerts, and data to the consumers via any number of media including email, SMS, customer applications, or dashboards.  Alerts are also natively produced within the Stream Processor.

Queries on the Storage safety layer will be batch used for creating advanced analytics to be embedded in the Stream Processor or to answer ad hoc inquiries, for example to develop new predictive models.

 

Do You Really Need a Stream Processor?

As you plan your IoT platform you should consider whether a Stream Processor is actually required.  For certain scenarios where the message to the end user is required only infrequently or for certain sensor uses it may be possible to skip the added complexity of a Stream Processor altogether.

 

When Real Time is Long

When real time is fairly long, for example when notifying the end user of any new findings can occur only once a day or even less often it may be perfectly reasonable to process the sensor data in batch.

From an architecture standpoint the sensor data would arrive at the Data Capture app (Kafka) and be sent directly to storage.  Using regular batch processing routines today’s data would be analyzed overnight and any important signals sent to the user the following day.

Batch processing is a possibility where ‘real time’ is 24 hours or more and in some cases perhaps as short as 12 hours.  Shorter than this and Stream Processing becomes more attractive.

It is possible to configure Stream Processing to evaluate data over any time period including days, weeks, and even months but at some point the value of simplifying the system outweighs the value of Stream Processing.

 

Four Applications of Sensor Data

There are four broad applications of sensor data that may also impact your decision as to whether or not to incorporate Stream Processing as illustrated by these examples.

Sensor Direct:  For example, reading the GPS coordinates directly from the sensor and dropping them on to a map can readily create a ‘where’s my phone’ style app.  It may be necessary to join static data regarding the user (their home address in order to limit the map scale) and that could be accomplished external to a Stream Processor using a standard table join or it could be accomplished within a Stream Processor.

Expert Rules:  Without the use of data science, it may be possible to write rules that give meaning to the inbound stream of data.  For example, when combined with the patient’s static data an expert rule might be to summon medical assistance if the patient’s temperature reaches 103°.

Predictive Analytics: The next two applications are both within the realm of data science.  Predictive analytics are used by a data scientist to find meaningful information in the data.

Unsupervised Learning:  In predictive analytics unsupervised learning means applying techniques like clustering and segmentation that don’t require historical data that would indicate a specific outcome.  For example, an accelerometer in your FitBit can readily learn that you are now more or less active than you have been recently, or that you are more or less active than other FitBit users with whom you compare.  Joining with the customer’s static data is a likely requirement to give the reading some context. 

The advantage of unsupervised learning is that it can be deployed almost immediately after the sensor is placed since no long period of time is required to build up training data. 

Some unsupervised modeling will be required to determine the thresholds at which the alerts should be sent.  For example, a message might only be appropriate if the period of change was more than say 20% day-over-day, or more than one standard deviation greater than a similar group of users. 

These algorithms would be determined by data scientists working from batch data and exported into the Stream Processor as a formula to be applied to the data as it streams by.

Supervised Learning:  Predictive models are developed using training data in which the outcome is known.  This requires some examples of the behavior or state to be detected and some examples where that state is not present. 

For example we might record the temperature, vibration, and power consumption of a motor and also whether that motor failed within the next 12 hours following the measurement.  A predictive model could be developed that predicts motor failure 12 hours ahead of time if sufficient training data is available. 

The model in the form of an algebraic formula (a few lines of C, Java, Python, or R) is then exported to the Stream Processor to score data as it streams by, automatically sending alerts when the score indicates an impending failure. 

The benefits of sophisticated predictive models used in Stream Processing are very high.  The challenge may be in gathering sufficient training data if the event is rare as a percentage of all readings or rare over time meaning that much time may pass before adequate training data can be acquired.

Watch for our final installment, Lesson 3.

Read more…

By Bill Vorhies.

Bill is Editorial Director for our sister site Data Science Central and has practiced as a data scientist and commercial predictive modeler since 2001. This article originally appeared here

Summary: This is the first in a series of articles aimed at providing a complete foundation and broad understanding of the technical issues surrounding an IoT or streaming system so that the reader can make intelligent decisions and ask informed questions when planning their IoT system. 

In This Article

In Lesson 2

In Lesson 3

Is it IoT or Streaming

Stream Processing – Open Source

Three Data Handling Paradigms – Spark versus Storm

Basics of IoT Architecture – Open Source

What Can Stream Processors Do

Streaming and Real Time Analytics

Data Capture – Open Source with Options

Open Source Options for Stream Processors

Beyond Open Source for Streaming

Storage – Open Source with Options

Spark Streaming and Storm

Competitors to Consider

Query – Open Source Open Source with Options

Lambda Architecture – Speed plus Safety

Trends to Watch

 

Do You Really Need a Stream Processor

 

 

Four Applications of Sensor Data

 

 

In talking to clients and prospects who are at the beginning of their IoT streaming projects it’s clear that there’s a lot of misunderstanding and gaps in their knowledge.  You can find hundreds of articles on IoT and inevitably they focus on some portion of the whole without an overall context or foundation.  This is understandable since the topic is big and far ranging not to mention changing fast. 

So our intent is to provide a broad foundation for folks who are starting to think about streaming and IoT.  We’ll start with the basics and move up through some of the more advanced topics, hopefully leaving you with enough information to then begin to start designing the details of your project or at least equipped to ask the right questions.

Since this is a large topic, we’ll spread it out over several articles with the goal of starting with the basics and adding detail in logical building blocks.

 

Is It IoT or Is It Streaming?

The very first thing we need to clear up for beginners is the nomenclature.  You will see the terms “IoT” and “Streaming” used to mean different things as well as parts of the same thing.  Here’s the core of the difference:  If the signal derives from sensors it’s IoT (Internet of Things).  The problem is that there are plenty of situations where the signal doesn’t come from sensors but are handled in essentially the same way.  Web logs, click streams, streams of text from social media, and streams of stock prices are examples of non-sensor streams that are therefore not “IoT”.

What they share however is that all are data-in-motion streams of data. Streaming is really the core concept and we could just as easily have called this “Event Stream Processing”, except that focusing on streaming leaves out several core elements of the architecture such as how we capture the signal, store the data, and query it.

In terms of the architecture, the streaming part is only one of the four main elements we’ll discuss here.  Later we’ll also talk about the fact that although the data may be streaming, you may not need to process it as a stream depending on what you think of as real time.  It’s a little confusing but we promise to clear that up below.

The architecture needed to handle all types of streaming data is essentially the same regardless of whether the source is specifically a sensor or not so throughout we’re going to refer to this as “IoT Architecture”.  And since this is going to be a discussion that focuses on architecture, if you’re still unclear about streaming in general you might start with these overviews: Stream Processing – What Is It and Who Needs It and Stream Processing and Streaming Analytics – How It Works”.

 

Basics of IoT Architecture – Open Source

Open source in Big Data has become a huge driver of innovation.  So much so that probably 80% of the information available on-line deals with some element or package for data handling that is open source.  Open source is also almost completely synonymous with Apache Institute.  So to understand the basics of IoT architecture we’re going to start by focusing on open source tools and packages.

If you’re at all familiar with IoT you cannot have avoided learning something about SPARK and Storm, two of the primary Apache open source streaming projects but these are only part of the overall architecture.  Also, later in this series we’ll turn our attention to the emerging proprietary non-open source options and why you may want to consider them.

Your IoT architecture will consist of four components: Data Capture, Stream Processing, Storage, and Query.  Depending on the specific packages you choose some of these may be combined but for this open source discussion we’ll assume they’re separate.

 

Data Capture – Open Source

Think of the Data Capture component as the catchers mitt for all your incoming sources be they sensor, web streams, text, image, or social media.  The Data Capture application needs to:

  1. Be able to capture all your data as fast as it’s coming from all sources at the same time.  In digital advertising bidding for example this can easily be 1 million events per second.  There are applications where the rate is even higher but it’s unlikely that yours will be this high.  However, if you have a million sensors each transmitting once per second you’re already there.
  2. Must not lose events.  Sensor data is notoriously dirty.  This can be caused by malfunction, age, signal drift, connectivity issues, or a variety of other network, software and hardware issues.  Depending on your use case you may be able to stand some data loss but our assumption is that you don’t want to lose any.
  3. Scale Easily:  As your data grows, your data capture app needs to keep up.  This means that it will be a distributed app running on a cluster as will all the other components discussed here.

Streaming data is time series so it arrives with at least three pieces of information: 1.) the time stamp from its moment of origination, 2.) sensor or source ID, and 3.) the value(s) being read at that moment.

Later you may combine your streaming data with static data, for example about your customer, but that happens in another component.

 

Why Do You Need a Message Collector At All?

Many of the Stream Processing apps including SPARK and Storm can directly ingest messages without a separate Message Collector front end.  However, if a node in the cluster fails they can’t guarantee that the data can be recovered.  Since we assume your business need demands that you be able to save all the incoming data, a front end Message Collector that can temporarily store and repeat data in the case of failure is considered a safe architecture.

 

Open Source Options for Message Collectors

In open source you have a number of options.  Here are some of the better known Data Collectors.  This is not an exhaustive list.

  • FluentD – General purpose multi-source data collector.
  • Flume – Large scale log aggregation framework.  Part of the Hadoop ecosystem.
  • MQ (e.g. RabbitMQ) There are a number of these lightweight message brokers deriving from the original IBM MQTT (message queuing telemetry transport, shortened to MQ).
  • AWS Kinesis – The other major cloud services also have open source Data Collectors.
  • Kafka – Distributed queue publish-subscribe system for large amounts of streaming data.

 

Kafka is Currently the Most Popular Choice

Kafka is not your only choice but it is far and away today’s most common choice used by LinkedIn, Netflix, Spotify, Uber, and AirBNB among others.

Kafka is a distributed messaging system designed to tolerate hardware, software, and network failures and to allow segments of failed data to be essentially rewound and replayed, providing the needed safety in your system.  Kafka came out of LinkedIn in 2011 and is known for its ability to handle very high throughput rates and to scale out.

If your stream of data needed no other processing, it could be passed directly through Kafka to a data store.

 

Storage – Open Source

Here’s a quick way to do a back-of-envelope assessment of how much storage you’ll need.  For example:

Number of Sensors

1 Million

Signal Frequency

Every 60 seconds

Data packet size

1 Kb

Events per sensor per day

1,440

Total events per day

1.44 Billion

Events per second

16,667

Total data size per day

1.44 TB per day

 

Your system will need two types of storage, ‘Forever’ storage and ‘Fast’ storage.

Fast storage is for real time look up after the data has passed through your streaming platform or even while it is still resident there.  You might need to query Fast storage in just a few milliseconds to add data and context to the data stream flowing through your streaming platform, like what were the min and max or average readings for sensor X over the last 24 hours or the last month.  How long you hold data in Fast storage will depend on your specific business need.

Forever storage isn’t really forever but you’ll need to assess exactly how long you want to hold on to the data.  It could be forever or it could be a matter of months or years.  Forever storage will support your advanced analytics and the predictive models you’ll implement to create signals in your streaming platform, and for general ad hoc batch queries.

RDBMS is not going to work for either of these needs based on speed, cost, and scale limitations.  Both these are going to be some version of NoSQL.

 

Cost Considerations

In selecting your storage platforms you’ll be concerned about scalability and reliability, but you’ll also be concerned about cost.  Consider this comparison drawn from Hortonworks:

 

For on premise storage a Hadoop cluster will be both the low cost and best scalability/reliability option.  Cloud storage also based on Hadoop is now approaching 1¢ per GB per month from Google, Amazon, and Microsoft.

 

Open Source Options for Storage

Once again we have to pause to explain nomenclature, this time about “Hadoop”.  Many times, indeed most times that you read about “Hadoop” the author is speaking about the whole ecosystem of packages that are available to run on Hadoop. 

Technically however Hadoop consists of three elements that are the minimum requirements for it to operate as a database.  Those are: HDFS (Hadoop file system – how the data is stored), YARN (the scheduler), and Map/Reduce (the query system).  “Hadoop” (the three component database) is good for batch queries but has recently been largely overtaken in new projects by SPARK which runs on HDFS and has a much faster query method. 

What you should really focus on is the HDFS foundation.  There are other open source alternatives to HDFS such as S3 and Mongo, and these are viable options.  However almost universally what you will encounter are NoSQL database systems based on HDFS.  These options include:

  • Hbase
  • Cassandra
  • Accumulo
  • SPARK
  • And many others.

We said earlier that RDBMS was non-competitive based on many factors, not the least of which is that the requirement for a schema-on-write is much less flexible than the NoSQL schema-on-read (late schema).  However, if you are committed to RDBMS you should examine the new entries in NewSQL which are RDBMS with most of the benefits of NoSQL.  If you’re not familiar, try one of these refresher articles here,here, or here.

 

Query – Open Source

The goal of your IoT streaming system is to be able to flag certain events in real time that your customer/user will find valuable.  At any given moment your system will contain two types of data, 1.) Data-in-motion, as it passes through your stream processing platform, and 2.) Data-at-rest, some of which will be in fast storage and some in forever storage.

There are two types of activity that will require you to query your data:

Real time outputs:  If your goal is to send an action message to a human or a machine, or if you are sending data to a dashboard for real time update you may need to enhance your streaming data with stored information.  One common type is static user information.  For example, adding static customer data to the data stream while it is passing through the stream processor can be used to enhance the predictive power of the signal.  A second type might be a signal enhancement.  For example if your sensor is telling you the current reading from a machine you might need to be able to compare that to the average, min, max, or other statistical variations from that same sensor over a variety of time periods ranging from say the last minute to the last month.

These data are going to be stored in your Fast storage and your query needs to be completed within a few milliseconds.

Analysis Queries:  It’s likely that your IoT system will contain some sophisticated predictive models that score the data as it passes by to predict human or machine behavior.  In IoT, developing predictive analytics remains the classic two step data science process: first analyze and model known data to create the predictive model, and second, export that code (or API) into your stream processing system so that it can score data as it passes through based on the model.  Your Forever data is the basis on which those predictive analytic models will be developed.  You will extract that data for analysis using a batch query that is much less time sensitive.

Open Source Options for Query

In the HDFS Apache ecosystem there are three broad categories of query options.

  1. Map/Reduce:  This method is one of the three legs of a Hadoop Database implementation and has been around the longest.  It can be complex to code though updated Apache projects like Pig and Hive seek to make this easier.  In batch mode, for analytic queries where time is not an issue Map/Reduce on a traditional Hadoop cluster will work perfectly well and can return results from large scale queries in minutes or hours.
  2. SPARK:  Based on HDFS, SPARK has started to replace Hadoop Map/Reduce because it is 10X to 100X faster at queries (depending on whether the data is on disc or in memory).  Particularly if you have used SPARK in your streaming platform it will make sense to also use it for your real time queries.  Latencies in the milliseconds range can be achieved depending on memory and other hardware factors.
  3. SQL:  Traditionally the whole NoSQL movement was named after database designs like Hadoop that could not be queried by SQL.  However, so many people were fluent in SQL and not in the more obscure Map/Reduce queries that there has been a constant drumbeat of development aimed at allowing SQL queries.  Today, SQL is so common on these HDFS databases that it’s no longer accurate to say NoSQL.  However, all these SQL implementations require some sort of intermediate translator so they are generally not suited to millisecond queries.  They do however make your non-traditional data stores open to any analysts or data scientists with SQL skills.

Watch for Lessons 2 and 3 in the next weeks.

Read more…

Article: A Brief History of Field Programmable Devices (FPGAs)-data-analytics-alone-cannot-deliver-effective-automation-solutions-industrial-iot-min-jpg

By Akeel Al-Attar. This article first appeared here

Automated analytics (which can also be referred to as machine learning, deep learning etc.) are currently attracting the lion’s share of interest from investors, consultants, journalists and executives looking at technologies that can deliver the business opportunities being afforded by the Internet of Things. The reason for this surge in interest is that the IOT generates huge volumes of data from which analytics can discover patterns, anomalies and insights which can then be used to automate, improve and control business operations.


One of the main attractions of automated analytics appears to be the perception that it represents an automated process that is able to learn automatically from data without the need to do any programming of rules. Furthermore, it is perceived that the IOT will allow organisations to apply analytics to data being generated by any physical asset or business process and thereafter being able to use automated analytics to monitor asset performance, detect anomalies and generate problem resolution / trouble-shooting advice; all without any programming of rules!

In reality, automated analytics is a powerful technology for turning data into actionable insight / knowledge and thereby represents a key enabling technology for automation in Industrial IOT. However, automated analytics alone cannot deliver complete solutions for the following reasons:

i- In order for analytics to learn effectively it needs data that spans the spectrum of normal, sub normal and anomalous asset/process behaviour. Such data can become available relatively quickly in a scenario where there are tens or hundreds of thousands of similar assets (central heating boilers, mobile phones etc.). However, this is not the case for more complex equipment / plants / processes where the volume of available faults or anomalous behaviour data is simply not large enough to facilitate effective analytics learning/modelling. As a result any generated automated analytics will be very restricted in its scope and will generate a large number of anomalies representing operating conditions that do not exist in the data.

ii- By focussing on data analytics alone we are ignoring the most important asset of any organisation; namely the expertise of its people in how to operate plants / processes. This expertise covers condition / risk assessment, planning, configuration, diagnostics, trouble-shooting and other skills that can involve decision making tasks. Automating ‘Decision making’ and applying it to streaming real-time IOT data offers huge business benefits and is very complementary to automated analytics in that it addresses the very areas in point 1 above where data coverage is incomplete, but human expertise exists.

Capturing expertise into an automated decision making system does require the programming of rules and decisions but that need not be a lengthy or cumbersome in a modern rules/decision automation technology such as Xpertrule. Decision making tasks can be represented in a graphical way that a subject matter expert can easily author and maintain without the involvement of a programmer. This can be done using graphical and easy to edit decision flows, decision trees, decision tables and rules. From my experience in using this approach, a substantial decision making task of tens of decision trees can be captured and deployed within a few weeks.

Given the complementary nature of automated analytics and automated decisions, I would recommend the use of symbolic learning data analytics techniques. Symbolic analytics generate rules/tree structures from data which are interpretable and understandable to the domain experts. Whilst rules/tree analytics models are marginally less accurate than deep learning or other ‘blackbox models’, the transparency of symbolic data models offer a number of advantages:

i- The analytics models can be validated by the domain experts
ii- The domain experts can add additional decision knowledge to the analytics models
iii- The transparency of the data models gives the experts insights into the root causes of problems and highlights opportunities for performance improvement.

Combining automated knowledge from data analytics with automated decisions from domain experts can deliver a paradigm shift in the way organisations use IOT to manage their assets / processes. It allows organisations to deploy their best practice expertise 24/7 real time throughout the organisation and rapidly turn newly acquired data into new and improved knowledge.

Below are example decision and analytics knowledge from an industrial IOT solution that we developed for a major manufacturer of powder processing mills. The solution monitors the performance of the mills to diagnose problems and to detect anomalous behaviour:

The Fault diagnosis tree below is part of the knowledge captured from the subject matter experts within the company

Article: A Brief History of Field Programmable Devices (FPGAs)-fault-diagnosis-tree-min-jpg



The tree below is generated by automated data analytics and relates the output particle size to other process parameters and environmental variables. The tree is one of many analytics models used to monitor anomalous behaviour of the process.

Article: A Brief History of Field Programmable Devices (FPGAs)-automated-data-analytics-min-jpg



The above example demonstrates both the complementary nature of rules and analytics automation and the interpretability of symbolic analytics. In my next posting I will cover the subject of the rapid capture of decision making expertise using decision structuring and the induction of decision trees from decision examples provided by subject matter experts.

Read more…

iot security

By Ben Dickson. This article originally appeared here.

A recent DDoS attack staged against a brick-and-mortar jewelry store highlights just how devastating the negligence of IoT security can become. The attack, as reported by SC Magazine, involved a 35,000 HTTP request per second flood carried out by an IoT botnetof more than 25,000 compromised CCTV cameras scattered across the entire globe, causing the shop’s servers to go down.

As detailed by cybersecurity firm Succuri, the attack is unusual because it has only used IoT devices and also because of its uncommonly lengthy duration. After the initial wave, when the servers were brought back online, a second, bigger attack, with a 50k HTTP RPS, was conducted, which lasted for several days.

A separate report by Computer Weekly details how the LizardStresser malware is creating IoT botnets by exploiting vulnerable devices, and is mounting massive 400 gigabits-per-second DDoS attacks without using amplification techniques.

This is just a glimpse of the opportunities that the Internet of Insecure Things is providing for malicious actors who are always looking for new ways to break into networks to defraud organizations of their cash and valuable assets, or to harm opponents and competitors.

You’ve been warned about IoT botnets before

While the rise in DDoS attacks based on IoT botnets is new, it wasn’t unexpected. In fact, after 2015 became the year of proof-of-concept attacks against the Internet of Things, it had been predicted that IoT devices would become a very attractive target for bot herdersin 2016.

As Dark Reading’s Ericka Chickowski said in this post, “2016 is going to be the year that attackers make a concerted effort to turn the Internet of Things (IoT) into the Botnet of Things.”

Researchers from Incapsula first warned about IoT botnets last year after detailing an attack they discovered which they tracked back to CCTV cameras at a retail store close to their office. And with insecure IoT devices becoming connected to the internet at a chaotic pace, hackers have good reason to give up general purpose computing devices, such as desktop and laptop computers, to go after the easier targets.

What makes IoT device such easy prey for botnet malware?

There are many reasons that IoT devices – and in this case CCTVs – make very attractive targets for bot herders. As Igal Zeifman, senior digital strategist from Imperva, detailed in the Incapsula blog post, “Security cameras are among the most prevalent and least protected IoT devices. Moreover, many have high upload connections, meant to support their remote streaming functionality.”

What makes it easy to conscript CCTVs ­– and other IoT devices for that matter – into botnets? According to Chris Hodson, CISO for EMEA region at cloud security company Zscaler, who spoke with SC Magazine, it’s because the security development lifecycle for IoT devices is often expedited or bypassed due to strict deadlines around time to market or the cost of the hardware.

This is a point that I’ve also raised on several occasions: one of the fundamental problems with IoT security is that the developers often come from an unconnected background, such as embedded systems, which means they have the knowhow to provide functionality but aren’t versed in the principles to write secure code for connected environments. In other cases, security is advertently neglected for the sake of meeting release deadlines of cost requirements.

Researchers at Arbor Networks summed up the prevalence of IoT botnet malware in four reasons:

  • The operating system of IoT devices is usually a stripped-down version of Linux, which means malware can be easily compiled for the target architecture.
  • IoT devices usually have full access to internet and aren’t subject to bandwidth limitations or filtering – which is very true in the case of CCTVs.
  • Minimal operating systems running on IoT devices don’t leave much room for security features such as auditing, which lets attackers compromise and exploit the devices without leaving trace.
  • There’s a lot of hardware and software reuse in IoT development, which means a lot of security-critical components become shared between devices. (Just take a look at “House of Keys” research by SEC Consult, which shows how the reuse HTTPS certificates and SSH keys endangers millions of devices.)

The part that concerns consumers is the carelessness in dealing with IoT device security. Since IoT devices aren’t as personal as, say, smartphones or PCs, users tend to “install and forget” IoT devices. Bad practices such as not changing passwords, or worse, leaving devices installed with factory-default passwords are epidemic in IoT ecosystems, which makes it very easy to find administrative access to the device and install IoT botnet malware into it.

What can be done about the IoT botnets?

I just wanted to raise the challenge of IoT botnets in this post. The response will be the subject of a future article. But very briefly, a lot can be done to mitigate the threat of IoT botnets in the future. For one thing, security should become a major factor in IoT development. As Cesare Garlati, chief security strategist at prpl foundation told SC, “The very fact that patching isn’t high on the priority list for admins is testament to why security in devices like CCTV cameras needs to be ‘baked in’ at the chip or hardware layer.”

We’ve already seen the efficiency of hardware security in the headaches that Apple gave the FBI in the San Bernardino iPhone case. Having devices that are secure at the hardware level will go a long way into hardening our defenses against exploits, including IoT botnets.

Moreover, we should also recognize that some IoT devices can’t be secured at the device level and therefore must be secured at the network level. Deploying network security solutions, like the ones I’ve described in this TNW article can help a lot in providing security against IoT botnets for devices that are inherently insecure.

These are just two tips at fighting back against the rising tide of IoT botnets. I’m sure that a lot of you readers out there have brilliant ideas and innovations that can help deal with this situation. Since I’ll be writing about this very soon, I’m eager to know what you’re doing to deal with the IoT botnet threat. Leave a comment, or better yet contact me, to share your ideas.

FEATURED IMAGE: SAVASYLAN/SHUTTERSTOCK

Read more…

IoT Central Digest, July 11, 2016

Here is the latest issue of the IoT Central Digest. This digest features stories about farming, security, applications and developers. Our community continues to grow and more members are contributing pieces for discussion and knowledge. If you're interested in being featured, we always welcome your contributions on all things IoT Infrastructure, IoT Application Development, IoT Data and IoT Security, and more. All members can post on IoT Central. Consider contributing today. Our guidelines are here.

5 Innovative Ways IoT Can Help Farms

Read more…

A smart, highly optimized distributed neural network, based on Intel Edison "Receptive" Nodes

Training ‘complex multi-layer’ neural networks is referred to as deep-learning as these multi-layer neural architectures interpose many neural processing layers between the input data and the predicted output results – hence the use of the word deep in the deep-learning catchphrase.

While the training procedure of large scale network is computationally expensive, evaluating the resulting trained neural network is not, which explains why trained networks can be extremely valuable as they have the ability to very quickly perform complex, real-world pattern recognition tasks on a variety of low-power devices.

These trained networks can perform complex pattern recognition tasks for real-world applications ranging from real-time anomaly detection in Industrial IoT to energy performance optimization in complex industrial systems. The high-value, high accuracy recognition (sometimes better than human) trained models have the ability to be deployed nearly everywhere, which explains the recent resurgence in machine-learning, in particular in deep-learning neural networks.

These architectures can be efficiently implemented on Intel Edison modules to process information quickly and economically, especially in Industrial IoT application.

Our architectural model is based on a proprietary algorithm, called Hierarchical LSTM, able to capture and learn the internal dynamics of physical systems, simply observing the evolution of related time series.

To train efficiently the system, we implemented a greedy, layer based parameter optimization approach, so each device can train one layer at a time, and send the encoded feature to the upper level device, to learn higher levels of abstraction on signal dinamic.

Using Intel Edison as layers "core computing units", we can perform higher sampling rates and frequent retraining, near the system we are observing without the need of a complex cloud architecture, sending just a small amount of encoded data to the cloud.

Read more…

5 Innovative Ways IoT Can Help Farms

With the global population expected to reach 9.7 billion people by 2050, innovation in the agricultural industry is more important than ever. This growth brings with it a need for increased food production and a dwindling availability of arable land. Projections show that feeding a population near 9 billion people would require raising global food production by some 70%. To provide for such steep demands, old farming techniques are simply no longer adequate. Thankfully, the agriculture industry is a burgeoning sector within the Internet of Things, and farmers globally are ready to reap the benefits.

Let’s look at a few ways IoT is helping the agriculture industry around the world.

 

  1. Smart Ag Is Environmentally Friendly

Agriculture is responsible for a significant environmental footprint, accounting for 80% to 90% of US water consumption, releasing massive quantities of chemical pesticides, and producing more greenhouse gas emissions with animal agriculture alone than all combined transportation. IoT technology can maximize production capabilities and minimize required resources in order to reduce the industry’s environmental harm. Sensors can be implemented to test agricultural factors such as soil for moisture and nutrient levels to ensure resources are being used as efficiently as possible. This way water and pesticides can be reserved from unnecessary use and alternates can be implemented.

  1. IoT Provides Precision Control for Farmers

The agriculture industry reaches far and wide, and each sector involves too much labor for one person to accomplish alone. As a result, much of the agriculture industry relies heavily on trusting intuition and human judgment. This also means that workers are nearly irreplaceable during illness or absence. Implementation of IoT technology can allow for real-time access to information that otherwise would take too much time or effort to obtain. Managers can have remotely controlled decision-making capabilities at their fingertips rather than having to wait for reports and then send out orders to workers. For example, rather than using valuable workers, drones are now available to monitor crops or apply treatment to specific areas. Also, companies like Lecia have developed GPS-guided combines and other agricultural IoT technologies. Aeris is helping to make solutions like these possible through cellular connectivity, perfect for remote real-time access to critical data.

  1. Farms Are More Productive With IoT

As agricultural businesses gain more insight and control over their operations, they are able to make better business decisions and thus increase productivity. If a farm can use drones or sensors to monitor fields or cattle, for example, then the experience of the farmer can be utilized to make decisions while the manual labor previously needed to monitor these areas can be better repurposed elsewhere. IoT technology can also be applied to agricultural machinery, allowing for preventative maintenance and more accurate reports in the case of a malfunction, saving time and money. As smarter decisions are made regarding resources, productivity will improve.

  1. Smart Farming Saves Money

According to the USDA, some of the top expenses of agricultural businesses include feed, labor, fuel, chemicals, and farm supplies and repairs, all expenditures that can be reduced with the help of IoT. Implementing IoT technology can allow businesses to make better decisions about efficiently using resources including assets, labor, and capital, significantly reducing operational costs. Replacing and improving past techniques will be the only way to maintain a competitive advantage with rising demands and ever-improving technology.

  1. IoT Provides Transparency for Consumers

As more data is made available to the public, consumers demand high quality products more emphatically than ever. Concerns continue to emerge regarding the environmental footprint, personal health effects, and other details surrounding food production. Utilizing IoT technology in production is the only way to provide consumers with the data and transparency that is now the standard expectation, and thus maintain a competitive advantage in the industry.

When you’re ready to connect your agriculture devices to the Internet of Things, contact Aeris for a customized IoT solution.

Read more…

Originally Posted by: Shawn Wasserman

Shodan search results show that over half a million devices use the 10-year-old OpenSSH 4.3 software. This puts all these devices at risk.

Shodan search results show that over half a million devices use the 10-year-old OpenSSH 4.3 software. This puts all these devices at risk.

One doesn’t have to look too far to realize how vulnerable the Internet of Things (IoT) can be. It just takes a quick search on IoT search engines like BullGuard and Shodan.io.

During a presentation at PTC LiveWorx 2016, Rob Black, senior director of product management at PTC, outlined how black hat hackers could get into over half a million connected devices using an old software known as OpenSSH 4.3.

OpenSSH is a secure shell (SSH) protocol used to allow users access to networks from a remote location. It’s harmless, even useful, if used by the right user in a controlled way.

Unfortunately, a popular version of the software, OpenSSH 4.3, has been out for about a decade. As a result, it has developed a laundry list of vulnerabilities that hackers can use to gain access to systems.

According to the Shodan IoT device search engine, over half a million devices on the ‘net still use this outdated software.

“Half a million devices are on the open Internet with 10-year-old software that allows you to tunnel inside to their network. Who thinks that’s good?” Black rhetorically questioned. “This is one example. One search. One software. One version of a software. There are millions of exposed resources on the Internet.”

The scary thing is that Black explained that some search results will bring up IoT devices associated with power plants and wind tunnels. According to AdaptiveMobile, a mobile network security company, up to 80 percent of connected devices on the IoT do not have the security measures they need to protect us. Once you find a device on Shodan, you can see many characteristics on that device which will help hackers get into it.

These attacks can even prove deadly depending on the IoT application. Take an integrated clinical environment (ICE) like an IoT-enabled hospital. Without proper security, many types of attacks have the potential to risk lives. According to a report published by the Industrial Internet Consortium, these attacks fall into five categories.

 

Five IoT hacking attacks that can risk lives. Examples from an integrated clinical environment (ICE). (Table from the Industrial Internet Consortium.)
Five IoT hacking attacks that can risk lives. Examples from an integrated clinical environment (ICE). (Table from the Industrial Internet Consortium.)

Engineers are designing these IoT devices, sensors and edge points. To ensure that hackers are kept at bay, these engineers need to understand and learn from their software engineer and IT cousins.

“From a design point of view, engineers need to learn about hacking security. You need security at the edge point to make an intelligent analytic device,” said Michael Wendenburg, CEO at Michael Wendendenburg Online Redaktion. “If you hack into that point, you hack into all this data. Engineers are not prepared for that.”

Black agreed, saying, “It’s our role as practitioners of IoT is to really manage those devices that we have in a smart way.”

How Do IoT and Cloud Security Differ?

Black explained that unlike in cloud security, humans may not be in the loop when it comes to IoT security. It’s not feasible for millions of users to be there to hit “Okay” to update software in billions of devices.

Black explained that unlike in cloud security, humans may not be in the loop when it comes to IoT security. It’s not feasible for millions of users to be there to hit “Okay” to update software in billions of devices.

An engineer might think that as long as the cloud system utilized by the IoT device is secure, then all is well. However, there are differences between an IoT system and a cloud system.

Black explained that on the cloud, users and applications are both managed. There are security tools and permissions put into place. On the operations side, servers will be secured and ports will be closed and audited. This takes a lot of testing, but it’s been done before. IoT security, on the other hand, adds complexity.

“Cloud security has been around for a long time and there are lots of good strong practices and management around cloud applications. For IoT, the key difference is we connect things,” clarified Black. “A lot of the challenge is the number of devices to manage and the differences between these devices.”

“There are a bunch of new issues out there like rogue sensors and rogue data sources,” said Andy Rhodes, division head of IoT at Dell. “If you’re orchestrating a turbine or a damn and someone hacks into that and changes the settings, then there are catastrophic issues.”  

Here are some other key differences between cloud and IoT applications:

  • IoT has a stronger potential for damage as water mains can be shut off, power plants can become critical and cars made unresponsive on the road.
  • IoT has a diverse number of devices, operating systems and protocols making it hard to consolidate and standardize as companies grow and products change.
  • Human interactions with all the devices is not scalable. For instance, humans many not be there to hit “Okay” for an update.

The key is to work together. Engineers and IT professionals need to demolish their silos and learn from one another to make the IoT ecosystem secure. However, just because the IT crew has the ecosystem covered on the cloud doesn’t mean the devices and sensors are secure.

“IT [Information Technology] knows how to do security and a lot of this is still traditional IT security working alongside the OT [Operations Technology] people to understand how to secure the sensors as well,” described Rhodes. “You need [security on the device] and network security on the IT side because data flows two ways so you have to secure both ends of that spectrum.”

How to Manage Your Connected Device

Black demonstrating an IoT security architecture.

Black demonstrating an IoT security architecture.

With current IoT trends, if your device isn’t connected to the Internet, it soon will be. Otherwise, it will not keep up with the 30 billion other connected devices Gartner expects to see in the market by 2020.

So the question may not be whether to get into the IoT market given all the security risks. It should be a question of how to manage connected devices with all these security risks.

Black demonstrated what a simple IoT architecture might look like. It includes devices within a firewall, wireless devices outside the firewall and having those devices connecting into the IoT platform. Then, all of this will be used in an application that will use the data from the devices to perform a function. All of these systems, applications and development tools used to make the system must be made secure.

The issue is that because all of these different systems are under the control of various organizations on the vendor, customer and public levels, it can be confusing to establish who is really responsible for all of this IoT security.

“I argue that for IoT we have a shared security responsibility,” noted Black. “This is not a one-entity responsibility. It is shared between the providers of the infrastructure, service, platform, application and the end customers.”

Importance of User Roles on IoT Security

Given all of the organizations and users that might be associated with one IoT system, defining roles for these organizations and users is of high importance.

Each user and organization will have different roles, which will define levels of control over the IoT system. For instance, you don’t want to give your customers visibility into and control over all of the IoT devices on your ecosystem. This could make the data of your other customers insecure, as competitors might gain insights due to the information on your system and the lack of roles governing the system.

However, a maintenance team that services all the devices sent to customers will need to see which devices from each customer will be up for servicing.

The key takeaway is that as your system grows on the IoT, much of this role management should be automated. Otherwise, the role management will not scale with the IoT system if a human remains in the role assignment loop.

“From a visibility and permission standpoint, what you really want are mechanisms to drive that behavior,” instructed Black. “When new devices are added, if you have a manual process, that is not going to scale when you [have] tens of thousands of devices. You are going to need a system that drives this behavior automatically. You just need to set the rules beforehand to ensure the users are put in the right groups.”

Division of Systems is Key to a Secure IoT Ecosystem

The division of permissions shouldn’t just be between roles. It should also be between systems within the IoT device itself. Engineers must design some systems and subsystems to be independent and separate from all other systems. This will ensure that if a hacker compromises your device, they will not be able to take control of key systems.

After all, there is no reason for an entertainment system in a car to be linked to the steering, brakes and accelerator of a car. As the WIRED video below shows, though, this was the case with the Jeep Cherokee. As a result, hackers were able to mess with one reporter’s drive on the highway with hilarious outcomes—but the joke isn’t funny anymore if people actually get hurt.

“The way some of these systems are designed, if you have access to this you have access to multiple design elements in the car,” said Frank Antonysamy, head of engineering and manufacturing solutions at Cognizant. “The way we are dealing with this is to isolate as much as possible and then get the data.”

“When you look at it from a system design [perspective], in an automobile for example, there is still a fair amount of isolation written into the design,” said Antonysamy. “Because I have access to my control panel doesn’t mean I have access to the accelerator. That kind of design-based isolation is critical at least until we get a zero-vulnerability scenario.”

Eric van Gemeren, vice president of R&D at Flowserve, explained that the automobile industry and other IoT device creators can learn a lot from the process industry on the separation of systems within a design.

“In the process industry, it’s different from having a car that’s IoT-enabled and someone can hack into it,” said van Gemeren. “In the process industry, there are well-established IEC [International Electrotechnical Commission] and ISO [International Organization for Standardization] standards for safety and compliance. The control communication network is always separate and independent from the diagnostics and asset management network. It’s very clear that when you design that solution, there are certain features and functions that will never be available through wireless, in a discrete controlled domain, with an entirely different protocols and with robust security on top of it.”

“A lot of the stuff we are talking about in the IoT space is all about gathering outbound asset information,” added van Gemeren. “You can’t send back control information or directions that can hijack the device.”

In other words, van Gemeren explained that if a safety system like fire suspension sprinklers were installed in a process plant, they will need to be on an isolated system.

Do Your Devices Need to Talk to Other Devices?

Black explained the scenarios in which you need to use device-to-device

Black explained the scenarios in which you need to use device-to-device

When people think about the IoT, many of them think of connected devices communicating with each other over the Internet.

Though there are situations when the data should be sent to the cloud, there are also situations where it is faster and more efficient for devices to talk to each other directly.

“You could go up to the cloud and negotiate up there and bring it back down but that is not using bandwidth efficiently and what happens if you lose network connectivity? Will your devices fail? Do you want them to be dependent on the network?” asked Black.

When connected device need to talk directly, you will need a way to authenticate the devices mutually as well as a method of authorizing the devices to an appropriate level of interactions.

“It doesn’t make sense for one car to have the authorization to turn on the windshield wipers for another car,” joked Black.

The Importance of Provisioning and Approval of an IoT Device

This brings us to another key step in setting up a secure IoT system: ensuring your processes can set up provisioning and approval for device-to-device communication, data ownership, de-provisioning and more.

“Any process that runs off of administration approval will fail on an IoT scale,” remarked Black. This is similar to the creation of roles the human needs to be out of the loop. Black added, “You can’t design a process based on admin approval—it might work for a hundred devices but it won’t work on a large-scale system.”

Unfortunately, you can’t just let all devices interconnect without a provisioning and approval process either. Take the Superfish scandal, for example. The program was intended to provide advertisers with a way to show ads based on a user’s Internet searches.

This sounds innocuous enough until you realize that, at the time, all Lenovo laptops had the same self-signed certification key for all the laptops that shipped out with the program. This allowed for man-in-the-middle hacking attacks that could intercept the Internet communications of any Lenovo laptop with the Superfish program still installed.

“Ensuring trust when you’re bootstrapping a device is challenging even big laptop manufacturers can make mistakes,” said Black. “We need to think through some of those processes to see how do we get secrets onto a device. You need a well-defined mechanism for establishing trust on your device.”

One method Black suggested to get your devices onto your IoT system with secure provisioning and approval is to use your enterprise resource planning (ERP) system. If your ERP system were connected to the IoT system, then the provisioning and approval process will expect to see the device. Not only would this system be secure, it can also be made scalable as there will be no need to have a human in the loop.

The Importance of De-Provisioning When You Re-Sell a Connected Device

Black explained the importance of factory resets and de-provisioning when selling used devices.

Black explained the importance of factory resets and de-provisioning when selling used devices.

There is a lot of confidential information that can be stored on a connected device. Therefore, if users aren’t careful, they could be giving a hacker everything they need to get into the system when re-selling these devices.

The average user would know enough to delete their personal and business data from the device, but there still might be information on the re-sold device that can open doors to hackers.

For instance, the device might store digital keys that were used to encrypt the data you were sending and receiving from the Internet. If you were to sell that equipment without changing those keys, then whomever you sold that equipment to could now be able to decrypt all of the data you sent and received while operating the device. Assuming the hacker intercepted that data in full knowledge that you were to sell the equipment, they now have gathered a lot of information on your personal or business operations.

As a result, engineers should design easy to use de-provisioning procedures for the users of their devices.

Whose Data Is It Anyway? Where the Contract’s Made Up and Protection Should Matter.

Black asked the question: Whose data is it anyway?

Black asked the question: Whose data is it anyway?

One point of contention for the development of IoT security is the question of who owns the data.

Is it the device manufacturer, systems operator, device operator or the maintenance operator?

Will the answer be dependent on the IoT device application?

These questions need answers if robust security measures are to be put into place. Otherwise, the right information might end up in the wrong hands.

“We’ve seen a range of responses about data ownership and a lot revolves around privacy,” said Colm Pendergast, director of IoT technology at Analog Devices. “To a large extent, it will come down to negotiations between various partners in an ecosystem.”

“[Who owns the data] is a question that is always on the table,” said Chris May, account executive at ARIDEA SOLUTIONS. “It depends on the type of data being acquired. If it’s general weather data, then people are not very concerned. The weather is the weather… When you get to environmental data, it’s a completely different story. They are very protective of that data. [What] If the wrong person gets that data and they don’t understand how to interpret it? [What] if they can’t understand it’s a sensor being recalibrated and they think a water shed was contaminated? It would be massive lawsuits.”

It appears that though 54 percent of surveyed consumers might be comfortable sharing their data with companies, the reverse is not always true.

Alternatively, Black used an example of a medical device company. If the company is sold, then it makes sense for whomever buys the company to also own the data. After all, it will, in theory, be using said data to service the same clients. It isn’t in the client’s interest for the data to start at point zero.

However, does the answer of selling data ownership change with the scenario? What if, instead of a company being sold, it’s a house? Who owns all the data of the smart home—the previous tenants or the incoming tenants? It might be useful for the new tenants to know the power usage history of the house so they can budget their expenses, but do you want strangers to have data like that?

“When you think about how many different entities are involved with an IoT implementation, there are a lot of them,” said Black. “Some of them probably have rights to some of that data and some it’s probably better if they don’t have it.”

Before security walls are put up for an IoT device, these questions must be answered. Otherwise, an owner of the data might be cut off from their property. This can lead to some serious legal ramifications. On the other hand, not understanding where the line in the sand is for data can also open up security risks.

“If there was one single challenge that people are concerned about and has slowed IoT deployments is the question of security and integrating security solutions all over that technology stack. It is one of the bigger challenges,” said Pendergast.

However, one solutions to the IoT data question may not lie with the engineers, programmers or designers. It might be in the hands of public relations educating the public about IoT security and what data is and isn’t being collected.

“We deal with the medical device market and we constantly face the issue that we can’t send patient data—and we are a cloud-based platform, so that is a challenge,” said Puneet Pandit, CEO of Glassbeam. “We are not taking the patient data; we are taking the operation data. I think that is a constant question. There is a lot of education that has to be done in the industry to clarify what IoT data means at the end of the day. People have created security barriers for all the right reasons, but in the context of IoT you are taking machine and operational data and that isn’t something that is included on data privacy.”

Reducing IoT Attack Surfaces: Do You Need Access to the Open Web?

Shodan is only able to show the IoT devices that are on the open web. The number, as well as types, of devices that it can find is certainly scary.

“[Security is] still the top-two or -three concern of customers when you read surveys and speak to them,” said Rhodes. “What you’ve basically done is you’ve opened up a surface of attack either as a gateway or the things themselves.”

Does your device need to be on the open web? Do multiple surfaces of attack need to exist? The answer is no—not if engineers design the device to be the one to initiate communications.

“Different IoT solutions have the capability to perform device-initiated communication,” said Black. “That means that from a connection standpoint, if your device initiates communications, then that device is exclusively paired with one server on the cloud. That device is only going to communicate with that server.”

In other words, the device won’t be generally available on the Internet.

“It’s something to think about. Can I communicate with this device from every [access point] on the earth or is it tied to a single server? Because you are really reducing your attack surface with that kind of capability,” Black explained. “You reduce your attack surface so you are not worried about everything in the world. You are only connected to a very limited set of servers.”

If your device can connect to any endpoint on the Internet, then any hacker at any location could in theory send a command to that device. However, if the device is connected only to one server via a device-initiated communication, then only that server can send commands. The theory is that your server will be within internal IT infrastructures and securities.

However, there is a downside to device-initiated connectivity. You will have to rely on the device to connect to the system in order to initiate an update or collect data. In other words, you can lose connections to the device as soon as a customer changes firewall securities or the network is interrupted. 

As a result, if engineers chooses to use device-initiated connections for an IoT system, then they will need to inform the customer. The customer will need to understand if the firewall and network connection isn’t interfering with the connection.

“We’ve seen a lot of software partners changing their architecture to support intermittent connectivity,” said Gerald Kleyn, director of engineering at Hewlett Packard Enterprise (HPE). “In some cases, if the weather gets bad and [satellite communication] goes down, then when it comes back up it starts releasing things that have been stored on the edge back up to the cloud.”

What to Do When You Find a Vulnerability on Your Connected Device

The longer your device is in the real world, the more likely it is that a vulnerability will be found. As a result, engineers will need to design software update compatibility into their devices.

“You need a software distribution mechanism that will work for all of your devices that’s scalable, secure, flexible and efficient,” said Black. “It needs to be flexible because all your devices are different, so they need different processes and procedures.”

“You need to be able to say, if the install isn’t going right, that you need to hold back and notify your system. You need to be able to say, ‘do this for North America first, or Europe or everyone but that customer that doesn’t want updates,’” added Black. “Without a plan, you will be sad when the next Heartbleed comes out. You are going to have to patch. So what is the mechanism you are going to utilize?”

This all must seem very complicated, but much of this IoT security issues will be answered when you choose the IoT platform to run, manage design the system. Black says that when choosing your IoT platform, keep these three main security challenges in mind:

  1. Managing the complex interactions between devices and user
  2. Patching security updates to your devices in an easy and secure fashion
  3.  Reducing the risk by mitigating cyber-attacks form finding your device

You can view the original post Here

Read more…

In a new update to the Worldwide Semiannual Internet of Things Spending Guide, the International Data Corporation (IDC) forecasts that U.S. organizations will invest more than $232 billion in Internet of Things (IoT) hardware, software, services, and connectivity this year. And IDC expects U.S. IoT revenues will experience a compound annual growth rate (CAGR) of 16.1% over the 2015-2019 forecast period, reaching more than $357 billion in 2019.

The industries leading the way in U.S. IoT investments are Manufacturing and Transportation at $35.5 billion and $24.9 billion, respectively, in 2016. However, Cross-Industry investment, which represents use cases common to all industries, will approach $31 billion this year. The IoT use cases receiving the greatest levels of investment from U.S. organizations across these three industry segments are:

  • Manufacturing Operations, which supports digitally-executed manufacturing, or how manufacturers use intelligent and interconnected I/O (input output) tools – e.g. sensors, actuators, drives, vision/video equipment etc. – to enable the different components in the manufacturing field – e.g. machine tools, robots, conveyor belts etc.– to autonomously exchange information, trigger actions and control each other independently.
  • Freight Monitoring, which uses radio frequency identification (RFID), global positioning system (GPS), GPRS, and GIS technologies to create an intelligent, Internet-connected transportation system. This system carries out the intelligent recognition, location, tracking and monitoring of freight and cargo through exchanging information and real-time communications via wireless, satellite or other channels.
  • Smart Buildings, which utilize advanced automation and integration to measure, monitor, control, and optimize building operations and maintenance. The key concept is optimization – meaning the deployment of a set of integrated control systems capable of adapting in real time to both internal policies and external signals. These systems manage how building equipment operates to use energy in the most efficient and cost-effective way.

Looking across all U.S. industries, these three IoT use cases will receive the greatest levels of investment throughout IDC's forecast. The next three largest IoT use cases in terms of U.S. revenue will be Remote Health Management, Smart Grid (Electricity), and Smart Home. The IoT use cases that will experience the greatest revenue growth in the U.S. over the 2015-2019 forecast period are In-Store Contextualized Marketing, Connected Vehicles, and Insurance Telematics.

"A use case represents a detailed composition of a technology investment that is made to produce a set of end user benefits," said Marcus Torchia, research manager, IoT for IDC's Customer Insights and Analysisteam. "The long term opportunity for IoT vendors is helping to identify and create immediate and residual benefits for end users through their technologies. We see strong opportunities across many industries. For example, in highly instrumented verticals like manufacturing and transportation, large data sets are used to optimize operational processes and extend the life of high capital cost assets. In other sectors like healthcare and consumer, IoT technology is being used to produce benefits that improve quality of life."

While Manufacturing and Transportation will lead the U.S. in terms of overall IoT investments, the Insurance, Retail, and Healthcare industries will see IoT spending levels increasing by 135%, 101%, and 96%, respectively, over the forecast period. In addition to driving some of the largest IoT investments, the Cross Industry segment will also see revenue growth of more than 100% through 2019.

Read more…

Guest post by Jules Oudmans. This article first appeared here.

An important, crucial, aspect in IIoT applications is the communication layer. This layer is responsible for relaying sensor values from the field to northbound processing of this data and the southbound control.

In my previous blog I concluded that IIoT is reality, but headaches are ahead choosing the right protocol and communications provider, especially when your IIoT solution requires long-range support, will be deployed in multiple countries and needs cross border coverage.

Protocols, protocols, protocols …

The myriad of protocols in IoT land is still growing and can truly be overwhelming. In below table I have limited the protocols listed to only those that support long-range. Next to this the table shows data rates and bi-directional capability, these are important qualifiers for typical IIoT solutions and may help you choose the right protocol … but continue reading!!

What Protocol … No Sorry What Provider .. or Protocol !??

The vast majority of today’s M2M communication relies on 2G and 3G networks. These networks reliably provide, relatively low-cost, long range and border stretching networks for IIoT applications. They are offered by a wide variety of providers and support roaming. A truly excellent choice for your IIoT solution … were it not that 2G/3G networks are expected to disappear in the next 5 years. This is where the headache starts ... as today no cross-border roaming IIoT provider is out there supporting a protocol with an expected useful lifetime equal to a smartphone.

You could off course develop your IIoT solution – sensors, middleware, processing nodes et cetera. – to support multiple protocols but this is unpractical and costly.

When we limit ourselves to an IIoT deployment that requires long range connectivity, is low- cost, will be deployed in multiple countries and will be available after 2020 than, today, we can choose between:

  • SigFox: A protocol with country coverage in certain EU countries and countries under rollout such as Brazil, Germany, UK and the US; and
  • LoRa: A protocol offered by multiple providers. Some country covering networks are available today and some network providers cover regions or cities.

SigFox

SigFox is available in countries where rollout is requested (chicken and egg). Today roaming is not supported, but besides this all data goes over SigFox managed servers(!). The latter something that certain companies will not want.

LoRaWAN

LoRaWAN is growing in popularity, LoRa networks are predominantly rolled out by the NTCs – National Telephone Companies – but you also see new communication providers popping up such as DIGIMONDO in Germany, Wireless Things in Belgium et cetera. This because there are no frequency auctions for LoRa. So far so good, but LoRa also has a small caveat: roaming is under development – expected this year.

Conclusion

With new communication providers popping up out of nowhere and NTCs pushing rollouts of LoRa like there is no tomorrow there is a lot of turbulence in the IIoT communications space.

Today no cross-border roaming IIoT provider is out there supporting a protocol with a ‘long’ lifespan. Today LoRa is, in Europe, one of the best alternatives to focus on.

Closing Notes

In this post I have not taken LTE-M into consideration as it is becoming available this year(1):

LTE-M rollout will likely be fast as it can utilize existing mobile phone infrastructures. I recommend you to read the Ericsson White Paper: Cellular networks for massive IoTand to keep an eye on this space. But also don’t lose track on Ingenu (promising(2)),NWAVE and NB-IoT. Some expect that NB-IoT will ‘crush’ LoRa and SigFox(3) .. just furthering the headache.

Weightless was left out of the consideration in this article as it is only available in a few different European cities and is more mid than long range .. but hey it may well suite your IIoT needs!

Due to the turbulence and changes in communications land this article very likely needs to be revisited in 3-to-6 months from now.

Finally: If you are looking to set-up a private LoRaWAN network, or wanting to play around with LoRa possibilities there is not much stopping you. For approximately 280$ you can have your own LoRa network – have a look at TheThingsNetwork.

UREASON

UREASON has been at the forefront of IoT /IoE, reasoning over real-time streaming data and events in the manufacturing industry and telecom. We apply an ensemble of techniques – best fitting the requirements – and a wealth of knowledge focused on providing a tailored response to the environment of our customers.

Our capabilities in the Industrial Internet of Things field include:

  • Feasibility studies and Proof of Concepts including hardware prototyping and field tests;
  • Support and roll-out of IIoT solutions in Operational Safety and Predictive Maintenance;
  • Recommendations for human-cyber physical systems, augmented reality and Internet of Things technologies; and
  • Support in Machine Learning and Big-Data initiatives supporting IIoT applications.

References

(1): Cellular IoT alphabet soup, Ericsson Research Blog, 2016

(2): RPMA Technology for the Internet of Things, Ingenu

(3): Vodafone to 'Crush' LoRa, Sigfox With NB-IoT, 2016

Read more…

The Next Frontier for Developers: IoT

Development for the Internet of Things has grown substantially over the past 12 months according to the newly released Global Developer Population and Demographics Study from Evans Data Corp.  

The number of developers currently working on IoT applications has increased 34% since last year to just over 6.2 million today. In addition, the increase of development for mobile devices, up 14% since last year, has led to smartphones being the most commonly connected IoT platform.  

The study, which combines the industry’s most exhaustive developer population model with the results of Evans Data’s biannual Global Development Survey also provides fresh population data for the four major regions: North America, APAC, EMEA, and Latin America and for more than 40 countries. Population numbers for adoption of the hottest tech areas are also included.  

“We're seeing how in the space of just a year, the possibilities introduced by the Internet of Things has attracted many developers.” said Michael Rasalan, Director of Research for Evans Data Corp.“This transition to IoT, while not without barriers, is rapid, because developers are able to leverage existing knowledge and expertise in complementary technologies like cloud and mobile, to create entirely new use cases. We're also seeing developers branch out from concepts centered on wearables to applications for more complex tasks, seen in the industrial space.”  

For the general developer population, estimates and projections for growth to 2021 show APAC leading the pack with nine hundred thousand more developers than EMEA. Growth in India and China are predicted to keep APAC’s population the highest globally for the next several years.  

The full report can be found here.

Read more…

Drones – A hacker’s playground

Original Post from: Vulpoid

Unmanned Aerial Vehicles (UAVs) offer new perspectives, both from a civilian and a military standpoint; yet, they present vulnerabilities having the potential to lead to disastrous consequences regarding public safety if exploited successfully, as evidenced by recent hacks. These repercussions can be prevented by implementing best practices, continuously assessing the technologies used and most importantly by remaining aware of the environment, of the weaknesses that may be exploited and of the threats that may emerge. The purpose of this article is not to provide countermeasures or solutions, but to outline flaws and vulnerabilities to better understand and address potential threats and threat actors.

timeline

Figure 1 UAVs hacks disclosure timeline

As shown by recent hacks, several professional Unmanned Aerial Vehicles (UAV) used byarmed forces, governments, police departments and the private sector are vulnerable to critical attacks which exploit both technical vulnerabilities and design flaws. This can lead to UAVs being spied on, made inoperable or controlled by the attacker unbeknownst to the UAV’s owner.

ecosystem

Figure 2 Drone’s ecosystem vulnerabilities

op_anar

Figure 3 Operation Anarchist base location

From a military intelligence perspective, it’s a godsend to gather valuable information. The GCHQ/NSA joint Operation Anarchist[1] during which Israeli drones’ scrambled video signals were intercepted and reconstructed, providing the US and UK a clear view of Israeli drones’ position, movements, payload and video footage is the perfect example. The Operation Anarchist – which started in 1998, lasted more than a decade and was disclosed only in late December 2015 – was run from the Troodos Mountains, Cyprus, from where encrypted video signals between Israeli drones and their bases were intercepted and unscrambled using open-source software tools.

The obvious drawback however for governments is that the same techniques can be used against them and become a serious threat, particularly when it comes to State security and notably for law enforcement agencies. While entry-level drones present vulnerabilities, their main purpose seems to be to reduce cost. IBM researcher Nils Rodday proved that high-end drones were also vulnerable when he studied professional quadcopters used by law enforcement agencies in the context of his Master’s Thesis[2]  in 2015. He showcased the results of the hacks during the RSA Conference 2016[3]. He also analyzed the quadcopters and discovered that the on-board chips lacked encryption implementation[4] which allowed him to hijack the drone by emulating the commands sent to the UAV through the controlling application. Furthermore, he took advantage of the weak encryption (WEP) used to cipher the link between the drone and its controller.

drone_archi

Figure 4 Drone flow architecture, by Nils Rodday, used in his work

In addition, concerns regarding homeland security have emerged, as shown by the case of the Mexican drug cartel who, in late December 2015, managed to control the US Customs and Border Patrol (CBP) drones’ movements[5]. This allowed the cartel to reroute the US CBP’s drones and to illegally cross the US-Mexican border, enabling them to smuggle drugs and people without being detected. The cartel used GPS jamming and GPS spoofing techniques which respectively disrupted the Command and Control (C&C) link, preventing the drone from receiving GPS signals, overriding legitimate ones and replacing them with fake ones, thus making it deviate from its original route.

gps_jam_spoof

Figure 5 GPS jamming / GPS spoofing

IDF

Figure 6 IDF Drones System hack

Moreover, warfare methods continually evolve and actors integrate new technologies in their arsenal, leveraging on them during actual conflict as evidenced by the recent hack of the Israeli Defense Forces’[6]  (IDF) drone surveillance system. The hack was perpetrated by a member of the Islamic State who gained access to HD footage from IDF’s drones hovering above the Gaza Strip for at least 2 years, starting in 2012 but potentially up until the arrest in February 2016. As a matter of fact, using only commonly available tools such as a satellite dish and a radio receiver[7], the hacker was able to intercept IDF’s drones’ video streams and managed to decode them, thus providing the Islamic State with a clear view of IDF’s drones video footage.

As evidenced by the aforementioned examples, attacks take place in several heterogeneous contexts and originate from actors belonging to different domains and with different levels of skill. In all the above mentioned cases, these events highlight weaknesses and vulnerabilities in the technologies used by UAVs along with flaws in the processes that were put in place by the victims regarding information handling. Military, law enforcement and governments are critical targets; however, the private sector is not spared as drones are being considered more widely by corporations for new services – Amazon Prime Air is a great example of that and may result in very annoying hacking opportunities to say the least. Consequences may be a matter of national security and public safety, therefore implementing best practices, setting up proper countermeasures – such as spread spectrum modulation in the case of signal jamming – and using state-of-the-art technologies proves itself crucial, yet being aware of the threat landscape one’s facing along with one’s own kill-chain is fundamental in order to avoid and mitigate such cases at best.

View the original post by clicking: Here

Read more…

Originally Posted by: Mimi Spier

The Internet of Things (IoT) is here to stay—and rapidly evolving. As we try to make sense of IoT’s impact on our lives and businesses, we also continue grappling with the security challenges.

As the IoT security landscape evolves, here are five key insights for designing and implementing IoT deployments for your enterprise.

5 IoT insights vmware airwatch

1. Protect Your People

IoT has opened up a world of possibilities in business, but it has also opened up a host of ways to potentially harm employees and customers. A security breach is not limited to stealing credit card data, anymore. Anyone with the right access could breach firewalls or steal health records. A key challenge of the IoT world is providing the right access to the right people at the right time.

[Related: 5 Real Ways to Enable IoT Success in Your Enterprise]

2. Watch Your Things

As millions of “things” start joining the enterprise network, it also expands the surface area for hackers to breach your system. All these devices will be leveraging public Wi-Fi, cloud, Bluetooth networks, etc., which will create multiple points of vulnerabilities. Your system needs to be designed for security from the bottom up to account for:

A) Device level: better quality devices

B) Data level: encryption and cryptology

C) Network level: certificates and firewalls

D) Application level: login/authorized access

3. Poor Quality of Things

The standards for IoT hardware and software are still evolving, which means until we have any established guidelines, we need to account for a vast range in the quality of “things.” Some of these may be very sophisticated and hardy, while others may be of the cheap disposable variety. Which devices you pick may depend upon factors like cost, usage and the use case itself. However, be warned that lower-quality devices have been used to gain entry to a secure network.

“By 2020, more than 25% of identified attacks in enterprises will involve the Internet of Things (IoT), although the IoT will account for less than 10% of the IT security budget.” Gartner

4. Is Your Network Ready?

One of the biggest challenge for any IT department implementing company-wide IoT projects will be assessing and managing bandwidth. As millions of devices join your network at increasing rates, scaling your network’s bandwidth will be an ongoing struggle. Your bandwidth needs must remain elastic, so you can support your enterprise needs, while minimizing costs. It is critical to minimize exposure of your networks by using, for example, micro-segmentation.

5. Data Is Your Friend

As with protecting any system, predictive maintenance is the way to stay a step ahead of breaches. The usual ways of pushing out timely security patches and software upgrades will continue to be helpful. However, one big advantage of IoT is the sheer amount of data it generates. You can track operational data to create alerts based on anomalies in the system. For example, if someone logs into the system from Atlanta and then, 30 minutes later, logs in again from Palo Alto, the system should raise a red flag.

You can view the original post by clicking Here.

Read more…

Guest post by Ruud Wetzels. This article originally appeared here

This is a bold claim perhaps, certainly as it is made by a data scientist with no particular knowledge of machines, trains or planes, or any experience in running a factory. Yet I am convinced that in your data we can find clues for improving your maintenance process, be it in the form of improved up-time, reduced costs or extended operational lifetime of your equipment. The reason I'm confident is that nowadays the power of computers to find patterns and correlations in vast amounts of data far exceeds the ability of any human to detect such clues.

Killer-app

The technique is called predictive maintenance, and it has been named the killer-app of the Internet of Things. I could agree to that. Predictive maintenance is about something tangible and familiar from everyday practice, it's relatively easy to start with without a need for significant investments up-front, and the results become evident fairly quickly. In other words: predictive maintenance has the potential to quickly yield real benefits, both operational and financial.

It all starts with data

As most machines and equipment in use today are fitted with sensors that gather data, there is usually an abundance of data to work with. Even if this sensory data is not yet collected for maintenance purposes, often the amount of data that is available - say in maintenance logbooks or financial databases - is sufficient to get started. And of course external data about anything that might be relevant can be included, weather data for instance.

The art of data analytics

The next steps with predictive maintenance are to feed all available data into a model and identify the appropriate algorithms to analyze those data with. This choice of algorithms by the way is really what constitutes the art of data analytics, it is the very step that determines the quality of the results. These algorithms then turn all those data into a single number: the probability that a certain event - in the context of maintenance: a failure or a breakdown - will happen within a future time slot, say the next 72 hours. Provided that the input data is cleaned-up and ready for use, such a model could be up and running in a matter of weeks.

Making tradeoffs

What remains to be done is to establish a threshold for intervention: at what probability level of failure will you intervene and perform the appropriate maintenance activity? This is how the maintenance process can be tailored to specific business objectives, which in my view is one of the major benefits that predictive maintenance offers. It allows railway companies and airlines for example to make an explicit trade-off between customer satisfaction (i.e. run on schedule) and safety (i.e. accept delays in case predictive maintenance is warranted), a trade-off that will probably be different in both cases.

 A leap of faith or an evidence-backed claim?

At this point the maintenance process has in effect been formalized. It is no longer run based on experience, intuition or a fixed set of instructions. Instead it is governed by an algorithm that produces a single number. To many companies, this is a somewhat scary thought, at least initially. Whether companies are willing to adopt such a formalized process depends to a large extent on human factors: how much affinity and understanding do management and maintenance engineers have of data analytics and what it can accomplish? In case the step to predictive maintenance requires too big a leap of faith, it can be run in parallel to the traditional maintenance process for some time, without making any interventions based upon its predictions. This way companies can gather enough evidence that I'm confident will back up my claim: predictive maintenance is the superior way of doing maintenance.

Photo courtesy of Teruhide Tomori 

Read more…
A security-first approach to developing IoT device software is critical, a key ingredient is an end-to-end threat assessment and analysis. A threat assessment includes taking stock of the various physical connections, potential losses/impacts, threats and the the difficulty of the attack. Importantly, addressing these threats needs to be prioritized based on likelihood and potential impact.
Read more…

Credit: ShutterstockCredit: Shutterstock

By Ben Dickson. This article originally appeared here.

The huge benefit that the Internet of Things (IoT) brings to different industries and domains is driving its growth and adoption at an unrelenting pace. Soon billions of connected devices will be spread across smart homes and cities, harvesting data, sending it to huge repositories for analysis and processing, and carrying out commands sent from smart apps and machine-learning-based systems.

While larger numbers of smart devices will unlock bigger opportunities for efficiency, energy and cost saving and revenue increase, they’ll also trail along some serious challenges and difficulties, some which are notably not addressable with current technological and communication infrastructure.

What’s wrong with centralized communications?

As is, all IoT ecosystems depend on client/server communications, centralized trust brokers and protocols such as SSL/TLS or mechanisms such as the Public Key Infrastructure (PKI) to identify network nodes and control communications.

These technologies have proven their worth for communications between generic computing devices for years, and will continue to respond to the needs of small, closed IoT ecosystems, like smart homes. But with the growth of IoT, centralized networks will soon become the bottleneck and cause lags and failures in critical exchanges because of too much network traffic, to say nothing of the extra investment they’ll require in terms of hubs and communications hardware. Imagine what would happen if your smart defibrillator failed to receive a command because your dishwasher, toaster, fridge, kettle and lights are having a nice M2M chat and have clogged up the network.

Decentralizing IoT networks

A solution would be to decentralize IoT networks in order to improve speed and connectivity. In many cases, substituting over-the-internet connectivity for local communication between devices will help increase speed and efficiency. After all why should a command exchange between a smartphone and light-switch have to go through the internet?

However achieving decentralization will present its own set of challenges, namely in the realm of security. And we know that IoT security is much more than just about protecting sensitive data. How do you make ensure security in communications between devices?

Devices would have to be able to communicate in a peer-to-peer manner and ensure security and integrity without the intervention of or dependence on a centralized trust center. The proposed system would have to protect the network and ecosystem against device spoofing and man-in-the-middle (MittM) attacks and make sure each command and message that is exchanged between nodes in a network are coming from a trusted and authenticated source and received by the right recipient.

How blockchain addresses the problem

Fortunately, the decentralization problem has already been solved in another popular technology: Bitcoin. The famous cryptocurrency is powered by a less-known (but no less exciting) technology named blockchain. The blockchain is a data structure that allows the creation and maintenance of a transaction ledger which is shared among the nodes of a distributed network. Blockchain uses cryptography to allow participants to manipulate the ledger without going through a central authority.

The decentralized, secure and trustless nature of the blockchain make it an ideal technology to power communication among nodes in IoT networks. And it is already being embraced by some of the leading brands in enterprise IoT technologies. Samsung and IBM announced their blockchain-based IoT platform called ADEPT at the Consumer Electronics Show (CES) last year.

When adapted to IoT, the blockchain will use the same mechanism used in financial Bitcoin transactions to establish an immutable record of smart devices and exchanges between them. This will enable autonomous smart devices to directly communicate and verify the validity of transactions without the need for a centralized authority. Devices become registered in blockchains once they enter IoT networks, after which they can process transactions.

There are many use cases for blockchain-based communications. A paper published by IBM and Samsung describes how blockchain can enable a washing machine to become a “semi-autonomous device capable of managing its own consumables supply, performing self-service and maintenance, and even negotiating with other peer devices both in the home and outside to optimize its environment.”

Other IoT domains can benefit from blockchain technology. For instance, an irrigation system can leverage the blockchain to control the flow of water based on direct input it receives from sensors reporting the conditions of the crops. Oil platforms can similarly use the technology to enable communications between smart devices and adjust functionality based on weather conditions.

What are the challenges?

In spite of all its benefits, the blockchain model is not without its flaws and shortcomings. The Bitcoin crew itself is suffering from inner feuds over how to deal with scalability issues pertaining to the Blockchain, which are casting a shadow over the future of the cryptocurrency.

There are also concerns about the processing power required to perform encryption for all the objects involved in a blockchain-based ecosystem. IoT ecosystems are very diverse. In contrast to generic computing networks, IoT networks are comprised of devices that have very different computing capabilities, and not all of them will be capable to run the same encryption algorithms at the desired speed.

Storage too will be a hurdle. Blockchain eliminates the need for a central server to store transactions and device IDs, but the ledger has to be stored on the nodes themselves. And the ledger will increase in size as time passes. That is beyond the capabilities of a wide range of smart devices such as sensors, which have very low storage capacity.

Other challenges are involved, including how the combination of IoT and blockchain technology will affect the marketing and sales efforts of manufacturers.

It’s still too early to say that blockchain will revolutionize and conquer the IoT industry. But it sure looks like a promising offer especially if its challenges can be met. We’ll see more of this in the coming months and years, as IoT continues to grow and become more and more ingrained in our lives.

Read more…

Upcoming IoT Events

More IoT News

Arcadia makes supporting clean energy easier

Nowadays, it’s easier than ever to power your home with clean energy, and yet, many Americans don’t know how to make the switch. Luckily, you don’t have to install expensive solar panels or switch utility companies…

Continue

Answering your Huawei ban questions

A lot has happened since we uploaded our most recent video about the Huawei ban last month. Another reprieve has been issued, licenses have been granted and the FCC has officially barred Huawei equipment from U.S. networks. Our viewers had some… Continue

IoT Career Opportunities