Big Data cloud the Internet of things: what the IoT platform and why they are needed

Image 1

The most intelligent part of the data analysis work is done in the cloud using special big Data tools combined into a common platform. Let’s talk about the features of IoT platforms and the technologies on which these cloud solutions are based. We have also prepared for you a brief overview of the most popular platforms for the Internet of Things.

What is an IoT platform and why it is needed

IoT platform is an internet of things system software for connecting and remotely accessing end devices (sensors, controllers, etc.) to the cloud. The purpose of the IoT platform is to provide seamless integration of various hardware through special interfaces, communication protocols, network topologies, as well as storage, processing and data mining.

There are the following 8 mandatory components of a full-fledged IoT platform:

1. Connectivity & normalization – information of different protocols and data formats in a single software interface ensures accurate transmission of information and interaction with all terminal devices.
2. Device management – ensuring correct and uninterrupted operation of end devices, their configuration and updating of software applications on them and edge gateways.
3. Database – providing scalable and reliable storage of information.
4. Processing & action management – monitoring the current and forecasting the future state of technological equipment on the basis of data from the end devices that are installed on it, as well as the development of commands to change the state of the equipment and the transmission of these signals to the actuator.
5. Analytics – integration and clustering of data, as well as forecasting the values of the required parameters, including using machine Learning methods (Machine Learning);
6. Visualization – a visual representation of the collected, processed and analyzed information in the form of graphs, charts, tables and other understandable representations;
7. Additional tools – tools that help developers and DevOps engineers to expand the functionality of the platform of the Internet of Things using the graphical interface and programming;
8. External interfaces – APIs, SDKs, and gateways for integration with other services, systems, and platforms.

Image 2 Required IoT Platform Components

Key technologies Big Data cloud platforms for the Internet of things

The reference architecture of IOT platforms is described by the international standard “ISO/IEC 30141:2018. The Internet of things is a reference architecture.” This standard, which has not yet been fully translated into Russian, is necessary to provide a single framework for DevOps engineers, developers of IoT platforms and IOT applications to create reliable, secure and fault-tolerant solutions. I IoT products within the framework of the reference architecture allows businesses to strengthen the effect of the introduction of new technologies, reducing the risks of their use.

Most modern IOT platforms provide real-time data mining using the following big data tools:

* aggregation and filtering of data flows (storm, samza);

• support for batch operations with the accumulated big data set (hadoop and spark tools);

* Machine learning for streaming and batch data (Spark, MLlib).

It Platforms also use the following big Data technologies:

* TCP TCP / IP CoAP, HTTP/HTTPS;

• messaging protocols in the concept of “publisher-subscriber” (MQTT, for amqp Protocol XMPP, DDS), implemented in software brokers in RabbitMQ server Apache Qpid, Apache in particular, ActiveMQ, as well as in Apache Kafka, which is considered the most scalable queue management tool;

* fast download of streaming data from the gateway and end devices (Apache NiFi, Apache MiNiFi, Apache Flume).

Most Popular IoT Platforms

The most popular cloud platforms for creating Internet of Things systems are Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform, Artik from Samsung Electronics, Cisco Cloud Connect, Salesforce Cloud, Watson and BlueMix from IBM, OpenStack. In 2018, they were joined by Kubernetes, an open container orchestration system for automating the deployment, scaling, and management of cloud applications. Kubernetes can not be called a specialized solution for the Internet of Things – the popularity of this technology is associated with its spread in the field of Big Data systems due to the convenience of managing microservice applications.

Image 3 The most popular cloud platforms among developers of Internet of Things according to the Eclipse Foundation

The most important criteria for distinguishing Internet of Things platforms from each other are the following:

• scalability – the maximum number of end devices that can connect to the platform, the complexity of such an extension, and the ability to effectively balance the load on the servers;
• ease of use – flexibility of integration APIs and ease of program code management;
• database – in what form/volume are stored large and small data received from end devices, the presence of hybrid cloud storage, etc.;
• deployment options – public or private cloud;
• security – encryption, user access control and other means of information protection.