Lack of Data Ingestion Framework Support Can Leave You As Misguided As A 16th-Century Explorer

By Tony Lapolito

This is the (in)famous Waldseemüller Map, first drawn in 1507. Legend says Mr. Waldseemüller decided to label the newly found continent America (after Amerigo Vespucci). The map was a big hit, and as it was (hand) copied - each cartographer copied “America,” and the name stuck.  That is why today I live in North America and not North Columbia.

Waldseemuller Map Free Image-2

What is astounding is how little was known about the world then - especially the area between Europe and Japan.  Today we know the names of Magellan, Drake, Hudson, Cook, Cortez, and others (yes, one of my favorite grade school topics). However, we do not talk about the names of Knight, Franklin, the Corte-Real brothers (all looking for the Northwest passage), and the 100’s other explorers that went missing because they were ‘flying blind.’ They all made terrible decisions based on a lack of insight not found on the maps of that time.

When managing connected devices - you want complete insight - missing data can be as fatal as sailing off the edge of the earth. You can be fully aware of what is going on only through the visibility of all of your devices and all the pertinent information.

In today's connected device economy, data management is becoming increasingly complex. Data is generated from various device types - all speaking different languages. So it is not sufficient to say - we have most of the protocols covered. That leads to holes in your maps (insights).

To effectively manage this data, you must have the right DeviceOps platform, including support for all ingestion protocols. Ingestion protocols transfer data from external sources into a data management system. A variety of protocols can be used for this purpose, each with its own benefits and drawbacks. These include RESTful APIs, webhooks, messaging protocols (e.g., MQTT, AMQP), file-based protocols (e.g., FTP, SFTP, SCP), database protocols (e.g., JDBC, ODBC), and streaming protocols (e.g., Kafka, Kinesis).]

Sometimes it can become alphabet soup. What’s BACnet - you won’t care until you are working with a building management system.  OPC UA - huh? You’ll care if you are working in industrial automation. The point is when scanning vendors' data sheets - you might just see a bunch of obscure protocols that you rarely think twice about until your device isn’t supported and you now can’t integrate that portion of your fleet into your systems.

So why is it essential for your DeviceOps platform to support multiple ingestion protocols? 

Increased Flexibility: By supporting multiple ingestion protocols, organizations have greater flexibility in managing different device types.

Improved Interoperability: Supporting multiple protocols allows for better interoperability with different systems. This is especially important in today's interconnected world, where data needs to be shared across various platforms and applications.

Enhanced Scalability and Performance: Different ingestion protocols have different performance characteristics, and some may be better suited for handling large volumes of data.

Future-Proofing: Organizations can future-proof their data management systems by supporting multiple ingestion protocols. As new protocols emerge, they can add support for them without having to replace their existing infrastructure.

While there are many benefits to supporting multiple ingestion protocols, there are also some challenges. For example, supporting numerous protocols can increase the complexity and development costs of supporting all these protocols in your DeviceOps platform. In addition, there is a constant need to add new protocols as they become relevant - even the ones that are not so relevant.  

Does EdgeIQ support every known protocol?  No, but we are committed to Universal Data Ingestion.  Our architecture is designed to quickly add new protocols to ingest, normalize and integrate different device types as data sources.

Some of the protocol’s support by EdgeIQ include:

Shell Polling
Cloud HTTP Ingest
GCP Pub/Sub
Assetlink Cloud Polling
Inmarsat Cloud Polling
ORBCOMM Cloud Polling
D-Bus Signal
Linux Dev
HTTP Client
HTTP Server
ICMP Polling
Shell Polling
SNMP Polling
TCP Client
TCP Server
UDP Client
UDP Server

So if you want to put your name on the map, like Vespucci, and not have your career go missing - when selecting a DeviceOps platform, ensure that you can support your current device data protocols and those you might be adding in the future. New data ingestion protocols are being developed all the time. By finding a vendor committed to supporting a broad range of protocols, you can ensure that you are always using the best possible technology.

You might not be able to see beyond the horizon, but one thing for sure is that the future is unknown, and surely someone will walk into your office and say we need to support this totally unknown and unexplored protocol.

Want help with DeviceOps and business transformation? Should we be partners? Or would you like to join the team and be a part of our mission? Whatever the reason, let’s connect!

Related Articles }}


Introduction of Lightweight M2M (LwM2M) in EdgeIQ's DeviceOps

EdgelQ, a leading provider of DeviceOps solutions for the connected product economy, is excited to ... }}


The Basics of Zero Touch Provisioning

At EdgeIQ, we provide DeviceOps software for the lifecycle management of any connected product ... }}


Disruption Requires Disruption

At EdgeIQ, we provide DeviceOps software for the lifecycle management of any connected product ... }}


A Voice from the Industry: Anand Rau, IoT and Telematics Expert

At EdgeIQ, we provide DeviceOps software for the lifecycle management of any connected product ... }}


Zero Touch Provisioning: Claiming and the Application Ecosystem

At EdgeIQ, we provide DeviceOps software for the lifecycle management of any connected product ...