1.1 INTRODUCTION
1.1.0 Definition
Distributed-SCADA-Networking (D-S-N) is an open computer
architecture for Supervisory Control And Data Acquisition (SCADA)
consisting of targeted protocols and conventions.
1.1.1 Purpose and Strategy
Typical of all laudable free-market ventures, D-S-N seeks to
drive down development and operational costs while increasing
worth, utility and robustness; in this case, for SCADA. Its
strategy would consolidate commercial off the shelf (COTS)
instruments, embedded controllers and supervisory computers of
disparate manufacture into SCADA systems safely using the
Internet and other emerging technologies. D-S-N intends to expand
component choice and link the new eclectic within a specialized
network hierarchy. This is not off the rack protocol, altered to
fit; D-S-N was tailored for SCADA.
1.1.2 D-S-N, A New Idea?
At its core, D-S-N is a simple idea: it divides complex
applications into small focused modules with standardized
interfaces; then it provides communication among them. Each
module is given an address used by D-S-N executive software to
route information packets among modules. The result is a
virtual private network -- the dVPN layer.
This idea is closely related to object-oriented programing (OOP).
D-S-N modules are objects. They collaborate using messages and
each chooses its appropriate method based on these data. Readers
will recognize even more, not exactly new, OOP pieces below this
surface. Yet, D-S-N is new: a new amalgamation of tested ideas
forming an alloy delivering coherent, comprehensive
consolidation. It is object-oriented system. A big idea for
SCADA.
1.1.3 D-S-N is Generalized Architecture.
D-S-N is a generalized architecture with minimal overhead. It
structures SCADA systems large and small. D-S-N describes obvious
universal hierarchies with useful ambiguity, providing enough
definition to scale the scope of most projects while allowing
extension when needed. Four billion devices are embodied without
extension.
1.1.4 SCADA Is Not Always The Name.
Control and data acquisition systems are not always called SCADA.
Complex scientific experiments are not called SCADA. Nor are
fast-food ovens, fryers, cash registers and drive-through
windows. While power distribution and oil refineries do use the
term, the instant examples are all supervisory control and data
acquisition systems. The Mars Rover is a REMOTE terminal unit
(RTU) in a SCADA system. The name notwithstanding, the "D-S-N
Overview" is written for technical people purchasing, managing,
maintaining or designing such systems. It intends to be
worthwhile for them. For others...
1.2 THE BIG PICTURE
1.2.1 SCADA Is Incompatible COTS.
SCADA is system: always a plurality; each device does part of a
bigger job. SCADA is commercial off the shelf embedded
controllers, servos, instruments, valves, relays, displays,
solenoids, variable speed drives and substations to solar cells
to windmills to flashlight batteries. So, SCADA is largely COTS
and COTS is intrinsically incompatible. D-S-N intends to
consolidate it.
1.2.2 SCADA Is Scattered COTS.
SCADA is usually distributed. Classically, it is: a group of RTUs
collecting data and controlling devices connected to other places
where data are analyzed, stored and monitored; virtually always,
there are supervisory points for directive and corrective action.
So, SCADA must have communication links. And, of course, it does.
Systems usually include some of these: RS-232/422, BlueTooth,
FireWire, USB, cellular, PSTN, radio modems, microwave,
satellites and even Internet links. But, could the link protocols
be more diverse? Easy SCADA network transparency and
consolidation is promised by D-S-N's dVPN layer.
1.2.3 D-S-N Masks Vendor Incompatibilities.
The good thing about standards is: there are so many of them.
COTS vendors do use interface standards -- almost all of them.
Beyond the sarcasm, vendor diversity is at least necessary and,
on balance, good for developers and users alike -- even with
interfaces issues. D-S-N converts vendor specific connection into
an appropriate system-wide internal standard. It converts
immediately: in the software module managing the COTS device.
Thereafter, the system uses the application's notion of state and
engineering values. This reduces and often eliminates device
protocol compromise.
Not just the external device connection needs standardization;
the software interface between the module and dVPN must also be
portable. D-S-N standardizes this interface. Another operating
system runs old code. A new location may change a module's
address but not its source code. Again, D-S-N consolidates.
1.2.4 COTS Becomes Portable.
The combination of immediate encapsulation and standardized
internal interface yields transparent portable objects: modules
sharing methods and data -- perhaps, a class. The brand-change
penalty for exchanging instruments, embedded controllers and
RTUs, as well as human-machine interfaces is reduced when modules
are "class-mates." OOP classes are freely exchanged and
rearranged. The next project can inherit these objects -- as
written and already tested.
1.2.5 D-S-N Dynamically Organizes Existing Links and Computers.
D-S-N adds dVPN within current infrastructure. It transparently
uses: Internet, cellular, satellites, USB, etc. Diverse computers
(PICs to Crays) can join dVPN. Moreover, D-S-N's protocols and
conventions can dynamically reorganize dVPN connected
applications (plug-and-play).
1.2.6 Hierarchy Is Used To Address Dmail.
The dVPN is hierarchically structured with the equivalent of
countries, provinces and towns. Just like sending a letter, each
level contributes part of an address. A mail truck driver heading
East from Denver need not know about stairways in the building
housing Apartment 207, only how to get to the Chicago Post
Office. D-S-N "Dmail" works the same way. Part of the address
describes the big move. Another part describes how far down the
hall.
1.2.7 Dmail Moves Packets.
Standardized packet structures move commands and data to any
point in the network -- regardless of the link type -- even when
no link is apparent. Applications sending information a few
microns in memory use exactly the same code to cross an ocean.
Use any or all of the above links and network connections remain
facile, robust, portable and transparent. Dmail is the same
everywhere.
1.2.8 Security Is Supported.
D-S-N supports security. Both the protocol and conventions
provide sentinels to protect the veracity of the network and
catch perpetrators. Examples: counterfeit addressing is
prevented, standard signal packets exchange security check
information, only verified control modules can command an
application. Existing link encryptions, like SSL, are supported
and end-to-end encryption is supported using security check
information.
1.2.9 D-S-N Mitigates Network Variability (TIME).
Other systemic network issues are also addressed. Commonly, the
time at which data were collected is wrongly assumed to be when
it arrived at the collection point. This assumption can lead to
disappointing temporal precision due to network latency.
Transmission latency on network links of the past was often
predictable; then, the time at which data were generated was a
simple offset from time-of-arrival.
Today's shared networks exacerbate temporal issues with another
uncertainty. On the Internet, latency varies because it is
unpredictably delayed by other traffic. Even the route may
change. While the Internet's dynamic environment adds robustness
and ubiquity, it lacks temporal consistency.
Sending a command suffers like collecting data. Uncertain time of
arrival can confound precision state changes. Internet connected
SCADA systems must deal with this issue. New techniques do.
A basic architectural feature mitigates the effect of network
transport variability and latency. As detailed later, the time
issue becomes almost unimportant. D-S-N provides high-fidelity
temporal data correlation and process coordination. As you will
see, it is a simple solution.
1.2.10 D-S-N Supports Many Simultaneous Applications.
Once the first application establishes the virtual private
network, it can be used for many concurrent applications needing
linkage over common geography. Each system or subsystem has a
unique "DSYS" identifier used for control, plug-and-play
organization and trouble shooting. Except for network capacity
demand, there is no unintended interaction between applications.
DSYS hierarchy is independent of addressing hierarchy. Addresses
route packets; DSYS organizes applications. DSYS encourages
dividing large applications into more tightly focused subsystems
of specialized modules.
1.2.11 Distribute All Large Applications.
A decade of experience is compelling: D-S-N helps all large
applications. Programs not considered SCADA and not even
distributed beyond a single CPU, are served by D-S-N
architecture. Design, using small transparent connected modules,
urges elegant structure and demands crisp module definition with
unambiguous interface. Except for system level design and test,
concentration becomes modular. Need-to-know is modular. Each
module's performance is verified independently. Modular strategy
fosters multi-vendor development.
1.2.12 DSYS Error and Management Conventions
D-S-N error handling support reduces the effort needed to debug
and manage exceptions. The network environment itself provides an
error logger with an easy message connection. DSYS conventions
provide for an application manager and event handler -- again,
with easy standardized connection.
1.2.13 Portable, Transparent and Connected COTS is Obvious.
Development around transparency, portability and networking
issues stems from before the transistor. Again, D-S-N is not a
unique fundamental concept. Framing scattered pieces into
transparent portable modules then linking them within secure dVPN
is not and obscure direction for people skilled in the art. This
direction may be obvious but the map was elusive (more in the
history section below).
1.3 WHAT TO EXPECT FROM THE OVERVIEW
Using D-S-N is not as daunting as it might seem. The basic
protocol is defined, implemented and available as open royalty
free architecture.
The balance of the Overview explores the D-S-N promise and how it
delivers. It details the required core protocol, network
hierarchy, packet structures, addressing scheme, routing rules,
packet delivery scheme and the interface to the required
executive services.
The basic protocol is expanded using conventions suggesting a
good start: an application program interface (API), time stamp
conventions, event handling conventions, encapsulation
conventions, organizational numbering schema, signal packet
conventions and plug-and-play conventions using rendezvous
managers. These suggestions do not disqualify other efficacious
ideas.
Another important section discusses D-S-N and the Internet. There
are examples of systems and pointers to other documents with
expanded information.
1.4 OPEN ARCHITECTURE LOWERS RISK.
1.4.1 Does Free Software Exclude Profit?
M. K. Gandhi thought wealth without work was Humankind's greatest
blunder. When juxtaposed, work without wealth, it can be slavery
-- no less a blunder. Because the future of Humankind is
inextricably linked to the computer, D-S-N has a social
component. It intends to increase public wealth without stifling
private opportunity: make all of us better off -- yet, reward and
encourage smart hard individual work.
In the D-S-N world, profit is not a swear word. Developers have
and will continue to add freely to D-S-N's open platform. Most
also focus on proprietary efforts where they bring distinctive
value. D-S-N has a foot in both venues. For now, egalitarian,
enlightened self-interest, greed and more cosmic issues are set
aside.
1.4.2 D-S-N Adds Transparency Through Open Architecture.
Open architecture is transparency. Users have better visibility
into their systems. Less is hidden in the sealed box. Common
platforms have more people finding and fixing shared problems; it
is not often your neighbor's valve-job improves your engine's
performance. Moreover, transparency helps manage supplier
diversity. Among other things, it becomes easier to divide
requirements among evolving suppliers in a modular fashion.
1.4.2 What About Copyleft?
This royalty free architecture is licensed under Free Software
Foundation Copyleft agreements (gnu.org/copyleft). D-S-N
documentation (including protocol and convention) is licensed
under GNU FDL, the OpenLibraries under GNU LGPL and much
executive code under GNU GPL.
1.5 D-S-N SUPPORT
D-S-N.org is dedicated to open architecture maximizing
interoperability and unfettered development. With the help of
user-groups it manages and publishes interface protocols that are
available to the public, royalty free (see D-S-N.org).
1.6 D-S-N HISTORY
Since the advent of the microprocessor, D-S-N has tried to sprout
from scientific instrument, industrial control and power
transmission design. D-S-N is the kind of project developers want
to do -- if it could be funded. Usually, resources are tightly
restricted to an immediate problem. Coherent, comprehensive
consolidated architecture with generalize solutions would be
nice, but the instant mission is solved cheaper; so, a nice tune
is written. Composing a D-S-N was not funded. Alone, each
resulting tune is pleasant enough -- but, playing them together
is a cacophony not a symphony.
Occasionally, circumstance allows another possibility. In the
early nineteen nineties, D-S-N's evolutional form was influenced
by several year's of instrument development at the National
Center for Atmospheric Research (NCAR).
There, the need became
clearer. For the next decade, many following projects -- with
world-wide research institutions including: the U. S.
Environmental Protection Agency (EPA),
U. S. Department of
Defense (DOD) and National Oceanic
and Atmospheric Administration
(NOAA) -- helped refine and validate D-S-N
architecture. Within a
scarce concomitant, D-S-N took root. It was nurtured by projects
and clients bringing both public and private funding. It was my
privilege to be there.